Sorry to comment on an old post, but some other people might come across this, or it might still be relevant to you. I remember this happening to myself on a server I was hosting with some friends. It was something to do with a mod mismatch, and it was super annoying to figure out. Something about load order or versions were out of sync or something, and it would cause the map to propagate differently for the other people, and apparently the only thing about the world and colony data that's synced by the server is other player's locations.

If I recall the ultimate fix ended up being to send everyone a zip with the exact mod files the server was running.