Yes. I only could solve it by building small settlements not too far away from each other. I start with one big village, then build one or two small settlements (one storage, hut, fire, totem, well...). Then you can build a little bit further and further.

If the villages are too far away the villagers just spend too much time walking between them. It would be nice to be able to found a second main village that is independent from the first.

The main problem that I am having now is that the small places are super fragile against raiders...

I've found satellite settlements seem to work better if I don't go too far away and if I focus on increasing my population based within the new settlement right away. Having more people based in the satellite town seems to make things more efficient and provides more defenders to fight raiders. Usually I'll build up my resource/food production by quite a bit but hold off on adding houses until there's a significant buffer, then build the new houses all in the new settlement.
Also try stockpiling construction resources so you can build your storage, hearth, well, several huts, etc almost all at once, then wait until a time of year when your people aren't either planting or harvesting to start the build.

The AI seems to be built to manage a single colony. Two colonies, very close to each other, are manageable, after that it becomes more complicated ...

Have a good day !

If it is possible they could give villagers permanent homes and then create a "collection Area" type banner to set a town radius so that villagers with homes within the banners radius will work industries within that radius?

Or maybe a village center? They could add the build and collection area tabs to it to allow the placement of buildings and collection areas for that specific village. Production and livestock limits could be controlled from here as well. It could upgrade into a Forum type building.

Besides my OCD hates seeing the trader at different camp fires each time a remove one. :P

Imagine being able to build a mining town that trades for food and tools from the city?

Or villagers dancing around the fire at the village center or debating in the Forum? :)

I love the game as is and its only getting better, but what I love most is whats possible.