following up here: I had a similar issue - it turns out the colonists were hoarding weapons. One guy was carrying around 15+ bows, and another colonist had another 20 or ranged weapons in his inventory. I made them drop the bows and unloaded simple sidearms, and the lag went away.

This may be a mod conflict, and I'm still attempting to replicate this on-demand, but with little success. Posting for visibility though.