You should list your computer specs, as even my 16-core PC* (3950x) with a 3090 and 64gb of RAM will slow down quite a bit when I knock a large building down. Of course, with ANYTHING huge it doesn't matter how much processor you can afford to throw at it, unless you own a huge super-computer it's NOT going to be real time even with 10x i9's or R9's.
This game seems to be REALLY good at loading all cores, in-fact it's one of the best ones I've seen when it comes to that, along with BeamNG Drive (a driving / crashing / soft-body physics vehicle simulator).
So yeah, get those extra cores, keep them cool and properly fed with power (good power supply and motherboard), don't run out of RAM, and have a decent enough GPU to handle the effects - and you might come out a few FPS ahead.
Or make slightly less big kabooms. That's a whole lot cheaper.
*My PC is 6 years old next month with a 2~2.5 year old GPU in it, and I've doubled the ram up to it's now current amount. I'm sure you can afford something newer than this, but if you run with at-least these specs you'll have a great experience even with half the cores (real cores not E-cores).
...Unless of-course, you have knocked down an entire 5+ story office complex/building involving millions of voxels. Each one to be processed affects the others in some ways, quickly expanding the computations outside of cache memory on the CPU. Thus, things slow, quite a bit actually. So while your mileage may vary, keep kabooms not-so-big if you don't like slowdowns.
e.g. Don't fire 300 exploding pianos at a structure and wonder why things get slow. That's a given.
What's also a given? This game, and what you can do, is awesome.