I have no idea if this will fix your issue, but if you're using Windows 11, make sure that .NET Host has a Graphics preference of High performance in the Settings app. Your hardware should be powerful enough to run the ultra preset at 60+ FPS. My laptop with a Ryzen 7 7840U and integrated graphics runs the ultra preset at around 40 FPS, and my desktop PC with a Ryzen 5 5600X and RX 580 4 GB runs the ultra preset at above 60 FPS.

Disable smooth lighting, that setting has been causing performance issues for me.

I've found that with the ultra settting, the enhanced glow affect can have a major performance impact. I have a 7800x3d and 4070ti and my frame rate drops to the high 40s - low 50s whenever I enter certain areas, especially when it comes to using this mod in conjunction with the Remnants mod

Smooth Lighting makes my game go from 60+ (Locked) to barely 30-35. Definitely some kind of issue happening there

Try throwing some optimization mods at it, I can recommend: High FPS Support, Nitrate and Not quite Nitrate