First, I want to say that I'm not 100% sure this will help, but it might. It's worth a shot.

TLDR: Enable Triple Buffering.
It's not going to completely fix it. It's just going to make it a bit more subtle.

I sometimes had these odd frame drops when I looked at certain objects (mostly candles) and when I looked at distant objects.
The way I fixed this was to see what graphical option that caused this issue. First I thought it was shadows. Turned that down to medium and it worked. But a moment later I realized it wasn't gone. I tried to turn character detail down. Exact same result. Lastly, I enabled Triple Buffering. That seemed to be it! I'm not exactly sure this is actually the proper fix. I'll update when I get some more results!
Hope it helped! I know it was very short. I tried to cut the bs :-)