The more of the map you've discovered the lower the fps get with this mod enabled. I've found a few ways to get more fps without removing the minimap again (Because this mod is very useful).

1. You can change the update rate of the map. Everytime the minimap updates (and you already discovered most of your map) your game freezes for a short amount of time. If you set the update rate to 'once a second' your game will stutter every second, that's still better than 20fps all the time, hu?

2. The more your minimap shows, the lower the fps get. If you zoom in a bit it helps (doesn't make a big difference, though).

3. Try to deactivate the map if you don't need it. You will get used to finding your way without it and when you need it you can just open it again. (If it's closed the fps are not affected by it).

4. The biggest problem are mods that add icons on the map. Try to disable every icon you don't need. Incase you're a modder: Make a mod to disable the icons of twigs, grass etc. They are the reason for the fps issues.

I hope it helps and sorry for my broken english.