This is the first report about this that I'm hearing. Couple questions for you:

1) Does the server you play run use any custom ini changes for the bar fridge?
2) Are you using a stack mod or just default ark items?

Considering that honey, wyvern milk, and prime have the shortest spoil timers I'm thinking it could be that once the server restarts the fridge doesn't actually become powered unless someone gets within render distance of the structure - No power means that the spoil multipliers go back to their default short spoil times.

Some things that can be done to help test this (if you feel so inclined to help out):

1) Add a fresh milk or stack of honey and note down the spoil timer.
2) restart the server like you normally would
3) Wait 5-10 minutes before you log back in and then take a look at the spoil timers remaining on those items you just added.

If everything is working as it should then there should only be 5-10 minutes off the spoil timers (the time you waited after the server restarted). If you see 20+ minutes missing from the items then it likely means that the structure is not detecting its power state on reload and then I can investigate if this is something on the mods end or on arks end.

No stack mods, and I haven't changed the ini for the bar fridge or made changes to the game.ini for stacks. The global spoil timer is set to x2. I'm thinking you might be right on this one. In previous games we used the Redwood Feeding Trough but I've always felt it was a bit too cheaty.

I'll have to try the test you suggested as well as testing it on Ragnarok as currently we're actively playing on Valguero and I wanted to eliminate the possibility of it being a map bug.