I mean forts are strong but I haven't had that many issues being unable to win. Win with cost yes but never was afraid of actually losing

I tried another faction and I was able to beat the forts, maybe its just the quest fort for the Dust lords thats overtuned. A fort this early in the quest chain should not be something you need high tier units for, I bascially had no chance with a lvl 5 hero and lvl 2 units with specialisation.

I do think something might be bugged with Fortresses when playing Last Lords. Played a long game with the Kin and Fortresses worked as I would expect. Started new game with Last Lords. Played my first Fortress, not the early mission-based one, just a lower-level fortress with 711 rated army. My Army was at least 40 points stronger but roughly equivalent. I got annihilated. I noticed their units armor at least doubled when I tooltipped before starting the fight and when I was in the Fortress fight. Their units did considerable damage to mine, too. Their Tyrant was barely damaged by any of my units, Hero included. However, an AI with an equivalent army to mine was about a turn behind mine on the map. After my army was killed, the AI walked in, suffered light to moderate damage, just as I would have expected to based on the their stats. I tried the battle at least half dozen times and only once did I get near winning but the battle had a turn 8 draw.

An easy thing to remember is the early forts you only need to kill the lord. Check for a kill the target objective and you can just focus them down.

I think the strength estimate on the forts is fairly accurate. If you don't have a minimum of 20% more strength in your army, I recommend waiting until you do.

Cleared the fortress easily with Last Lords.
Maybe it's scaling with difficulty?

I dont usually have much trouble with the fortresses.

Don't forget to research the techs that lets you upgrade your units... and do the upgrades asap. I found that that helped me the most when taking on forts in mid-game.