IMHO voting for best or worst Background Perk (or perks in general) is a play style issue.
Privileged can easily be countered with a Neural Torso.
But in case of a Heavy this means a different combat role as he can’t Jet Jump anymore. This may lead to a new (strong?) Heavy build. I’ve done it before (because I didn’t had a spare Heavy torso armor): equipped with light and speedy armor using a Deceptor MG at close range (maybe combined with Berserker’s Dash or Assault’s Quick Aim).

Well if he has quick aim, use a harrower then instead of Deceptor.. Yea neural torso works, but lose jump jets.. I'd rather keep the jets and just not select Privileged

Deceptor is available early game. Harrower is strong but late game.
I like it when the game encourages me to think about different builds/solutions.

True, very true

Or select privileged when the heavy strength is so high he can take it.
I also use Technician head and leg armour to reduce weight by 2 and increase rather than decrease accuracy that the heavy helmet and legs give. Tech amor is fairly high and gives an accuracy boost. Heavy armor subtracts accuracy

On other class like Assault and Berserkers it's less of an issue

As mar.sch mention, status of perk scales from playstyle. Privileged is pretty strong perk if you setup properly, but also ruin, if missplayed. Build wisely.