Well if you just have heavy infantry retinue you will almost surely get murdered in the long range face. It is adviseable to recruit some archers or light infantry as well for the skrimish phase.
Generally speaking a mixture is most of the time a good idea. Light cavalry for example is important to kill as many troops when the enemy flank is fleeing.

Battles are devided into three phases; Skirmish, Melee, Pursue. Different unit types are better or worse in different phases. Every battle starts in Skirmish phase and the skirmish phase will usually last at least 10 days unless one of the armies on a flank begin to route then that flank would switch to pursue. Most people suggest large groups of archers as a general rule because they are the second best (horse archers are better but are only available to some groups) in the skirmish phase and if you have excessive numbers there will be usually only be the skrimish followed by a short Pursue phase in a battle. This combined with the fact that storming holdings is based on skrimish damage make archer type damage desired for quick effective wars. It tends to be better to have more balanced armies when you don't a extreame numerical superiorty as the battle will likely draw out into the Melee phase and you are less likely to be taking holding by assult.

When I am large enough, I like my armies to be 4 melee retinues to 1 cavalry retinue, which helps out in the skirmish and pursui and doesn't hurt to much in the melee. and didn't they change it so storming holdings now requires melee?

Culture specific Retinues have better bonuses than normal retinues(Which have bonuses over levy) But Yeah all HI doesnt work unless you severely overpower them.