I've seen this error reported on every version of AoM going back to 2006, mods or no mods. Available RAM seems inconsequential. On the other hand, you're crowding the screen the highest-poly units ever ported to AoM and EE isn't optimized by any stretch of the imagination. I'm actually surprised you got as many units rendered as you did before it crashed.

Damn, so its an engine limitation. I was afraid that was the case. :(
I wish they'd relinquish the rights so someone could make this in unity or unreal.