So I've been animating with Eleanor for a while and animating with her was unbearable at some point. I decided to go check the console and I saw "invalid pixel shader index (out of range)" being spammed every milisecond. With that SFM's memory overflows.

What was the problem - no "$bumpmap" on the "eye" .vmt file. After I assigned her a bump - SFM and the model started to work fine.

Also, for those who wants her skirt to not appear invisible at certain angle - add "$nocull 1" at "red" .vmt file

Hope it helps!