At long last the infamous bug can be fixed.

When you are getting the infinite download glitch, while still in play exit out of garrys mod (dont quit, just minimize). Go to your steam client, and hover over Library. You should see a little menu with three options. Press "downloads." Navigate to Garrys Mod and you see that "workshop content" is locked. This is our problem. Press the button at the far right with an arrow pointing up. Pressing the button unlocks the workshop content and when you click back on gmod, all your outfits should have downloaded. It is important to press the button on the far right, pressing the "Play" button on the left instead tries to launch gmod again.

While this does not permanently stop the infinite downloading glitch, it can be done every time it happens and will work all the time.