Make sure you don't have the self-illum layer in mask 1 (channel 4) white. If it's white, your whole model will be bright. You might also need to adjust some specular highlight channels.

I dont really got the whole masks thing yet (how to edit them like on the guide), but I did covered the Alpha channel of mask1 with black and left a tiny spot unpainted as the Valve guide said, it still made my model bright.

Can you show me? I can't really do much other than offer advice on what I think is wrong.

Like this

http://psteamcommunity.yuanyoumao.com/sharedfiles/filedetails/?id=153455359

Wow, that's really odd. I'm almost certain part of your mask textures are not being used properly. Try mimicking an existing juggernaut sword (dig through some of the game vmt files to see how valve set the channels up).

Well, I looked around Jugg's default sword materials and used his mask1 on my model.
It did work and it now has shadows and lights.

Now I want to know HOW I can edit them properly without having to copy other files.
Do you sugest any tutorials?

http://psteamcommunity.yuanyoumao.com/sharedfiles/filedetails/?id=153533275

There aren't really any tutorials to look at other than what valve provided. The pdf literally explains exactly what each channel does in terms of shading and lighting. My advice would be to re-read it and learn what each channel actually does. Looks good, though.

Yes, I got to understand what they do (materials, reflections, self-illumination, all that stuff), my question is how I edit each mask layer so I can get the same result as the Valve models.

Its the only thing I lack to know (aside some other stuff I might need to know to make some pants, but that comes later): how I edit the RGB/A masks separately.

Oh. Well that's not too bad. I'm assuming you use photoshop (if not, whatever program you use should be similar):

In photoshop you should see a panel in the bottom right labeled "channels".[dl.dropboxusercontent.com] If you don't see it, go to window > channels. From here you can edit individual channels of each mask.

Channel 1 = Red
Channel 2 = Green
Channel 3 = Blue
Channel 4 = Alpha (transparency)

Normally channels are used to control the amounts of RGB in each part of the image (with the last channel being reserved for alpha transparency), but the source engine can read these channels to be masks based on how much portions are colored white (white = full power, black = no power, grey = in-between). You can edit these channels the same way you edit anything else in photoshop.

Hm, I dont have Photoshop. Even gave up trying to find it by other means.

I'm using Gimp, its the closest I have. I got to edit the Alpha channel, now I'm having the problem where I cant edit the other channels without editing the alpha channel along them.

Like, I want to place something on the R Channel, nothing shows up. Only when I select to edit the alpha channel along it something shows.

Sorry to bother this much, neither TF2 or Dota tutorials talked about this in depth.

I don't personally use Gimp, so I don't really know what could be wrong. I did a quick google search that perhaps might help you: http://www.youtube.com/watch?v=ukuc256YjCs

Oh shit I got it. Thanks for the looking this for me, but I got it by looking around Valve textures.

It turns out that to work the mask2 properly, you need to give the Alpha channel a low oppacity white color, then you can edit the other channels without having to edit the Alpha channel along it.

Thank you so much man. I may be back later when I try to create some pants.