The difference in appearance of color comes from the difference between a matte and glossy surface.

Matte means the light is reflected in a diffuse way, like for paper or clay. Glossy means the light is reflected in a way that looks shiny like a mirror or polished metal.

An object has color because the body of the material absorbs some colors of light and reflects others. The colors you see are the reflected colors. In addition to the colors absorbed by and reflected from the bulk of the material, there is also the light reflected from the surface. A matte surface reflects white light in all directions. A glossy surface reflects white light only in certain directions, which is why you see a bright glare off of glossy objects.

The surface of fabric has a rough texture (rough on a very small scale) which gives it a matte appearance. The color you see when you look at fabric is the combination of the bulk color plus the white light. When the fabric gets wet, the water creates a smoother, glossier surface. The glossy surface changes how the white light is reflected and makes it easier to see the underlying color, thereby making the fabric color look more intense.

References:
http://www.physicsforums.com/showthread.php? t=423358
http://www.cs.cmu.edu/afs/cs/academic/class/15462 -f09/www/lec/lec8.pdf