Vision also uses receptors, nerves, and places in our brains. The receptors for color are called cones and they are in the retina in the back of our eyes. It is pretty common for males to have cones that dont allow them to see all the colors that others can see. Often males cant tell red from green. Some people cant see color at all. You are right that two people may see something different but call it the same thing. If a kindergarten teacher has a few circles on the wall labeled with their color, we might both learn the names of the colors even if you are just learning to call different shades of gray by color names. You would not even know what you were missing. Thats why they have to use special tests for colorblindness. You can try some at:
Some people are born without pain receptors. This may sound great, but can you think of problems it would cause?