"Colour" is a subjective interpretation we make based on how our eye re-acts to a specific light frequency. What appears as one hue to one person may look different to another, especially if they are partially color blind. What I see as "red" my son sees as brown. Looks to him the same as what I see as brown looks. You are correct that we distinguish different colors based on the specific light frequencies (or mix thereof) that are reflected by a particular substance and we humans have a consensus that certain frequencies we associate with specific color names.