Yes, they must. However, the reason why they are called ultraviolet is because their frequencies are beyond the frequencies of violet, which is the color of the highest frequencies of visible light.

Infrared waves occur at frequencies below those of red visible light. This is the reason why the human eyes cannot see those bands, because their frequencies are out of the ones that the human eyes can see.

Theoretically, yes, but there is one problem with that, that being that water is opaque to both the ultraviolet and the infrared. Near infrared and near ultraviolet (very close to being visible) probably do have bands just as you describe them, but farther afield the light gets absorbed.

Rainbows do have UV and IR bands. Rainbows are the result of water droplets acting as a prism to separate light (electromagnetic radiation). The only difference between UV and IR light versus visible light is the wavelength (and also the frequency). People can't see these bands because our eyes are not sensitive to the UV or IR. However, it's probably worth nothing that these bands might suffer from other effects in a rainbow. For example, IR light may be absorbed by atmospheric molecules, and the UV light may be scattered a bit more. In fact, I believe you can photograph the UV bands with a camera, provided that there are no optics in the camera that filter UV light.

Yes, they do. A chemistry professor who answered a question just like yours on the internet says that you can see the ultraviolet band in a black and white photograph of a rainbow because the film is sensitive to UV light. I haven't tried this myself, but maybe you can try it at home! Next time you see a rainbow, take a photograph using color film and take another using black and white film and then compare the photos. You should see an extra band in the black and white photo. Just make sure that you use a camera without a UV filter. Have fun!