Great question. The quick answer is that the colder a liquid, the more gas it can dissolve or "contain" as you aptly put it. So a cold glass of water has more oxygen stored in it than a warm glass. Now, this might seem contrary to your observations because when you fill up a clear glass with hot water you can usually see a bunch of bubbles in it. However, you see those bubbles because they are escaping from the water. To put it another way, at higher temperatures, the water is less soluble to air, which is primarily nitrogen and oxygen amongst other things. The solubility of a liquid to a gas refers to the maximum amount of gas that the liquid can contain/dissolve. The greater the solubility, the more gas a liquid can contain. So for hot water, which is less soluble than cold water, the dissolved oxygen is released.

In general, hot water dissolves fewer gases (like oxygen or carbon dioxide) but more solids (like salt or sugar) than cold water does. Gases are more likely to escape into the air at high temperatures, and they are not replaced (in other words, re-dissolving into the water from the air) as quickly. In chemistry terms, we say the equilibrium has shifted toward the gas phase from the liquid phase.

Cold water. Gases in general are more soluble in colder water. One way to remember it is that if you open a 2-liter bottle of soda and leave it out, it will go flat much more quickly than if you put it in the fridge. That's because if you leave it out it'll be warmer and more of the dissolved CO₂ will escape.