Great question. The quick answer is that the colder a liquid, the more gas it can dissolve or "contain" as you aptly put it. So a cold glass of water has more oxygen stored in it than a warm glass. Now, this might seem contrary to your observations because when you fill up a clear glass with hot water you can usually see a bunch of bubbles in it. However, you see those bubbles because they are escaping from the water. To put it another way, at higher temperatures, the water is less soluble to air, which is primarily nitrogen and oxygen amongst other things. The solubility of a liquid to a gas refers to the maximum amount of gas that the liquid can contain/dissolve. The greater the solubility, the more gas a liquid can contain. So for hot water, which is less soluble than cold water, the dissolved oxygen is released.
In general, hot water dissolves fewer gases (like oxygen or carbon dioxide) but more solids (like salt or sugar) than cold water does. Gases are more likely to escape into the air at high temperatures, and they are not replaced (in other words, re-dissolving into the water from the air) as quickly. In chemistry terms, we say the equilibrium has shifted toward the gas phase from the liquid phase.