Great question! Let's start with what a heat reservoir is. A heat reservoir is basically some system that in thermal contact with some other system, such that the exchange of heat does not (significantly) change the temperature of the reservoir. In other words, a heat reservoir is a sink and source of thermal energy, but essentially stays at the same temperature. A perfect heat reservoir is an idealistic construct (i.e., there isn't a material that would not change temperature upon exchange of heat), but you can get pretty close to one!

What makes a good heat reservoir?
The basic property in any good heat reservoir is the ability to absorb heat without heating up too much (or conversely, to expel heat without cooling down too much). This property is known as heat capacity, and has units of Joules/Kelvin; it may be interpreted as the the amount of energy required to change the temperature of your material by 1 Kelvin (or whatever units of temperature you use). Heat capacity depends on how much stuff you have (e.g., it take more energy to heat up more water), so it is more useful to define a specific heat capacity, which is has units of Joules/Kelvin/Kilogram. Specific heat capacity may be interpreted as the amount of energy required to the change the temperature of 1 kg of your material by 1 Kelvin (or whatever units of energy, mass, and temperature you use). Compare this definition to that of heat capacity to notice the difference.

An important thing to note is that there is an implicit assumption that we are only dealing with specific heat capacity. Heat capacity deals with the heating up of materials. However, you probably know that water boils off once the temperature hits 100 C. At this point, you are no longer heating up the water, you are cause a change in phase from liquid to gas. Measuring how much energy it takes to make a change in phase is latent heat (in this case, of vaporization).

Why are water and air used as heat reservoirs? Here is a table of specific heat capacities of several materials, including air and water.

Why is water a particularly good heat reservoir?
It is because it has such a high specific heat capacity. That is, it takes a lot of energy to change the temperature of water. Perhaps, this is why a watched pot never boils... Although it has a much higher specific heat capacity, a reason hydrogen is not used is because it can be quite explosive.

Basically, you can build a large enough reservoir that surrounds the actual system you are interested in. Once in thermal contact (via conduction), your water or air can put in or take out heat from the system. With enough water or air (and possibly some convection to spread the heat around the reservoir), you can maintain the temperature of the reservoir. As you can see from the above table, air is not the most ideal heat reservoir, so water is often the material choice. In fact, it is this same property that water is used as a cooling agent for a number of applications, like in nuclear reactors.

And if you are curious, where does the specific heat capacity come from?
Or perhaps differently worded, what about water makes it have such a high specific heat capacity?
There are two factors: 1) small molecular mass and 2) more ways to store energy.

The first one is easier to grasp. A smaller mass means there are more molecules per unit of mass (remember, specific heat capacity is measured with respect to mass!), so there are more molecules to absorb heat per kilogram for instance.

The second is trickier. The first thing to recognize is that a molecule is not stationary. It moves around, rotates, and vibrates. This ability to move around, rotate, and vibrate is encapsulated in an idea known as a degree of freedom. You can imagine not all molecules have the same amount of freedom to move around as other do. The more degrees of freedom, the higher the heat capacity. That is, the more degrees of freedom, the more ways you can store energy.

Water is not only a relatively light molecule, but it has a bent structure - The angle between the hydrogens is not 180 degrees. This bent structure gives water more degrees of freedom than for instance nitrogen gas (N₂), where the angle between the nitrogens is 180 degrees. In fact, it's this bent structure that gives water many of its peculiar properties (like why ice floats in water, even though it's a solid).

For something to be called a heat reservoir, it must be able to give or take a lot of energy from another object without changing temperature itself. Usually this is accomplished by a heat reservoir having a lot of mass which means it has a lot of molecules. An object’s temperature will rise as more energy is added which means that on average the molecules in the object are moving faster. If the object has more molecules in it, it will take more energy for all of the molecules to move faster and will therefore not raise in temperature as much as a smaller object.

Water and air are probably the two easiest and cheapest substances to get a lot of so it is easy to make a heat reservoir out of them. Additionally, water has the ability to absorb a lot of energy before rising in temperature which makes it a good heat reservoir. However, if you have enough of any substance it could potentially act as a heat reservoir though air and water are probably some of the best choices.

This is a good question! Water, air, and other materials all exhibit a property called "specific heat." Different materials have different specific heats. The specific heat of a material is the amount of thermal energy (i.e. "heat") it takes to raise the temperature of a unit mass of the material by 1 degree. Since different materials can have different masses, and we want to be able to accurately compare specific heats of materials, we say "unit mass," which could be a gram or a mole, for example. If we want to know how much thermal energy it takes to raise the temperature of a specific example of material (for example, the swimming pool at a school gym, or a pond in someone's garden), then we can look at heat capacity, which is an extensive quantity. Extensive means that the it depends on how much of the material there is.

It turns out that water has a very high specific heat and heat capacity, and therefore can absorb a lot of thermal energy as its temperature increases. That means that at any given temperature, a body of water has a certain amount of thermal energy stored. This thermal energy can then be released slowly as the temperature of that body of water decreases.

The specific heat of air is about 1/4 that of water. This means that when you raise the temperature of a body of water by 1 degree, and you raise the temperature of a body of air that has the same mass as the water by 1 degree, the air's thermal energy only increased by 1/4 the amount the water's thermal energy increased. However, it still absorbs thermal energy when its temperature increases and therefore can still act like a "heat reservoir!"

Water and air, like everything else on earth is made of atoms. Atoms bond together to form molecules. For example, chemically pure water is made of two Hydrogens (H) and one Oxygen (O) that are bound together forming the H₂O molecule. You can imagine the H₂O molecule as two tiny balls of H attached to O by springs in a V-shaped geometry.

Each of these springs can compress and decompress, a process by which they can store "vibrational energy". In addition the molecule can rotate about an axis, which stores rotational energy. It is also free to move in any direction, which store kinetic energy. These different ways of storing energy are called different degrees of freedom. It is the energy that is stored within these degrees of freedom, that we call as "heat". More the heat stored, more the atoms vibrates, and faster the molecules move and rotate. Molecules often bump into each other, and when they do, they exchange heat energy.

This is how water and any other substance stores heat and act as "heat reservoir". Air is a mixture of many gases, such as water vapor, Oxygen, Nitrogen, etc.. Each of these molecules can be imagined as little vibrating and moving balls that store energy.

When water is cooled, you are essentially removing the stored energy. This cause the molecules to move and vibrate slower and slower. On cooling, you reach a point where the molecules cannot move at all and are stuck to a specific position. This is when you form ice.

When water is heated, you are adding energy. This causes the molecule to vibrate and move vigorously. At this point (called the boiling point), some molecules start jumping out of the vessel to form water vapor, which is the gas phase.

Water is composed of water molecules, which when heated up, either on your stovetop in the case of a pot of water or by the sun in the case of a lake, start moving faster. They can translate faster, rotate faster and vibrate a little faster too. This extra ability for motion that they have means that they have more energy: more kinetic and rotational energy. We can measure this energy by measuring the temperature of the water. Higher temperature means more energy exists.

Have you noticed how it easier to heat up oil in a pan than water? To increase the temperature of the water by 1 degree, it takes much more energy compared to increasing the temperature of the oil. This means that we can add and remove big chunks of energy from water without changing its temperature much! So, water is a great heat reservoir! ...and oil is not, because its temperature fluctuates a lot as we add and remove energy. Oil is really good at transferring heat, so we use it in car engines to take the energy away so that we don't melt our engine. In scientific language, how much energy a substance can absorb before it changes its temperature by one degree is called "heat capacity". Water has a large heat capacity, larger than oil. If you think of the sea and the lakes, their temperature does not change very much between day and night or winter and summer, exactly because water has a large heat capacity. This is why the changing of the temperature of the oceans is such a big deal.

What we call humidity is the water contained in the air. Death Valley today has humidity 64%, and Santa Barbara 82%. Since Santa Barbara has more water in the air, it should be able to keep the temperature more constant than Death Valley throughout the day. That's why desert temperature fluctuates a lot during the day. It is not the only factor, of course. Air can act as a heat reservoir too. But because it doesn't have many molecules in space, it is very dilute, there are not many molecules to take up the energy by translating and rotating faster. Therefore, if we want a heat reservoir, it's not a good idea to use air, and we should prefer water.

If you want to read more, a similar question was answered previously in ScienceLine

This is actually due to the nature of heat. Heat is actually the energy stored in the vibrations of the atoms of a material. The temperature of a material that we measure is directly related to the speed of the vibrations of the atoms that make up that material. Heat can travel in a variety of ways, one of which is where vibrating atoms bump into their neighbors and cause them in turn to vibrate. This method of transferring heat is called conduction. Another important concept to understanding heat is a concept called specific heat. This means that different materials respond differently when heat energy is added to them. The difference between water and air is a good example of this. If you were to add exactly the same amount of heat energy to an equal weight of both water and air, their temperatures would not go up the exact same amount. The air's temperature would go up significantly more than the water's temperature. This means that the air molecules would be vibrating more quickly than the water molecules. In short, it takes more energy to get the water molecules vibrating at the same speed. This means that water can act as a heat reservoir, because it can absorb more heat than other materials, while having its temperature change by a small amount. This is why blacksmiths plunge red hot metal into water, and it is also related to why the climate near oceans is much more moderate than it is inland.