What's going on is that the water in the straw is pushed into the straw by the air pressure outside of the straw. As long as the pressure outside is able to overcome the force of gravity, the liquid will stay in the straw.

A column of water 32 feet (9.8 meters - there is a reason why this is a familiar number and it has to do with the density of water being 1) in height generates one atmosphere of pressure. As a consequence, if you have your column of water in your straw or thermometer, or what have you that is more than 32 feet in height, cavitations will occur and the water will fall down out of the straw creating "bubbles" of vacuum behind it, as gravity is overpowering the air pressure. Anything less than 32 feet tall will hold - unless of course the pressure of the water at the bottom of the container is enough to rupture the container (14.7 lbs/sq. inch, or 100 KPa, is a lot of pressure for most containers!).

It's great to see how interested in science you are over the summer. Awesome.

In the case of a straw with liquid, it is both the vacuum your finger creates as well as "capillary forces" that keep the liquid from spilling out. Capillary forces are due to the liquid being attracted to the walls of the straw. Since the straw has a lot of surface area exposed to the liquid inside, these capillary forces are important and contribute to holding the liquid inside. If you were to increase the diameter of the straw, more liquid is held inside the straw. At some point, the increasing weight of the liquid inside the straw could not be supported by both the vacuum and the capillary forces so I would not expect to see this on a larger scale.