Let's first talk about what happens when you remove your finger and the water flows out. As the water is flowing out the bottom of the straw, air is replacing it by flowing in from the top. The straw is never truly "empty", there is always either air or water in it. When you cover the top of the straw air cannot enter and there is nothing to replace the water if it were to flow out. Because of this, the water stays in the straw. These ideas are related to the concept of "pressure". When the top of the straw is uncovered, the air pressure is equal at the top and bottom, so gravity pulls the water down. When the top is covered, there is no air pressure there. The air pressure at the bottom actually counteracts gravity and the water stays in the straw. Pressure is an important topic in many areas of science and engineering.