Marshmallows have sugar. Sugar contains carbon, hydrogen, and oxygen. Water is hydrogen and oxygen. The hydrogen and oxygen leave the marshmallows as water, leaving the carbon behind. Eventually this would turn the marshmallows into charcoal if you could remove all of the water, although this isn't normally possible without a fire.

Water molecules go from where there are more water molecules to where there are fewer water molecules. That's what molecules do, if they are free to move. You can do an experiment by putting a drop of food color into a glass of water and watching the food color molecules move from where there are a lot of them to where there are not a lot of them. What happens to the food color if you wait a long time?

A fresh marshmallow will have more water than the air, unless the air is quite moist. That's what you're thinking, when you write about water moving out of the marshmallow.

I'm guessing stale marshmallows are just those that have lost water. My son's marshmallows are gooey, and they stick together. He says they come from the store that way.