Air is denser at sea level than at higher altitudes because at sea level, the gravity of the earth is stronger because the air particles are closer to the mass of the earth (gravity is distance-dependent; the shorter the distance between two things, the greater the gravity).

Gravity is one factor. The other major factor is that at sea level, there is more air pushing down from above, compacting the air particles and making the air denser.

Air pressure is due to the weight of air above you. The higher you go, the less air there is above you, so the less pressure you experience. When pressure is low, the air is compressed less, so it is less dense.

The air at sea level is compressed by the weight of the air above it, which forces it to be denser. Higher up, there is less pressure, so the air expands to have a lower density.

interesting article

This article says the air high up presses down on the air at sea level:
"At sea level, because air is compressible, the weight of all that air above us compresses the air around us, making it denser. As you go up a mountain, the air becomes less compressed and is therefore thinner. ... Try using our barometric pressure calculator to see how air pressure changes at high altitudes".