An air bubble is, as you note, filled with air; so when it bursts, the air must go somewhere. If the air bubble is under water, it couldn't burst in the same sense as a soap bubble or an air bubble that rises to the top of the water before bursting. The air in the bubble could get dissolved in the water, in which case the bubble would disappear.

This could happen if you heated water to the point where air bubbles were forming and then cooled it. I assume at least some of the air bubbles would disappear without rising to the top and then bursting. The other possibility I can see is that the air bubbles could break into smaller bubbles - or join up to form larger bubbles.

You could probably observe some of those possibilities if you heated the water longer, to where the bubbles were actually starting to move in the water. That would be harder to observe, though.