Yes, the deeper you go into the Earth, the less gravitational force acting on you, ultimately reducing your weight. Say you go 100 meters below the surface of the Earth. There, the mass in the shell above you (100 meters thick) pulling you in one direction exactly cancels out the mass in the on the other side of the Earth (also 100 meters thick) pulling you in the opposite direction. In essence, the Earth has lost a bit of its mass and that is why it pulls you down with less force.