So, the general way to think of this is that an object in motion will stay moving the same direction (inertial force) unless​ a force is applied to it (applied force). So, for an orbit, the diagram would look something like this for a satellite around Earth example:

The inertial force is in a direction tangential to the orbit. The applied force of gravity is perpendicular to that, back toward Earth. The resulting direction of movement is somewhere between those two, and is the orbit

So, you can think of an orbit more as falling in a circle. The objects will orbit forever UNLESS there are more forces at play. For example, space very near to Earth is not a perfect vacuum (although it is very close), so some of the inertial force is slowly lost by essentially wind resistance or friction in space, causing objects in low earth orbit to slowly fall back to Earth as the orbits decay. Therefore, the useful objects in orbit are either replaced or "boosted back into orbit" every now and then.

On Wikipedia, they discuss doing this for the Hubble space telescope. read here