This is a great question and the story of the discovery of the mechanism of ATP synthase is a fascinating one.

Determining the structure of a protein or a protein-complex like ATP synthase at atomistic-scale resolution is typically done by light scattering techniques . For example x-ray crystallography ( ray crystallography " ) is the most common method to determine 3D structures, but this method requires that you first crystallize the molecule.

By measuring the x-ray diffraction pattern, researchers can back calculated the structure that is compatible with the scattering. This method was developed in the 1950s and won the Nobel Prize in chemistry in 1962. As you can imagine the X-rays scattering pattern is often very complicated and the development of computers and now machine learning algorithms have advanced the field since the 1960s.

The newest method that is now getting a lot of press is called Cryo-EM ( Cryo EM ) and allows atomistic resolution structures to be determined without requiring crystal structures. As the name implies, this involves cooling your sample to cryogenic temperatures. This method won the Nobel Prize in Chemistry this year (2017)! . You can read a bit about the prize ( here ).

The field of determining all the structural components is called structural biology and is a big area of research in biology. Once all the pieces are known and their 3D structures are determined, the next step is to figure out how the pieces work together. This requires some artistry and the clever designing of experiments to test various hypotheses that scientist come up with from looking at the (static) structures provide by x-ray crystallography or cryo-em.

My favorite experiment demonstrating the turbine motion of the ATPsynthase was performed by a group of scientist from Japan. They fixed the ATPsynthase complex to a glass plate and were able to attach a long fluorescently labeled filament to the gamma sub-unit. When viewed under a fluorescence microscope you can see the actin filaments rotating counterclockwise in discrete 120 degree steps. (Actually the experimenters were running the ATPsynthase motor in reverse!, by adding ATP as an energy source, the motor was hydrolyzing ATP and using the energy as a hydrogen pump) In the mitochondria the motor runs the other way and uses the electrochemical potential gradient of hydrogen ions to synthesize ATP and the motor runs clockwise.

If you have access to a library, you can read the original paper. Noji, Yasuda, Yoshida, and Kinosita Jr, “Direct observation of the rotation of F1-ATPase”, Nature, 386, 299-302 (March 1997), but I think the movie is even better than the paper and it is on youtube ( here ) A really clever experiment.

Good question. There are a couple of sources of data on the shape of proteins:

a. Proteins can form crystals. Using x-ray crystallography (so shining x-rays through the crystal and seeing how they diffract ), it is possible to see what the structure of the crystal is and, with it, the shape and location of the atoms that make up the protein.

b. Electron microscopes, while they don't have the resolution to see individual ATP molecules because they're too small, they can see the protein molecules as they're quite large (by molecular standards). This means that, yes, we do know what the ATP synthase turbine looks like , that it has a pore that the proton can pass through only if it ratchets a quarter-rotation every time a proton comes through.

Finally, once we know what the molecules look like between x-ray crystallography and electron microscopy, we can build computer models of the proteins and watch what happens from the physics when we play with them. This is all theory, but it does explain their behavior very accurately, so we're pretty confident that the theory is either correct or pretty close.

As you've noticed, the resolution of even electron microscopes is not high enough to determine the structure of molecule complexes like ATP synthase . However, biochemists have some cool tricks for getting around this, like X-ray crystallography. In X-ray crystallography, a crystal made up of the protein/molecule of interest is blasted with X-rays, and an image is obtained. This diffraction pattern (or series of dots caused by some x-rays amplifying/cancelling each other out) can be analyzed to determine what the original structure looked like! Scientists can look at the diffraction pattern under different conditions, which give clues as to its function. Additionally, scientists use bioinformatics (a word for information compiled over the years about certain protein domains, etc) to hypothesize what role each domain will play; for example, whether or not that domain is likely to bind ATP.