Rutherford did a lot of experiments with radiation of different kinds. One that he liked to use involved "alpha particles." It wasn't known what these were at the time, but they are two protons and two neutrons in a ball. This is just the nucleus of a helium atom, but he didn't know that. He did know, that you can move alpha particles by putting them in an electric field. They have a positive charge. At the time, the common idea of the atom was the "plum pudding model" (you may have heard of this), meaning that electrons ("plums" which had already been discovered) are randomly scattered throughout the atomic pudding, and the pudding had a slightly positive charge. Rutherford decided to test this idea by firing alpha particles at gold atoms. If the alpha particles went straight through the "pudding" it would support a plum pudding model.

Instead, most alpha particles went straight through, but a small fraction bounced straight back, the ones that hit the nucleus. Because positive charges repel, this meant that a small region of concentrated positive charge, the nucleus, existed within atoms. This tiny ball was reflecting some of the positive alpha particles back. Funny enough, Rutherford won the Nobel prize even before that discovery.

Here's a diagram of the experiment:
see here