Yes there is such a place the Gabon natural reactor in the southern part of Africa. Two billion years ago in what is now the Gabon Republic in equatorial Africa, a reach deposit of uranium at a site known as Oklo developed a natural chain reaction. In the process it produced radioactive waste identical to that produced by nuclear power generation today.
In 1972, the French government announced the discovery of the remains of this natural fission reactor. The French researched the sites and shared findings in 1975 at a conference sponsored by the International Atomic Energy Agency. Although the detailed aspects are still being studied, the initial results show that the waste did not move from where it was generated. It stayed tightly bound in the rocks where it was created. The waste did not enter the ground water but remained totally isolated from the environment-unknown until explorers went to the site in search of uranium.
Studies at Oklo are continuing and scientists are looking for other sites of "natural reactors" because of the unique opportunity they provide to study potential migration of fission products.
HERE IS A WEB SITE where you can learn more:
http://starfire.ne.uiuc.edu/ne201/course/topics/nuclear_waste/natural_reactor.html

Radioactive isotopes do exist in nature. (Isotopes are any of 2 or more forms of the same element, which weigh different amounts. The number of protons is fixed for any element, but the number of neutrons in the nucleus can vary, thus producing isotopes. Uranium exists in nature as 2 radioactive isotopes. U-235 has a half-life of 710 million years and U-238 has a half-life of 4.5 billion years.) Radioactive isotopes decay at consistent rates. Geologists use the ratios of radioactive "parents" to their resulting stable products to determine the ages of rocks and minerals.
The specific case you are talking about took some special circumstances.
In the Oklo uranium mine of Gabon a natural reactor appears to have run for between 500,000 and 2,000,000 years. It happened about 1.85 billion years ago when there was more U-235. Uranium-rich sand and mud accumulated along with organic carbon is a basin about 2.1 billion years ago. The sediments were buried to shallow depths, and the uranium ore sustained nuclear fission reactions that heated the deposit to about 400C (750F).
There were apparently 17 sites that became natural reactors and the nine of these that have been carefully studied produced about 17,800 megawatts of power. Now that would help California!
My source for most of this information was page 6 Natural Disasters by Patrick Abbott, published by McGraw-Hill.

Actually, the reactor was formed not too far from the surface -- Yes, many of the isotopes used in nuclear reactors are found in nature,however, rarely are they in high enough concentrations to foster cascade reactions.
Elements such as Uranium 235, and Thorium both exhibit neutron release and can be triggered to fission by neutron capture. This process occurs in nature all the time and was first observed in the lab in the late 19th century.
However, to make a cascade or chain reaction, the neutrons released from a random decay must have some realistic probability of hitting another nucleus of a fissable material. The problem is that neutrons exit a fission event at very high speeds and since they are neutral, interact with matter relatively weakly -- further they decay in about 10 minutes to a proton and an electron. It turns out that a few materials function as "moderators" which lower the energy of the neutrons to thermal levels where they are far more likely to cause a fission reaction. The first moderator used in a nuclear reactor was carbon (graphite bricks) -- however, ordinary water works quite well.
You can imagine a large area of very high concentration uranium ore which is close enough to the surface to become saturated with water.
As the water moderates the randomly produced neutrons, the probability of a cascade reaction increases -- producing more reactions and a rise in temperature. As the temperature increases, the water will tend to leave due to its increased pressure and cease moderation activities.
Effectively you have a self moderated reaction with might last for millennia. The evidence for such a reactor would be chemical and isotopic changes in the material -- i.e. chemical changes due to ionizing radiation and excess quantities of high neutron stable and low live active isotopes.
In a reactor, many isotopes are created that are not commonly observed in nature. Typically, such isotopes have half lives that are less that a billion years or so... any element with a shorter half life would have decayed long ago and so will not be found on the earth's crust. Examples are Plutonium and Technetium which are produced with ease in a reactor, but which are not found on the earth. (Tc is found spectrally in the sun...)
However, it is possible for long life isotomes (e.g. U235, U238, Th251...) which have several billion year half lives to decay to other species with much shorter half lives. This is the mechanism for production of radium and radon gas-- both of which have relatively short half lives but are nonetheless found on and in the earth.
References on the Net:
http://www.physics.isu.edu/radinf/oklo.htm
http://www.curtin.edu.au/curtin/centre/waisrc/OKLO/index.shtml
Cowan, A Natural Fission Reactor, Scientific American, 7/76: 3647
http://www.ymp.gov/factsheets/doeymp0010.htm