That is an interesting question! You are correct that most of the heat flux of the earth (heat lost from an area of the surface over time) is thought to originate from the decay of radioactive elements. When radioactive elements decay to their daughter products they emit energy, often in the form of gamma radiation, which can be harmful to humans. The reason that we are much more concerned about things like nuclear explosions than background radiation from rocks is because the energy released is much greater. It all comes down to the concentration of radioactive elements in a nuclear weapon or energy reactor versus in typical rocks in the earth’s crust.

Nuclear weapons and reactors use highly concentrated nuclear fuel, such as uranium. In the case of uranium, the fuel has also been enriched in the isotope ²³⁵U (uranium with 92 protons and 143 neutrons in the nucleus); this is because ²³⁵U is more likely to decay by fission than ²³⁸U (uranium with 92 protons and 146 neutrons in the nucleus). In nature, ²³⁵U makes up less than 1% of all uranium, but weapon and reactor grade uranium contains more than 20% ²³⁵U.

The concentration of uranium in average continental crust is 1.7 parts per million (that’s 0.00017%; Wedepohl, 1995). That means that there is 1.7 grams of uranium in 1 million grams of rock, and only about 0.7% of it is fissile ²³⁵U. Let’s do a simple calculation to determine how rock (average crust) it would take to produce enough ²³⁵U for a nuclear weapon.

concentration of U density of typical crust ²³⁵U in natural uranium

(1.7g U / 106 g rock) x (2.8x106 g rock / 1 m³) x (0.007 g ²³⁵U / 1 g U) = 0.03332 g ²³⁵U/m³

This means that there is only about 0.033 grams of ²³⁵U in a cubic meter of rock in the earth’s crust. The critical mass of ²³⁵U (the amount needed to cause a sustained nuclear reaction) is 52 thousand grams (wikepedia.org).

concentration ²³⁵U critical mass

(1 m³ / 0.03332 g ²³⁵U) x (5.2 x 104 g ²³⁵U) = 1,560,000 m³ rock

Over 1.5 million cubic meters of rock contains amount of uranium-235 used in a nuclear weapon. The result of this calculation is actually surprising to me; 1.5 m³ is a lot of rock, but it’s not that much. However, nuclear reactions require that the fuel is very concentrated; that is, that the 52 thousand grams of ²³⁵U is closely packed together. The low concentration in average crust means that the intensity of radiation is very low.

The element potassium is a more important source of heat in the crust than uranium. The isotope potassium-40 (19 protons and 21 neutrons in the nucleus) only makes up about ~0.01% of natural potassium, but the element is over ten thousand times more abundant in the crust. The type of decay that 40K undergoes emits much less energy than ²³⁵U fission, and is not harmful to humans in the concentration of natural crust.

references
Wedephohl, K.H. (1995). The composition of the continental crust. Geochimica et Cosmochimica Acta, 59, 1217–1232.

There is lots of radioactive decay inside the earth but it is spread over a huge volume, and much of the radiation is shielded by the crust so we get only a very low-level dose which is not a big health concern. Nuclear energy disasters can potentially generate lots of radiation in a very small space so the dose people receive is very large and can cause health problems.

So you're curious about radiation, that's excellent. First we have to clear up a few language issues that scientists are sometimes not so careful about. Scientists sometimes use certain words to mean very specific things, but it's not always obvious what they mean.

"Radiation" is a great example of this. Strictly speaking, "Radiation" refers to energy that can be carried through a vacuum (ie: nothingness) by either particles or waves. This doesn't seem to help us much because it's pretty vague, but it does show why the word can be misunderstood sometimes. The light coming from the sun is radiation, and so is the heat coming from it. When you toast bread in a toaster, it's being heated up largely through radiation, nuclear power plants also create radiation, but it is of a very different kind. When we are talking about radiation it is important to know what kind of radiation we are talking about. Some of it is harmless, like the electromagnetic radiation coming from a light bulb (which we call light). Some of it can be dangerous, like the radiation coming from plutonium. We can generally classify radiation as ionizing radiation (the harmful kind) or non-ionizing radiation (the harmless kind). The difference between the two is that ionizing radiation carries enough energy to cause real changes in the matter that they interact with. It's like the difference between a spitball and a bullet. Ionizing radiation is powerful enough to knock the electrons right off of an atom, sometimes powerful enough to disrupt the protons and neutrons in an atom.

So let's get back to your question. The earth does indeed emit radiation, most of it is thermal radiation, in other words, radiation in the form of heat. This is the same kind of radiation that heats up your toast in the toaster. So most of it is not immediately harmful to humans. But you are right, the earth does emit some amount of harmful, ionizing radiation. Most of this is produced through radon gas, which is naturally found in the ground. As a matter of fact, most of the radiation that US citizens will ever encounter comes from this naturally occurring radon gas. And if there is too much of it, it can pose a health risk, but we know that there is a certain amount of it that does not seem to produce any ill effects.

Now we have to talk about something called dosage, or radiation dosimetry, which is the study of how big of a dose of ionizing radiation someone can take without any negative side effects. The truth is that ionizing radiation is everywhere. Grass and trees are slightly radioactive in this way because of the carbon that goes into them. The truth is that some tiny amount of the carbon in the air is radioactive... 0.0000000001% to be exact! Bananas are radioactive too and so are you! Both bananas and people contain potassium (it's a natural building block for bones), and potassium is naturally gives off tiny amounts of ionizing radiation. Of course none of this ionizing radiation is anything to worry about, because it all depends on the dose. Just like you can eat a cookie and it's yummy, but eat 40 cookies and you might have a stomach ache. Eat nothing but cookies for years and years and you might face more serious side effects. With radiation it is all about the dose.

People are afraid of nuclear explosions because they worry that they might be exposed to large amounts of the dangerous ionizing radiation. In the rare cases when nuclear power plants do fail, there is a risk of people being exposed to dangerous amounts of radiation, but in general every reasonable safety measure is taken to ensure that even in the case of disasters, people are shielded from harm. You may have heard of some nuclear power plants melting down in Fukushima, Japan. These did recently cause some amount of harm, but it must be understood that the Fukushima disaster was about the worst of the worst that could happen. In the United States there are some nuclear power plants, but they are well built and well maintained and the risk of harm coming to anyone from them is absolutely tiny. In life, there are always risks. What we as a society need to do is decide which risks are acceptable and which are not. This means that people need to learn about and understand the risks, and decide whether it's worth it. As a scientist and a former nuclear worker (someone who works in a place where they may be exposed to ionizing radiation) I decided long ago that it was worth it. But I encourage you to read more and make up your own mind. Here are a few links you may find interesting and helpful to you. Thanks for the question.

You are correct: most of the radioactivity we are exposed to on a daily basis is of natural origin, either radon gas (a decay product of uranium in the Earth's interior) or carbon 14 (created by high-energy sunlight and cosmic rays). Close proximity to a nuclear blast will expose you to a great deal more harmful radiation than you would ordinarily, but that doesn't make natural sources any less dangerous.

Due to the propaganda against nuclear weapons during the Cold War, many people are more afraid of artificial nuclear energy than is appropriate - which, again, isn't to say that the danger of that isn't real either.

The INTENSITY of radiation is VERY, VERY low due to natural radioactive decay of U, Th, K, and other radioactive elements. The intensity of emitted radiation in CONCENTRATED elements that have a short half-life is very, very large and kill organisms in a few minutes. So it all has to do with the INTENSITY of radiation... in other words the number of radioactive emissions per unit time.

The graphic found in this link explains very clearly your question as to why people are afraid of nuclear meltdowns at nuclear power plants or nuclear explosions more than they are afraid of natural background radiation. The answer is that we do not get immediately sick and die from lower radiation dosages but we can get sick very quickly and die from higher radiation dosages

It's all about the dose. Over several million years, most advanced organisms (including mammals, like humans!) have evolved ways to detect and repair the radioactive damage and mutations that occur. Cellular organisms that weren't able to repair this damage were less likely to reproduce and have viable offspring, so this led to an evolutionary pressure. Mammals, being relatively complicated multicelullar organisms, have many ways to deal with mutations and ensure the affected cells are destroyed.