This is a great question and it gets right at the one of the key mysteries in developmental biology - how do cells "know" when to stop dividing (and when to resume)? It isn't a very far leap to see how this can impact health and disease as well, as it is a question especially relevant to wound healing and cancer. There are entire courses and fields of research built around this question, but I'll try to distill it down to a few key points here.

During early development, cells are indeed dividing in a rapid, albeit controlled way. Most, however, eventually differentiate (take on specific fates/identities/functional roles) and either dramatically slow their cell division cycles or stop dividing altogether. In mammals, it is thought that a main regulator of this is cell-cell contacts and a strong link to differential gene expression.

So how are cells replenished if they are no longer dividing? They can't live forever, right? Well, some types can in fact re-start their cell cycles and emerge from a quiescent state back into a rapidly dividing state. However, in most tissues, there are small populations of stem cells that maintain tissue and organ homeostasis by responding to feedback signals that trigger division and thus production of brand new cells - the new cells have a limited division potential and eventually differentiate.

Under normal (healthy) circumstances, this is a rather slow and steady process of "maintenance." However, in response to infection, damage due to trauma or some other disruption, there can be a more dramatic response. However, this is temporary and tightly regulated.

So what can go wrong? Well, in general, if the cells keep dividing inappropriately, that is called a neoplasia - and is the start of cancer. Now why or how could that happen? For most carcinomas (the predominant type of cancer), it's thought there are two main ways. First, a differentiated cell might acquire mutations in key genes that inappropriately causes (re)activation of cell division and de-differentiation in an unregulated way. In fact, the cell may become more "stem like" in terms of its gene expression patterns and behaviors. Second, there may be a mutation in a stem cell that leads to continues division (this is the case in a lot of leukemias and lymphomas).

Finally, although it's clearly a key question for human development, health, and disease, a great deal of what we know comes from studying organisms that give us more ways to gain experimental insight, such as fruit flies and nematodes. The regulation of cell division - and the genes that encode the proteins involved - are highly conserved across all multicellular organisms. In fact, nematodes are particularly relevant to your question - they are "eutelic" organisms, which means that every adult animal has the exact same number of somatic cells and researchers know the exact lineage of each and every cell!

Besides division, cells also go through a process called differentiation. When a cell differentiates, it becomes more and more specialized and eventually loses the ability to make another undifferentiated cell, or self-renewal. A terminally differentiated cell can no longer divide.

However, as in many other things in biology, this process is not perfect. Some differentiated cell can regain the ability to divide through mutation. When combined with many other factors, those cells will grow uncontrollably and there indeed will be many cells. Those cells are also known as cancer cells.

While it is true that in your body there are many cells in the process of dividing, cell division is actually a highly regulated process. This regulation is controlled by chemical signals between cells, most notably in the form of proteins known as cyclins.

Cyclins can activate enzymes responsible for the synthesis of the cell cycle, and are also important in cell cycle arrest, wherein cells no longer divide. Eventually these aging cells will accrue too much DNA damage to be viable and will be replaced by newer daughter cells. There is such a thing as constantly dividing cells: these are known as cancer. Such cells are not under the same cell cycle regulation as the rest of your somatic cells, and thus grow out of control and cause harm to your more regulated tissue. In fact, many scientists right now are searching for a way to selectively induce cell cycle arrest in these tumor cells as a potential therapeutic. If you're interested in this, you could be one of them! Hope this helps.

Healthy cells only divide when hormones or other signals tell them to divide. When this system fails, and the cells divide uncontrollably, we call that 'cancer'. Cancer cells will eventually take over the host's body, killing it, and themselves in the process.