National Geographic Magazine did a wonderful article recently on the evolution of whales and addressed this exact question. See the November 2001 Issue, p. 64.

Basically, the article states that about 65 million years ago, the ancestor of the whales walked on land and was related to the modern-day horses and cows (a group of hoofed mammals called ungulates). Recent molecular data has indicated that the whales closest living relative on land is the hippo! It's that surprising! However, they think that hippos and whales began diverging (evolving very differently from one another) about 63 million years ago. A lot can happen in 63 million years.

In addition to molecular evidence indicating that whales evolved from land-dwelling ancestors, go to your local museum and look at any whale skeleton. You'll see two tiny sets of bones on either side of the back bone near the rear of the animal. These sets of bones are what is left of the whale's hind legs and feet! These sets of bones are called "vestigial hind limbs".

Yes, this is actually true! It sounds a little bit unreal but of course these whales looked quite different from the whales we know nowadays. Scientist study bones and fossils to find out how life has evolved here on Earth (This means they like to draw a family trees of the major animal group). For a long time they did not know which land mammals are most closely related to whales and dolphins, because whales are so unlike any land mammals (Their legs are paddles and they have nostrils atop their heads). But scientists knew that whales descended (evolved) from four-footed land mammals because whales and dolphins still have some features of land mammal: they use lungs to breathe air and give birth to young that are nursed by milk.

About some 50 million years ago whales evolved from hoofed mammals and became marine animals. This happened very fast. It took only about 8 million years (This is fast for evolution). Now, just last year scientist have found fossils of animals that roamed the land that is now Pakistan about 50 million years ago. These animals looked a little bit like wolves but they were land-living, four legged whales. These animals are now thought to be true whales ancestors on land. How did the scientists know that? Well, they found several strange bones in the ears of these fossils that only occur in whales.