It isn't impossible at all! (Hint; I'll tell you tomorrow...)
Each of us travels forward through time. The rate at which time passes for us is always the same, one second at a time. However, if someone is moving very, very fast (close to the speed of light), one second of time measured by them will correspond to many seconds of time measured by someone at rest. This is a consequence of Einstein's theory of special relativity. General relativity (a much more complicated theory) also couples gravity with the rate at which time passes.
Say that you take off in a spaceship and travel at nearly the speed of light while I remain on earth. If you could see me (maybe through a telescope) I would be running around as if my life were a movie, and someone had pressed the fast-forward button. If I could observe you, I would see you moving very slowly. However, both of us would look and feel normal to ourselves.
Traveling backwards in time is a different story. There are many very serious paradoxes involved in travelling back in time; for example, you could meet yourself, and tell the younger "you" to not ever travel back in time! Physicists say that these situations "violate causality," meaning that the cause/effect relationship between events is messed up. These possible paradoxes suggest that it might be impossible to travel backwards in time.