To understand this we need some background: Electrons in atoms are distributed in electron shells. The electrons in a shell have the same energy. There is a certain number of electrons that can be fit in each of the successive shells of the atoms. The higher (outermost) shells will not start to fill up with electrons until the lower shells are completely full.
For example, the first shell, the closest to the nucleus, can fit two electrons, the second can fit eight, the third eighteen, and so on. Atoms have a marked tendency to be surrounded by "full shells" of electrons, even if that implies that the whole atom will bear an electric charge, that is, will become an ion. The atoms that naturally have their outermost shell completely filled are extremely reluctant to engage in any chemical reaction. They are called noble gases, and that's why chemist say that elements have a tendency to acquire a noble gas configuration.
This filling of the outermost shell can be accomplished in basically in two ways: either by losing electrons when there are relatively few electrons in that outer shell or by gaining electrons, when only a few electrons will complete the shell. For example, the sodium atom has 11 electrons. Two in the first shell, eight in the second and only one in the third. It is then very likely that sodium will get rid of that last electron, becoming the ion sodium one plus, with a full shell of eight electrons. On the other hand, the fluorine atom has nine electrons, again two in the first shell and seven in the second shell.
Can you guess what is the most natural way fluorine enters a chemical reaction? Of course, by accepting one electron, becoming fluoride ion and therefore having a full eight-electron second shell.

If everything is clear so far, let's talk about hydrogen, a very special element indeed. In its nucleus there is only one proton. So its only electron shell has only one electron. To gain the noble gas configuration, hydrogen can gain one electron, becoming the hydride ion, with a negative charge and the configuration of helium, a noble gas with two electrons in its only electron shell. Hydrogen behaves that way in the presence of strong bases, which are electron donors. But also the hydrogen atom can gain stability by ejecting its only electron, and becoming a bare proton (generally associated with a molecule). This ion of charge one plus is formed in the presence of strong acids, which have a tendency to gain electrons.

In fact, it is quite not accurate to say that hydrogen gains or loses an electron in acid/base. What is more accurate is to DEFINE an acid as a proton donor (the acid gives up a proton) and a base as a proton acceptor (the base takes up a proton). Protons are the positive hydrogen ion and are written H⁺. Remember that the hydrogen atoms comprises a proton in the nucleus and an electron whizzing around the proton. One can remove the electron from the hydrogen atom according to the reaction:

H = H⁺ + e^- (where we write the electron as e- to show that it has a negative charge).

Now a typical acid easily gives up its proton.
An example is HCl:
HCl = H⁺ + Cl^- : HCl is a strong acid.

A typical base easily takes up a proton:
NH3 + H⁺ = NH4⁺ : NH3 is a strong base.