Some of the elements in the periodic table are named after their common English names, but as you point out correctly, many of them are not. Usually they are the first letters of the element in another language- either the language of the person who discovered them, or sometimes latin.

Your example of potassium was discovered by Sir Humphrey Davy in England in 1807. The name comes from the English word "potash" (pot ashes) and the Arabic word "qali" meaning alkali (since potassium, when dissolved in water, forms an alkaline solution and is in the column of the periodic table called the alkali metals. The origin of the symbol K comes from the Latin word "kalium".

There is an excellent web-site:

webelements
which lets you click on a particular element and find out a lot of information about it, including where it's name comes from.

By the way, it's also very interesting that, in addition to discovering potassium, Sir Humphrey Davy invented the Davy lamp. This was the first safety lamp, in which the flame was enclosed so that it couldn't ignite any surrounding flammable gases. It was widely used by miners and saved many lives by preventing explosions in mines.