How did the concept of Time Travel originated?

In the 19th century, there was the use of the fictional concept of Time Travel, be it for novels, movies or anything. It was everywhere. But how did people came up with such an unusual concept and why after such a long time scientists believe that this concept may not be fictional? 

Time travel usually refers to traveling through the time, whether in past or future, by various methods such as by traveling faster than light (Special Theory of Relativity by Albert Einstein), using Wormholes etc. 

As we all know that our world is a Three Dimensional world or the 3-D world. We perceive every object in three dimensions, namely, Length, Breadth and Height. In the recent years, there has been a debate on the possibility of a number of more dimensions in space than we know. Some scientists believe to have known as much as 21 dimensions. Now-a-days, even time is considered to be the 4th dimension of the world. 

Now, we consider ourselves to be living in the three dimensional world, which comprises of length, breadth and height. And, we also know that we, on our will, can move backward or forward in these three dimensions freely, any time and at our will. 

Now, a group of scientists believe, that if we consider time as the 4th dimension, then like the other three dimensions, we should also be able to move in this 4th dimensions, freely, at any time and at our own will. On this supposition, originated the idea of time travel and scientists began researching on the possibility of time travel. And have even come up with a few theoretical concepts of time travel. 

One such concept of time travel is called the Twin Paradox. It states that if we travel at the speed of light to distant star, then slow down, turn back, and then travel back to earth with the speed of light, we would find ourselves to be aged-less, or would have traveled back in time. However, since till now we have not been able to achieve the speed of light, the concept of time travel remains hypothetical.