I hear the word "literally" used by many people around me, and even my fellow English majors use "literally" loosely.
You are not being literal if you are being figurative. In order to place a literal modification on a word, it first has to be a figure of speech before you can put "literally" before or after it. You are being literal if you are driving down some western interstate highway, your car breaks down just between two giant rock formations, you put your four-ways on, and you call your long-lost friend you were about to visit to say, "Here's the thing. I'm LITERALLY stuck between a rock and a hard place."
You are also being literal if you walk into a Barnes & Noble, find a book, look at its cover without opening it, and scoff, "What pretentious, elitist drivel," which at that point, you LITERALLY judged a book by its cover.
It is not just in the figurative context that I noticed the word being used, but also when emphasizing something, like when some girl says, "Oh, she's LITERALLY the worst!" Unless you are saying that she is the worst competently in a context where that word usually refers to the person's moral character, then replace it with "really." Even the Oxford Dictionary succumbed to stating that it is the case in an informal sense. Well, I would hope to use "literal" in the more formal sense.
"Literally" is formally used when trying to draw comparison between two meanings of the same phrase or word. Those two meanings are meant to be both a figure of speech AND relevant to the real-world topic being discussed. So, "literally" would mean that it is a non-exaggerated use of a commonly exaggerated phrase.
There are articles I wrote, such as this and this, in which I have used "literal," or any other conjugation of that word in what I concluded was its most appropriate context. I would hope that I am not being judgmental when I say that people use "literally" wrong, but what would the point of my English major be if I did not pay close attention to word choice and context?
If I have not won anyone over, then at least remember the Boy who Cried "Literally." I would just hope you do not literally laugh your guts out.