Caving is a sport not many people get the chance to pursue. However, it is a sport that gives you the chance to go to places no one has been before, or places few people have gone. There are different kinds of caves, including solution caves, which are most common, and talus caves, such as Purgatory Chasm, in Sutton, Massachusetts. Caving is not a dangerous sport, as long as you play it safe.
Chances are, you've never been in a cave before. Even if you have, you probably don't know every single inch of the cave you are about to explore. Caves hold all sorts of gems, waiting to be explored. Go out and explore, folks.
2. Buddy System
Make sure to have at least two other people with you so that in the chance that someone gets hurt, someone can stay with the hurt person while the other goes to seek help. Typically, you want about 5 people in your group. Make sure you are caving with experienced cavers when you first start out!
3. Proper Clothing
Caves are cold. Caves are wet and slippery. Objects can fall (Yell "ROCK!" if you dislodge something beneath you and it starts to fall. Be prepared. Go into the cave with as little as possible. If you get cold, and you will, don't stop moving. Stay out of water as much as possible. Wear multiple thin layers (wool or polypropylene are best). Wear one pair of pants over long-johns, then add coveralls. Wear heavy gloves. Wear hiking boots and heavy, wool socks. And lastly, have a garbage bag of dry clothes to change into when you are done, and leave that in your car. (Use the garbage bag to carry all your muddy clothes afterword.)
You need at least three independent sources of light at all times. Your hardhat, which you should wear at ALL times while in the cave, should have a mounted headlight that is carbide or battery powered. You should have two water-proof flashlights. Carry spare batteries for your flashlights and headlight, and extra bulbs. Utilize a fanny pack (I did).
Know your limits. NEVER jump. Always have two limbs on the surface of the cave at all times (ie. keep your feet secure and move both of your arms). Use the cave to your advantage. Wedge your feet into crevasses to move easier. Don't shine your lights in other people's eyes. Crawls are often necessary. Be careful not to disturb any bats or fragile rock formations on the ceiling and walls of the cave. You might get stuck, but you will be okay. Breathe and wiggle and maybe request some assistance from the friends you brought with you.
Personally, I went caving once while at science camp, and once at Purgatory Chasm, though that experience was hardly caving. The cave we decided to explore was just a small "room" cage. Caving, while at science camp, was thrilling. I got stuck, I climbed vertically, I explored, and most importantly, I had fun. And that is the most important part of caving. It's all about having fun.