From the outside, most horseback riding might look like a person is just sitting on a horse while the animal does all the work, but in reality, the rider is giving the horse specific cues to control this 11 hundred pound animal that could kill the rider at any second if it wanted to. The reason for this misconception of horseback riding being "easy" mostly comes people watching professionals on television or people riding horses that are so well trained they could probabably walk around the arena without anyone on their back.
Some would even dare to describe horseback as not a sport in the first place, but if these people were to ride a newly trained horse they would understand that the strength needed to stay on the animal is immense. Not only do you need to use your legs just to stay on the horse, you also use them to give certain cues. While riding the strength of your core muscles must also be strong in order to not look like a dead fish flopping around. Riders are constantly fighting gravity, and the force given off by the horse to look stable and elegant. That's just the physical demands on the horse. The barn work surrounding horses also requires fitness and strength. Carrying bales of hay or buckets of water across the barn isn't easy. Neither is setting up jumps, or really any other work related to horses.
Horseback riding is competitive if you're in the horse world, you probably know the name of anyone at least semi-decent on a district-wide, or even state-wide level. Going into a ring with 20 other horses who have their own minds and do what they want is not easy. Neither is other events in horseback riding. For example, a rider that competes in jumping classes must memorize a complicated pattern (usually right before they get into the ring) or risk disqualification. Not to mention these competitions include many rules and regulations that, if not followed can also cause disqualification.
Riders are controlling an animal that has it's own mind and ideas. A lot of horseback riding is convincing the horse to do what you want it to. There are many cases where the horse, who no longer feels like training will literately just walk back to the barn if their rider cannot control them. At least a human teammate you can reason with, a horse however is a literal animal that most of the time has no idea what you're trying to say.
The fact that a horse isn't human teammate also moves into the dangers of horseback riding. When you fall in a normal sport, you're usually hitting the ground from your own feet. While in horseback riding, you can fall 5 feet, which happens more often than you think. Not to mention the horse can stomp on the rider once they hit the ground. Researchers at the National Trama Database found that equestrian sports contributed to the highest percentage of traumatic brain injuries for adults.
Overall, horseback riding is a dangerous sport that requires both athletic skill and intelligence. Riders are tough and dedicated to their sport. Both riders and horses alike deserve much more credit than they recieve.