When you ride, your body works the whole time: balance, strength in your arms and core, reactions in a fraction of a second. We usually only notice when something hurts.
Try it after your next ride. Sit down for a moment and ask how your back is, your hands, your neck. What did your body do today?
And if your body no longer puts up with everything it did at twenty, that is no reason to be annoyed. You adjust the tour.