If you hold yourself steady in the saddle with your stomach and back, you don't need your arms to hang on. The bars stay loose, and the ride gets calmer.
Ten minutes, two to three times a week, is enough. The classic is the forearm plank: start with 20 to 30 seconds and build up slowly. On your next ride, notice whether you hold on with your knees and core or with your hands.
If you have health issues, get advice first.