First off I'm not a horse expert and my math isn't completely worked out, but this is my opinion on the subject.
There are four ways of viewing rider size for riding.
The simple minded one is via weight. Rather than 1/5th the weight. RQ roughly works out to 1/3rd the weight for average human size on the modern horse. It breaks down on smaller mounts however.
The close to the book method is to use pluses and minuses for every d6 or +-6 off of 4d6+12. This works fine for near horse size, not so well for rather smaller: 2d6+6 would be -3, 6d6+36 would be +6, which sort of covers trolls.
Use half the stats, which covers pygmies down to ducks and pixies for smaller. Larger works as well, but rapidly gets to non humans.
Use a combo. Half the stats for smaller, pluses for larger for generally human sized. Larger species such as trolls could use double their size stats.
This doesn't cover ancient species Such as Roman horses were about eighty three percent as high as modern horses. So a modern horse is about eight feet long (not high), and a Roman horse is closer to six foot nine feet long. Nor the fact that using weight instead doesn't work well.