OK first off bringing any giant size creates into any combat system designed for human to human size combat is going to be diffiult and require compromise. Remmeber susyems are easy abstractions to contain a story.
I agree that the RQ3 system created pretty unkillable beast, the RQ 2 system makes them feel a little tissue page thin a times, and they could easily fall victim to a lucky peseant with a pitch fork. ( Hyoerbol used ).
We aren't limited to those two solutions and there could be more. That again will be compomises but may scale better, within the effetcive range;
One possibility that jumps to mind is to make the bonsu on HP bonus for size( and/or power) a gradient rather than linear, a couple of options below;
Linear Progression Hybrid
13 - 16 +1 +1 +1
17 - 20 +2 +3 +3
21 - 24 +3 +6 + 5
25 -28 +4 +10 + 7
29 -32 +5 +15 + 10
32 -36 + 6 +21 +13
37 -40 + 7 + 28 + 17
There is obvious a place where this scales and becomes silly, but i think thats well oustide what we create rules for.
I would note that HP location boundaries are set to increase every 3 extra hps, that probably does not translate to large monsters and ypu need to look at increasign ti every 4 or 5hp.
The other less nuanced suggestion si to just give massive monsters an extra hp special ability where you add 5, 10 or 15hp as a special ability.