I only noticed that as a problem on TVs that handle HDR poorly. They don't have enough peak brightness or enough dimming zones. For instance, TCL makes three lines of TV, sort of like the old Sears model of Good, Better, Best. On my 4 or 5 year end low price mode, HDR looked awful (My neighbor likes the TV now). The midrannge model from the next model year ithat I have at my Lake home now looks pretty good in HDR all the time. The midrange TCL QM7 model from last year at my other home looks spectacular, as I remember. I haven't seen it for 6 months.
If I had a TV that does HDR poorly, I would configure the Gemini to disble all HDR.