Most likely because you’re switching to performance mode when gaming. That would turn off the fan limit, I’m pretty sure.
Rather than forcing the fan speed, have you checked what your GPU is doing when it spins fans up randomly? Is something using it? Is the temperature spiking?
btop can tell you some of that, installed by default. nvtop can tell you which process is using it.