You don’t have to hack the firmware you just have to use a hacked driver. And what do you mean external transcodes are limited to 720p? I can watch 4K at work streaming from home on Plex. I also use my 1660 super for reencoding using tadarr and literally just tonight I was working on switching from Blue Iris to frigate and using my video card for AI. Also I do believe three is the limit of transcodes you can do on a consumer card not one.
I don’t relay anything through plex servers, I can easily access my media from anywhere without their help. You don’t seem to know what you are talking about.