It depends on the phone, some work just fine if you remove the battery while they are on, some can be made to run like that just by moving the pins that run to the input of the battery to the output of the battery, some can be made with a small soldering, and some are not able to be modded.
I found this : https://forum.xda-developers.com/galaxy-s3/accessories/how-to-samsung-galaxy-s-iii-battery-t3417303
Thats the good thing about popular phone models :)
RE: An easy, but very helpful tip : If you have an old phone you no longer use crunching GRC, remove the casing to improve cooling, and with that, perfomance.