Can I put a bigger battery in my Samsung Galaxy Note 20?

To add to Dan's excellent analysis, the short answer is no, there are almost no cases where you can interchange batteries between one phone model and another. Batteries are easy enough to design that each new model custom designs the battery used in it to fit the available space.

With older battery technologies, there were huge differences in power density such that you could get batteries with much higher power ratings that fit in the same space. That's not the case with lithium ion batteries; it's basically just a case of, the bigger the battery, the more storage (in mAh) it has. Then there are other considerations such as the dimensions of the battery (will it even fit in the space?) and the connection between the battery and the phone (will the connector reach, and is the plug the same; i.e., will it even plug in?). Then you get into the issues with the BMS, or Battery Management System, the small circuitry that comes mounted on the battery that provides things like temperature and voltage readings to the phone. Those have to be completely compatible or the phone is going to take issue with not being able to talk to the BMS.

Finally, you do realize that you're taking this entire gamble on the chance of getting 5% more battery life, right? That's all the difference there is between a 4300 mAh battery and a 4500 mAh one.

Finally, don't fall for any hype about so-called "gold" batteries that supposedly offer huge increases in power. Our own inestimable @oldturkey03 has done the research and concluded that those batteries are nothing but a scam.

The myth of the "high capacity" batteries - iFixit Repair Guide

All in all, my own opinion is that it's not going to be worth it, but of course it's your call as to whether you want to try it or not. Given that the two phones are very close in generations and size, there is a non-zero chance that what you want to do will work and it would certainly be interesting to find out if it does, but I'd suggest you go into it with low expectations and a willingness to part with a few bucks for a battery that might or might not work in your phone.

If you do decide to go ahead with it, be sure to let us know how it turns out!