How to change rechargable batteries

I'm guessing you have the most recent model that is charged via lightning cable? I'm guessing there might be a little confusion from other fixers, as most of us are used to the older model of the Magic Mouse that had the standard user-serviceable battery compartment. From the sound of your problem, it sounds like the rechargeable battery built in has reached the end of its effective lifespan. If it is the newer version, I recommend updating your question to specify that your device is the Magic Mouse 2 to avoid confusion.

You can find the teardown guide for the Magic Mouse 2 here.

It appears that the battery pack in this model can be replaced - however, adhesive is used in addition to screws to hold the mouse together, requiring a tool like the iOpener or something similar to take it apart. If you are feeling adventurous, you can give it shot - the battery can most likely be purchased directly from Apple (if I asked about MBP sata cable from them before, and it was possible to purchase the part by itself.) - although considering the use of adhesive in the Mouse's design, its worth considering paying for one of the Apple certified techs handle the replacement install for you.