"Rusty" sounds like a battery may have leaked and corroded the battery connection terminals or water has penetrated the controller at one time and done the same.
Remove the battery from the controller and then disassemble the controller and clean the terminals (and anything else) using a Q-Tip slightly moistened (not dripping) with Isopropyl Alcohol 99%+ (available at electronics parts stores).
Once the terminals are bright and clean (if there's any left that hasn't been corroded away) re-assemble and insert a new battery and check if it works OK.
Here's the Xbox 360 Wireless Controller Bottom Panel Replacement guide that may help.
Here's the rest of the controller repair guides if you have to disassemble it further. Xbox 360 Wireless Controller Repair