PSP-2000 could only turn on if battery is not inserted

As the question above said, my PSP-2000 will not turn on if battery is inserted, it only displays the green light (power), however if turned on using the Charger, it turns on properly and battery can now be used and charged.