This is where I screwed up so I recommend extra caution for other newbies. Only the black part is supposed to come off. Don't be an idiot like me and pry off too much...

weeelll I think I have a broken Vita now. Make sure you're not an idiot like me who doesn't know which parts to pull in which direction when trying to free the battery cable.