To further clarify lemerise's comment, I found the power icon (with a magnifying glass) between the fan and the orange optical drive cable. Worked like a charm!
Not only is it useful to loosen up the speaker next to the SATA connector I found it helpful to use a pen to mark the exact location of the edges of the SATA connector on top of the new drive, so I could guide the connector in place. It also helps to put the bumpers in after you get this SATA connector in place.
And DON’T do what I did, which is to remove the left speaker - which can easily lead to breaking the cable between the power on button and the power supply. The cable is routed through a hidden slot at the very bottom of the left speaker