Lock button unresponsive - when locked often does not wake the screen.

If I lock the phone and then try to unlock it there are periods where the screen is totally unresponsive. I know the phone is on as I receive notification sounds, but even getting notifications does not turn on the screen as it should (ambient display is on). It happens even when the phone is fully charged or charging