Depending on the replacement switches you purchased, you may need to shave the arm down that pushes on the switch. This is the part of the bumper that extends downward and physically presses the switch itself.

This was required for my device as I had leftover Xbox 360 switches that seem to be more tactile and durable than the Xbox One variants.