Here is a link with step by step instructions. The only thing you should have to take apart is popping off the top cover. https://ps4.wonderhowto.com/how-to/manually-eject-stuck-disc-playstation-4-console-0149510/
It's possible I suppose, but very very difficult. I'm not even sure where you would find just the glass. You would also need UV Glue, and other tools. In my opinion it would be a very good idea just to replace the entire assembly.