Can I diagnose No TV output using multimeter?

I would say a multimeter is a solid diagnostic investment. And It's definitely possible it could help. I use mine regularly for things like this. You'll want something with pretty fine point leads for the Switch. Or something with replaceable leads. iFixit sells one that's super reasonable, but there are plenty of options around if you don't have one already. My Fluke 115 was a bit more of an investment.

You can use the list of known good diode mode readings at logi.wiki and compare to your Switch to locate any inconsistencies. Even if the port looks good, there are some pins that connect to the board under the connector. Which you also wouldn't be able to test directly with a multimeter. But they lead to the P13 USB/Display chip so you would be able to test the lines that way. Both the port and the P13 chip are regular points of failure for this issue.

To use a multimeter to get these readings, you would need to put it in diode mode. Whatever multimeter you buy, you'll want to check for the symbol in the attached image. Put the red probe on a point on the board that's a known ground, the anchor spots on the charge port, for example. The black probe would go on the spot you're trying to measure.