It seems to be a fairly rare issue, I can't find many other people talking about it (or at least people posting and having answers), and all my friends don't seem to have this issue. I can also join public lobbies just fine, but when my friends start lobbies it says they are "not in a lobby" and I can't join them.

So far I have tried:
Verifying game cache
Reinstalling the game
Disabling firewall
Restarting computer
Reinstalling VR drivers
Restarting router
Plugging my computer directly into my router (normally I use a switch)

I have also launched both directly from steam and through the Steam VR home and Steam VR menu. I am running out of options.

My PC specs are:

Ryzen 7 2700x
32gb 2866MHz RAM
EVGA GTX 1070
The game is installed to a Western Digital Blue 500gb m.2 sata SSD
ASUS B450 prime
Windows 10
My computer is going through a network switch, which then goes to my router. All via ethernet.
My ISP is verizon, and I have their gigabit fibre optic package.
HTC Vive