While I think there is still a fair part that Vive can improve their streaming solution, by tweaking my setup, I was able to improve my experience from basically unusable to near to if not perfect.
The first thing you need to know is that speed does not equal connection quality. My network speed also was perfectly fine the whole time, but the difficult part here is not pure sped, it's more routing, response time/ping and stability/sync.
For example, you're talking about "near zero latency", but then about ~10ms. In terms of network delay in a LAN, everything over 1ms is not near zero.
You saying "I dont connect my computer via LAN [Ethernet] to the router" basically means that you double the latency, you could have when using Ethernet instead of Wi-Fi.
"The connection drastically reduces down, picture quality" sound like you have enabled 'dynamic bitrate' in the settings of vive streaming hub. My guess as a software developer would be, that one lag spike results in it being scaled down and the upscaling is probably not aggressive enough to regulate it back up. Try turning it off. This will not help against the initial lag, but could solve the bad picture quality afterward.
One of my problems actually was too many devices connected to the same router, and I'm talking about 0 additional Wi-Fi connections, just a lot of traffic over Ethernet. This never had any impact on my speed, but seems to make problems with routing the VR stream stable enough.