This is the way.

Done correctly, all of the devices will be on one network (i.e. they can all see each other, and there is a single device - the primary router - that is providing NAT/DHCP services to all devices), AND there will be two wi-fi options in the house with two SSID’s, one from the primary and one from the secondary. So you would get the benefit of a faster wireless connection for the Quest.

One minor question for OP: do you really need the secondary router and that network switch? I would guess the secondary router would have enough ethernet ports.

Is this a double NAT situation? I’m unclear on why you need two networks. If you put the secondary router in bridge mode or access point mode, would that solve your problem? Is there a reason you can’t set up the VR thing on the main router?

Or, why not connect your media server to the secondary network?