You could run the reverse proxy on your VPS

This may be the way I need to do it, unfortunately (it seems a fair bit more complicated for someone with pretty limited knowledge on this kind of networking).

Ideally I’d like to use your second approach, with CF Tunnels (something I’ve looked into a bit in the past), but from what I understand, I’d run the risk of violating their TOS and being blocked if too much video passes through their server, which is fairly likely to happen when running a service like Emby through CF.

Thanks! Those are great tips, and definitely sound like the type of thing I’m looking for, and I’m sure I’d have no problem setting them up, aside from one issue:

My main concern, is using these services while behind CG-NAT; currently the only way I know how to access my network from outside at all is to go through my VPS using OpenVPN clients, since I don’t have a public IP.

I’d ideally like to do away with the VPN entirely, so I don’t need to set up client apps to give new devices access, but adding the extra layer of CG-NAT on top of those services makes this all more confusing for me, since most of the information I’ve found online doesn’t involve CG-NAT.