How a remote desktop works inside a browser
A browser controlling a computer with nothing installed on the viewing side looks like it should require a plugin. It does not, and understanding the four pieces involved explains most of the behaviour you notice — including why it occasionally takes a few seconds to start.
The four pieces
Capture. The host application asks the operating system for the screen and receives a stream of frames. On a desktop app this is a normal system capability; in a browser it is the same API a video call uses to share your screen.
Encoding. Frames are compressed as video — usually H.264 or VP8/VP9 — because sending raw pixels is out of the question. A 1080p desktop at 30 fps is about 1.5 Gbps uncompressed and roughly 3 to 8 Mbps encoded.
Transport. WebRTC carries the stream. It is the same machinery behind browser video calls, which is why it works without plugins: every modern browser already has it.
Input. Your clicks and keystrokes travel back over a WebRTC data channel, and the host replays them as synthetic input events on the operating system.
Why connecting takes a moment: ICE
Neither machine knows how to reach the other at the start. Both are behind routers, both have private addresses, and the internet cannot deliver to either directly. WebRTC solves this with ICE — a negotiation that gathers every possible way each side might be reachable and then tests them.
- Signalling. Both sides connect to a small server and exchange descriptions of what they support and how they might be reached. This is what your 6-digit code identifies: which two parties should be introduced. The signalling server is a matchmaker, not a pipe.
- STUN. Each side asks a STUN server "what address do I appear to have from outside?". The answer is the public address and port the router has assigned, which the other side might be able to reach.
- Candidate pairs. The two sides now have a list of possible addresses each — local network, public, relayed — and they try the combinations in parallel, preferring the fastest.
- TURN, if nothing works. When no direct path can be found, traffic is relayed through a TURN server. It always works and it costs an extra hop of latency, plus real bandwidth that someone pays for.
That whole exchange is why connecting is not instantaneous and why the first second or two can feel uncertain. It also explains why a session sometimes starts slightly laggy and then improves: ICE can switch to a better path after the connection is already usable.
What the servers see
Worth being precise about, because "it goes through their server" means different things for different servers.
- The signalling server sees that two parties want to be connected, their session code, and their candidate addresses — so IP addresses and timing. It does not carry the screen or your keystrokes.
- A STUN server sees a single request asking what your public address is. Nothing else passes through it.
- A TURN relay, when used, does carry every packet — but WebRTC media is encrypted with DTLS-SRTP between the two endpoints, and the relay is not one of them. It forwards ciphertext it cannot read. What it does know is that two addresses exchanged traffic, how much, and for how long.
So the practical summary: metadata is visible to infrastructure, content is not. That is a meaningfully different situation from a service that decrypts your session server-side, and it is worth checking which kind you are using — see who can see a remote session.
What this design is good and bad at
| Good at | Bad at |
|---|---|
| Working through NAT with no configuration | Fine text at low bitrate — video compression softens it |
| Video and motion, because it is a video codec | Guaranteed sub-20 ms input, because encode and decode cost time |
| Zero install on the viewing device | Long sessions on a phone — encode/decode drains battery |
| Encrypted transport as a requirement, not an option | Anything needing device redirection: printers, drives, smart cards |
More guides
Never put RDP straight on the internet — and what to do instead
Port 3389 is scanned continuously, credential stuffing against it is automated, and it is the most common entry point for ransomware. Four safer ways to reach the same machine.
Why a remote connection will not establish, and how to tell which layer failed
NAT, CGNAT, symmetric NAT and corporate firewalls each break the connection in a different place. A diagnosis order that identifies the layer in a few minutes.
Why the remote cursor lags, and which of the six causes is yours
Network round-trip, upstream bandwidth, encoder delay, frame rate, resolution and relay hops each add lag in a different way. How to tell them apart and what actually helps.
Try RemoteFrames
Install the host on the computer you want to reach, then enter its 6-digit code in any browser. No account — two free 10-minute sessions a day.
Try RemoteFrames