Encryption on the wire: c2s, s2s and TLS

A message between two people on different servers never travels over a single connection. The client opens a stream to the server that holds the account, and that server opens a second stream to the server at the far end. Each connection is negotiated separately, and each is protected, or not, on its own terms. So the familiar question, whether XMPP is encrypted, splits in two: what guards the hop from client to server, and what guards the hop from server to server. RFC 6120 answers both with the same instrument, TLS, applied at two points on the route.

Two hops, two handshakes

The client to server stream and the server to server stream are both XML streams, and the specification runs them through the same machinery. Whichever side initiates, it opens a TCP connection and sends a stream header; the receiving entity answers with a header of its own and a list of stream features. Where the server supports TLS, the features list says so, and the connection is upgraded where it stands. The method is a STARTTLS extension modelled on extensions already familiar from IMAP, POP3 and ACAP, which is why the negotiation can be read in the stream itself rather than inferred from the outside.

Why does the stream restart?

The exchange is short. The initiating entity sends a <starttls/> command, and the receiving entity replies with either <proceed/> or <failure/>. The TLS handshake is deemed to begin immediately after the closing angle bracket of <proceed/>, and between command and reply the parties must not send so much as whitespace between elements, because the security layer depends on byte precision. Once TLS is in place, both parties must restart the stream: fresh headers, a fresh features list. The restart is not ceremony. What the server offers next can depend on whether the channel is protected. SASL negotiation ends the same way, with a second restart.

What travels inside the protected channel

Authentication comes second, and the order is deliberate. STARTTLS must be completed before SASL begins, partly to safeguard the credentials exchanged during SASL, and partly because a certificate presented during TLS can later justify the SASL EXTERNAL mechanism, where the far side is identified from its certificate alone. A server that insists on TLS before accepting a particular mechanism must not advertise that mechanism until TLS has completed. The client keeps its own ordered list of preferred mechanisms and works down it, skipping anything offered that is not on the list. The ordering rules in RFC 6120 pin all of this down.

Can a server insist on encryption?

Support for STARTTLS is required in every XMPP client and server implementation; that part is not negotiable. Whether a deployment tolerates an unprotected stream is left to its administrator, who may make TLS mandatory for client connections, for server connections, or for both. When it is mandatory, the server marks the feature with an empty <required/> child element and should advertise nothing else at the first stage. SASL sits at the other extreme: it is always mandatory, and no stream proceeds without it. The specification sets the floor; what each public server enforces is a decision left to the person running it.

Certificates that name a service

On the server to server hop the certificate does the introducing. A server certificate should include the domainparts of the services hosted on the machine, and the identifier types are prescribed: DNS-ID and SRV-ID support are required in software, clients verifying only the _xmpp-client service type, servers both _xmpp-client and _xmpp-server. The older XmppAddr type survives in software for compatibility but is no longer encouraged in certificates, and a wildcard is permitted only as the complete left-most label. The name need not match the machine: in the specification’s worked example, im.example.com lives, after SRV lookups, on a machine called x.example.com. One more reason to keep the DNS records a federated server publishes and its certificate in step.

Renegotiation, and why XMPP rarely bothers

TLS itself allows either party to start a new handshake, to refresh encryption keys or to protect client credentials by authenticating the server first. The specification’s answer is mostly to decline. Support for renegotiation is optional, and entities are discouraged from attempting it, because streams in XMPP are inexpensive to establish and a stream reset serves the first purpose without a new handshake. An entity that meets an attempt it does not support must close the TCP connection at once, without a stream error: the violation happened at the TLS layer, below anything XMPP could describe.

What the channel does not hide

TLS protects the stream from tampering and eavesdropping, but it is channel encryption between adjacent parties. Both hops can be encrypted and the two servers still read every stanza that crosses them. The developer community has been working on end to end encryption to raise the security bar further, a different layer with different assumptions.

Read a failure and you know the layer

The next time a client refuses to connect, the shape of the failure says where it died. A STARTTLS failure is an XMPP event: the server returns <failure/>, closes the stream and cuts the TCP connection, over causes as prosaic as a malformed command, a feature that was never offered, or an internal error. A TLS error such as bad_certificate or handshake_failure is different: it is generated and handled inside the handshake itself and never surfaces as a STARTTLS failure. Telling the two apart tells you which layer to blame. Check one thing on any server you use: TLS completes before a password moves.

rfc-editor.org

The link above leads to rfc-editor.org, where RFC 6120, the XMPP Core specification, is published in full. The document holds the STARTTLS negotiation of Section 5, the SASL negotiation of Section 6 and the certificate rules of Section 13.7, in numbered clauses with their MUST and SHOULD keywords and worked examples of stream headers. Everything compressed here is there at length, in the wording the IETF issued.