WebRTC / SIP over WebSocket registration fails in Kamailio

What you're seeing

WebSocket connection to 'wss://sip.example.com:8443/' failed:
JsSIP:Transport close() | WebSocket closed wss://sip.example.com:8443 (code: 1006)

Code 1006 means the connection died before or during the upgrade. If the socket opens but REGISTER gets no reply, or calls to the browser never arrive, the handshake worked and the problem is in SIP routing.

Likely causes

Most common first.

1. No working xhttp:request route. The HTTP upgrade is handled in event_route[xhttp:request] by ws_handle_handshake(). With no such route, or with checks in it that reject the request, the browser gets 403/404 and closes the socket.
2. tcp_accept_no_cl isn't set. The upgrade is an HTTP GET with no Content-Length. Without tcp_accept_no_cl=yes, Kamailio doesn't accept it as a complete message and the handshake never happens.
3. The Host check rejects the browser. The example route checks that the Host header (name and port) is one of Kamailio's own addresses. If sip.example.com:8443 isn't a listen address or alias, every handshake is refused.
4. The browser doesn't trust the certificate. If the certificate is self-signed, expired or for another name, the browser drops the connection during TLS and Kamailio sees almost nothing. Open https://host:port in the browser to see the certificate error.
5. Contacts aren't reachable after registration. Browsers register with an invalid Contact such as abcd.invalid. Unless the registration is fixed up for NAT/WebSocket, lookup() returns an address Kamailio can't route to, so incoming calls fail even though REGISTER got 200 OK.

How to fix

1. Listen, accept HTTP, load the modules

enable_tls=yes
tcp_accept_no_cl=yes
listen=tls:203.0.113.10:8443
alias=sip.example.com:8443

loadmodule "sl.so"
loadmodule "tls.so"
loadmodule "xhttp.so"
loadmodule "websocket.so"
modparam("tls", "config", "/etc/kamailio/tls.cfg")

2. Handle the upgrade

This follows the websocket module's documented example, with an optional Origin check:

event_route[xhttp:request] {
    set_reply_close();
    set_reply_no_connect();

    if ($Rp != 8443) {
        xhttp_reply("403", "Forbidden", "", "");
        exit;
    }

    if ($hdr(Upgrade) =~ "websocket"
            && $hdr(Connection) =~ "Upgrade"
            && $rm =~ "GET") {

        if ($hdr(Host) == $null || !is_myself("sip:" + $hdr(Host))) {
            xlog("L_WARN", "WS: bad Host $hdr(Host) from $si\n");
            xhttp_reply("403", "Forbidden", "", "");
            exit;
        }

        # optional: only your web app may connect
        if ($hdr(Origin) != "https://app.example.com") {
            xlog("L_WARN", "WS: rejected Origin $hdr(Origin) from $si\n");
            xhttp_reply("403", "Forbidden", "", "");
            exit;
        }

        if (ws_handle_handshake()) {
            exit;
        }
    }

    xhttp_reply("404", "Not Found", "", "");
}

If the host name contains a hyphen, quote it in alias. Test the upgrade without a browser. A working endpoint answers HTTP/1.1 101 Switching Protocols. Leave out the sip sub-protocol and Kamailio answers 400:

curl -i -N --http1.1 \
  -H "Connection: Upgrade" -H "Upgrade: websocket" \
  -H "Sec-WebSocket-Version: 13" -H "Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==" \
  -H "Sec-WebSocket-Protocol: sip" \
  https://sip.example.com:8443/

3. Fix up WebSocket registrations

In route[NATDETECT], treat WebSocket clients as NATed, so the registrar stores where the connection really is. FLT_NATS and the received_avp settings are already in the stock config; is_first_hop() needs siputils:

# stock: modparam("nathelper|registrar", "received_avp", "$avp(RECEIVED)")

route[NATDETECT] {
    # force_rport() is already called in the stock REQINIT
    # 19 = private Contact, Via mismatch, port mismatch; 64 = WebSocket
    if (nat_uac_test("19") || nat_uac_test("64")) {
        if (is_method("REGISTER")) {
            fix_nated_register();
        } else if (is_first_hop()) {
            set_contact_alias();
        }
        setflag(FLT_NATS);
    }
    return;
}

Behind nginx or another reverse proxy, the proxy must pass the upgrade (proxy_http_version 1.1, Upgrade and Connection headers) and have an idle timeout longer than the client's keepalive. Once registration works, WebRTC media needs rtpengine with ICE and DTLS-SRTP. See one-way audio.