docs(mqtt-auth): record TLS for app users via NPM + WebSockets
TLS for the phone app went live on the VPS on 2026-09-30: - Mosquitto got a second, non-published listener on 8083 with `protocol websockets`; port 1883 stays plain TCP for the boards (ESP32s can't spare RAM for TLS). - The mosquitto service joined the external Docker network npm_npmnet so NPM (NPMplus) can reach mosquitto:8083 by name; it stays on `default` to reach the Console backend at 172.20.0.1:8000. - NPM proxy host mqtt.bellsystems.net -> http://mosquitto:8083 terminates TLS and renews the Let's Encrypt cert. proxy_read/send_timeout 3600s added so NPM doesn't drop idle MQTT connections after 60s. NPMplus has no "Websockets Support" toggle (always on). - Verified end to end: a paho client over wss://mqtt.bellsystems.net:443 (path /mqtt) authenticated via the Console backend and received a heartbeat. Chosen over native 8883 because NPM already owns 80/443 and certificate renewal, so there is no extra cert handling on the host, and 443 also gets through networks that block 8883. The doc now gives the app's final transport (wss, 443, /mqtt, never 1883, keepalive < 3600s), the listener/network/NPM layout, the end-to-end test, rollback steps, and a new known gap: the backend's port 8000 is published on 0.0.0.0, so the /mqtt/auth/* endpoints are reachable from the internet. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5.5
@@ -23,9 +23,13 @@ The Mosquitto / go-auth config lives on the VPS, **not in this repo**.
|
||||
| Username | `app_<firebase_uid>` |
|
||||
| Password | The user's current **Firebase ID token** (`getIdToken()`) |
|
||||
| Client ID | Must start with `app_<firebase_uid>_`, e.g. `app_<uid>_<installId>` |
|
||||
| Transport | **TLS only** (`mqtts://` 8883 or `wss://`): the password is a bearer token |
|
||||
| Transport | **MQTT over secure WebSockets**: `wss://mqtt.bellsystems.net:443`, path `/mqtt` |
|
||||
| Wildcards | Not allowed (`+` / `#` are denied) |
|
||||
|
||||
**Never connect the app to port 1883.** It is plain TCP for the boards only. The broker
|
||||
would still check the login there, but by then the token has already been sent
|
||||
unencrypted. Keep the MQTT keepalive below 3600 s (e.g. 30–60 s).
|
||||
|
||||
Any other client ID is refused on every topic. The prefix rule stops one user from
|
||||
reusing another user's client ID to kick them off.
|
||||
|
||||
@@ -126,7 +130,22 @@ When those lines stop appearing, set `MQTT_ALLOW_LEGACY_PASSWORD=false`.
|
||||
# foreign topics -> must say "All subscription requests were denied."
|
||||
docker exec mosquitto mosquitto_sub -h localhost -i acl-test-foreign -u "$A" -P "$PW" -t "vesper/$B/#" -v -W 5
|
||||
|
||||
5. **Enable a TLS listener** in Mosquitto for the app (certificate on the VPS).
|
||||
5. **TLS for the app: done (2026-09-30).** See "Current VPS broker setup" below.
|
||||
Test it from the VPS host (should print `connect: Success` and a `got:` line):
|
||||
|
||||
docker exec -i bellsystems-backend python - <<'EOF'
|
||||
import time, paho.mqtt.client as m
|
||||
from mqtt.auth import _derive_password
|
||||
A = "PV25L22BP01R01"
|
||||
c = m.Client(m.CallbackAPIVersion.VERSION2, client_id="wss-test", transport="websockets")
|
||||
c.ws_set_options(path="/mqtt")
|
||||
c.tls_set()
|
||||
c.username_pw_set(A, _derive_password(A))
|
||||
c.on_connect = lambda c, u, f, rc, p: (print("connect:", rc), c.subscribe(f"vesper/{A}/#"))
|
||||
c.on_message = lambda c, u, msg: print("got:", msg.topic)
|
||||
c.connect("mqtt.bellsystems.net", 443, 30)
|
||||
c.loop_start(); time.sleep(8); c.loop_stop()
|
||||
EOF
|
||||
|
||||
### Current VPS broker setup (as of 2026-09-30)
|
||||
|
||||
@@ -137,15 +156,38 @@ When those lines stop appearing, set `MQTT_ALLOW_LEGACY_PASSWORD=false`.
|
||||
- Already set correctly: `backends files,http`, `http_host 172.20.0.1` (Docker bridge),
|
||||
`http_port 8000`, both URIs, `http_method post`, `params_mode form`,
|
||||
`response_mode status`, `cache true`, `cache_reset true`.
|
||||
- `allow_anonymous false` ✓. Only listener: `listener 1883 0.0.0.0` (plain TCP, used by
|
||||
the boards). **No TLS listener yet.** One must be added for the app (a second
|
||||
listener, e.g. 8883 with `certfile`/`keyfile`) while keeping 1883 for the boards.
|
||||
- Still to change: `auth_cache_seconds` and `acl_cache_seconds` are **300**. Lower
|
||||
both to 60 before app users go live, or a block/unassign can take up to 5 minutes
|
||||
to apply.
|
||||
- `allow_anonymous false` ✓. `auth_cache_seconds 60`, `acl_cache_seconds 60` ✓.
|
||||
`auth_opt_hasher` is not set (a harmless "defaulting to PBKDF2" warning at start).
|
||||
- Listeners:
|
||||
- `listener 1883 0.0.0.0`: plain TCP, **boards only** (ESP32 can't spare RAM for TLS).
|
||||
Published on the host as `1883:1883`.
|
||||
- `listener 8083` + `protocol websockets`: plain WebSockets, **not published** on the
|
||||
host. Only reachable from the Docker network `npm_npmnet`.
|
||||
- TLS is terminated by **NPM** (NPMplus, container `npm-npm-1`), which also renews the
|
||||
Let's Encrypt certificate. Proxy host: `mqtt.bellsystems.net` → `http://mosquitto:8083`,
|
||||
Force SSL + HTTP/2 on. Advanced config:
|
||||
|
||||
proxy_read_timeout 3600s;
|
||||
proxy_send_timeout 3600s;
|
||||
|
||||
(Without these, NPM drops idle MQTT connections after 60 s.) NPMplus has no
|
||||
"Websockets Support" toggle; WebSockets are always on.
|
||||
- Compose file: `/home/bellsystems/stacks/mosquitto/docker-compose.yml`. The service is
|
||||
on `default` (reaches the backend at `172.20.0.1:8000`) **and** the external network
|
||||
`npm_npmnet` (so NPM can reach `mosquitto:8083`). Removing `default` breaks all
|
||||
board logins.
|
||||
- Rollback for the TLS change: `.bak` copies of `mosquitto.conf` and `docker-compose.yml`
|
||||
sit next to the originals. Restore them, run `docker compose up -d` in that folder,
|
||||
then delete the NPM proxy host.
|
||||
|
||||
## Known gaps / decisions
|
||||
|
||||
- **Console backend port 8000 is public.** It is published as `0.0.0.0:8000` and the VPS
|
||||
has no firewall (Docker-published ports bypass host firewall rules anyway), so anyone
|
||||
can call `/mqtt/auth/user` and `/mqtt/auth/acl`. The passwords can't realistically be
|
||||
guessed, so it's not urgent, but these endpoints are meant only for Mosquitto.
|
||||
To be fixed separately, carefully, so nginx/NPM keep reaching the Console.
|
||||
|
||||
- **Old-firmware boards use a different command topic.** Broker logs (2026-09-30) show
|
||||
some boards (e.g. `PV26B02BP01R01`, `BSVSPR-26C20B-STD10R-2KCDPH`) subscribing to
|
||||
`vesper/{serial}/control`, not `vesper/{serial}/control/command`. The app ACL only
|
||||
|
||||
Reference in New Issue
Block a user