bellsystems-cp/docs/mqtt-app-user-auth.md

MQTT Authentication for Phone-App Users

Why this exists

The remote phone app (FlutterFlow + Firebase Auth) talks to Vesper boards over MQTT. We did not want to create an MQTT account per app user, so app users log into Mosquitto with their Firebase identity, and the Console backend decides what each one may do, based on which devices are assigned to them.

Mosquitto uses mosquitto-go-auth with two backends:

• files (passwd file on the VPS): admin, bonamin, NodeRED.
• http (this Console): POST /mqtt/auth/user (on CONNECT) and POST /mqtt/auth/acl (on SUBSCRIBE, PUBLISH and every message delivery). Code: backend/mqtt/auth.py, backend/mqtt/app_users.py.

The Mosquitto / go-auth config lives on the VPS, not in this repo.

Connection contract for the app (read this if you build the app)

Setting	Value
Username	app_<firebase_uid>
Password	The user's current Firebase ID token (getIdToken())
Client ID	Must start with app_<firebase_uid>_, e.g. app_<uid>_<installId>
Transport	TLS only (mqtts:// 8883 or wss://): the password is a bearer token
Wildcards	Not allowed (+ / # are denied)

Any other client ID is refused on every topic. The prefix rule stops one user from reusing another user's client ID to kick them off.

Allowed topics, only for serials assigned to that user:

Action	acc	Topics
Publish	2	vesper/{serial}/control/command
Subscribe / receive messages	4 / 1	vesper/{serial}/control/ack, vesper/{serial}/status/heartbeat, vesper/{serial}/status/playback

Everything else is denied (403).

Token lifetime: Mosquitto checks the password only at CONNECT. When the app reconnects, it must fetch a fresh ID token (getIdToken() refreshes an expired one). A connection stays open past token expiry. Blocking or unassigning still takes effect on that connection, because every publish or delivery goes through the ACL (within about 60 s of the go-auth ACL cache).

Which field is the device serial

The serial in vesper/{serial}/... is the device doc's serial_number: it is flashed into NVS and the firmware uses it as its MQTT id. Legacy docs only have it in device_id. users.service.device_serial_of() holds that rule; always use it.

How access is decided

• users/{doc}.device_serials: [string] lists the serials a user may reach. It mirrors the devices' user_list and is written in the same atomic Firestore batch by every Console path that edits user_list:
  • users.service.assign_device / unassign_device
  • POST / DELETE /api/devices/{id}/user-list (device Manage tab)
  • PUT /api/devices/{id} when the body contains user_list (devices.service.update_device)
• Users are resolved by the uid field (a query), never by doc ID. Console- created users get random doc IDs (.add()); FlutterFlow uses the uid as the doc ID.
• A user is refused when status == "blocked".
• Lookups are cached in-process for 60 s (mqtt/app_users.py). Assign/unassign, update, block/unblock and delete call invalidate(uid). If you add a new code path that changes user_list, device_serials or status, it must do the same. The cache is per process: if uvicorn ever runs multiple workers, invalidation only reaches one of them (entries still expire after 60 s).
• app_ usernames never go through the device HMAC or the legacy password.

Legacy "vesper" password

Boards on pre-HMAC firmware still use the shared password vesper. MQTT_ALLOW_LEGACY_PASSWORD (default true) controls it. It is only accepted for device-shaped usernames. Each use is logged at WARNING, once per board per hour:

MQTT legacy password accepted for <serial> — board still on pre-HMAC firmware

When those lines stop appearing, set MQTT_ALLOW_LEGACY_PASSWORD=false.

Rollout checklist

1. Deploy the backend.

2. Backfill device_serials for existing assignments (run it right after the deploy):

   cd backend
   python scripts/backfill_user_device_serials.py          # dry run, writes nothing
   python scripts/backfill_user_device_serials.py --apply  # writes
   

   It only writes the device_serials field on users docs. It is idempotent: a second run finds nothing to change. Until it runs, the Console shows existing users with no devices, and app users are denied everything.

3. go-auth config on the VPS (R = required, r = recommended):

   auth_opt_backends files, http                  # R  files first, then http
   auth_opt_http_host <host reaching the backend> # R
   auth_opt_http_port 8000                        # R
   auth_opt_http_getuser_uri /mqtt/auth/user      # R
   auth_opt_http_aclcheck_uri /mqtt/auth/acl      # R
   auth_opt_http_params_mode form                 # R  endpoints read form fields; go-auth's default is json
   auth_opt_http_response_mode status             # r
   auth_opt_http_method POST                      # r
   auth_opt_http_timeout 5                        # r
   auth_opt_disable_superuser true                # r  no superuser endpoint exists or is needed
   auth_opt_cache true                            # r
   auth_opt_cache_type go-cache                   # r
   auth_opt_auth_cache_seconds 60                 # r
   auth_opt_acl_cache_seconds 60                  # r
   auth_opt_auth_jitter_seconds 10                # r
   auth_opt_acl_jitter_seconds 10                 # r
   allow_anonymous false                          # R
   

4. Files-backend ACL: verified OK (2026-09-30). The VPS config sets no auth_opt_files_acl_path (nor acl_file). A live test confirmed the files backend does not grant ACLs to everyone, so every ACL check reaches the Console. If an ACL file is ever added, only lines under user admin / user bonamin / user NodeRED are safe. A pattern line, or a topic line placed before the first user line, applies to every user (app users included) and must not cover vesper/.... Re-run the isolation test after any broker config change (on the VPS host):

   A=PV25L22BP01R01; B=PV26B02BP01R01
   PW=$(docker exec bellsystems-backend python -c "from mqtt.auth import _derive_password; print(_derive_password('$A'))")
   # own topics -> should print messages
   docker exec mosquitto mosquitto_sub -h localhost -i acl-test-own -u "$A" -P "$PW" -t "vesper/$A/#" -v -W 5
   # foreign topics -> must say "All subscription requests were denied."
   docker exec mosquitto mosquitto_sub -h localhost -i acl-test-foreign -u "$A" -P "$PW" -t "vesper/$B/#" -v -W 5
   

5. Enable a TLS listener in Mosquitto for the app (certificate on the VPS).

Current VPS broker setup (as of 2026-09-30)

• Container mosquitto, image iegomez/mosquitto-go-auth:latest.
• Config: host /home/bellsystems/stacks/mosquitto/config/mosquitto.conf, mounted at /etc/mosquitto/mosquitto.conf (Docker images usually use /mosquitto/config; this one does not). Passwd file: /home/bellsystems/stacks/mosquitto/passwd.
• Already set correctly: backends files,http, http_host 172.20.0.1 (Docker bridge), http_port 8000, both URIs, http_method post, params_mode form, response_mode status, cache true, cache_reset true.
• Still to change: auth_cache_seconds and acl_cache_seconds are 300. Lower both to 60 before app users go live, or a block/unassign can take up to 5 minutes to apply.

Known gaps / decisions

• FlutterFlow writes to user_list. If the app changes user_list directly in Firestore (e.g. a claim flow), it must also update device_serials the same way (ArrayUnion / ArrayRemove of the serial on the user doc). Otherwise that user cannot reach the device over MQTT. The app owner will handle this in the app.
• The device_users subcollection on devices (older way of recording device users, read first by devices.service.get_device_users) is intentionally ignored by the backfill and the MQTT ACL. Left as is by decision.
• The two Console assignment paths store user_list entries in different formats (path strings vs DocumentReferences). This predates this work; all readers accept both.
• ACL denies are logged at INFO, which is not shown by default. Auth denies and legacy-password logins are WARNING.

Tests

cd backend
pip install pytest    # not in requirements.txt
python -m pytest tests

tests/test_mqtt_auth.py covers the auth and ACL endpoints. tests/test_device_serials_sync.py covers the device PUT sync. Firebase and Firestore are mocked.