create_user wrote the profile with .add() (random doc ID). On first login the
FlutterFlow app looks for users/{uid}, doesn't find it, and creates a second,
bare doc - so every console-created user ended up duplicated, and devices
assigned in the console pointed at the doc the app never reads.
Now the profile is written to users/{uid} with created_time set, and the email
is lowercased to match what Firebase Auth stores. If the Firestore write
fails, the just-created Auth account is deleted so no orphan is left.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
DeviceUpdate accepts user_list, so a device PUT could add or remove users
without touching their device_serials - leaving the MQTT app ACL stale
(a removed user would keep access; an added user would be denied).
update_device now diffs the old vs new user_list and, in the same atomic
batch as the device write, ArrayUnion/ArrayRemoves the device's serial on
each added/removed user, then invalidates their MQTT ACL cache entries.
Dangling user references are skipped (updating a missing doc would fail
the whole batch). PUTs without user_list take the old single-update path.
Covered by tests/test_device_serials_sync.py (fake Firestore).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
First pytest suite in the backend (backend/tests/, run from backend/ with
`python -m pytest tests`). firebase_admin.verify_id_token and Firestore
are mocked, so no network access is needed. 82 cases:
- /user devices: HMAC ok / wrong / other serial's HMAC, kiosk HMAC,
legacy password with flag on/off, legacy rejected for non-device-shaped
usernames and for app_ users, HMAC rejected for app_ users, legacy-login
log rate limiting.
- /user app users: valid token (asserts check_revoked=True), token for a
different uid, revoked, expired, blocked user, unknown uid, empty uid,
and that a denied token never appears in logs.
- /acl app users: acc 1/2/4 allow/deny per topic, unsupported acc values,
wildcards, foreign serial, malformed topics, wrong clientid prefixes
(incl. uid-prefix collision), blocked/unknown users, cache hit +
invalidate, cache expiry.
- /acl devices/kiosk/superuser: unchanged behaviour.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The shared legacy password is still needed for boards on pre-HMAC
firmware, but it was accepted for any username. Now:
- controlled by MQTT_ALLOW_LEGACY_PASSWORD (config.py, default true;
documented in .env.example) so it can be switched off without a deploy,
- only accepted for device-shaped usernames (uppercase alphanumeric
segments joined by "-", optional "-kiosk"), never for app_ users or
any other shape,
- every successful legacy login is logged at WARNING with the username,
rate-limited to once per username per hour, so the boards still
depending on it are visible before the flag is turned off.
HMAC auth is unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
POST /mqtt/auth/acl now handles "app_<uid>" users:
- topic must be exactly vesper/{serial}/<a>/<b> with serial in the user's
device_serials (resolved by the users doc `uid` field),
- publish (acc 2): only control/command,
- subscribe (acc 4) and read/delivery (acc 1): only control/ack,
status/heartbeat, status/playback,
- wildcard topics (+ / #) are denied,
- clientid must start with "app_<uid>_" so one user cannot reuse another
user's client id to kick them off,
- blocked users are denied; anything else (incl. other acc values) is 403.
Lookups go through mqtt/app_users.py's 60s TTL cache so per-message
checks don't hit Firestore every time; assign/unassign/block invalidate it.
Also fixes the acc comment: mosquitto passes 1 = read (delivery),
2 = write (publish), 4 = subscribe - not "1 = subscribe, 3 = both".
Device/kiosk ACL is unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The remote FlutterFlow app connects to Mosquitto as "app_<firebase_uid>"
with a Firebase ID token as the password, so no per-user MQTT accounts
need to exist anywhere.
For app_ usernames, POST /mqtt/auth/user now:
- verifies the token with firebase_admin.auth.verify_id_token
(check_revoked=True),
- requires the decoded uid to equal the uid in the username,
- requires a users doc with that `uid` field (queried, not by doc id)
whose status is not "blocked" (same meaning as users.service.block_user).
It returns 200/403 and logs the deny reason - never the token.
app_ usernames never fall through to the HMAC / legacy "vesper" check.
Device and kiosk auth are unchanged. App users are still denied every
topic by the existing ACL until the app ACL lands in the next commit.
Both handlers are now plain `def` so the blocking Firestore / Firebase
calls run in FastAPI's threadpool instead of stalling the event loop.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One-off script to populate the new device_serials array for assignments
made before the assign/unassign sync existed. Treats each device's
user_list as the source of truth and sets every user's device_serials to
exactly the matching serials, so re-running it is a no-op.
Dry run by default; --apply writes (batched, <=400 per commit).
It also reports users whose doc ID != uid field (MQTT resolves users by the
uid field), users with no uid, devices with users but no serial, and
user_list entries pointing at non-existent users. user_list entries are
accepted as DocumentReferences, "users/{id}" paths or raw doc IDs, and an
entry that matches a uid field rather than a doc ID is mapped to its doc.
Dry run against current data: 13 devices, 13 users, 11 users to update,
0 doc-ID/uid mismatches, 1 dangling user_list entry
(Cx2Va72sUzDbr1T8Ebbh on BSVSPR-26I047-STD10R-88YFJP).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds a `device_serials: [string]` array to Firestore `users` docs so the
MQTT ACL (and get_user_devices) can answer "which boards may this user
reach?" without streaming the entire devices collection.
- The serial is the value used in MQTT topics vesper/{serial}/...: the
device doc's `serial_number` (flashed into NVS, used by the firmware as
its MQTT id), falling back to the legacy `device_id` for old docs.
Centralised in users.service.device_serial_of().
- assign_device / unassign_device now write the device's user_list and the
user's device_serials (ArrayUnion/ArrayRemove) in one atomic batch.
- The device Manage tab endpoints (POST/DELETE /api/devices/{id}/user-list)
also edit user_list, so they get the same batched sync - otherwise the
most common assignment path would silently leave device_serials stale.
- get_user_devices resolves devices via device_serials with chunked
Firestore "in" queries instead of a full collection scan. Requires the
backfill script (next commit) to be run for existing assignments.
- New mqtt/app_users.py: resolves users by the `uid` FIELD (not doc id -
create_user uses .add(), FlutterFlow uses uid as doc id) with a 60s
in-process TTL cache. Assign/unassign, update, block/unblock and delete
invalidate that uid's entry.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Missed from the previous Reset Stats commit — these are the
request/response schemas for POST /devices/{id}/reset-stats.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
New POST /devices/{id}/reset-stats + ResetStatsModal lets staff clear
QA/bench test data accumulated before a device ships to a customer:
Firestore bell/playback counters and Postgres history tables (logs,
heartbeats, commands, boot events, alert events, ping samples,
diagnostics reports, control/reports, and opt-in live alerts). The two
firmware-side resets (telemetry.reset_boot_data, logs.clear) go over
the existing MQTT command/ack flow instead, since they need the device
online and duplicating that round-trip server-side would just be a
second, inferior implementation.
Also includes two incidental cleanups in devices/router.py: audit-log
entries for create/update now use console_name in their label (missed
by the earlier console_name commit), and add/remove-device-user rename
their local Firestore client from `db` to `fs` to stop shadowing the
`db: AsyncSession` dependency param in the same function scope.
AddDeviceUserModal's results list also gets a max-height + scroll so a
long match list doesn't grow the modal off-screen.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The inventory list and detail page previously only showed raw
customer_id / user_list references. list_devices now batch-resolves
customer names and user display names/emails (via Firestore get_all(),
avoiding an N+1 round trip per device) and exposes them as
customer_name / users on DeviceInventoryItem. Search now matches
against device name, console name, customer name, and assigned users
in addition to serial/owner/batch.
Also adds hw_types (multi-select board type) and has_users filters to
GET /manufacturing/devices, and carries console_name through for
display. DeviceInventoryDetail is updated to show and use all of this
(customer name, assigned users, new UI components for signal/charts).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sysadmin/admin-only settings page for configuring how long device log
history is kept before pruning. GET/PUT /api/settings/log-retention,
new LogRetentionSettings page, nav entry, and route.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
New users can now be created with a password, which creates a real
Firebase Auth account (so they can log into the mobile app immediately)
alongside the Firestore profile document. UserCreate is now
UserProfile + password (request-only, never persisted or echoed back);
deleting a user also removes their Auth account.
- backend/users: split UserCreate into UserProfile (persisted shape)
and UserCreate (adds password), wire firebase_auth create/delete
- CreateUserModal: new lightweight modal for creating a user from
other flows (e.g. device onboarding) without leaving the page
- UserForm: adds the password field for new users; also fixes
useToast() being used undestructured (toast.success(...) was being
called on the hook's return value instead of its .toast method)
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Lets staff set a friendly name for a device independent of whatever
name the end user gave it in the app. console_name is never shown to
app users and never synced from/to device_name; every display label
across search, equipment/helpdesk name resolution, device search, and
the Manage tab's issue linker now falls back through console_name ->
device_name -> serial rather than device_name alone.
Also includes an incidental one-line fix in devices/service.py: the
nested-struct deep-merge in update_device() was missing the newly
added device_health_settings key.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
subscrStart, warrantyStart, and maintainedOn are written to Firestore as
Timestamps like the other date fields here, but were missing from
_TIMESTAMP_FIELD_NAMES, so they weren't being converted back to ISO
strings on read.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Two-tier (warning/critical) threshold config per device — restarts/week,
RSSI floor, free-heap floor, CPU temp ceiling, plus an offline timeout
used for both a client-computed health status icon and the Health tab
chart's gap detection. Purely advisory for now: no server-side
email/push alerting infra exists yet, so email_on_threshold and
push_on_crash_boot are placeholders for that future work.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Two efforts that landed together because the v2 topic work extends
tables the health-telemetry effort added days earlier in the same
files/functions, making them impractical to separate cleanly:
Health/diagnostics telemetry (schema, Jul 13-17):
- New Postgres tables: device_alert_events, device_boot_events,
device_ping_samples, device_diagnostics_reports, plus a `source`
column on device_logs to distinguish log origins
- Query/service layer in pg_mqtt.py and database/__init__.py for
inserting and listing this history, plus a "latest metrics" endpoint
combining most-recent diagnostics + ping RTT per device
- mqtt/router.py gains list endpoints for alert/boot/ping/diagnostics
history, consumed by the upcoming Health tab
MQTT v2 topic migration (Sep 21):
- Heartbeat payload flattened per vesper_mqtt_topic_spec_v2.md, adding
rssi/free_heap/state/ok fields
- Command replies move to control/ack, device-initiated events to
control/reports; mqtt/client.py subscribes to the new topic set and
runs a ping_loop (wired up in main.py) for RTT sampling
- mqtt/logger.py and pg_mqtt.py updated to parse and persist the new
payload shape alongside the legacy fields for backwards compatibility
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
melody.uid is never actually populated anywhere in MelodyForm.jsx, so keying
local .bsm storage on it (as the previous commit did) would silently break
for every existing melody. pid is the correct key anyway: it identifies the
underlying archetype binary, and multiple melodies legitimately share one
pid (each remaps the same note sequence to different bells/speed/duration
via its own settings). Deletion is now share-aware — a melody's binary is
only removed from disk once no other melody still references its pid.
Also adds backend/scripts/migrate_melody_binaries_to_local.py to backfill
existing melodies from their old Firebase URLs to local storage, with
--dry-run support and a warning list for pids whose melodies point at
different source files (a pre-existing data issue, flagged for manual
review via each melody's playback button rather than silently resolved).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
ESP32 devices can't spare the 40KB+ RAM a TLS client needs, so Firebase
Storage's HTTPS-only download URLs were blocking melody downloads. Binaries
are now written to local disk (./data/melody_binaries) and served through a
new unauthenticated /api/melodies/download/{pid} route, exposed publicly on
a separate melodies.bellsystems.net vhost (plain HTTP, no TLS) so the main
console domain can stay HTTPS-only with no exceptions. Preview audio still
uses Firebase Storage since it's only ever fetched by the HTTPS admin UI.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- Manufacturing router now uses shared/audit.log_action (Postgres) instead
of the separate manufacturing/audit.py (SQLite mfg_audit_log), so all
manufacturing events appear in the Log Viewer
- Added log_action calls to 5 previously unlogged endpoints: lifecycle
patch, lifecycle create, lifecycle delete, flash asset upload, flash
asset note
- Removed the now-redundant /manufacturing/audit-log endpoint
- Log Viewer restricted to sysadmin only: backend uses require_sysadmin
(was require_admin_or_above), frontend adds role guard on the page
- Fixed Action badge column clipping: table-layout auto + whiteSpace nowrap
so the column sizes to fit the widest badge (Status Change)
- Added device_batch entity type to Log Viewer entity labels and filters
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add deploy-host.sh for webhook-triggered docker redeploy
- Update docker-compose.yml and nginx.conf for auto-pull setup
- Fix vite.config.js and admin router for deployment environment
- Fix NVS CRC seed to use 0xFFFFFFFF to match esp_rom_crc32_le
- Add dual-panel flash UI: esptool log + live 115200 serial monitor
- Auto-reset device via RTS after flash (no manual power cycle needed)
- Clean up Header.jsx debug title text
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
bonaminandClaude Opus 5.5