feat(mqtt): store F-070 crash detail and group crashes fleet-wide

Firmware F-070 adds a fuller `crash` object (backtrace, backtrace_corrupted,
elf_sha256) and a new `pre_crash` snapshot (uptime, heap stats, optional
abort_msg) to boot_report and to telemetry.get_boot_history entries.

- Migration c9d0e1f2a3b4: device_boot_events gains crash / pre_crash JSONB,
  stored verbatim so later additive fields need no migration. The legacy
  crash_* columns are still filled; old rows and old firmware are unchanged.
  JSON is bound as CAST(CAST(:x AS TEXT) AS JSONB): a bare JSONB cast makes
  asyncpg JSON-encode the already-encoded string a second time.
- boot_report ingestion passes both objects through.
- telemetry.get_boot_history replies (control/ack) are merged into boot
  history whoever sent the command: an entry matches an existing row on
  boot_count + reset_reason + time within 30 min (boot_count alone is not
  unique - the counter gets reset), and only fills crash/pre_crash the row
  lacks; unmatched entries are boots we never saw live and are inserted at
  the device's timestamp; entries without ts are skipped.
- GET /api/mqtt/crash-groups: fault boots grouped by abort_msg with hex
  addresses stripped, else task + exception cause, else reset reason. When
  abort_msg is present, pc/exc_cause describe abort() itself and are
  ignored for grouping.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-09-30 16:28:40 +03:00
co-authored by Claude Opus 5.5
parent 46c5c0a846
commit 963516ece0
7 changed files with 365 additions and 25 deletions
+13
View File
@@ -9,7 +9,9 @@ from mqtt.models import (
AlertEventListResponse, BootEventListResponse, PingSampleListResponse,
DiagnosticsReportListResponse, LatestMetricsResponse,
LatestDiagnosticsEntry, LatestPingEntry, DeviceReportListResponse,
CrashGroupListResponse,
)
from mqtt.crash_groups import group_crashes
from mqtt.client import mqtt_manager
from mqtt import presence
import database as db
@@ -177,6 +179,17 @@ async def get_device_boot_events(
return BootEventListResponse(events=events, total=total)
@router.get("/crash-groups", response_model=CrashGroupListResponse)
async def get_crash_groups(
since: Optional[datetime] = Query(None, description="ISO timestamp — only crashes at/after this time"),
until: Optional[datetime] = Query(None, description="ISO timestamp — only crashes at/before this time"),
_user: TokenPayload = Depends(require_permission("mqtt", "view")),
):
"""Fault boots across the whole fleet, grouped by crash signature."""
events = await db.get_crash_events(since=since, until=until)
return CrashGroupListResponse(groups=group_crashes(events), total_crashes=len(events))
@router.get("/ping-samples/{device_serial}", response_model=PingSampleListResponse)
async def get_device_ping_samples(
device_serial: str,