fix: key melody binary storage on pid instead of uid, add migration script

melody.uid is never actually populated anywhere in MelodyForm.jsx, so keying
local .bsm storage on it (as the previous commit did) would silently break
for every existing melody. pid is the correct key anyway: it identifies the
underlying archetype binary, and multiple melodies legitimately share one
pid (each remaps the same note sequence to different bells/speed/duration
via its own settings). Deletion is now share-aware — a melody's binary is
only removed from disk once no other melody still references its pid.

Also adds backend/scripts/migrate_melody_binaries_to_local.py to backfill
existing melodies from their old Firebase URLs to local storage, with
--dry-run support and a warning list for pids whose melodies point at
different source files (a pre-existing data issue, flagged for manual
review via each melody's playback button rather than silently resolved).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

.stitch/DESIGN.md

@@ -0,0 +1,133 @@
+# Design System: BellSystems Console Design
+**Project ID:** 18406618574074411899
+
+---
+
+## 1. Visual Theme & Atmosphere
+
+**"The Digital Observatory"** — A high-fidelity, immersive enterprise command center aesthetic that rejects the typical boxed-in SaaS feel. The mood is best described as **Atmospheric Depth**: dark, spacious, and precision-focused. Inspired by high-end editorial design and command center interfaces, data is treated as a premium asset surfaced through tonal layering rather than structural lines.
+
+The base is built on **Midnight Navy** — a deep blue-black that evokes an infinite canvas — with UI elements that appear to float and glow rather than sit flat. Hierarchy is established exclusively through surface tone shifts; hard borders are forbidden. The result feels like peering through a high-resolution window into enterprise data, where clarity comes from contrast in depth rather than weight.
+
+Overall density is **medium-high** — information-rich layouts with generous vertical whitespace between elements, but no wasted screen real estate.
+
+---
+
+## 2. Color Palette & Roles
+
+### Core Surfaces (darkest to lightest)
+| Name | Hex | Role |
+|---|---|---|
+| **Abyss** | `#0a0e14` | Deepest well; nested content backgrounds |
+| **Midnight** | `#10141a` | Main application viewport / page background |
+| **Void Navy** | `#181c22` | Sidebar, header, and secondary navigation surfaces |
+| **Deep Slate** | `#1c2026` | Default card and information module background |
+| **Elevated Slate** | `#262a31` | High cards, hovered table rows |
+| **Island** | `#31353c` | Active states, selected rows, top-layer containers |
+| **Frosted Glass** | `#353940` | Floating modals and dropdowns (at 80% opacity with blur) |
+
+### Accent & Brand Colors
+| Name | Hex | Role |
+|---|---|---|
+| **Indigo Glow** | `#c0c1ff` | Primary accent; CTA text, key metrics, active nav indicator |
+| **Lavender Soft** | `#8083ff` | Primary container fills |
+| **Violet Pulse** | `#d2bbff` | Secondary accent; gradient pair to Indigo Glow |
+| **Deep Indigo** | `#6001d1` | Secondary container fills |
+| **Royal Indigo** | `#494bd6` | Inverse primary; used on light-over-dark contexts |
+| **Aqua Sky** | `#7bd0ff` | Tertiary accent; data visualization, device status indicators |
+| **Ocean** | `#009bd1` | Tertiary container |
+
+### Semantic / State Colors
+| Name | Hex | Role |
+|---|---|---|
+| **Coral Error** | `#ffb4ab` | Error text and destructive action labels |
+| **Crimson** | `#93000a` | Error container backgrounds |
+| **Emerald** (Tailwind) | `emerald-*` | Online / active device status badges |
+| **Amber** (Tailwind) | `amber-*` | Warning / pending device status badges |
+
+### Text Colors
+| Name | Hex | Role |
+|---|---|---|
+| **Cloud** | `#dfe2eb` | Primary text; body copy, data values, headings |
+| **Mist** | `#c7c4d7` | Secondary / muted text; labels, metadata |
+| **Ghost** | `#908fa0` | Placeholder text, disabled states, divider fill |
+| **Boundary** | `#464554` | Ghost border fallback for inputs (used at low opacity) |
+
+---
+
+## 3. Typography Rules
+
+**Single font family throughout: Inter** (geometric sans-serif). Used for headlines, body, and labels alike — unity is achieved through weight and tracking variation rather than font switching.
+
+| Level | Size | Weight | Tracking | Usage |
+|---|---|---|---|---|
+| **Display** | 56px / 3.5rem | Bold (700) | Tight (negative) | Hero KPI metrics, total counts |
+| **Headline** | 24px / 1.5rem | SemiBold (600) | Normal | Page titles, major section headings |
+| **Title** | 16px / 1.0rem | Medium (500) | Normal | Card titles, module headers, tab labels |
+| **Body** | 14px / 0.875rem | Regular (400) | Normal | Default text, table rows, descriptions |
+| **Label** | 11px / 0.6875rem | SemiBold (600) | Wide (+0.1em) | Sidebar category headers, metadata chips — All Caps |
+
+**The Editorial Rule:** Sidebar category headers use the Label style in all-caps with wide letter-spacing (+0.1em). This creates a "system-level" authority that contrasts with fluid body text and signals structural navigation. Body text must never be all-caps.
+
+---
+
+## 4. Component Stylings
+
+### Buttons
+- **Primary CTA:** Gradient fill from Indigo Glow (`#c0c1ff`) to Violet Pulse (`#d2bbff`). White text. Softly rounded corners (matching `lg` radius — just barely rounded, not pill). On hover, a high-glow indigo shadow emanates beneath the button.
+- **Secondary / Outline:** Island background (`#31353c`) with an extremely faint ghost border — outline-variant (`#464554`) at 20–30% opacity. Mist text (`#c7c4d7`).
+- **Tertiary / Ghost:** Fully transparent background. Mist text. On hover: Cloud text with elevated background tint.
+- **Destructive:** Coral Error text (`#ffb4ab`) on a Crimson container (`#93000a` at low opacity). Used for delete and irreversible actions. Never bright red.
+
+### Cards & Containers
+- **Default Card:** Deep Slate background (`#1c2026`). No visible border — separation is purely tonal. Features a **subtle top-edge inner glow**: `inset 0px 1px 0px rgba(192, 193, 255, 0.05)` — mimics a ceiling light reflecting off the card's glass surface.
+- **Corner Rounding:** Minimal — just enough to soften, not enough to feel playful. Approximately 4px (the `lg` token = 0.25rem). Cards feel rectangular and purposeful, not bubbly.
+- **Elevation principle:** No drop shadows on standard cards. Hierarchy comes from the surface color step (Void Navy → Deep Slate → Elevated Slate).
+- **Floating Modals / Dropdowns:** Frosted Glass background (`rgba(53, 57, 64, 0.8)`) with `backdrop-filter: blur(12px)`. This "Glassmorphism" effect keeps the user contextually anchored to the underlying page while the modal floats above. Shadow: `0px 8px 24px rgba(13, 17, 23, 0.6)` — navy-tinted, never pure black.
+
+### Inputs & Forms
+- **Default state:** Deep Slate background. Ghost border at very low opacity (near invisible). Text in Cloud color.
+- **Focus state:** Indigo Glow (`#c0c1ff`) border at 40% alpha creates a soft halo glow — not a hard ring. The inner glow on the input also subtly strengthens.
+- **Placeholder text:** Ghost color (`#908fa0`).
+- **Select / Dropdown:** Same surface as inputs; opens as a Frosted Glass panel with blur.
+- **No hard outlines at rest** — inputs feel embedded in the surface until interacted with.
+
+### Status Badges
+- **Shape:** Fully pill-shaped (maximum border-radius / `full` = 0.75rem).
+- **Style:** Functional color (Emerald, Amber, Coral) at approximately 15% background opacity, with the same color at full 100% opacity for the label text. Creates a "glowing ink" effect — the badge appears illuminated from within.
+- **Examples:** Online → soft emerald glow; Warning → soft amber glow; Error/Offline → soft coral glow.
+
+### Navigation Sidebar (224px wide)
+- **Background:** Void Navy (`#181c22`) — one step lighter than the main viewport.
+- **Active item indicator:** A `3px` vertical bar on the far-left edge using Indigo Glow (`#c0c1ff`). The text weight also increases slightly. No background highlight on active items — the light bar IS the indicator.
+- **Inactive items:** Mist text (`#c7c4d7`) at normal weight.
+- **Category headers:** Label-style — all-caps, 11px, SemiBold, wide tracking. Ghost color (`#908fa0`).
+- **Item padding:** Comfortable vertical padding (~9.6px / 0.6rem) for breathing room between items.
+- **No dividers** between nav sections — spacing does the work.
+
+### Data Tables
+- **Row separation:** No horizontal dividers. Alternating subtle tonal rows (Island `#31353c` on hover) and consistent vertical gap rhythm.
+- **Header row:** Mist text (`#c7c4d7`), Label-style capitalization, slightly smaller than body.
+- **Hovered row:** Elevated Slate (`#262a31`) or Island (`#31353c`) background.
+- **Selected row:** Island background with Indigo Glow left border accent.
+
+### Scrollbars
+- Slim — 4px wide track.
+- Thumb: Boundary color (`#464554`), with 2px border-radius.
+- Track: Transparent.
+- Overall feel: Nearly invisible unless sought out.
+
+---
+
+## 5. Layout Principles
+
+**Whitespace is structure.** The design never uses lines or dividers to separate sections — space does that job. When content feels disconnected, the solution is always to add vertical breathing room, never to draw a border.
+
+- **Page content area:** Fills the viewport to the right of the 224px sidebar. Padding inside the content area is generous — approximately 24–32px on all sides.
+- **Section spacing:** Major sections within a page are separated by approximately 24px of vertical space. Sub-sections by 16px.
+- **Card grid:** Cards sit in fluid grids with 16px gaps. Cards never touch each other.
+- **Alignment:** Strong left-edge alignment for all content. Data tables, card headers, and page titles all share the same left origin point.
+- **No horizontal rules / `<hr>` elements:** Surface color transitions and whitespace define the visual structure entirely.
+- **Modals:** Centered in the viewport, overlaid on a dark scrim. The page content behind is still readable through the frosted glass effect, maintaining spatial context.
+- **The "No Raw Border" rule:** Any element requiring a visible boundary for accessibility (e.g., an active input) must use the ghost border approach — Boundary color (`#464554`) at 20% opacity maximum. Full-opacity borders are strictly prohibited.
+- **Mobile / responsive:** The sidebar collapses to a drawer on narrow viewports. Cards reflow to single-column. The design's depth relies on background layers, so it translates naturally to smaller screens.

CLAUDE.md

@@ -0,0 +1,383 @@
+# CLAUDE.md — BellSystems Control Panel v2
+# Instructions for Claude Code
+
+> Read this file at the start of every session.
+> This is the v2 project — a clean rebuild. The old v1 code lives in `frontend/src/_archive/` for reference only.
+> Also read `DESIGN.md` before writing any UI code.
+
+---
+
+## Project Structure
+
+```
+C:\development\bellsystems-cp-v2\
+│   CLAUDE.md                    ← you are here
+│   DESIGN.md                    ← design rules, component contracts, page layout spec
+│   docker-compose.yml
+│
+├── backend/                     ← FastAPI backend — DO NOT MODIFY
+│
+└── frontend/
+    ├── src/
+    │   ├── _archive/            ← v1 reference code — READ ONLY, never import from here except auth
+    │   ├── assets/
+    │   │   ├── global-icons/    ← action SVGs (edit, delete, download, etc.)
+    │   │   ├── side-menu-icons/ ← sidebar navigation SVGs
+    │   │   ├── comms/           ← communication type SVGs
+    │   │   ├── other-icons/     ← misc SVGs
+    │   │   └── customer-status/ ← CRM status SVGs
+    │   ├── components/
+    │   │   ├── ui/              ← design system components (the ONLY place to source UI)
+    │   │   ├── layout/          ← Sidebar, Header, MainLayout
+    │   │   └── shared/
+    │   ├── hooks/
+    │   ├── lib/
+    │   ├── modals/              ← all modal components live here, grouped by domain
+    │   ├── pages/               ← one file per page, grouped by domain
+    │   ├── providers/
+    │   ├── router/
+    │   │   └── index.jsx        ← all routes defined here
+    │   ├── styles/
+    │   │   ├── tokens.css       ← ALL design tokens (colors, fonts, spacing, shadows)
+    │   │   ├── components.css   ← ALL component-level styles
+    │   │   └── global.css       ← base resets, typography, scrollbar, .page-wrapper
+    │   └── main.jsx             ← app entry point
+    └── vite.config.js
+```
+
+---
+
+## Project Overview
+
+Bespoke SaaS Admin Console for BellSystems. Manages Devices, Customers (CRM),
+Manufacturing, Firmware, MQTT, Melodies, Staff, and more.
+
+- **Backend:** FastAPI at `backend/` — never modify
+- **Archive:** `frontend/src/_archive/` — v1 reference, read-only
+- **Active code:** `frontend/src/` (everything except `_archive/`)
+- **Design rules:** `DESIGN.md` at project root — read before writing any UI
+- **Style Guide:** live at `/dev/styleguide` — shows every component with every variant
+- **API client:** `frontend/src/lib/api.js` wraps `_archive/api/client.js`
+
+---
+
+## Import Alias
+
+`@/` maps to `frontend/src/`:
+
+```js
+import Button from '@/components/ui/Button'             ✅
+import Select from '@/components/ui/Select'             ✅
+import { useAuth } from '@/hooks/useAuth'               ✅
+import MainLayout from '@/components/layout/MainLayout' ✅
+```
+
+Never use relative `../` paths except inside `providers/` and `hooks/` when referencing `_archive/`.
+
+---
+
+## The Golden Rules
+
+- **Never modify `_archive/`** — it is read-only reference material
+- **All new code goes in `frontend/src/`** — no exceptions
+- **No `/v2/` prefix anywhere** — routes start from `/`, imports start from `@/`
+- **Read `DESIGN.md` before writing any UI code**
+- **Source every UI element from `@/components/ui/`** — no raw HTML elements for styled things
+- **Use only CSS tokens** — never raw hex, rgb, or pixel values in component or page files
+- **Use `.masonry-grid` for all content pages with multiple variable-height sections** — never `display: grid` with fixed columns for card layouts. See DESIGN.md §11.
+
+---
+
+## Available UI Components
+
+Every component lives in `frontend/src/components/ui/`. These are the ONLY components to use.
+Check the live Style Guide at `/dev/styleguide` to see all variants and states.
+
+| Component       | Import path                          | Purpose                                      |
+|-----------------|--------------------------------------|----------------------------------------------|
+| `Button`        | `@/components/ui/Button`             | All interactive actions                       |
+| `StatusBadge`   | `@/components/ui/StatusBadge`        | Coloured status pills                         |
+| `FormField`     | `@/components/ui/FormField`          | Every text/email/password/textarea input      |
+| `Select`        | `@/components/ui/Select`             | Custom dropdown (used inside FormField type="select") |
+| `Modal`         | `@/components/ui/Modal`              | All overlay dialogs                           |
+| `ConfirmDialog` | `@/components/ui/ConfirmDialog`      | Destructive / confirmation prompts            |
+| `DataTable`     | `@/components/ui/DataTable`          | All tabular data with sorting/selection       |
+| `Pagination`    | `@/components/ui/Pagination`         | Page controls beneath DataTable               |
+| `Spinner`       | `@/components/ui/Spinner`            | Loading indicators                            |
+| `PageHeader`    | `@/components/ui/PageHeader`         | Page title block — every page starts with this |
+| `Card`          | `@/components/ui/Card`               | Contained content sections                   |
+| `Tabs`          | `@/components/ui/Tabs`               | Tabbed navigation within a page              |
+| `Toast`         | `@/components/ui/Toast`              | Transient notifications (via `useToast`)     |
+| `SearchBar`     | `@/components/ui/SearchBar`          | Search inputs with debounce                  |
+| `Breadcrumbs`   | `@/components/ui/Breadcrumbs`        | Navigation trail on detail pages             |
+| `Icon`          | `@/components/ui/Icon`               | Inline SVG icons by name                     |
+
+---
+
+## Folder Structure — Pages & Modals
+
+Folders mirror the sidebar section hierarchy exactly.
+
+```
+frontend/src/pages/
+├── auth/                        ← unauthenticated routes (login)
+├── dashboard/                   ← General section
+├── bellcloud/                   ← Bell Cloud section
+│   ├── devices/
+│   │   └── notes/
+│   ├── users/
+│   ├── melodies/
+│   │   └── archetypes/
+│   └── mqtt/
+├── crm/                         ← Headquarters section
+│   ├── comms/
+│   │   └── mail/
+│   ├── customers/
+│   │   └── tabs/
+│   ├── orders/
+│   ├── quotations/
+│   └── products/
+├── engineering/                 ← Engineering section
+│   ├── manufacturing/
+│   ├── firmware/
+│   └── developer/
+├── public/                      ← Public / unauthenticated pages
+│   ├── cloudflash/
+│   └── serial/
+├── settings/                    ← Console Settings section
+│   └── staff/
+└── dev/                         ← Internal dev tools (StyleGuide)
+```
+
+```
+frontend/src/modals/
+├── bellcloud/
+│   ├── devices/
+│   ├── melodies/
+│   └── users/
+├── crm/
+│   └── products/
+├── engineering/
+│   └── manufacturing/
+└── shared/
+```
+
+---
+
+## Page Layout — How Every Page Is Structured
+
+Every authenticated page is rendered inside `MainLayout`, which provides:
+- **Sidebar** — fixed left, `224px` wide (`--sidebar-width`)
+- **Header** — fixed top, `56px` tall (`--header-height`)
+- **Content area** — the remaining viewport space
+
+Inside the content area, every page uses the `.page-wrapper` class (defined in `global.css`):
+
+```css
+.page-wrapper {
+  flex: 1;
+  display: flex;
+  flex-direction: column;
+  padding: var(--space-12);     /* 48px on all sides — desktop */
+  gap: var(--space-6);          /* 24px between top-level sections */
+  min-width: 0;
+}
+
+@media (max-width: 768px) {
+  .page-wrapper {
+    padding: var(--space-8);    /* 32px on mobile */
+    gap: var(--space-4);
+  }
+}
+```
+
+**This is the consistency guarantee**: because every page uses `.page-wrapper`, the `<PageHeader>` title on every page starts at exactly the same position — 48px from the top and 48px from the left edge of the content area. Never override these paddings. Never add extra wrappers around `page-wrapper` that introduce additional offset.
+
+### Content width modes
+
+Pages fall into two modes — choose based on how much content the page has:
+
+**Full-width** (default — lists, tables, dashboards):
+```jsx
+<div className="page-wrapper">
+```
+
+**Centered** (forms, settings, pages with very few items):
+```jsx
+<div className="page-wrapper page-wrapper--centered">
+```
+Centered mode caps each direct child at `--content-max-width-sm` (640px) by default and centers it horizontally. Override when needed:
+```jsx
+<div className="page-wrapper page-wrapper--centered"
+     style={{ '--page-content-max-width': 'var(--content-max-width-md)' }}>
+```
+Available tokens: `--content-max-width-xs` (480px), `--content-max-width-sm` (640px), `--content-max-width-md` (800px), `--content-max-width-lg` (1024px).
+
+---
+
+## Rules for Every Page
+
+### Before writing any code
+1. Read `DESIGN.md` — confirm tokens and components to use
+2. Check `frontend/src/components/ui/` — use existing components only
+3. Check the Style Guide at `/dev/styleguide` for the correct variant/props
+4. Check `frontend/src/_archive/` for the equivalent v1 page — copy API calls and data shape only, never styling
+
+### Page template
+
+```jsx
+// frontend/src/pages/[domain]/PageName.jsx
+
+import PageHeader from '@/components/ui/PageHeader'
+import { useAuth } from '@/hooks/useAuth'
+// Import ONLY from @/components/ui/ — never raw HTML elements for styled things
+
+export default function PageName() {
+  // 1. Auth
+  const { user } = useAuth()
+
+  // 2. State & data fetching
+
+  // 3. Event handlers
+
+  // 4. Render — always handle: loading, error, empty, data states
+  return (
+    <div className="page-wrapper">
+      <PageHeader title="Page Title" subtitle="Optional description">
+        {/* Action buttons — use <Button variant="primary"> etc. */}
+      </PageHeader>
+
+      {/* Page content — use Card, DataTable, Tabs, etc. */}
+    </div>
+  )
+}
+```
+
+### Styling rules
+- **Tailwind** for layout only: `flex`, `grid`, `items-center`, `min-w-0`, etc.
+- **CSS token variables** for ALL colors, spacing, typography — `var(--token-name)`
+- **No** `.module.css` or per-page scoped CSS files
+- **No** `style={{ }}` inline styles except for genuinely dynamic values (e.g. calculated widths)
+- **No** raw hex, rgb, or pixel values anywhere
+
+### Toolbar buttons — matching SearchBar height
+
+`.btn` has `line-height: 1` while `.searchbar-input` has `line-height: var(--line-height-base)` (1.5). This makes buttons shorter than the search bar by default. Whenever a `<Button>`, `<SegmentedControl>`, or `<IconButtonGroup>` sits in the same toolbar row as a `<SearchBar>`, all buttons must use `padding-top/bottom: var(--space-3)` and `line-height: var(--line-height-base)`.
+
+**Already handled automatically (no extra props needed):**
+- `SegmentedControl` — `.seg-ctrl__btn` in `components.css` enforces `--space-3` padding and `var(--line-height-base)` globally.
+- `IconButtonGroup` — `.icon-btn-group__btn` in `components.css` enforces `--space-3` padding globally.
+
+**Must be overridden manually — standalone `<Button>` in the same row as a `<SearchBar>`:**
+
+```jsx
+// Option A — inline style prop on the button:
+<Button
+  size="md"
+  style={{ paddingTop: 'var(--space-3)', paddingBottom: 'var(--space-3)', lineHeight: 'var(--line-height-base)' }}
+>
+  Label
+</Button>
+
+// Option B (preferred when multiple buttons share a toolbar) — scoped CSS block:
+<>
+  <style>{`
+    .my-toolbar .btn {
+      padding-top: var(--space-3) !important;
+      padding-bottom: var(--space-3) !important;
+      line-height: var(--line-height-base) !important;
+    }
+  `}</style>
+  <div className="my-toolbar" style={{ display: 'flex', alignItems: 'center', gap: 'var(--space-2)' }}>
+    <IconButtonGroup … />
+    <Button variant="primary" size="md">Compose</Button>
+  </div>
+</>
+```
+
+### Date & time formatting
+- **Greek/European date style everywhere** — DD/MM/YYYY, never US-style MM/DD/YYYY
+- **All date/time formatting must use `@/lib/formatters`** — never use raw `toLocaleDateString()`, `Intl.DateTimeFormat`, `toLocaleString()`, or `toISOString().slice()` in pages or modals
+- **For `datetime-local` input values** — use `toDatetimeLocal(iso)` and `nowLocal()` from formatters. Never use `new Date(x).toISOString().slice(0, 16)` — it converts to UTC and shifts the time (timezone bug)
+- **Currency** — use `fmtEuro(n)` from formatters (Greek locale: `1.250,00 €`)
+
+Available formatters (`import { ... } from '@/lib/formatters'`):
+
+| Function            | Output example                     | Use for                              |
+|---------------------|------------------------------------|--------------------------------------|
+| `fmtDate`           | `05/03/2026`                       | Short numeric dates (tables, lists)  |
+| `fmtDateMedium`     | `5 Mar 2026`                       | Medium dates (cards, details)        |
+| `fmtDateLong`       | `5 March 2026`                     | Long dates (headings, summaries)     |
+| `fmtDateFull`       | `Wednesday, 5 March 2026`          | Dashboard, full context              |
+| `fmtDateTime`       | `5 March 2026, 2:30 pm`            | Date + 12h time                      |
+| `fmtDateTimeMedium` | `5 Mar 2026, 14:30`                | Date + 24h time (compact)            |
+| `fmtDateTimeFull`   | `Wed, 5 Mar 2026, 2:30 pm`         | Emails, comms                        |
+| `fmtTime24`         | `14:30:05`                         | Time with seconds                    |
+| `fmtRelative`       | `5 minutes ago`                    | Relative timestamps                  |
+| `toDatetimeLocal`   | `2026-03-05T14:30`                 | `datetime-local` input values        |
+| `nowLocal`          | `2026-03-05T14:30`                 | Current time for form defaults       |
+| `toDateInput`       | `2026-03-05`                       | `date` input values                  |
+| `fmtEuro`           | `1.250,00 €`                       | Euro currency                        |
+
+### Data fetching
+- Use `frontend/src/lib/api.js` (wraps `_archive/api/client.js`)
+- Every data-fetching component must handle **loading**, **error**, and **empty** states
+
+### Modals
+- Never defined inside page files
+- Live in `frontend/src/modals/[sidebar-section]/[domain]/ModalName.jsx` — mirror the pages folder structure
+- Pass data via props, actions via callbacks
+
+---
+
+## Missing Components — Stop and Ask
+
+If building a page requires a UI component that does not exist in
+frontend/src/components/ui/, Claude Code must STOP and say:
+
+"I need a [ComponentName] component which doesn't exist yet.
+Please build it and add it to the StyleGuide before I continue."
+
+Do NOT:
+- Invent an inline one-off component inside a page file
+- Use raw HTML elements styled with inline CSS as a substitute
+- Proceed and leave a placeholder
+
+The StyleGuide at frontend/src/pages/dev/StyleGuide.jsx is the source
+of truth for what components exist and how they look. Every component
+used in a page must have a visible example there first.
+
+## Building a New Page — Checklist
+
+- [ ] File in correct `frontend/src/pages/[domain]/` folder
+- [ ] Root element is `<div className="page-wrapper">` — nothing else, nothing wrapping it
+- [ ] First child inside `page-wrapper` is `<PageHeader title="...">`
+- [ ] Only components from `frontend/src/components/ui/` used
+- [ ] No raw hex colors or pixel spacing values anywhere
+- [ ] Loading state implemented
+- [ ] Error state implemented
+- [ ] Empty state implemented
+- [ ] Mobile responsive (375px minimum)
+- [ ] Modals in `frontend/src/modals/`
+- [ ] Route added to `frontend/src/router/index.jsx`
+
+---
+
+## API Client
+
+```js
+// frontend/src/lib/api.js
+export { default } from '../_archive/api/client'
+```
+
+All pages import from `@/lib/api`, never directly from `_archive`.
+
+---
+
+## Auth
+
+`frontend/src/hooks/useAuth.js` re-exports from the archive AuthContext.
+`frontend/src/providers/AuthProvider.jsx` re-exports the AuthProvider.
+
+These are the ONLY two files permitted to import from `_archive/auth/`.
+All other files use `@/hooks/useAuth`.

DESIGN.md

@@ -0,0 +1,627 @@
+# DESIGN SYSTEM — BellSystems Control Panel v2
+
+> Single source of truth for all UI/UX decisions.
+> Read before writing any page, component, or modal.
+> Never override these rules inline. Change the rule here first, then propagate.
+>
+> Live reference: `/dev/styleguide` — every component, every variant, every state.
+
+---
+
+## 1. Core Philosophy
+
+- **Consistency over creativity.** Every page must feel like it belongs to the same product.
+- **Tokens over hardcoded values.** Never write a raw color, spacing value, or font size. Always `var(--token)`.
+- **Components over repetition.** If you write the same pattern twice, it becomes a shared component.
+- **Page layout is global.** All pages share the same padding and spacing anchors. Content always starts at the same position.
+- **Accessible by default.** ARIA labels, keyboard navigation, visible focus states on all interactive elements.
+
+---
+
+## 2. Page Layout Anatomy
+
+Every authenticated page lives inside `MainLayout`, which provides:
+
+```
+┌─────────────┬────────────────────────────────────────────┐
+│             │  HEADER  (height: 56px / --header-height)  │
+│             │────────────────────────────────────────────┤
+│             │                                            │
+│   SIDEBAR   │          CONTENT AREA                      │
+│  (224px /   │  ┌──────────────────────────────────────┐  │
+│ --sidebar-  │  │  .page-wrapper                       │  │
+│   width)    │  │  padding: 48px (--space-12)          │  │
+│             │  │  gap: 24px between sections          │  │
+│             │  │                                      │  │
+│             │  │  <PageHeader>   ← always first       │  │
+│             │  │  <content...>                        │  │
+│             │  └──────────────────────────────────────┘  │
+└─────────────┴────────────────────────────────────────────┘
+```
+
+### The consistency guarantee
+
+`.page-wrapper` is defined once in `global.css`. It is the only wrapper used on every page:
+
+```css
+.page-wrapper {
+  flex: 1;
+  display: flex;
+  flex-direction: column;
+  padding: var(--space-12);  /* 48px — desktop */
+  gap: var(--space-6);       /* 24px between direct children */
+  min-width: 0;
+}
+/* Mobile: padding drops to --space-8 (32px), gap to --space-4 (16px) */
+```
+
+**Rules:**
+- Every page's root element is `<div className="page-wrapper">` — no exceptions
+- Never add extra padding, margin, or wrapper divs that shift content relative to `.page-wrapper`
+- Never override `.page-wrapper` padding per-page
+- The `<PageHeader>` is always the first child inside `.page-wrapper`
+- This ensures that on every page, the title starts at exactly 48px from the top-left corner of the content area
+
+### Content width modes
+
+Every page falls into one of two modes:
+
+#### 1. Full-width (default)
+Content expands to fill the entire available area. Use this for all data-heavy pages: lists, tables, dashboards, detail views.
+
+```jsx
+<div className="page-wrapper">
+  {/* content fills the full content area */}
+</div>
+```
+
+#### 2. Centered (narrow content)
+For pages with a small number of elements that would look lost spanning the full viewport — e.g. settings forms, auth pages, single-entity configuration screens.
+
+```jsx
+<div className="page-wrapper page-wrapper--centered">
+  {/* every direct child is capped at --content-max-width-sm (640px) and centered */}
+</div>
+```
+
+Default max-width is `--content-max-width-sm` (640px). Override per-page only when necessary:
+
+```jsx
+<div
+  className="page-wrapper page-wrapper--centered"
+  style={{ '--page-content-max-width': 'var(--content-max-width-md)' }}
+>
+```
+
+Available width tokens:
+
+| Token                       | Value  | Use                                     |
+|-----------------------------|--------|-----------------------------------------|
+| `--content-max-width-xs`    | 480px  | Tiny forms, login, auth                 |
+| `--content-max-width-sm`    | 640px  | Small forms, simple settings (default)  |
+| `--content-max-width-md`    | 800px  | Medium forms, detail-light pages        |
+| `--content-max-width-lg`    | 1024px | Moderate-width constrained pages        |
+
+**Rules:**
+- Never use `page-wrapper--centered` on a list page, data table page, or any page where content should grow with the viewport
+- Never hardcode a pixel `max-width` in a page file — always use a token
+
+---
+
+## 3. Color Tokens
+
+All colors are CSS custom properties defined in `frontend/src/styles/tokens.css`.
+The system is **dark-first**: `:root` = dark theme. `[data-theme="light"]` overrides exist as a placeholder.
+
+### Rule: never write a raw color value in any component or page file. Always `var(--token)`.
+
+### Background Surfaces (7-step tonal ladder)
+
+| Token                  | Value        | Use                                                    |
+|------------------------|--------------|--------------------------------------------------------|
+| `--color-bg-abyss`     | `#0a0e14`    | Deepest well: code blocks, input backgrounds           |
+| `--color-bg-base`      | `#10141a`    | Page background (viewport fill)                        |
+| `--color-bg-void`      | `#181c22`    | Sidebar, header                                        |
+| `--color-bg-surface`   | `#1c2026`    | Default card / panel background                        |
+| `--color-bg-elevated`  | `#262a31`    | Raised cards, hovered rows, dropdowns                  |
+| `--color-bg-island`    | `#31353c`    | Active states, selected rows, pressed buttons          |
+| `--color-bg-float`     | `rgba(53,57,64,0.80)` | Glassmorphism: modals, floating panels      |
+
+### Brand / Primary (Indigo Glow)
+
+| Token                      | Value                        | Use                                  |
+|----------------------------|------------------------------|--------------------------------------|
+| `--color-primary`          | `#c0c1ff`                    | CTAs, active nav, key accent          |
+| `--color-primary-hover`    | `#d2bbff`                    | Hover, gradient endpoint              |
+| `--color-primary-container`| `#8083ff`                    | Container fills                       |
+| `--color-primary-subtle`   | `rgba(128,131,255,0.12)`     | Hover backgrounds, tinted areas       |
+| `--gradient-primary`       | `linear-gradient(135deg, #c0c1ff, #d2bbff)` | Primary button fill   |
+
+### Semantic / State Colors
+
+| Token                  | Value                     | Use                                            |
+|------------------------|---------------------------|------------------------------------------------|
+| `--color-success`      | `#4ade80`                 | Online, active, confirmed                      |
+| `--color-success-bg`   | `rgba(74,222,128,0.12)`   | Success badge / button resting background      |
+| `--color-warning`      | `#fbbf24`                 | Pending, needs attention                       |
+| `--color-warning-bg`   | `rgba(251,191,36,0.12)`   | Warning badge / button resting background      |
+| `--color-danger`       | `#ff5c5c`                 | Error text, destructive actions                |
+| `--color-danger-bg`    | `rgba(255,92,92,0.12)`    | Danger badge / button resting background       |
+| `--color-info`         | `#7bd0ff`                 | Informational, aqua-sky accent                 |
+| `--color-info-bg`      | `rgba(123,208,255,0.12)`  | Info badge background                          |
+
+### Text Colors (4-step hierarchy)
+
+| Token                     | Value       | Use                                               |
+|---------------------------|-------------|---------------------------------------------------|
+| `--color-text-primary`    | `#dfe2eb`   | Body copy, data values, headings                  |
+| `--color-text-secondary`  | `#c7c4d7`   | Labels, metadata, inactive nav                    |
+| `--color-text-muted`      | `#908fa0`   | Placeholders, disabled, category headers          |
+| `--color-text-inverse`    | `#10141a`   | Text on primary/accent backgrounds (dark on light)|
+| `--color-text-accent`     | `#c0c1ff`   | Active nav items, links                           |
+
+### Borders
+
+| Token                   | Value                      | Use                                        |
+|-------------------------|----------------------------|--------------------------------------------|
+| `--color-border`        | `rgba(70,69,84,0.20)`      | Resting inputs, card outlines              |
+| `--color-border-strong` | `rgba(70,69,84,0.45)`      | Secondary buttons, stronger dividers       |
+| `--color-border-focus`  | `rgba(192,193,255,0.40)`   | Focus ring halo on inputs                  |
+
+---
+
+## 4. Typography
+
+Two-font system. Three families total.
+
+### Font Families
+
+| Token                    | Font                | Role                                              |
+|--------------------------|---------------------|---------------------------------------------------|
+| `--font-family-display`  | `Barlow Condensed`  | H1, H2, page titles, modal titles                 |
+| `--font-family-base`     | `Onest`             | All UI text, body, labels, buttons, table rows    |
+| `--font-family-mono`     | `JetBrains Mono`    | Serial numbers, IDs, code, API keys, terminal     |
+
+**Why this pairing:**
+- `Barlow Condensed` has an industrial/engineering quality — feels like instrument panel labelling. Makes page titles immediately distinctive.
+- `Onest` is a Ukrainian geometric grotesque with slightly unusual proportions and excellent numerics. Clean at 14px. Not the overused Inter/Space Grotesk.
+- `JetBrains Mono` is the standard for developer-facing data.
+
+### Font Sizes
+
+| Token              | Value      | Use                                          |
+|--------------------|------------|----------------------------------------------|
+| `--font-size-xs`   | `0.6875rem` (11px) | Labels, sidebar category headers, chips |
+| `--font-size-sm`   | `0.75rem` (12px)   | Captions, helper text, table headers    |
+| `--font-size-base` | `0.875rem` (14px)  | Body text, table rows (default)         |
+| `--font-size-md`   | `1rem` (16px)      | Card titles, module headers             |
+| `--font-size-lg`   | `1.125rem` (18px)  | Section subheadings                     |
+| `--font-size-xl`   | `1.5rem` (24px)    | Page headings (h1/h2)                   |
+| `--font-size-2xl`  | `3.5rem` (56px)    | Hero KPI numbers, dashboard metrics     |
+
+### Font Weights
+
+| Token                     | Value | Use                                    |
+|---------------------------|-------|----------------------------------------|
+| `--font-weight-normal`    | 400   | Body copy                              |
+| `--font-weight-medium`    | 500   | Emphasized body, table values          |
+| `--font-weight-semibold`  | 600   | Headings, button labels, field labels  |
+| `--font-weight-bold`      | 700   | Strong emphasis, hero metrics          |
+
+### Typography Usage Rules
+
+- **Page titles (`<PageHeader>`):** `Barlow Condensed`, `1.75rem`, weight 600 (handled by `.v2-page-header-title`)
+- **Modal titles:** `Barlow Condensed`, `1.125rem`, weight 600 (handled by `.v2-modal-title`)
+- **H1, H2 globally:** `Barlow Condensed`, `var(--font-size-xl)`, weight 600 — set in `global.css`
+- **H3–H6:** `Onest` (body font), normal heading weights
+- **Card titles:** `Onest`, `--font-size-base`, weight 600
+- **Table headers:** `Onest`, `--font-size-sm`, weight 600, uppercase, `--tracking-wide`
+- **Body / cell text:** `Onest`, `--font-size-base`, weight 400
+- **Muted / helper text:** `Onest`, `--font-size-sm`, `--color-text-muted`
+- **Serials, IDs, codes:** `JetBrains Mono`, `--font-size-sm`
+
+### Letter Spacing
+
+| Token               | Value     | Use                                        |
+|---------------------|-----------|--------------------------------------------|
+| `--tracking-normal` | `0em`     | Default                                    |
+| `--tracking-tight`  | `-0.01em` | Barlow Condensed headings                  |
+| `--tracking-wide`   | `0.08em`  | Uppercase labels, sidebar category headers |
+| `--tracking-display`| `-0.02em` | Hero KPI numbers at 56px                   |
+
+---
+
+## 4b. Date, Time & Currency Formatting
+
+All dates use **Greek/European style** (day-first). Never use US-style MM/DD/YYYY anywhere in the app.
+
+All formatting is centralized in `frontend/src/lib/formatters.js`. Never use raw `toLocaleDateString()`, `Intl.DateTimeFormat`, `toLocaleString()`, or `toISOString().slice()` in pages or modals — always import from `@/lib/formatters`.
+
+### Available formatters
+
+| Function            | Output example                     | Use for                              |
+|---------------------|------------------------------------|--------------------------------------|
+| `fmtDate`           | `05/03/2026`                       | Short numeric dates (tables, lists)  |
+| `fmtDateMedium`     | `5 Mar 2026`                       | Medium dates (cards, details)        |
+| `fmtDateLong`       | `5 March 2026`                     | Long dates (headings, summaries)     |
+| `fmtDateFull`       | `Wednesday, 5 March 2026`          | Dashboard, full context              |
+| `fmtDateTime`       | `5 March 2026, 2:30 pm`            | Date + 12h time                      |
+| `fmtDateTimeMedium` | `5 Mar 2026, 14:30`                | Date + 24h time (compact)            |
+| `fmtDateTimeFull`   | `Wed, 5 Mar 2026, 2:30 pm`         | Emails, comms                        |
+| `fmtRelative`       | `5 minutes ago`                    | Relative timestamps                  |
+| `fmtEuro`           | `1.250,00 €`                       | Euro currency (Greek locale)         |
+
+### Form input helpers
+
+| Function            | Output example          | Use for                                          |
+|---------------------|-------------------------|--------------------------------------------------|
+| `toDatetimeLocal`   | `2026-03-05T14:30`      | Populating `datetime-local` inputs (local time)  |
+| `nowLocal`          | `2026-03-05T14:30`      | Current time for form defaults                   |
+| `toDateInput`       | `2026-03-05`            | Populating `date` inputs                         |
+
+### Critical rule: no `toISOString().slice()` for form inputs
+
+`toISOString()` converts to **UTC**, which shifts the time by the user's timezone offset (e.g. 3 hours for Greece). Always use `toDatetimeLocal()` or `nowLocal()` instead.
+
+---
+
+## 5. Spacing System
+
+4px base unit. All spacing must use tokens — no arbitrary pixel values.
+
+| Token        | Value  | Common use                                       |
+|--------------|--------|--------------------------------------------------|
+| `--space-1`  | 4px    | Tight gaps, icon padding                         |
+| `--space-2`  | 8px    | Between label and input, inline gaps             |
+| `--space-3`  | 12px   | Table cell padding, compact button padding       |
+| `--space-4`  | 16px   | Between form fields, mobile page padding         |
+| `--space-5`  | 20px   | Tab item spacing                                 |
+| `--space-6`  | 24px   | **Page padding**, card padding, section gap      |
+| `--space-8`  | 32px   | Between major sections                           |
+| `--space-10` | 40px   | Large section gap                                |
+| `--space-12` | 48px   | Extra large spacing                              |
+| `--space-16` | 64px   | Maximum spacing, hero sections                   |
+
+---
+
+## 6. Border Radius & Shadows
+
+### Border Radius
+
+| Token          | Value    | Use                                       |
+|----------------|----------|-------------------------------------------|
+| `--radius-sm`  | 4px      | Tags, small chips, select option rows     |
+| `--radius-md`  | 6px      | Buttons, inputs, table badges             |
+| `--radius-lg`  | 8px      | Cards, panels, dropdown menus             |
+| `--radius-xl`  | 12px     | Modals, large containers                  |
+| `--radius-full`| 9999px   | Status badge pills, avatars               |
+
+### Shadows
+
+| Token                   | Value                                    | Use                                |
+|-------------------------|------------------------------------------|------------------------------------|
+| `--shadow-card`         | `inset 0 1px 0 rgba(192,193,255,0.05)`  | Card top-edge glass reflection     |
+| `--shadow-sm`           | `0 2px 8px rgba(10,14,20,0.40)`         | Subtle lift                        |
+| `--shadow-md`           | `0 4px 16px rgba(10,14,20,0.50)`        | Elevated cards                     |
+| `--shadow-lg`           | `0 8px 24px rgba(13,17,23,0.60)`        | Modals, dropdowns                  |
+| `--shadow-focus`        | `0 0 0 3px rgba(192,193,255,0.20)`      | Focus ring glow                    |
+| `--shadow-primary-glow` | `0 4px 16px rgba(192,193,255,0.28)`     | Primary button hover halo          |
+| `--shadow-danger-glow`  | `0 4px 16px rgba(255,92,92,0.40)`       | Danger button hover halo           |
+| `--shadow-success-glow` | `0 4px 16px rgba(74,222,128,0.35)`      | Success button hover halo          |
+
+---
+
+## 7. Component Rules
+
+### Button
+
+Import: `@/components/ui/Button`
+
+**Variants:**
+
+| Variant          | Resting state                         | Hover state                                              |
+|------------------|---------------------------------------|----------------------------------------------------------|
+| `primary`        | Indigo→violet gradient, dark text     | `+brightness(1.06)` + `--shadow-primary-glow` halo      |
+| `secondary`      | Island bg, ghost border               | Elevated bg + focus border + subtle indigo glow          |
+| `ghost`          | Transparent                           | Elevated bg + whisper indigo glow                        |
+| `danger`         | Coral tint bg, coral text             | **Solid coral fill**, dark text + `--shadow-danger-glow` |
+| `success`        | Emerald tint bg, emerald text         | **Solid emerald fill**, dark text + `--shadow-success-glow` |
+| `table-actions`  | Fully transparent, muted text         | Island bg + strong border (identical to `secondary`) — also activates on `tr:hover` |
+
+**Sizes:** `sm`, `md` (default), `lg`
+
+**Rules:**
+- Never use a raw `<button>` element for a styled action
+- Always pass `loading` prop for async actions (shows spinner, disables interaction)
+- Icon-only buttons must have `aria-label`
+- Active/press state: `filter: brightness(0.94)`, glow removed
+
+---
+
+### FormField
+
+Import: `@/components/ui/FormField`
+
+Wraps every form control: label + input/textarea/select + hint + error message.
+Never place a raw `<input>` on a page.
+
+**Types:** `text`, `email`, `password`, `number`, `tel`, `url`, `textarea`, `select`
+
+**For `type="select"`**: pass `<option>` elements as children. FormField uses the custom `Select` component internally — the native `<select>` is never rendered.
+
+```jsx
+<FormField label="Status" name="status" type="select" value={val} onChange={handleChange}>
+  <option value="">Choose…</option>
+  <option value="active">Active</option>
+  <option value="inactive">Inactive</option>
+</FormField>
+```
+
+**Input appearance:** "cutout" inset-shadow treatment — the field appears recessed into the surface. Background: `--color-bg-abyss`. Focus: `--color-border-focus` ring.
+
+---
+
+### Select (standalone)
+
+Import: `@/components/ui/Select`
+
+Fully custom dropdown replacing native `<select>`. Floating menu via portal, keyboard navigation, checkmark on selected item. Usually consumed via `FormField type="select"`. Use directly when you need a select outside a form label context.
+
+---
+
+### DataTable
+
+Import: `@/components/ui/DataTable`
+
+**Always include:** column headers, loading skeleton, empty state, pagination.
+**Rows:** alternate tint via `--color-tint-row` (`rgba(192,193,255,0.015)`). Hover: `--color-bg-island`.
+**Status columns:** always `<StatusBadge>` — never raw text.
+**Row actions:** last column, right-aligned, use portal-based action menu.
+
+---
+
+### Modal
+
+Import: `@/components/ui/Modal`
+
+Sizes: `sm` (480px), `md` (640px — default), `lg` (800px), `xl` (60vw/60vh), `xxl` (85vw/85vh), `full` (calc(100vw/100vh − 64px)).
+
+**Rules:**
+- Always has: title, close (×) button, footer action buttons
+- Closes on Escape + backdrop click unless `persistent={true}`
+- Destructive prompts use `<ConfirmDialog>` instead
+- Modal JSX never lives inside a page file — always in `frontend/src/modals/[domain]/`
+
+---
+
+### ConfirmDialog
+
+Import: `@/components/ui/ConfirmDialog`
+
+Wraps `<Modal size="sm">` with a centred icon + message. Use for any action that is destructive or hard to reverse.
+
+Variants: `danger` (coral circle + triangle icon), `primary` (indigo circle + info icon).
+
+---
+
+### PageHeader
+
+Import: `@/components/ui/PageHeader`
+
+**Always the first element inside `.page-wrapper`.** Creates the page title block.
+
+Props: `title` (required), `subtitle`, `breadcrumbs`, `children` (action buttons slot).
+
+The title renders as `<h1>` with class `.v2-page-header-title` — uses `Barlow Condensed` at `1.75rem` / weight 600.
+
+```jsx
+<PageHeader title="Device Inventory" subtitle="All registered Bell units">
+  <Button variant="primary">Add Device</Button>
+</PageHeader>
+```
+
+---
+
+### Card
+
+Import: `@/components/ui/Card`
+
+Variants: `flat` (default), `elevated`, `outlined`.
+
+Props: `title`, `subtitle`, `footer`, `padding` (bool, default true), `children`.
+
+Card header has a faint indigo gradient ceiling (`linear-gradient` from top).
+
+---
+
+### Tabs
+
+Import: `@/components/ui/Tabs`
+
+Variants: `line` (default — underline indicator), `pill` (filled background).
+
+Props: `tabs` (array of `{key, label, icon?, count?}`), `active`, `onChange`, `variant`.
+
+Line variant uses a sliding indicator measured with `useLayoutEffect`. Pill variant uses filled backgrounds.
+
+Spacing: line tabs have `gap: --space-5` between items, pill tabs `gap: --space-4`.
+
+---
+
+### Toast
+
+Import: `@/components/ui/Toast` → `{ ToastProvider, useToast }`
+
+Setup: wrap the app (or router) with `<ToastProvider>`. Then in any component:
+
+```jsx
+const toast = useToast()
+toast.success('Saved', 'Device updated successfully.')
+toast.danger('Error', 'Failed to connect.')
+toast.warning('Warning', 'Firmware is outdated.')
+toast.info('Info', 'Sync in progress.')
+```
+
+Toasts auto-dismiss after 4000ms. Hover pauses the timer. Stack appears in the bottom-right corner.
+
+---
+
+### SearchBar
+
+Import: `@/components/ui/SearchBar`
+
+Supports controlled (`value` + `onChange`) or uncontrolled mode.
+Debounced by default (300ms). Clear button appears when text is present.
+Appearance matches the `FormField` cutout treatment.
+
+---
+
+### Breadcrumbs
+
+Import: `@/components/ui/Breadcrumbs`
+
+Use on detail pages only (not list pages). Items: array of `{ label, href? }`. Last item has no href — it is the current page.
+
+---
+
+### Spinner
+
+Import: `@/components/ui/Spinner`
+
+Props: `size` (`sm`, `md`, `lg`), `color` (defaults to `--color-primary`).
+
+Use inside loading states. Buttons show their own spinner via `loading` prop — do not add a separate `<Spinner>` inside buttons.
+
+---
+
+### StatusBadge
+
+Import: `@/components/ui/StatusBadge`
+
+Never use a raw `<span>` with a background color for status. Always `<StatusBadge>`.
+
+Variants: `success`, `warning`, `danger`, `info`, `neutral`.
+
+---
+
+### Icon
+
+Import: `@/components/ui/Icon`
+
+Renders an inline SVG by name. 35 named icons available (see Style Guide `/dev/styleguide` → Icon section for the full list).
+
+```jsx
+<Icon name="edit" size={16} />
+<Icon name="delete" size={20} color="var(--color-danger)" />
+```
+
+**Asset SVGs** (from `/assets/` folders) are displayed via `<img>` tags in the Style Guide, not via `<Icon>`. These are pre-rendered SVG files used for sidebar icons, comms icons, customer status icons, etc. Use them as image sources, not as Icon component names.
+
+---
+
+## 8. Icons
+
+Three sources:
+
+| Source                              | Use case                              | How to render               |
+|-------------------------------------|---------------------------------------|-----------------------------|
+| `<Icon name="..." />`               | Action icons, UI chrome               | `@/components/ui/Icon`      |
+| `assets/side-menu-icons/*.svg`      | Sidebar navigation                    | `<img src={...} />`         |
+| `assets/comms/*.svg`                | Communication type indicators         | `<img src={...} />`         |
+| `assets/customer-status/*.svg`      | CRM status icons                      | `<img src={...} />`         |
+| `assets/global-icons/*.svg`         | Legacy action icons (prefer `<Icon>`) | `<img src={...} />`         |
+| `assets/other-icons/*.svg`          | Misc UI icons                         | `<img src={...} />`         |
+
+Never add a new icon library (e.g. heroicons, lucide). Use the existing sources.
+
+---
+
+## 9. Theming Rules
+
+- Theme is controlled by `data-theme` attribute on `<html>`
+- Default is dark (`:root` = dark theme)
+- **Never** use Tailwind's `dark:` prefix — theming is handled entirely via CSS tokens
+- `[data-theme="light"]` overrides exist in `tokens.css` as a future placeholder
+
+---
+
+## 10. Responsive Breakpoints
+
+| Token              | Value  | Behaviour                                              |
+|--------------------|--------|--------------------------------------------------------|
+| `--breakpoint-sm`  | 640px  |                                                        |
+| `--breakpoint-md`  | 768px  | Sidebar collapses; page padding drops to `--space-4`  |
+| `--breakpoint-lg`  | 1024px | Full sidebar shown                                     |
+| `--breakpoint-xl`  | 1280px |                                                        |
+
+Mobile (`< 768px`): single column, sidebar hidden (drawer), tables may become card lists.
+
+---
+
+## 11. Section Layout — Masonry Grid
+
+**Default layout for ALL content pages with multiple variable-height sections.**
+
+Sections on a content page must flow like physical objects stacked in columns — the next section always drops into the shortest column. This is CSS column masonry.
+
+### How it works
+
+```
+Column 1    | Column 2    | Column 3
+────────────┼─────────────┼────────────
+Section A   | Section B   | Section C
+(300px)     | (250px)     | (350px)
+            │             │
+Section E   | Section D   |
+(200px)     | (200px)     |
+```
+
+Sections fill left-to-right across the top, then each new section drops into whichever column is currently shortest. This is automatic — the browser handles placement via CSS `columns`.
+
+### Usage
+
+```jsx
+{/* 2 columns */}
+<div className="masonry-grid masonry-grid--2">
+  <Card title="Account Info">…</Card>
+  <Card title="Profile">…</Card>
+  <Card title="Security">…</Card>   {/* auto-drops into shortest column */}
+</div>
+
+{/* 3 columns */}
+<div className="masonry-grid masonry-grid--3">
+  {sections.map(s => <Card key={s.id}>…</Card>)}
+</div>
+```
+
+Available variants: `masonry-grid--2`, `masonry-grid--3`, `masonry-grid--4`
+
+Responsive behaviour:
+- `--4` collapses to 3 cols at 1024px, 1 col at 768px
+- `--3` collapses to 2 cols at 1024px, 1 col at 768px
+- `--2` collapses to 1 col at 768px
+
+### Rules
+
+- **Use `.masonry-grid` by default** on all content pages with 2+ variable-height sections
+- **Do NOT** use `display: grid` with `gridTemplateColumns` for variable-height card layouts — this creates uneven whitespace when cards differ in height
+- **Do NOT** use `.masonry-grid` for DataTable pages — tables span full width on their own
+- **Do NOT** use `.masonry-grid` when sections must align horizontally (e.g. two fields that are semantically paired side-by-side within a card) — that's an internal card layout, not page-level masonry
+- The `Card` component already has `break-inside: avoid` so it will never be split across columns
+
+---
+
+## 12. What Claude Code Must NEVER Do
+
+- ❌ Write a hex color, `rgb()`, or `hsl()` value directly in any component or page file
+- ❌ Write a pixel spacing or size value that isn't a `--space-*` token
+- ❌ Use a raw `<button>`, `<input>`, or `<select>` for anything styled — always use the wrapper component
+- ❌ Create a `.module.css` or any per-page CSS file
+- ❌ Use Tailwind's `dark:` prefix — theming is via CSS tokens only
+- ❌ Place modal JSX inside a page file — modals live in `frontend/src/modals/`
+- ❌ Wrap `.page-wrapper` in additional divs that shift content alignment
+- ❌ Override `.page-wrapper`'s padding to make a single page "different"
+- ❌ Skip loading, error, and empty states on any data-fetching component
+- ❌ Import from `_archive/` anywhere except `@/lib/api.js`, `@/hooks/useAuth.js`, and `@/providers/AuthProvider.jsx`
+- ❌ Install a new icon library or introduce new SVG icons outside of `assets/`
+- ❌ Invent new color values not in `tokens.css`

backend/alembic.ini

@@ -0,0 +1,113 @@
+# A generic, single database configuration.
+
+[alembic]
+# path to migration scripts
+script_location = alembic
+
+# template used to generate migration file names; The default value is %%(rev)s_%%(slug)s
+# Uncomment the line below if you want the files to be prepended with date and time
+# file_template = %%(year)d_%%(month).2d_%%(day).2d_%%(hour).2d%%(minute).2d-%%(rev)s_%%(slug)s
+
+# sys.path path, will be prepended to sys.path if present.
+# defaults to the current working directory.
+prepend_sys_path = .
+
+# timezone to use when rendering the date within the migration file
+# as well as the filename.
+# If specified, requires the python>=3.9 or backports.zoneinfo library.
+# Any required deps can installed by adding `tzdata` to the `[alembic]` section
+# of pyproject.toml.
+# timezone =
+
+# max length of characters to apply to the "slug" field
+# truncate_slug_length = 40
+
+# set to 'true' to run the environment during
+# the 'revision' command, regardless of autogenerate
+# revision_environment = false
+
+# set to 'true' to allow .pyc and .pyo files without
+# a source .py file to be detected as revisions in the
+# versions/ directory
+# sourceless = false
+
+# version location specification; This defaults
+# to alembic/versions.  When using multiple version
+# directories, initial revisions must be specified with --version-path.
+# The path separator used here should be the separator specified by "version_path_separator" below.
+# version_locations = %(here)s/bar:%(here)s/bat:alembic/versions
+
+# version path separator; As mentioned above, this is the character used to split
+# version_locations. The default within new alembic.ini files is "os", which uses os.pathsep.
+# If this key is omitted entirely, it falls back to the legacy behavior of splitting on spaces and/or commas.
+# Valid values for version_path_separator are:
+#
+# version_path_separator = :
+# version_path_separator = ;
+# version_path_separator = space
+version_path_separator = os  # Use os.pathsep. Default configuration used for new projects.
+
+# set to 'true' to search source files recursively
+# in "version_locations" directory
+# New in Alembic version 1.10
+# recursive_version_locations = false
+
+# the output encoding used when revision files
+# are written from script.py.mako
+# output_encoding = utf-8
+
+# NOTE: The database URL is set programmatically in env.py from settings.
+# Do not set sqlalchemy.url here.
+
+
+[post_write_hooks]
+# post_write_hooks defines scripts or Python functions that are run
+# on newly generated revision scripts.  See the documentation for further
+# detail and examples
+
+# format using "black" - use the console_scripts runner, against the "black" entrypoint
+# hooks = black
+# black.type = console_scripts
+# black.entrypoint = black
+# black.options = -l 79 REVISION_SCRIPT_FILENAME
+
+# lint with attempts to fix using "ruff" - use the exec runner, execute a binary
+# hooks = ruff
+# ruff.type = exec
+# ruff.executable = %(here)s/.venv/bin/ruff
+# ruff.options = --fix REVISION_SCRIPT_FILENAME
+
+# Logging configuration
+[loggers]
+keys = root,sqlalchemy,alembic
+
+[handlers]
+keys = console
+
+[formatters]
+keys = generic
+
+[logger_root]
+level = WARN
+handlers = console
+qualname =
+
+[logger_sqlalchemy]
+level = WARN
+handlers =
+qualname = sqlalchemy.engine
+
+[logger_alembic]
+level = INFO
+handlers =
+qualname = alembic
+
+[handler_console]
+class = StreamHandler
+args = (sys.stderr,)
+level = NOTSET
+formatter = generic
+
+[formatter_generic]
+format = %(levelname)-5.5s [%(name)s] %(message)s
+datefmt = %H:%M:%S

backend/alembic/env.py

@@ -0,0 +1,44 @@
+import asyncio
+from logging.config import fileConfig
+from sqlalchemy.ext.asyncio import create_async_engine
+from alembic import context
+from config import settings
+
+# Import all models so Alembic can see them
+from database.models import Base  # noqa: F401 — triggers all ORM imports
+
+config = context.config
+if config.config_file_name is not None:
+    fileConfig(config.config_file_name)
+
+target_metadata = Base.metadata
+
+
+def run_migrations_offline() -> None:
+    url = settings.database_url
+    context.configure(url=url, target_metadata=target_metadata, literal_binds=True)
+    with context.begin_transaction():
+        context.run_migrations()
+
+
+def do_run_migrations(connection):
+    context.configure(connection=connection, target_metadata=target_metadata)
+    with context.begin_transaction():
+        context.run_migrations()
+
+
+async def run_async_migrations() -> None:
+    engine = create_async_engine(settings.database_url)
+    async with engine.begin() as conn:
+        await conn.run_sync(do_run_migrations)
+    await engine.dispose()
+
+
+def run_migrations_online() -> None:
+    asyncio.run(run_async_migrations())
+
+
+if context.is_offline_mode():
+    run_migrations_offline()
+else:
+    run_migrations_online()

backend/alembic/script.py.mako

@@ -0,0 +1,26 @@
+"""${message}
+
+Revision ID: ${up_revision}
+Revises: ${down_revision | comma,n}
+Create Date: ${create_date}
+
+"""
+from typing import Sequence, Union
+
+from alembic import op
+import sqlalchemy as sa
+${imports if imports else ""}
+
+# revision identifiers, used by Alembic.
+revision: str = ${repr(up_revision)}
+down_revision: Union[str, None] = ${repr(down_revision)}
+branch_labels: Union[str, Sequence[str], None] = ${repr(branch_labels)}
+depends_on: Union[str, Sequence[str], None] = ${repr(depends_on)}
+
+
+def upgrade() -> None:
+    ${upgrades if upgrades else "pass"}
+
+
+def downgrade() -> None:
+    ${downgrades if downgrades else "pass"}

backend/alembic/versions/244a0b0f35be_rename_entries_to_crm_entries.py

@@ -0,0 +1,83 @@
+"""rename_entries_to_crm_entries
+
+Revision ID: 244a0b0f35be
+Revises: 485d40e86e4b
+Create Date: 2026-04-15 20:05:20.835281
+
+"""
+from typing import Sequence, Union
+
+from alembic import op
+import sqlalchemy as sa
+from sqlalchemy.dialects import postgresql
+
+# revision identifiers, used by Alembic.
+revision: str = '244a0b0f35be'
+down_revision: Union[str, None] = '485d40e86e4b'
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    # 1. Drop FK from support_tickets -> entries before touching the entries table
+    op.drop_constraint('support_tickets_linked_entry_id_fkey', 'support_tickets', type_='foreignkey')
+    # 2. Drop dependent table first, then parent
+    op.drop_table('entry_links')
+    op.drop_table('entries')
+    # 3. Create new tables with crm_ prefix
+    op.create_table('crm_entries',
+    sa.Column('id', sa.UUID(), nullable=False),
+    sa.Column('type', sa.String(length=10), nullable=False),
+    sa.Column('title', sa.String(length=500), nullable=False),
+    sa.Column('body', sa.Text(), nullable=True),
+    sa.Column('status', sa.String(length=20), nullable=True),
+    sa.Column('severity', sa.String(length=10), nullable=True),
+    sa.Column('author_id', sa.String(length=128), nullable=False),
+    sa.Column('author_name', sa.String(length=255), nullable=True),
+    sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
+    sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
+    sa.PrimaryKeyConstraint('id')
+    )
+    op.create_table('crm_entry_links',
+    sa.Column('id', sa.UUID(), nullable=False),
+    sa.Column('entry_id', sa.UUID(), nullable=False),
+    sa.Column('entity_type', sa.String(length=20), nullable=False),
+    sa.Column('entity_id', sa.String(length=128), nullable=False),
+    sa.ForeignKeyConstraint(['entry_id'], ['crm_entries.id'], ondelete='CASCADE'),
+    sa.PrimaryKeyConstraint('id'),
+    sa.UniqueConstraint('entry_id', 'entity_type', 'entity_id')
+    )
+    # 4. Recreate FK on support_tickets pointing at new table
+    op.create_foreign_key(None, 'support_tickets', 'crm_entries', ['linked_entry_id'], ['id'], ondelete='SET NULL')
+
+
+def downgrade() -> None:
+    # ### commands auto generated by Alembic - please adjust! ###
+    op.drop_constraint(None, 'support_tickets', type_='foreignkey')
+    op.create_foreign_key('support_tickets_linked_entry_id_fkey', 'support_tickets', 'entries', ['linked_entry_id'], ['id'], ondelete='SET NULL')
+    op.create_table('entries',
+    sa.Column('id', sa.UUID(), autoincrement=False, nullable=False),
+    sa.Column('type', sa.VARCHAR(length=10), autoincrement=False, nullable=False),
+    sa.Column('title', sa.VARCHAR(length=500), autoincrement=False, nullable=False),
+    sa.Column('body', sa.TEXT(), autoincrement=False, nullable=True),
+    sa.Column('status', sa.VARCHAR(length=20), autoincrement=False, nullable=True),
+    sa.Column('severity', sa.VARCHAR(length=10), autoincrement=False, nullable=True),
+    sa.Column('author_id', sa.VARCHAR(length=128), autoincrement=False, nullable=False),
+    sa.Column('author_name', sa.VARCHAR(length=255), autoincrement=False, nullable=True),
+    sa.Column('created_at', postgresql.TIMESTAMP(timezone=True), autoincrement=False, nullable=False),
+    sa.Column('updated_at', postgresql.TIMESTAMP(timezone=True), autoincrement=False, nullable=False),
+    sa.PrimaryKeyConstraint('id', name='entries_pkey'),
+    postgresql_ignore_search_path=False
+    )
+    op.create_table('entry_links',
+    sa.Column('id', sa.UUID(), autoincrement=False, nullable=False),
+    sa.Column('entry_id', sa.UUID(), autoincrement=False, nullable=False),
+    sa.Column('entity_type', sa.VARCHAR(length=20), autoincrement=False, nullable=False),
+    sa.Column('entity_id', sa.VARCHAR(length=128), autoincrement=False, nullable=False),
+    sa.ForeignKeyConstraint(['entry_id'], ['entries.id'], name='entry_links_entry_id_fkey', ondelete='CASCADE'),
+    sa.PrimaryKeyConstraint('id', name='entry_links_pkey'),
+    sa.UniqueConstraint('entry_id', 'entity_type', 'entity_id', name='entry_links_entry_id_entity_type_entity_id_key')
+    )
+    op.drop_table('crm_entry_links')
+    op.drop_table('crm_entries')
+    # ### end Alembic commands ###

backend/alembic/versions/485d40e86e4b_initial_notes_and_tickets.py

@@ -0,0 +1,82 @@
+"""initial_notes_and_tickets
+
+Revision ID: 485d40e86e4b
+Revises: 
+Create Date: 2026-04-15 20:01:04.225959
+
+"""
+from typing import Sequence, Union
+
+from alembic import op
+import sqlalchemy as sa
+
+
+# revision identifiers, used by Alembic.
+revision: str = '485d40e86e4b'
+down_revision: Union[str, None] = None
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    # ### commands auto generated by Alembic - please adjust! ###
+    op.create_table('entries',
+    sa.Column('id', sa.UUID(), nullable=False),
+    sa.Column('type', sa.String(length=10), nullable=False),
+    sa.Column('title', sa.String(length=500), nullable=False),
+    sa.Column('body', sa.Text(), nullable=True),
+    sa.Column('status', sa.String(length=20), nullable=True),
+    sa.Column('severity', sa.String(length=10), nullable=True),
+    sa.Column('author_id', sa.String(length=128), nullable=False),
+    sa.Column('author_name', sa.String(length=255), nullable=True),
+    sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
+    sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
+    sa.PrimaryKeyConstraint('id')
+    )
+    op.create_table('entry_links',
+    sa.Column('id', sa.UUID(), nullable=False),
+    sa.Column('entry_id', sa.UUID(), nullable=False),
+    sa.Column('entity_type', sa.String(length=20), nullable=False),
+    sa.Column('entity_id', sa.String(length=128), nullable=False),
+    sa.ForeignKeyConstraint(['entry_id'], ['entries.id'], ondelete='CASCADE'),
+    sa.PrimaryKeyConstraint('id'),
+    sa.UniqueConstraint('entry_id', 'entity_type', 'entity_id')
+    )
+    op.create_table('support_tickets',
+    sa.Column('id', sa.UUID(), nullable=False),
+    sa.Column('customer_id', sa.String(length=128), nullable=False),
+    sa.Column('customer_name', sa.String(length=255), nullable=True),
+    sa.Column('device_id', sa.String(length=128), nullable=True),
+    sa.Column('device_serial', sa.String(length=64), nullable=True),
+    sa.Column('subject', sa.String(length=500), nullable=False),
+    sa.Column('status', sa.String(length=30), nullable=False),
+    sa.Column('priority', sa.String(length=10), nullable=True),
+    sa.Column('opened_via', sa.String(length=20), nullable=True),
+    sa.Column('linked_entry_id', sa.UUID(), nullable=True),
+    sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
+    sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
+    sa.ForeignKeyConstraint(['linked_entry_id'], ['entries.id'], ondelete='SET NULL'),
+    sa.PrimaryKeyConstraint('id')
+    )
+    op.create_table('ticket_messages',
+    sa.Column('id', sa.UUID(), nullable=False),
+    sa.Column('ticket_id', sa.UUID(), nullable=False),
+    sa.Column('sender_type', sa.String(length=10), nullable=False),
+    sa.Column('sender_id', sa.String(length=128), nullable=False),
+    sa.Column('sender_name', sa.String(length=255), nullable=True),
+    sa.Column('body', sa.Text(), nullable=False),
+    sa.Column('is_internal', sa.Boolean(), nullable=False),
+    sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
+    sa.ForeignKeyConstraint(['ticket_id'], ['support_tickets.id'], ondelete='CASCADE'),
+    sa.PrimaryKeyConstraint('id')
+    )
+    # ### end Alembic commands ###
+
+
+def downgrade() -> None:
+    # ### commands auto generated by Alembic - please adjust! ###
+    op.drop_table('ticket_messages')
+    op.drop_table('support_tickets')
+    op.drop_table('entry_links')
+    op.drop_table('entries')
+    # ### end Alembic commands ###

backend/alembic/versions/a1b2c3d4e5f6_add_category_to_crm_entries.py

@@ -0,0 +1,23 @@
+"""add_category_to_crm_entries
+
+Revision ID: a1b2c3d4e5f6
+Revises: 244a0b0f35be
+Create Date: 2026-04-16 09:00:00.000000
+
+"""
+from typing import Sequence, Union
+from alembic import op
+import sqlalchemy as sa
+
+revision: str = 'a1b2c3d4e5f6'
+down_revision: Union[str, None] = '244a0b0f35be'
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    op.add_column('crm_entries', sa.Column('category', sa.String(length=30), nullable=True))
+
+
+def downgrade() -> None:
+    op.drop_column('crm_entries', 'category')

backend/alembic/versions/b1c2d3e4f5a6_phase_0_schema_foundation.py

@@ -0,0 +1,507 @@
+"""phase_0_schema_foundation
+
+Adds all Phase 0 tables:
+  - _migration_runs        (migration tracking)
+  - audit_log              (staff action audit trail)
+  - crm_products
+  - crm_customers
+  - crm_orders
+  - crm_comms_log
+  - crm_media
+  - crm_sync_state
+  - crm_quotations
+  - crm_quotation_items
+  - staff
+  - console_settings
+  - public_features
+  - melody_drafts
+  - built_melodies
+  - mfg_audit_log
+  - device_alerts
+  - commands               (raw SQL — no ORM model)
+  - heartbeats             (raw SQL — no ORM model)
+  - device_logs            (partitioned by month — raw SQL)
+  - device_logs_2025_01 … device_logs_2026_06  (initial partitions)
+
+Revision ID: b1c2d3e4f5a6
+Revises: a1b2c3d4e5f6
+Create Date: 2026-04-17 00:00:00.000000
+"""
+from typing import Sequence, Union
+
+import sqlalchemy as sa
+from alembic import op
+from sqlalchemy.dialects.postgresql import ARRAY, JSONB
+
+# revision identifiers
+revision: str = "b1c2d3e4f5a6"
+down_revision: Union[str, None] = "a1b2c3d4e5f6"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    # ------------------------------------------------------------------ #
+    # _migration_runs
+    # ------------------------------------------------------------------ #
+    op.create_table(
+        "_migration_runs",
+        sa.Column("id",          sa.BigInteger(), primary_key=True, autoincrement=True),
+        sa.Column("script_name", sa.String(256), nullable=False),
+        sa.Column("ran_at",      sa.DateTime(timezone=True), nullable=False,
+                  server_default=sa.func.now()),
+        sa.Column("source_rows", sa.BigInteger(), nullable=False, server_default="0"),
+        sa.Column("dest_rows",   sa.BigInteger(), nullable=False, server_default="0"),
+        sa.Column("success",     sa.String(8),  nullable=False, server_default="ok"),
+        sa.Column("notes",       sa.Text(),     nullable=True),
+    )
+
+    # ------------------------------------------------------------------ #
+    # audit_log
+    # ------------------------------------------------------------------ #
+    op.create_table(
+        "audit_log",
+        sa.Column("id",           sa.BigInteger(), primary_key=True, autoincrement=True),
+        sa.Column("occurred_at",  sa.DateTime(timezone=True), nullable=False,
+                  server_default=sa.func.now()),
+        sa.Column("actor_id",     sa.String(128), nullable=False),
+        sa.Column("actor_name",   sa.String(255), nullable=False),
+        sa.Column("action",       sa.String(64),  nullable=False),
+        sa.Column("entity_type",  sa.String(64),  nullable=False),
+        sa.Column("entity_id",    sa.String(128), nullable=False),
+        sa.Column("entity_label", sa.String(500), nullable=True),
+        sa.Column("changes",      JSONB,           nullable=True),
+        sa.Column("meta",         JSONB,           nullable=True),
+    )
+    op.create_index("idx_audit_actor",    "audit_log", ["actor_id",   "occurred_at"])
+    op.create_index("idx_audit_entity",   "audit_log", ["entity_type","entity_id", "occurred_at"])
+    op.create_index("idx_audit_action",   "audit_log", ["action",     "occurred_at"])
+    op.create_index("idx_audit_occurred", "audit_log", ["occurred_at"])
+
+    # ------------------------------------------------------------------ #
+    # staff
+    # ------------------------------------------------------------------ #
+    op.create_table(
+        "staff",
+        sa.Column("id",              sa.String(128), primary_key=True),
+        sa.Column("firestore_id",    sa.String(128), nullable=True,  unique=True),
+        sa.Column("email",           sa.String(256), nullable=False, unique=True),
+        sa.Column("name",            sa.String(255), nullable=False),
+        sa.Column("role",            sa.String(64),  nullable=False, server_default="staff"),
+        sa.Column("permissions",     JSONB,           nullable=False, server_default="{}"),
+        sa.Column("hashed_password", sa.String(256), nullable=False),
+        sa.Column("is_active",       sa.Boolean(),   nullable=False, server_default="true"),
+        sa.Column("created_at",      sa.DateTime(timezone=True), nullable=False,
+                  server_default=sa.func.now()),
+        sa.Column("updated_at",      sa.DateTime(timezone=True), nullable=False,
+                  server_default=sa.func.now()),
+    )
+
+    # ------------------------------------------------------------------ #
+    # console_settings  &  public_features
+    # ------------------------------------------------------------------ #
+    op.create_table(
+        "console_settings",
+        sa.Column("key",        sa.String(128), primary_key=True),
+        sa.Column("value",      JSONB,           nullable=True),
+        sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False,
+                  server_default=sa.func.now()),
+    )
+    op.create_table(
+        "public_features",
+        sa.Column("key",        sa.String(128), primary_key=True),
+        sa.Column("value",      JSONB,           nullable=True),
+        sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False,
+                  server_default=sa.func.now()),
+    )
+
+    # ------------------------------------------------------------------ #
+    # crm_products
+    # ------------------------------------------------------------------ #
+    op.create_table(
+        "crm_products",
+        sa.Column("id",           sa.String(128), primary_key=True),
+        sa.Column("firestore_id", sa.String(128), nullable=True, unique=True),
+        sa.Column("name",         sa.String(500), nullable=False),
+        sa.Column("sku",          sa.String(128), nullable=True),
+        sa.Column("category",     sa.String(128), nullable=True),
+        sa.Column("description",  sa.Text(),      nullable=True),
+        sa.Column("unit_cost",    sa.Numeric(12, 2), nullable=False, server_default="0"),
+        sa.Column("currency",     sa.String(10),  nullable=False, server_default="EUR"),
+        sa.Column("unit_type",    sa.String(32),  nullable=False, server_default="pcs"),
+        sa.Column("is_active",    sa.Boolean(),   nullable=False, server_default="true"),
+        sa.Column("created_at",   sa.DateTime(timezone=True), nullable=False,
+                  server_default=sa.func.now()),
+        sa.Column("updated_at",   sa.DateTime(timezone=True), nullable=False,
+                  server_default=sa.func.now()),
+    )
+
+    # ------------------------------------------------------------------ #
+    # crm_customers
+    # ------------------------------------------------------------------ #
+    op.create_table(
+        "crm_customers",
+        sa.Column("id",                  sa.String(128), primary_key=True),
+        sa.Column("firestore_id",        sa.String(128), nullable=True, unique=True),
+        sa.Column("title",               sa.String(32),  nullable=True),
+        sa.Column("name",                sa.String(255), nullable=False),
+        sa.Column("surname",             sa.String(255), nullable=True),
+        sa.Column("organization",        sa.String(500), nullable=True),
+        sa.Column("religion",            sa.String(64),  nullable=True),
+        sa.Column("language",            sa.String(10),  nullable=False, server_default="el"),
+        sa.Column("folder_id",           sa.String(128), nullable=False, unique=True),
+        sa.Column("relationship_status", sa.String(64),  nullable=False, server_default="lead"),
+        sa.Column("nextcloud_folder",    sa.String(500), nullable=True),
+        sa.Column("contacts",            JSONB, nullable=False, server_default="[]"),
+        sa.Column("notes",               JSONB, nullable=False, server_default="[]"),
+        sa.Column("location",            JSONB, nullable=True),
+        sa.Column("tags",                ARRAY(sa.String()), nullable=False, server_default="{}"),
+        sa.Column("owned_items",         JSONB, nullable=False, server_default="[]"),
+        sa.Column("linked_user_ids",     ARRAY(sa.String()), nullable=False, server_default="{}"),
+        sa.Column("technical_issues",    JSONB, nullable=False, server_default="[]"),
+        sa.Column("install_support",     JSONB, nullable=False, server_default="[]"),
+        sa.Column("transaction_history", JSONB, nullable=False, server_default="[]"),
+        sa.Column("crm_summary",         JSONB, nullable=True),
+        sa.Column("created_at",          sa.DateTime(timezone=True), nullable=False),
+        sa.Column("updated_at",          sa.DateTime(timezone=True), nullable=False),
+    )
+    op.create_index("idx_crm_customers_rel_status", "crm_customers", ["relationship_status"])
+    op.create_index("idx_crm_customers_name",       "crm_customers", ["name", "surname"])
+    op.create_index("idx_crm_customers_tags",       "crm_customers", ["tags"],
+                    postgresql_using="gin")
+
+    # ------------------------------------------------------------------ #
+    # crm_orders
+    # ------------------------------------------------------------------ #
+    op.create_table(
+        "crm_orders",
+        sa.Column("id",                  sa.String(128), primary_key=True),
+        sa.Column("customer_id",         sa.String(128),
+                  sa.ForeignKey("crm_customers.id", ondelete="CASCADE"), nullable=False),
+        sa.Column("order_number",        sa.String(64),  nullable=False, unique=True),
+        sa.Column("title",               sa.String(500), nullable=True),
+        sa.Column("created_by",          sa.String(128), nullable=True),
+        sa.Column("status",              sa.String(64),  nullable=False,
+                  server_default="negotiating"),
+        sa.Column("status_updated_date", sa.DateTime(timezone=True), nullable=True),
+        sa.Column("status_updated_by",   sa.String(128), nullable=True),
+        sa.Column("items",               JSONB, nullable=False, server_default="[]"),
+        sa.Column("subtotal",            sa.Numeric(12, 2), nullable=False, server_default="0"),
+        sa.Column("discount",            JSONB, nullable=True),
+        sa.Column("total_price",         sa.Numeric(12, 2), nullable=False, server_default="0"),
+        sa.Column("currency",            sa.String(10), nullable=False, server_default="EUR"),
+        sa.Column("shipping",            JSONB, nullable=True),
+        sa.Column("payment_status",      JSONB, nullable=False, server_default="{}"),
+        sa.Column("invoice_path",        sa.String(500), nullable=True),
+        sa.Column("notes",               sa.Text(), nullable=True),
+        sa.Column("timeline",            JSONB, nullable=False, server_default="[]"),
+        sa.Column("created_at",          sa.DateTime(timezone=True), nullable=False),
+        sa.Column("updated_at",          sa.DateTime(timezone=True), nullable=False),
+    )
+    op.create_index("idx_crm_orders_customer", "crm_orders", ["customer_id"])
+    op.create_index("idx_crm_orders_status",   "crm_orders", ["status"])
+
+    # ------------------------------------------------------------------ #
+    # crm_comms_log
+    # ------------------------------------------------------------------ #
+    op.create_table(
+        "crm_comms_log",
+        sa.Column("id",             sa.String(128), primary_key=True),
+        sa.Column("customer_id",    sa.String(128),
+                  sa.ForeignKey("crm_customers.id", ondelete="SET NULL"), nullable=True),
+        sa.Column("type",           sa.String(32),  nullable=False),
+        sa.Column("mail_account",   sa.String(256), nullable=True),
+        sa.Column("direction",      sa.String(16),  nullable=False),
+        sa.Column("subject",        sa.String(500), nullable=True),
+        sa.Column("body",           sa.Text(),      nullable=True),
+        sa.Column("body_html",      sa.Text(),      nullable=True),
+        sa.Column("attachments",    JSONB, nullable=False, server_default="[]"),
+        sa.Column("ext_message_id", sa.String(500), nullable=True),
+        sa.Column("from_addr",      sa.String(500), nullable=True),
+        sa.Column("to_addrs",       sa.Text(),      nullable=True),
+        sa.Column("logged_by",      sa.String(128), nullable=True),
+        sa.Column("is_important",   sa.Boolean(), nullable=False, server_default="false"),
+        sa.Column("is_read",        sa.Boolean(), nullable=False, server_default="true"),
+        sa.Column("occurred_at",    sa.DateTime(timezone=True), nullable=False),
+        sa.Column("created_at",     sa.DateTime(timezone=True), nullable=False,
+                  server_default=sa.func.now()),
+    )
+    op.create_index("idx_crm_comms_customer", "crm_comms_log", ["customer_id", "occurred_at"])
+
+    # ------------------------------------------------------------------ #
+    # crm_media
+    # ------------------------------------------------------------------ #
+    op.create_table(
+        "crm_media",
+        sa.Column("id",             sa.String(128), primary_key=True),
+        sa.Column("customer_id",    sa.String(128),
+                  sa.ForeignKey("crm_customers.id", ondelete="SET NULL"), nullable=True),
+        sa.Column("order_id",       sa.String(128), nullable=True),
+        sa.Column("filename",       sa.String(500),  nullable=False),
+        sa.Column("nextcloud_path", sa.String(1000), nullable=False),
+        sa.Column("thumbnail_path", sa.String(1000), nullable=True),
+        sa.Column("mime_type",      sa.String(128),  nullable=True),
+        sa.Column("direction",      sa.String(16),   nullable=True),
+        sa.Column("tags",           JSONB, nullable=False, server_default="[]"),
+        sa.Column("uploaded_by",    sa.String(128),  nullable=True),
+        sa.Column("created_at",     sa.DateTime(timezone=True), nullable=False,
+                  server_default=sa.func.now()),
+    )
+    op.create_index("idx_crm_media_customer", "crm_media", ["customer_id"])
+    op.create_index("idx_crm_media_order",    "crm_media", ["order_id"])
+
+    # ------------------------------------------------------------------ #
+    # crm_sync_state
+    # ------------------------------------------------------------------ #
+    op.create_table(
+        "crm_sync_state",
+        sa.Column("key",   sa.String(128), primary_key=True),
+        sa.Column("value", sa.Text(),      nullable=True),
+    )
+
+    # ------------------------------------------------------------------ #
+    # crm_quotations
+    # ------------------------------------------------------------------ #
+    op.create_table(
+        "crm_quotations",
+        sa.Column("id",                       sa.String(128), primary_key=True),
+        sa.Column("quotation_number",         sa.String(64),  nullable=False, unique=True),
+        sa.Column("title",                    sa.String(500), nullable=True),
+        sa.Column("subtitle",                 sa.String(500), nullable=True),
+        sa.Column("customer_id",              sa.String(128),
+                  sa.ForeignKey("crm_customers.id", ondelete="CASCADE"), nullable=False),
+        sa.Column("language",                 sa.String(10),  nullable=False, server_default="en"),
+        sa.Column("status",                   sa.String(32),  nullable=False, server_default="draft"),
+        sa.Column("order_type",               sa.String(64),  nullable=True),
+        sa.Column("shipping_method",          sa.String(64),  nullable=True),
+        sa.Column("estimated_shipping_date",  sa.String(32),  nullable=True),
+        sa.Column("global_discount_label",    sa.String(128), nullable=True),
+        sa.Column("global_discount_percent",  sa.Numeric(8, 4), nullable=False, server_default="0"),
+        sa.Column("vat_percent",              sa.Numeric(8, 4), nullable=False, server_default="24"),
+        sa.Column("global_vat_percent",       sa.Numeric(8, 4), nullable=False, server_default="24"),
+        sa.Column("shipping_cost",            sa.Numeric(12, 2), nullable=False, server_default="0"),
+        sa.Column("shipping_cost_discount",   sa.Numeric(12, 2), nullable=False, server_default="0"),
+        sa.Column("install_cost",             sa.Numeric(12, 2), nullable=False, server_default="0"),
+        sa.Column("install_cost_discount",    sa.Numeric(12, 2), nullable=False, server_default="0"),
+        sa.Column("extras_label",             sa.String(256), nullable=True),
+        sa.Column("extras_cost",              sa.Numeric(12, 2), nullable=False, server_default="0"),
+        sa.Column("comments",                 JSONB, nullable=False, server_default="[]"),
+        sa.Column("quick_notes",              JSONB, nullable=False, server_default="{}"),
+        sa.Column("subtotal_before_discount", sa.Numeric(12, 2), nullable=False, server_default="0"),
+        sa.Column("global_discount_amount",   sa.Numeric(12, 2), nullable=False, server_default="0"),
+        sa.Column("new_subtotal",             sa.Numeric(12, 2), nullable=False, server_default="0"),
+        sa.Column("vat_amount",               sa.Numeric(12, 2), nullable=False, server_default="0"),
+        sa.Column("final_total",              sa.Numeric(12, 2), nullable=False, server_default="0"),
+        sa.Column("nextcloud_pdf_path",       sa.String(1000), nullable=True),
+        sa.Column("nextcloud_pdf_url",        sa.String(1000), nullable=True),
+        sa.Column("client_org",               sa.String(500), nullable=True),
+        sa.Column("client_name",              sa.String(500), nullable=True),
+        sa.Column("client_location",          sa.String(500), nullable=True),
+        sa.Column("client_phone",             sa.String(64),  nullable=True),
+        sa.Column("client_email",             sa.String(256), nullable=True),
+        sa.Column("is_legacy",                sa.Boolean(), nullable=False, server_default="false"),
+        sa.Column("legacy_date",              sa.String(32),   nullable=True),
+        sa.Column("legacy_pdf_path",          sa.String(1000), nullable=True),
+        sa.Column("created_at",               sa.DateTime(timezone=True), nullable=False),
+        sa.Column("updated_at",               sa.DateTime(timezone=True), nullable=False),
+    )
+    op.create_index("idx_crm_quotations_customer", "crm_quotations", ["customer_id"])
+
+    # ------------------------------------------------------------------ #
+    # crm_quotation_items
+    # ------------------------------------------------------------------ #
+    op.create_table(
+        "crm_quotation_items",
+        sa.Column("id",               sa.String(128), primary_key=True),
+        sa.Column("quotation_id",     sa.String(128),
+                  sa.ForeignKey("crm_quotations.id", ondelete="CASCADE"), nullable=False),
+        sa.Column("product_id",       sa.String(128),  nullable=True),
+        sa.Column("description",      sa.Text(),       nullable=True),
+        sa.Column("description_en",   sa.Text(),       nullable=True),
+        sa.Column("description_gr",   sa.Text(),       nullable=True),
+        sa.Column("unit_type",        sa.String(32),   nullable=False, server_default="pcs"),
+        sa.Column("unit_cost",        sa.Numeric(12, 4), nullable=False, server_default="0"),
+        sa.Column("discount_percent", sa.Numeric(8, 4),  nullable=False, server_default="0"),
+        sa.Column("vat_percent",      sa.Numeric(8, 4),  nullable=False, server_default="24"),
+        sa.Column("quantity",         sa.Numeric(12, 4), nullable=False, server_default="1"),
+        sa.Column("line_total",       sa.Numeric(12, 2), nullable=False, server_default="0"),
+        sa.Column("sort_order",       sa.Integer(),      nullable=False, server_default="0"),
+    )
+    op.create_index("idx_crm_quotation_items_quotation", "crm_quotation_items",
+                    ["quotation_id", "sort_order"])
+
+    # ------------------------------------------------------------------ #
+    # melody_drafts
+    # ------------------------------------------------------------------ #
+    op.create_table(
+        "melody_drafts",
+        sa.Column("id",         sa.String(128), primary_key=True),
+        sa.Column("status",     sa.String(32),  nullable=False, server_default="draft"),
+        sa.Column("data",       JSONB,           nullable=False),
+        sa.Column("created_at", sa.DateTime(timezone=True), nullable=False,
+                  server_default=sa.func.now()),
+        sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False,
+                  server_default=sa.func.now()),
+    )
+    op.create_index("idx_melody_drafts_status", "melody_drafts", ["status"])
+
+    # ------------------------------------------------------------------ #
+    # built_melodies
+    # ------------------------------------------------------------------ #
+    op.create_table(
+        "built_melodies",
+        sa.Column("id",                  sa.String(128), primary_key=True),
+        sa.Column("name",                sa.String(500), nullable=False),
+        sa.Column("pid",                 sa.String(128), nullable=False),
+        sa.Column("steps",               JSONB,           nullable=False),
+        sa.Column("binary_path",         sa.String(1000), nullable=True),
+        sa.Column("progmem_code",        sa.Text(),       nullable=True),
+        sa.Column("assigned_melody_ids", JSONB, nullable=False, server_default="[]"),
+        sa.Column("is_builtin",          sa.Boolean(), nullable=False, server_default="false"),
+        sa.Column("created_at",          sa.DateTime(timezone=True), nullable=False,
+                  server_default=sa.func.now()),
+        sa.Column("updated_at",          sa.DateTime(timezone=True), nullable=False,
+                  server_default=sa.func.now()),
+    )
+
+    # ------------------------------------------------------------------ #
+    # mfg_audit_log
+    # ------------------------------------------------------------------ #
+    op.create_table(
+        "mfg_audit_log",
+        sa.Column("id",            sa.BigInteger(), primary_key=True, autoincrement=True),
+        sa.Column("timestamp",     sa.DateTime(timezone=True), nullable=False,
+                  server_default=sa.func.now()),
+        sa.Column("admin_user",    sa.String(256), nullable=False),
+        sa.Column("action",        sa.String(128), nullable=False),
+        sa.Column("serial_number", sa.String(128), nullable=True),
+        sa.Column("detail",        sa.Text(),      nullable=True),
+    )
+    op.create_index("idx_mfg_audit_time",   "mfg_audit_log", ["timestamp"])
+    op.create_index("idx_mfg_audit_action", "mfg_audit_log", ["action"])
+
+    # ------------------------------------------------------------------ #
+    # device_alerts
+    # ------------------------------------------------------------------ #
+    op.create_table(
+        "device_alerts",
+        sa.Column("id",            sa.BigInteger(), primary_key=True, autoincrement=True),
+        sa.Column("device_serial", sa.String(128), nullable=False),
+        sa.Column("subsystem",     sa.String(128), nullable=False),
+        sa.Column("state",         sa.String(64),  nullable=False),
+        sa.Column("message",       sa.Text(),      nullable=True),
+        sa.Column("updated_at",    sa.DateTime(timezone=True), nullable=False,
+                  server_default=sa.func.now()),
+        sa.UniqueConstraint("device_serial", "subsystem", name="uq_device_alerts_serial_subsystem"),
+    )
+    op.create_index("idx_device_alerts_serial", "device_alerts", ["device_serial"])
+
+    # ------------------------------------------------------------------ #
+    # commands  (raw SQL — mirrors SQLite schema, no ORM model)
+    # ------------------------------------------------------------------ #
+    op.execute("""
+        CREATE TABLE commands (
+            id               BIGSERIAL PRIMARY KEY,
+            device_serial    TEXT NOT NULL,
+            command_name     TEXT NOT NULL,
+            command_payload  TEXT,
+            status           TEXT NOT NULL DEFAULT 'pending',
+            response_payload TEXT,
+            sent_at          TIMESTAMPTZ NOT NULL DEFAULT now(),
+            responded_at     TIMESTAMPTZ
+        )
+    """)
+    op.execute("CREATE INDEX idx_commands_serial_time ON commands(device_serial, sent_at DESC)")
+    op.execute("CREATE INDEX idx_commands_status ON commands(status)")
+
+    # ------------------------------------------------------------------ #
+    # heartbeats  (raw SQL — mirrors SQLite schema, no ORM model)
+    # ------------------------------------------------------------------ #
+    op.execute("""
+        CREATE TABLE heartbeats (
+            id               BIGSERIAL PRIMARY KEY,
+            device_serial    TEXT NOT NULL,
+            device_id        TEXT,
+            firmware_version TEXT,
+            ip_address       TEXT,
+            gateway          TEXT,
+            uptime_ms        BIGINT,
+            uptime_display   TEXT,
+            received_at      TIMESTAMPTZ NOT NULL DEFAULT now()
+        )
+    """)
+    op.execute("CREATE INDEX idx_heartbeats_serial_time ON heartbeats(device_serial, received_at DESC)")
+
+    # ------------------------------------------------------------------ #
+    # device_logs — partitioned by month on received_at
+    # ------------------------------------------------------------------ #
+    op.execute("""
+        CREATE TABLE device_logs (
+            id               BIGSERIAL,
+            device_serial    TEXT NOT NULL,
+            level            TEXT NOT NULL,
+            message          TEXT NOT NULL,
+            device_timestamp BIGINT,
+            received_at      TIMESTAMPTZ NOT NULL DEFAULT now(),
+            PRIMARY KEY (id, received_at)
+        ) PARTITION BY RANGE (received_at)
+    """)
+    op.execute("""
+        CREATE INDEX idx_device_logs_serial_time
+        ON device_logs(device_serial, received_at DESC)
+    """)
+    op.execute("""
+        CREATE INDEX idx_device_logs_level
+        ON device_logs(level, received_at DESC)
+    """)
+
+    # Create partitions: 2025-01 through 2026-06 (covers all existing data + near future)
+    partitions = [
+        ("2025_01", "2025-01-01", "2025-02-01"),
+        ("2025_02", "2025-02-01", "2025-03-01"),
+        ("2025_03", "2025-03-01", "2025-04-01"),
+        ("2025_04", "2025-04-01", "2025-05-01"),
+        ("2025_05", "2025-05-01", "2025-06-01"),
+        ("2025_06", "2025-06-01", "2025-07-01"),
+        ("2025_07", "2025-07-01", "2025-08-01"),
+        ("2025_08", "2025-08-01", "2025-09-01"),
+        ("2025_09", "2025-09-01", "2025-10-01"),
+        ("2025_10", "2025-10-01", "2025-11-01"),
+        ("2025_11", "2025-11-01", "2025-12-01"),
+        ("2025_12", "2025-12-01", "2026-01-01"),
+        ("2026_01", "2026-01-01", "2026-02-01"),
+        ("2026_02", "2026-02-01", "2026-03-01"),
+        ("2026_03", "2026-03-01", "2026-04-01"),
+        ("2026_04", "2026-04-01", "2026-05-01"),
+        ("2026_05", "2026-05-01", "2026-06-01"),
+        ("2026_06", "2026-06-01", "2026-07-01"),
+    ]
+    for suffix, start, end in partitions:
+        op.execute(f"""
+            CREATE TABLE device_logs_{suffix} PARTITION OF device_logs
+            FOR VALUES FROM ('{start}') TO ('{end}')
+        """)
+
+
+def downgrade() -> None:
+    # Drop in reverse dependency order
+    op.execute("DROP TABLE IF EXISTS device_logs CASCADE")   # drops all partitions too
+    op.execute("DROP TABLE IF EXISTS heartbeats CASCADE")
+    op.execute("DROP TABLE IF EXISTS commands CASCADE")
+    op.drop_table("device_alerts")
+    op.drop_table("mfg_audit_log")
+    op.drop_table("built_melodies")
+    op.drop_table("melody_drafts")
+    op.drop_table("crm_quotation_items")
+    op.drop_table("crm_quotations")
+    op.drop_table("crm_sync_state")
+    op.drop_table("crm_media")
+    op.drop_table("crm_comms_log")
+    op.drop_table("crm_orders")
+    op.drop_table("crm_customers")
+    op.drop_table("crm_products")
+    op.drop_table("public_features")
+    op.drop_table("console_settings")
+    op.drop_table("staff")
+    op.drop_table("audit_log")
+    op.drop_table("_migration_runs")

backend/alembic/versions/c3d4e5f6a7b8_phase_3_staff_ui_prefs.py

@@ -0,0 +1,32 @@
+"""phase_3_staff_ui_prefs
+
+Adds ui_prefs JSONB column to the staff table (Phase 3 — staff auth cutover).
+Also corrects permissions to be nullable (sysadmin/admin have NULL permissions).
+
+Revision ID: c3d4e5f6a7b8
+Revises: b1c2d3e4f5a6
+Create Date: 2026-04-17 00:00:00.000000
+"""
+from typing import Sequence, Union
+import sqlalchemy as sa
+from sqlalchemy.dialects.postgresql import JSONB
+from alembic import op
+
+revision: str = "c3d4e5f6a7b8"
+down_revision: Union[str, None] = "b1c2d3e4f5a6"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    op.add_column(
+        "staff",
+        sa.Column("ui_prefs", JSONB, nullable=False, server_default="{}"),
+    )
+    # permissions was NOT NULL DEFAULT '{}' — relax to nullable for sysadmin/admin
+    op.alter_column("staff", "permissions", nullable=True)
+
+
+def downgrade() -> None:
+    op.drop_column("staff", "ui_prefs")
+    op.alter_column("staff", "permissions", nullable=False)

backend/audit/router.py

@@ -0,0 +1,74 @@
+from datetime import datetime
+from typing import Optional
+
+from fastapi import APIRouter, Depends, Query
+from sqlalchemy import select, and_
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from database.postgres import get_pg_session
+from shared.orm import AuditLog
+from auth.dependencies import require_sysadmin
+from auth.models import TokenPayload
+
+router = APIRouter(prefix="/api/audit-log", tags=["audit-log"])
+
+_MAX_LIMIT = 200
+_DEFAULT_LIMIT = 50
+
+
+@router.get("")
+async def list_audit_log(
+    actor_id: Optional[str] = Query(None),
+    entity_type: Optional[str] = Query(None),
+    entity_id: Optional[str] = Query(None),
+    action: Optional[str] = Query(None),
+    from_date: Optional[datetime] = Query(None),
+    to_date: Optional[datetime] = Query(None),
+    limit: int = Query(_DEFAULT_LIMIT, ge=1, le=_MAX_LIMIT),
+    offset: int = Query(0, ge=0),
+    _user: TokenPayload = Depends(require_sysadmin),
+    db: AsyncSession = Depends(get_pg_session),
+):
+    filters = []
+    if actor_id:
+        filters.append(AuditLog.actor_id == actor_id)
+    if entity_type:
+        filters.append(AuditLog.entity_type == entity_type)
+    if entity_id:
+        filters.append(AuditLog.entity_id == entity_id)
+    if action:
+        filters.append(AuditLog.action == action)
+    if from_date:
+        filters.append(AuditLog.occurred_at >= from_date)
+    if to_date:
+        filters.append(AuditLog.occurred_at <= to_date)
+
+    stmt = (
+        select(AuditLog)
+        .where(and_(*filters) if filters else True)
+        .order_by(AuditLog.occurred_at.desc())
+        .offset(offset)
+        .limit(limit)
+    )
+    result = await db.execute(stmt)
+    rows = result.scalars().all()
+
+    return {
+        "entries": [
+            {
+                "id":           r.id,
+                "occurred_at":  r.occurred_at.isoformat(),
+                "actor_id":     r.actor_id,
+                "actor_name":   r.actor_name,
+                "action":       r.action,
+                "entity_type":  r.entity_type,
+                "entity_id":    r.entity_id,
+                "entity_label": r.entity_label,
+                "changes":      r.changes,
+                "meta":         r.meta,
+            }
+            for r in rows
+        ],
+        "limit":  limit,
+        "offset": offset,
+    }

backend/auth/dependencies.py

@@ -1,10 +1,14 @@
 from fastapi import Depends
 from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials
 from jose import JWTError
+from sqlalchemy import select
+from sqlalchemy.ext.asyncio import AsyncSession
+
 from auth.utils import decode_access_token
 from auth.models import TokenPayload, Role
+from database.postgres import get_pg_session
+from staff.orm import Staff
 from shared.exceptions import AuthenticationError, AuthorizationError
-from shared.firebase import get_db
 
 security = HTTPBearer()
 
@@ -37,18 +41,15 @@ def require_roles(*allowed_roles: Role):
     return role_checker
 
 
-async def _get_user_permissions(user: TokenPayload) -> dict:
+async def _get_user_permissions(user: TokenPayload, db: AsyncSession) -> dict | None:
-    """Fetch permissions from Firestore for the given user."""
+    """Fetch permissions from Postgres for the given user."""
     if user.role in (Role.sysadmin, Role.admin):
         return None  # Full access
-    db = get_db()
+    result = await db.execute(select(Staff).where(Staff.id == user.sub).limit(1))
-    if not db:
+    staff = result.scalar_one_or_none()
+    if staff is None:
         raise AuthorizationError()
-    doc = db.collection("admin_users").document(user.sub).get()
+    return staff.permissions
-    if not doc.exists:
-        raise AuthorizationError()
-    data = doc.to_dict()
-    return data.get("permissions")
 
 
 def require_permission(section: str, action: str):
@@ -58,17 +59,17 @@ def require_permission(section: str, action: str):
     """
     async def permission_checker(
         current_user: TokenPayload = Depends(get_current_user),
+        db: AsyncSession = Depends(get_pg_session),
     ) -> TokenPayload:
-        # sysadmin and admin have full access
         if current_user.role in (Role.sysadmin, Role.admin):
             return current_user
 
-        permissions = await _get_user_permissions(current_user)
+        permissions = await _get_user_permissions(current_user, db)
         if not permissions:
             raise AuthorizationError()
 
         if section == "mqtt":
-            if not permissions.get("mqtt", False):
+            if not permissions.get("mqtt", {}).get("access", False):
                 raise AuthorizationError()
             return current_user
 
@@ -89,11 +90,7 @@ def require_permission(section: str, action: str):
 # Pre-built convenience dependencies
 require_sysadmin = require_roles(Role.sysadmin)
 require_admin_or_above = require_roles(Role.sysadmin, Role.admin)
-
-# Staff management: only sysadmin and admin
 require_staff_management = require_roles(Role.sysadmin, Role.admin)
-
-# Viewer-level: any authenticated user (actual permission check per-action)
 require_any_authenticated = require_roles(
     Role.sysadmin, Role.admin, Role.editor, Role.user,
 )

backend/auth/router.py

@@ -1,59 +1,74 @@
-from fastapi import APIRouter
+from fastapi import APIRouter, Depends, Request
-from shared.firebase import get_db
+from sqlalchemy import select
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from database.postgres import get_pg_session
+from staff.orm import Staff
 from auth.models import LoginRequest, TokenResponse
 from auth.utils import verify_password, create_access_token
+from shared.audit import log_action
 from shared.exceptions import AuthenticationError
 
 router = APIRouter(prefix="/api/auth", tags=["auth"])
 
-
+_ROLE_MAP = {
-@router.post("/login", response_model=TokenResponse)
-async def login(body: LoginRequest):
-    db = get_db()
-    if not db:
-        raise AuthenticationError("Service unavailable")
-
-    users_ref = db.collection("admin_users")
-    query = users_ref.where("email", "==", body.email).limit(1).get()
-
-    if not query:
-        raise AuthenticationError("Invalid email or password")
-
-    doc = query[0]
-    user_data = doc.to_dict()
-
-    if not user_data.get("is_active", True):
-        raise AuthenticationError("Account is disabled")
-
-    if not verify_password(body.password, user_data["hashed_password"]):
-        raise AuthenticationError("Invalid email or password")
-
-    role = user_data["role"]
-    # Map legacy roles to new roles
-    role_mapping = {
     "superadmin":     "sysadmin",
     "melody_editor":  "editor",
     "device_manager": "editor",
     "user_manager":   "editor",
     "viewer":         "user",
-    }
+    "staff":          "user",
-    role = role_mapping.get(role, role)
+}
+
+
+@router.post("/login", response_model=TokenResponse)
+async def login(
+    body: LoginRequest,
+    request: Request,
+    db: AsyncSession = Depends(get_pg_session),
+):
+    result = await db.execute(
+        select(Staff).where(Staff.email == body.email).limit(1)
+    )
+    staff = result.scalar_one_or_none()
+
+    if staff is None:
+        raise AuthenticationError("Invalid email or password")
+
+    if not staff.is_active:
+        raise AuthenticationError("Account is disabled")
+
+    if not verify_password(body.password, staff.hashed_password):
+        raise AuthenticationError("Invalid email or password")
+
+    role = _ROLE_MAP.get(staff.role, staff.role)
 
     token = create_access_token({
-        "sub": doc.id,
+        "sub":   staff.id,
-        "email": user_data["email"],
+        "email": staff.email,
         "role":  role,
-        "name": user_data["name"],
+        "name":  staff.name,
     })
 
-    # Get permissions for editor/user roles
     permissions = None
     if role in ("editor", "user"):
-        permissions = user_data.get("permissions")
+        permissions = staff.permissions
+
+    await log_action(
+        db,
+        actor_id=staff.id,
+        actor_name=staff.name,
+        action="LOGIN",
+        entity_type="staff",
+        entity_id=staff.id,
+        entity_label=staff.email,
+        meta={"ip": request.client.host if request.client else None},
+    )
+    await db.commit()
 
     return TokenResponse(
         access_token=token,
         role=role,
-        name=user_data["name"],
+        name=staff.name,
         permissions=permissions,
     )

backend/builder/router.py

@@ -1,5 +1,6 @@
 from fastapi import APIRouter, Depends, HTTPException
 from fastapi.responses import FileResponse, PlainTextResponse
+from sqlalchemy.ext.asyncio import AsyncSession
 from auth.models import TokenPayload
 from auth.dependencies import require_permission
 from builder.models import (
@@ -9,6 +10,8 @@ from builder.models import (
     BuiltMelodyListResponse,
 )
 from builder import service
+from database.postgres import get_pg_session
+from shared.audit import log_action
 
 router = APIRouter(prefix="/api/builder/melodies", tags=["builder"])
 
@@ -54,8 +57,12 @@ async def get_built_melody(
 async def create_built_melody(
     body: BuiltMelodyCreate,
     _user: TokenPayload = Depends(require_permission("melodies", "edit")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
-    return await service.create_built_melody(body)
+    melody = await service.create_built_melody(body)
+    await log_action(db, _user.sub, _user.name or _user.email, "CREATE", "archetype",
+                     str(melody.id), melody.name or str(melody.id))
+    return melody
 
 
 @router.put("/{melody_id}", response_model=BuiltMelodyInDB)
@@ -63,16 +70,31 @@ async def update_built_melody(
     melody_id: str,
     body: BuiltMelodyUpdate,
     _user: TokenPayload = Depends(require_permission("melodies", "edit")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
-    return await service.update_built_melody(melody_id, body)
+    old = await service.get_built_melody(melody_id)
+    melody = await service.update_built_melody(melody_id, body)
+    _SKIP = {"updated_at", "id", "steps", "builtin_code"}
+    changes = {
+        k: {"old": getattr(old, k, None), "new": getattr(melody, k, None)}
+        for k in body.model_fields_set
+        if k not in _SKIP and getattr(old, k, None) != getattr(melody, k, None)
+    }
+    await log_action(db, _user.sub, _user.name or _user.email, "UPDATE", "archetype",
+                     melody_id, melody.name or melody_id, changes=changes or None)
+    return melody
 
 
 @router.delete("/{melody_id}", status_code=204)
 async def delete_built_melody(
     melody_id: str,
     _user: TokenPayload = Depends(require_permission("melodies", "delete")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
+    melody = await service.get_built_melody(melody_id)
     await service.delete_built_melody(melody_id)
+    await log_action(db, _user.sub, _user.name or _user.email, "DELETE", "archetype",
+                     melody_id, melody.name if melody else melody_id)
 
 
 @router.post("/{melody_id}/toggle-builtin", response_model=BuiltMelodyInDB)

backend/config.py

@@ -26,10 +26,15 @@ class Settings(BaseSettings):
     sqlite_db_path: str = "./data/database.db"
     mqtt_data_retention_days: int = 90
 
+    # Postgres
+    database_url: str = "postgresql+asyncpg://bellsystems_user:password@postgres:5432/bellsystems_db"
+
     # Local file storage
     built_melodies_storage_path: str = "./storage/built_melodies"
     firmware_storage_path: str = "./storage/firmware"
     flash_assets_storage_path: str = "./storage/flash_assets"
+    melody_binaries_storage_path: str = "./storage/melody_binaries"
+    melody_download_base_url: str = "http://melodies.bellsystems.net/download"
 
     # Email (Resend)
     resend_api_key: str = "re_placeholder_change_me"

backend/crm/comms_router.py

@@ -28,6 +28,16 @@ class MailListResponse(BaseModel):
     total: int
 
 
+@router.get("/latest-batch", response_model=dict)
+async def latest_comm_batch(
+    ids: str = Query(..., description="Comma-separated customer IDs"),
+    _user: TokenPayload = Depends(require_permission("crm", "view")),
+):
+    """Return the latest comm summary (id, type, occurred_at) keyed by customer_id."""
+    customer_ids = [i.strip() for i in ids.split(",") if i.strip()]
+    return await service.get_latest_comm_batch(customer_ids)
+
+
 @router.get("/all", response_model=CommListResponse)
 async def list_all_comms(
     type: Optional[str] = Query(None),

backend/crm/customers_router.py

@@ -2,16 +2,86 @@ import asyncio
 import logging
 from fastapi import APIRouter, Depends, Query, BackgroundTasks, Body
 from typing import Optional
+from sqlalchemy.ext.asyncio import AsyncSession
 
 from auth.models import TokenPayload
 from auth.dependencies import require_permission
 from crm.models import CustomerCreate, CustomerUpdate, CustomerInDB, CustomerListResponse, TransactionEntry
 from crm import service, nextcloud
 from config import settings
+from database.postgres import get_pg_session
+from shared.audit import log_action
 
 router = APIRouter(prefix="/api/crm/customers", tags=["crm-customers"])
 logger = logging.getLogger(__name__)
 
+# ── Diff helpers ──────────────────────────────────────────────────────────────
+
+_SCALAR_FIELDS = {
+    "name", "surname", "title", "organization", "religion", "language",
+    "relationship_status", "nextcloud_folder",
+}
+_SKIP_FIELDS = {"updated_at", "firestore_id", "id"}
+
+
+def _scalar_diff(old, new) -> dict:
+    result = {}
+    for f in _SCALAR_FIELDS:
+        ov = getattr(old, f, None)
+        nv = getattr(new, f, None)
+        if ov != nv:
+            result[f] = {"old": ov, "new": nv}
+    return result
+
+
+def _list_diff(field: str, old_list: list, new_list: list, label_fn) -> dict:
+    old_labels = {label_fn(i) for i in (old_list or [])}
+    new_labels = {label_fn(i) for i in (new_list or [])}
+    added   = new_labels - old_labels
+    removed = old_labels - new_labels
+    result = {}
+    if added:
+        result[f"{field}.added"] = {"old": None, "new": sorted(added)}
+    if removed:
+        result[f"{field}.removed"] = {"old": sorted(removed), "new": None}
+    return result
+
+
+def _customer_diff(old, new, changed_fields: set) -> dict:
+    changes = _scalar_diff(old, new)
+
+    # contacts — keyed by type+value string
+    if "contacts" in changed_fields:
+        changes.update(_list_diff(
+            "contacts",
+            old.contacts or [],
+            new.contacts or [],
+            lambda c: f"{c.get('type','?')}:{c.get('value','?')}" if isinstance(c, dict)
+                      else f"{getattr(c,'type','?')}:{getattr(c,'value','?')}",
+        ))
+
+    # location — flatten to individual sub-fields
+    if "location" in changed_fields:
+        old_loc = old.location or {}
+        new_loc = new.location or {}
+        if isinstance(old_loc, object) and not isinstance(old_loc, dict):
+            old_loc = old_loc.model_dump() if hasattr(old_loc, "model_dump") else {}
+        if isinstance(new_loc, object) and not isinstance(new_loc, dict):
+            new_loc = new_loc.model_dump() if hasattr(new_loc, "model_dump") else {}
+        for k in set(old_loc) | set(new_loc):
+            ov, nv = old_loc.get(k), new_loc.get(k)
+            if ov != nv:
+                changes[f"location.{k}"] = {"old": ov, "new": nv}
+
+    # tags
+    if "tags" in changed_fields:
+        old_tags = set(old.tags or [])
+        new_tags = set(new.tags or [])
+        if old_tags != new_tags:
+            changes["tags"] = {"old": sorted(old_tags), "new": sorted(new_tags)}
+
+    return changes
+
 
 @router.get("", response_model=CustomerListResponse)
 async def list_customers(
@@ -46,10 +116,14 @@ async def create_customer(
     body: CustomerCreate,
     background_tasks: BackgroundTasks,
     _user: TokenPayload = Depends(require_permission("crm", "edit")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
     customer = service.create_customer(body)
     if settings.nextcloud_url:
         background_tasks.add_task(_init_nextcloud_folder, customer)
+    label = " ".join(filter(None, [customer.name, customer.surname])) or customer.organization or customer.id
+    await log_action(db, _user.sub, _user.name or _user.email, "CREATE", "customer",
+                     customer.id, label)
     return customer
 
 
@@ -65,12 +139,19 @@ async def _init_nextcloud_folder(customer) -> None:
 
 
 @router.put("/{customer_id}", response_model=CustomerInDB)
-def update_customer(
+async def update_customer(
     customer_id: str,
     body: CustomerUpdate,
     _user: TokenPayload = Depends(require_permission("crm", "edit")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
-    return service.update_customer(customer_id, body)
+    old = service.get_customer(customer_id)
+    customer = service.update_customer(customer_id, body)
+    label = " ".join(filter(None, [customer.name, customer.surname])) or customer.organization or customer_id
+    changes = _customer_diff(old, customer, body.model_fields_set)
+    await log_action(db, _user.sub, _user.name or _user.email, "UPDATE", "customer",
+                     customer_id, label, changes=changes or None)
+    return customer
 
 
 @router.delete("/{customer_id}", status_code=204)
@@ -80,6 +161,7 @@ async def delete_customer(
     wipe_files: bool = Query(False),
     wipe_nextcloud: bool = Query(False),
     _user: TokenPayload = Depends(require_permission("crm", "edit")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
     customer = service.delete_customer(customer_id)
     nc_path = service.get_customer_nc_path(customer)
@@ -104,6 +186,10 @@ async def delete_customer(
             except Exception as e:
                 logger.warning("Could not rename NC folder for customer %s: %s", customer_id, e)
 
+    label = " ".join(filter(None, [customer.name, customer.surname])) or customer.organization or customer_id
+    await log_action(db, _user.sub, _user.name or _user.email, "DELETE", "customer",
+                     customer_id, label)
+
 
 @router.get("/{customer_id}/last-comm-direction")
 async def get_last_comm_direction(
@@ -117,12 +203,17 @@ async def get_last_comm_direction(
 # ── Relationship Status ───────────────────────────────────────────────────────
 
 @router.patch("/{customer_id}/relationship-status", response_model=CustomerInDB)
-def update_relationship_status(
+async def update_relationship_status(
     customer_id: str,
     body: dict = Body(...),
     _user: TokenPayload = Depends(require_permission("crm", "edit")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
-    return service.update_relationship_status(customer_id, body.get("status", ""))
+    customer = service.update_relationship_status(customer_id, body.get("status", ""))
+    label = " ".join(filter(None, [customer.name, customer.surname])) or customer.organization or customer_id
+    await log_action(db, _user.sub, _user.name or _user.email, "STATUS_CHANGE", "customer",
+                     customer_id, label, meta={"status": body.get("status", "")})
+    return customer
 
 
 # ── Technical Issues ──────────────────────────────────────────────────────────

backend/crm/orders_router.py

@@ -1,10 +1,13 @@
 from fastapi import APIRouter, Depends, Query
 from typing import Optional
+from sqlalchemy.ext.asyncio import AsyncSession
 
 from auth.models import TokenPayload
 from auth.dependencies import require_permission
 from crm.models import OrderCreate, OrderUpdate, OrderInDB, OrderListResponse
 from crm import service
+from database.postgres import get_pg_session
+from shared.audit import log_action
 
 router = APIRouter(prefix="/api/crm/customers/{customer_id}/orders", tags=["crm-orders"])
 
@@ -29,27 +32,35 @@ def get_next_order_number(
 
 
 @router.post("/init-negotiations", response_model=OrderInDB, status_code=201)
-def init_negotiations(
+async def init_negotiations(
     customer_id: str,
     body: dict,
     _user: TokenPayload = Depends(require_permission("crm", "edit")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
-    return service.init_negotiations(
+    order = service.init_negotiations(
         customer_id=customer_id,
         title=body.get("title", ""),
         note=body.get("note", ""),
         date=body.get("date"),
         created_by=body.get("created_by", ""),
     )
+    await log_action(db, _user.sub, _user.name or _user.email, "CREATE", "order",
+                     order.id, order.order_number or order.id, meta={"action_detail": "negotiations_started"})
+    return order
 
 
 @router.post("", response_model=OrderInDB, status_code=201)
-def create_order(
+async def create_order(
     customer_id: str,
     body: OrderCreate,
     _user: TokenPayload = Depends(require_permission("crm", "edit")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
-    return service.create_order(customer_id, body)
+    order = service.create_order(customer_id, body)
+    await log_action(db, _user.sub, _user.name or _user.email, "CREATE", "order",
+                     order.id, order.order_number or order.id)
+    return order
 
 
 @router.get("/{order_id}", response_model=OrderInDB)
@@ -62,22 +73,37 @@ def get_order(
 
 
 @router.patch("/{order_id}", response_model=OrderInDB)
-def update_order(
+async def update_order(
     customer_id: str,
     order_id: str,
     body: OrderUpdate,
     _user: TokenPayload = Depends(require_permission("crm", "edit")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
-    return service.update_order(customer_id, order_id, body)
+    old = service.get_order(customer_id, order_id)
+    order = service.update_order(customer_id, order_id, body)
+    action = "STATUS_CHANGE" if body.status is not None else "UPDATE"
+    _SKIP = {"updated_at", "id", "customer_id", "items", "timeline", "discount", "shipping", "payment_status"}
+    changes = {
+        k: {"old": getattr(old, k, None), "new": getattr(order, k, None)}
+        for k in body.model_fields_set
+        if k not in _SKIP and getattr(old, k, None) != getattr(order, k, None)
+    }
+    await log_action(db, _user.sub, _user.name or _user.email, action, "order",
+                     order_id, order.order_number or order_id, changes=changes or None)
+    return order
 
 
 @router.delete("/{order_id}", status_code=204)
-def delete_order(
+async def delete_order(
     customer_id: str,
     order_id: str,
     _user: TokenPayload = Depends(require_permission("crm", "edit")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
     service.delete_order(customer_id, order_id)
+    await log_action(db, _user.sub, _user.name or _user.email, "DELETE", "order",
+                     order_id, order_id)
 
 
 @router.post("/{order_id}/timeline", response_model=OrderInDB)

backend/crm/orm.py

@@ -0,0 +1,239 @@
+from datetime import datetime, timezone
+from sqlalchemy import (
+    BigInteger, Boolean, Column, DateTime, ForeignKey, Index, Integer,
+    Numeric, String, Text, UniqueConstraint,
+)
+from sqlalchemy.dialects.postgresql import ARRAY, JSONB
+from sqlalchemy.orm import relationship
+from database.postgres import Base
+
+
+def _now():
+    return datetime.now(timezone.utc)
+
+
+class CrmProduct(Base):
+    __tablename__ = "crm_products"
+
+    id           = Column(String(128), primary_key=True)   # Firestore doc ID
+    firestore_id = Column(String(128), unique=True)        # same as id during transition
+    name         = Column(String(500), nullable=False)
+    sku          = Column(String(128))
+    category     = Column(String(128))
+    description  = Column(Text)
+    unit_cost    = Column(Numeric(12, 2), nullable=False, default=0)
+    currency     = Column(String(10), nullable=False, default="EUR")
+    unit_type    = Column(String(32), nullable=False, default="pcs")
+    is_active    = Column(Boolean, nullable=False, default=True)
+    created_at   = Column(DateTime(timezone=True), nullable=False, default=_now)
+    updated_at   = Column(DateTime(timezone=True), nullable=False, default=_now, onupdate=_now)
+
+
+class CrmCustomer(Base):
+    __tablename__ = "crm_customers"
+    __table_args__ = (
+        Index("idx_crm_customers_rel_status", "relationship_status"),
+        Index("idx_crm_customers_name", "name", "surname"),
+        Index("idx_crm_customers_tags", "tags", postgresql_using="gin"),
+    )
+
+    id                  = Column(String(128), primary_key=True)   # Firestore doc ID
+    firestore_id        = Column(String(128), unique=True)
+    title               = Column(String(32))
+    name                = Column(String(255), nullable=False)
+    surname             = Column(String(255))
+    organization        = Column(String(500))
+    religion            = Column(String(64))
+    language            = Column(String(10), nullable=False, default="el")
+    folder_id           = Column(String(128), unique=True, nullable=False)
+    relationship_status = Column(String(64), nullable=False, default="lead")
+    nextcloud_folder    = Column(String(500))
+    contacts            = Column(JSONB, nullable=False, default=list)
+    notes               = Column(JSONB, nullable=False, default=list)
+    location            = Column(JSONB)
+    tags                = Column(ARRAY(String), nullable=False, default=list)
+    owned_items         = Column(JSONB, nullable=False, default=list)
+    linked_user_ids     = Column(ARRAY(String), nullable=False, default=list)
+    technical_issues    = Column(JSONB, nullable=False, default=list)
+    install_support     = Column(JSONB, nullable=False, default=list)
+    transaction_history = Column(JSONB, nullable=False, default=list)
+    crm_summary         = Column(JSONB)
+    created_at          = Column(DateTime(timezone=True), nullable=False)
+    updated_at          = Column(DateTime(timezone=True), nullable=False)
+
+    orders     = relationship("CrmOrder", back_populates="customer",
+                              cascade="all, delete-orphan", lazy="noload")
+    quotations = relationship("CrmQuotation", back_populates="customer",
+                              cascade="all, delete-orphan", lazy="noload")
+    comms      = relationship("CrmCommsLog", back_populates="customer",
+                              cascade="all, delete-orphan", lazy="noload")
+    media      = relationship("CrmMedia", back_populates="customer", lazy="noload")
+
+
+class CrmOrder(Base):
+    __tablename__ = "crm_orders"
+    __table_args__ = (
+        Index("idx_crm_orders_customer", "customer_id"),
+        Index("idx_crm_orders_status", "status"),
+    )
+
+    id                 = Column(String(128), primary_key=True)   # Firestore doc ID
+    customer_id        = Column(String(128), ForeignKey("crm_customers.id", ondelete="CASCADE"),
+                                nullable=False)
+    order_number       = Column(String(64), unique=True, nullable=False)
+    title              = Column(String(500))
+    created_by         = Column(String(128))
+    status             = Column(String(64), nullable=False, default="negotiating")
+    status_updated_date = Column(DateTime(timezone=True))
+    status_updated_by  = Column(String(128))
+    items              = Column(JSONB, nullable=False, default=list)
+    subtotal           = Column(Numeric(12, 2), nullable=False, default=0)
+    discount           = Column(JSONB)
+    total_price        = Column(Numeric(12, 2), nullable=False, default=0)
+    currency           = Column(String(10), nullable=False, default="EUR")
+    shipping           = Column(JSONB)
+    payment_status     = Column(JSONB, nullable=False, default=dict)
+    invoice_path       = Column(String(500))
+    notes              = Column(Text)
+    timeline           = Column(JSONB, nullable=False, default=list)
+    created_at         = Column(DateTime(timezone=True), nullable=False)
+    updated_at         = Column(DateTime(timezone=True), nullable=False)
+
+    customer = relationship("CrmCustomer", back_populates="orders")
+
+
+class CrmCommsLog(Base):
+    __tablename__ = "crm_comms_log"
+    __table_args__ = (
+        Index("idx_crm_comms_customer", "customer_id", "occurred_at"),
+    )
+
+    id             = Column(String(128), primary_key=True)
+    customer_id    = Column(String(128), ForeignKey("crm_customers.id", ondelete="SET NULL"),
+                            nullable=True)
+    type           = Column(String(32), nullable=False)       # email | sms | call | note | ...
+    mail_account   = Column(String(256))
+    direction      = Column(String(16), nullable=False)       # inbound | outbound
+    subject        = Column(String(500))
+    body           = Column(Text)
+    body_html      = Column(Text)
+    attachments    = Column(JSONB, nullable=False, default=list)
+    ext_message_id = Column(String(500))
+    from_addr      = Column(String(500))
+    to_addrs       = Column(Text)                             # JSON array as text or comma-sep
+    logged_by      = Column(String(128))
+    is_important   = Column(Boolean, nullable=False, default=False)
+    is_read        = Column(Boolean, nullable=False, default=True)
+    occurred_at    = Column(DateTime(timezone=True), nullable=False)
+    created_at     = Column(DateTime(timezone=True), nullable=False, default=_now)
+
+    customer = relationship("CrmCustomer", back_populates="comms")
+
+
+class CrmMedia(Base):
+    __tablename__ = "crm_media"
+    __table_args__ = (
+        Index("idx_crm_media_customer", "customer_id"),
+        Index("idx_crm_media_order", "order_id"),
+    )
+
+    id             = Column(String(128), primary_key=True)
+    customer_id    = Column(String(128), ForeignKey("crm_customers.id", ondelete="SET NULL"),
+                            nullable=True)
+    order_id       = Column(String(128))
+    filename       = Column(String(500), nullable=False)
+    nextcloud_path = Column(String(1000), nullable=False)
+    thumbnail_path = Column(String(1000))
+    mime_type      = Column(String(128))
+    direction      = Column(String(16))
+    tags           = Column(JSONB, nullable=False, default=list)
+    uploaded_by    = Column(String(128))
+    created_at     = Column(DateTime(timezone=True), nullable=False, default=_now)
+
+    customer = relationship("CrmCustomer", back_populates="media")
+
+
+class CrmSyncState(Base):
+    __tablename__ = "crm_sync_state"
+
+    key   = Column(String(128), primary_key=True)
+    value = Column(Text)
+
+
+class CrmQuotation(Base):
+    __tablename__ = "crm_quotations"
+    __table_args__ = (
+        Index("idx_crm_quotations_customer", "customer_id"),
+    )
+
+    id                       = Column(String(128), primary_key=True)
+    quotation_number         = Column(String(64), unique=True, nullable=False)
+    title                    = Column(String(500))
+    subtitle                 = Column(String(500))
+    customer_id              = Column(String(128), ForeignKey("crm_customers.id", ondelete="CASCADE"),
+                                      nullable=False)
+    language                 = Column(String(10), nullable=False, default="en")
+    status                   = Column(String(32), nullable=False, default="draft")
+    order_type               = Column(String(64))
+    shipping_method          = Column(String(64))
+    estimated_shipping_date  = Column(String(32))            # stored as DATE string
+    global_discount_label    = Column(String(128))
+    global_discount_percent  = Column(Numeric(8, 4), nullable=False, default=0)
+    vat_percent              = Column(Numeric(8, 4), nullable=False, default=24)
+    global_vat_percent       = Column(Numeric(8, 4), nullable=False, default=24)
+    shipping_cost            = Column(Numeric(12, 2), nullable=False, default=0)
+    shipping_cost_discount   = Column(Numeric(12, 2), nullable=False, default=0)
+    install_cost             = Column(Numeric(12, 2), nullable=False, default=0)
+    install_cost_discount    = Column(Numeric(12, 2), nullable=False, default=0)
+    extras_label             = Column(String(256))
+    extras_cost              = Column(Numeric(12, 2), nullable=False, default=0)
+    comments                 = Column(JSONB, nullable=False, default=list)
+    quick_notes              = Column(JSONB, nullable=False, default=dict)
+    subtotal_before_discount = Column(Numeric(12, 2), nullable=False, default=0)
+    global_discount_amount   = Column(Numeric(12, 2), nullable=False, default=0)
+    new_subtotal             = Column(Numeric(12, 2), nullable=False, default=0)
+    vat_amount               = Column(Numeric(12, 2), nullable=False, default=0)
+    final_total              = Column(Numeric(12, 2), nullable=False, default=0)
+    nextcloud_pdf_path       = Column(String(1000))
+    nextcloud_pdf_url        = Column(String(1000))
+    # Client snapshot fields (denormalised for PDF generation)
+    client_org               = Column(String(500))
+    client_name              = Column(String(500))
+    client_location          = Column(String(500))
+    client_phone             = Column(String(64))
+    client_email             = Column(String(256))
+    # Legacy quotation fields
+    is_legacy                = Column(Boolean, nullable=False, default=False)
+    legacy_date              = Column(String(32))
+    legacy_pdf_path          = Column(String(1000))
+    created_at               = Column(DateTime(timezone=True), nullable=False)
+    updated_at               = Column(DateTime(timezone=True), nullable=False)
+
+    customer = relationship("CrmCustomer", back_populates="quotations")
+    items    = relationship("CrmQuotationItem", back_populates="quotation",
+                            cascade="all, delete-orphan",
+                            order_by="CrmQuotationItem.sort_order", lazy="noload")
+
+
+class CrmQuotationItem(Base):
+    __tablename__ = "crm_quotation_items"
+    __table_args__ = (
+        Index("idx_crm_quotation_items_quotation", "quotation_id", "sort_order"),
+    )
+
+    id               = Column(String(128), primary_key=True)
+    quotation_id     = Column(String(128), ForeignKey("crm_quotations.id", ondelete="CASCADE"),
+                              nullable=False)
+    product_id       = Column(String(128))
+    description      = Column(Text)
+    description_en   = Column(Text)
+    description_gr   = Column(Text)
+    unit_type        = Column(String(32), nullable=False, default="pcs")
+    unit_cost        = Column(Numeric(12, 4), nullable=False, default=0)
+    discount_percent = Column(Numeric(8, 4), nullable=False, default=0)
+    vat_percent      = Column(Numeric(8, 4), nullable=False, default=24)
+    quantity         = Column(Numeric(12, 4), nullable=False, default=1)
+    line_total       = Column(Numeric(12, 2), nullable=False, default=0)
+    sort_order       = Column(Integer, nullable=False, default=0)
+
+    quotation = relationship("CrmQuotation", back_populates="items")

backend/crm/quotation_models.py

@@ -5,9 +5,10 @@ from pydantic import BaseModel
 
 class QuotationStatus(str, Enum):
     draft = "draft"
+    built = "built"
     sent = "sent"
     accepted = "accepted"
-    rejected = "rejected"
+    declined = "declined"
 
 
 class QuotationItemCreate(BaseModel):
@@ -39,6 +40,7 @@ class QuotationCreate(BaseModel):
     estimated_shipping_date: Optional[str] = None
     global_discount_label: Optional[str] = None
     global_discount_percent: float = 0.0
+    global_vat_percent: float = 24.0
     shipping_cost: float = 0.0
     shipping_cost_discount: float = 0.0
     install_cost: float = 0.0
@@ -70,6 +72,7 @@ class QuotationUpdate(BaseModel):
     estimated_shipping_date: Optional[str] = None
     global_discount_label: Optional[str] = None
     global_discount_percent: Optional[float] = None
+    global_vat_percent: Optional[float] = None
     shipping_cost: Optional[float] = None
     shipping_cost_discount: Optional[float] = None
     install_cost: Optional[float] = None
@@ -104,6 +107,7 @@ class QuotationInDB(BaseModel):
     estimated_shipping_date: Optional[str] = None
     global_discount_label: Optional[str] = None
     global_discount_percent: float = 0.0
+    global_vat_percent: float = 24.0
     shipping_cost: float = 0.0
     shipping_cost_discount: float = 0.0
     install_cost: float = 0.0

backend/crm/quotations_router.py

@@ -2,6 +2,7 @@ from fastapi import APIRouter, Depends, Query, UploadFile, File
 from fastapi.responses import StreamingResponse
 from typing import Optional
 import io
+from sqlalchemy.ext.asyncio import AsyncSession
 
 from auth.dependencies import require_permission
 from auth.models import TokenPayload
@@ -13,6 +14,8 @@ from crm.quotation_models import (
     QuotationUpdate,
 )
 from crm import quotations_service as svc
+from database.postgres import get_pg_session
+from shared.audit import log_action
 
 router = APIRouter(prefix="/api/crm/quotations", tags=["crm-quotations"])
 
@@ -72,11 +75,15 @@ async def create_quotation(
     body: QuotationCreate,
     generate_pdf: bool = Query(False),
     _user: TokenPayload = Depends(require_permission("crm", "edit")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
     """
     Create a quotation. Pass ?generate_pdf=true to immediately generate and upload the PDF.
     """
-    return await svc.create_quotation(body, generate_pdf=generate_pdf)
+    q = await svc.create_quotation(body, generate_pdf=generate_pdf)
+    await log_action(db, _user.sub, _user.name or _user.email, "CREATE", "quotation",
+                     str(q.id), q.quotation_number or str(q.id))
+    return q
 
 
 @router.put("/{quotation_id}", response_model=QuotationInDB)
@@ -85,19 +92,34 @@ async def update_quotation(
     body: QuotationUpdate,
     generate_pdf: bool = Query(False),
     _user: TokenPayload = Depends(require_permission("crm", "edit")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
     """
     Update a quotation. Pass ?generate_pdf=true to regenerate the PDF.
     """
-    return await svc.update_quotation(quotation_id, body, generate_pdf=generate_pdf)
+    old = await svc.get_quotation(quotation_id)
+    q = await svc.update_quotation(quotation_id, body, generate_pdf=generate_pdf)
+    _SKIP = {"updated_at", "id", "items", "pdf_path"}
+    changes = {
+        k: {"old": getattr(old, k, None), "new": getattr(q, k, None)}
+        for k in body.model_fields_set
+        if k not in _SKIP and getattr(old, k, None) != getattr(q, k, None)
+    }
+    await log_action(db, _user.sub, _user.name or _user.email, "UPDATE", "quotation",
+                     quotation_id, q.quotation_number or quotation_id, changes=changes or None)
+    return q
 
 
 @router.delete("/{quotation_id}", status_code=204)
 async def delete_quotation(
     quotation_id: str,
     _user: TokenPayload = Depends(require_permission("crm", "edit")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
+    q = await svc.get_quotation(quotation_id)
     await svc.delete_quotation(quotation_id)
+    await log_action(db, _user.sub, _user.name or _user.email, "DELETE", "quotation",
+                     quotation_id, q.quotation_number if q else quotation_id)
 
 
 @router.post("/{quotation_id}/regenerate-pdf", response_model=QuotationInDB)

backend/crm/quotations_service.py

@@ -42,6 +42,7 @@ def _float(d: Decimal) -> float:
 def _calculate_totals(
     items: list,
     global_discount_percent: float,
+    global_vat_percent: float,
     shipping_cost: float,
     shipping_cost_discount: float,
     install_cost: float,
@@ -50,21 +51,20 @@ def _calculate_totals(
 ) -> dict:
     """
     Calculate all monetary totals using Decimal arithmetic (ROUND_HALF_UP).
-    VAT is computed per-item from each item's vat_percent field.
+    VAT is a single global rate applied to items only (not shipping or install).
     Shipping and install costs carry 0% VAT.
     Returns a dict of floats ready for DB storage.
     """
-    # Per-line totals and per-item VAT
+    # Per-line totals (items only)
     item_totals = []
-    item_vat = Decimal(0)
     for item in items:
         cost = _d(item.get("unit_cost", 0))
         qty = _d(item.get("quantity", 1))
         disc = _d(item.get("discount_percent", 0))
         net = cost * qty * (1 - disc / 100)
         item_totals.append(net)
-        vat_pct = _d(item.get("vat_percent", 24))
+
-        item_vat += net * (vat_pct / 100)
+    items_net = sum(item_totals, Decimal(0))
 
     # Shipping net (VAT = 0%)
     ship_gross = _d(shipping_cost)
@@ -76,16 +76,17 @@ def _calculate_totals(
     install_disc = _d(install_cost_discount)
     install_net = install_gross * (1 - install_disc / 100)
 
-    subtotal = sum(item_totals, Decimal(0)) + ship_net + install_net
+    subtotal = items_net + ship_net + install_net
 
     global_disc_pct = _d(global_discount_percent)
     global_disc_amount = subtotal * (global_disc_pct / 100)
     new_subtotal = subtotal - global_disc_amount
 
-    # Global discount proportionally reduces VAT too
+    # VAT applies only to items portion, scaled by the global discount ratio
-    if subtotal > 0:
+    vat_pct = _d(global_vat_percent)
-        disc_ratio = new_subtotal / subtotal
+    if subtotal > 0 and items_net > 0:
-        vat_amount = item_vat * disc_ratio
+        items_ratio = items_net / subtotal
+        vat_amount = new_subtotal * items_ratio * (vat_pct / 100)
     else:
         vat_amount = Decimal(0)
 
@@ -109,14 +110,16 @@ def _calc_line_total(item) -> float:
 
 
 async def _generate_quotation_number(db) -> str:
-    year = datetime.utcnow().year
+    now = datetime.utcnow()
-    prefix = f"QT-{year}-"
+    yy = now.strftime("%y")
+    mm = now.strftime("%m")
+    prefix = f"QT-{yy}-{mm}-"
     rows = await db.execute_fetchall(
         "SELECT quotation_number FROM crm_quotations WHERE quotation_number LIKE ? ORDER BY quotation_number DESC LIMIT 1",
         (f"{prefix}%",),
     )
     if rows:
-        last_num = rows[0][0]  # e.g. "QT-2026-012"
+        last_num = rows[0][0]  # e.g. "QT-26-04-012"
         try:
             seq = int(last_num[len(prefix):]) + 1
         except ValueError:
@@ -174,13 +177,16 @@ async def list_all_quotations() -> list[dict]:
                 doc = fstore.collection("crm_customers").document(cid).get()
                 if doc.exists:
                     d = doc.to_dict()
-                    parts = [d.get("name", ""), d.get("surname", ""), d.get("organization", "")]
+                    name_parts = [d.get("name", ""), d.get("surname", "")]
-                    label = " ".join(p for p in parts if p).strip()
+                    full_name = " ".join(p for p in name_parts if p).strip()
-                    customer_names[cid] = label or cid
+                    org = (d.get("organization", "") or "").strip()
+                    customer_names[cid] = {"name": full_name or cid, "org": org}
             except Exception:
-                customer_names[cid] = cid
+                customer_names[cid] = {"name": cid, "org": ""}
     for item in items:
-        item["customer_name"] = customer_names.get(item["customer_id"], "")
+        info = customer_names.get(item["customer_id"], {"name": "", "org": ""})
+        item["customer_name"] = info["name"]
+        item["customer_org"] = info["org"]
     return items
 
 
@@ -222,6 +228,7 @@ async def create_quotation(data: QuotationCreate, generate_pdf: bool = False) ->
     totals = _calculate_totals(
         items_raw,
         data.global_discount_percent,
+        data.global_vat_percent,
         data.shipping_cost,
         data.shipping_cost_discount,
         data.install_cost,
@@ -236,7 +243,7 @@ async def create_quotation(data: QuotationCreate, generate_pdf: bool = False) ->
         """INSERT INTO crm_quotations (
             id, quotation_number, title, subtitle, customer_id,
             language, status, order_type, shipping_method, estimated_shipping_date,
-            global_discount_label, global_discount_percent,
+            global_discount_label, global_discount_percent, global_vat_percent,
             shipping_cost, shipping_cost_discount, install_cost, install_cost_discount,
             extras_label, extras_cost, comments, quick_notes,
             subtotal_before_discount, global_discount_amount, new_subtotal, vat_amount, final_total,
@@ -247,7 +254,7 @@ async def create_quotation(data: QuotationCreate, generate_pdf: bool = False) ->
         ) VALUES (
             ?, ?, ?, ?, ?,
             ?, 'draft', ?, ?, ?,
-            ?, ?,
+            ?, ?, ?,
             ?, ?, ?, ?,
             ?, ?, ?, ?,
             ?, ?, ?, ?, ?,
@@ -259,7 +266,7 @@ async def create_quotation(data: QuotationCreate, generate_pdf: bool = False) ->
         (
             qid, quotation_number, data.title, data.subtitle, data.customer_id,
             data.language, data.order_type, data.shipping_method, data.estimated_shipping_date,
-            data.global_discount_label, data.global_discount_percent,
+            data.global_discount_label, data.global_discount_percent, data.global_vat_percent,
             data.shipping_cost, data.shipping_cost_discount, data.install_cost, data.install_cost_discount,
             data.extras_label, data.extras_cost, comments_json, quick_notes_json,
             totals["subtotal_before_discount"], totals["global_discount_amount"],
@@ -317,7 +324,7 @@ async def update_quotation(quotation_id: str, data: QuotationUpdate, generate_pd
 
     scalar_fields = [
         "title", "subtitle", "language", "status", "order_type", "shipping_method",
-        "estimated_shipping_date", "global_discount_label", "global_discount_percent",
+        "estimated_shipping_date", "global_discount_label", "global_discount_percent", "global_vat_percent",
         "shipping_cost", "shipping_cost_discount", "install_cost",
         "install_cost_discount", "extras_label", "extras_cost",
         "client_org", "client_name", "client_location", "client_phone", "client_email",
@@ -352,6 +359,7 @@ async def update_quotation(quotation_id: str, data: QuotationUpdate, generate_pd
     totals = _calculate_totals(
         items_raw,
         float(merged.get("global_discount_percent", 0)),
+        float(merged.get("global_vat_percent", 24)),
         float(merged.get("shipping_cost", 0)),
         float(merged.get("shipping_cost_discount", 0)),
         float(merged.get("install_cost", 0)),

backend/crm/router.py

@@ -3,11 +3,14 @@ from fastapi.responses import FileResponse
 from typing import Optional
 import os
 import shutil
+from sqlalchemy.ext.asyncio import AsyncSession
 
 from auth.models import TokenPayload
 from auth.dependencies import require_permission
 from crm.models import ProductCreate, ProductUpdate, ProductInDB, ProductListResponse
 from crm import service
+from database.postgres import get_pg_session
+from shared.audit import log_action
 
 router = APIRouter(prefix="/api/crm/products", tags=["crm-products"])
 
@@ -35,28 +38,47 @@ def get_product(
 
 
 @router.post("", response_model=ProductInDB, status_code=201)
-def create_product(
+async def create_product(
     body: ProductCreate,
     _user: TokenPayload = Depends(require_permission("crm", "edit")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
-    return service.create_product(body)
+    product = service.create_product(body)
+    await log_action(db, _user.sub, _user.name or _user.email, "CREATE", "product",
+                     product.id, product.name)
+    return product
 
 
 @router.put("/{product_id}", response_model=ProductInDB)
-def update_product(
+async def update_product(
     product_id: str,
     body: ProductUpdate,
     _user: TokenPayload = Depends(require_permission("crm", "edit")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
-    return service.update_product(product_id, body)
+    old = service.get_product(product_id)
+    product = service.update_product(product_id, body)
+    _SKIP = {"updated_at", "id", "photo_url"}
+    changes = {
+        k: {"old": getattr(old, k, None), "new": getattr(product, k, None)}
+        for k in body.model_fields_set
+        if k not in _SKIP and getattr(old, k, None) != getattr(product, k, None)
+    }
+    await log_action(db, _user.sub, _user.name or _user.email, "UPDATE", "product",
+                     product_id, product.name, changes=changes or None)
+    return product
 
 
 @router.delete("/{product_id}", status_code=204)
-def delete_product(
+async def delete_product(
     product_id: str,
     _user: TokenPayload = Depends(require_permission("crm", "edit")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
+    product = service.get_product(product_id)
     service.delete_product(product_id)
+    await log_action(db, _user.sub, _user.name or _user.email, "DELETE", "product",
+                     product_id, product.name)
 
 
 @router.post("/{product_id}/photo", response_model=ProductInDB)

backend/crm/service.py

@@ -305,6 +305,33 @@ async def get_last_comm_timestamp(customer_id: str) -> str | None:
     return None
 
 
+async def get_latest_comm_batch(customer_ids: list[str]) -> dict[str, dict]:
+    """Return a dict of customer_id → {id, type, occurred_at} for the latest comm per customer.
+    Uses a single SQL query — no N+1 regardless of list size.
+    """
+    if not customer_ids:
+        return {}
+    db = await mqtt_db.get_db()
+    placeholders = ",".join("?" * len(customer_ids))
+    rows = await db.execute_fetchall(
+        f"""
+        SELECT customer_id, id, type, COALESCE(occurred_at, created_at) AS ts
+        FROM crm_comms_log
+        WHERE customer_id IN ({placeholders})
+        AND customer_id IS NOT NULL AND customer_id != ''
+        ORDER BY ts DESC
+        """,
+        customer_ids,
+    )
+    # Keep only the first (latest) row per customer
+    result: dict[str, dict] = {}
+    for row in rows:
+        cid = row[0]
+        if cid not in result:
+            result[cid] = {"id": row[1], "type": row[2], "occurred_at": row[3]}
+    return result
+
+
 async def list_customers_sorted_by_latest_comm(customers: list[CustomerInDB]) -> list[CustomerInDB]:
     """Re-sort a list of customers so those with the most recent comm come first."""
     timestamps = await asyncio.gather(

backend/database/__init__.py

@@ -1,7 +1,7 @@
-from database.core import (
+# MQTT live data — Phase 5: all functions now backed by Postgres
+from database.pg_mqtt import (
     init_db,
     close_db,
-    get_db,
     purge_loop,
     purge_old_data,
     insert_log,
@@ -16,8 +16,14 @@ from database.core import (
     upsert_alert,
     delete_alert,
     get_alerts,
+    partition_manager_loop,
+    ensure_current_partitions,
 )
 
+# SQLite connection — still used by melodies, builder, manufacturing, and crm
+# modules that have not yet been cut over to Postgres.
+from database.core import get_db
+
 __all__ = [
     "init_db",
     "close_db",
@@ -36,4 +42,6 @@ __all__ = [
     "upsert_alert",
     "delete_alert",
     "get_alerts",
+    "partition_manager_loop",
+    "ensure_current_partitions",
 ]

backend/database/core.py

@@ -208,6 +208,7 @@ async def init_db():
         "ALTER TABLE crm_quotation_items ADD COLUMN description_en TEXT",
         "ALTER TABLE crm_quotation_items ADD COLUMN description_gr TEXT",
         "ALTER TABLE built_melodies ADD COLUMN is_builtin INTEGER NOT NULL DEFAULT 0",
+        "ALTER TABLE crm_quotations ADD COLUMN global_vat_percent REAL NOT NULL DEFAULT 24",
     ]
     for m in _migrations:
         try:

backend/database/models.py

@@ -0,0 +1,23 @@
+from database.postgres import Base  # noqa: F401 — Base must be imported for Alembic autogenerate
+
+# Import all ORM models here so Alembic autogenerate detects them.
+# Add each new model file as it is created.
+
+# --- Existing ---
+from notes.orm import Entry, EntryLink                              # noqa: F401
+from tickets.orm import SupportTicket, TicketMessage               # noqa: F401
+
+# --- Phase 0 ---
+from shared.orm import MigrationRun, AuditLog                      # noqa: F401
+from crm.orm import (                                              # noqa: F401
+    CrmProduct, CrmCustomer, CrmOrder,
+    CrmCommsLog, CrmMedia, CrmSyncState,
+    CrmQuotation, CrmQuotationItem,
+)
+from staff.orm import Staff                                         # noqa: F401
+from settings.orm import ConsoleSetting, PublicFeature             # noqa: F401
+from melodies.orm import MelodyDraft, BuiltMelody                  # noqa: F401
+from manufacturing.orm import MfgAuditLog                          # noqa: F401
+from devices.orm import DeviceAlert                                 # noqa: F401
+# NOTE: device_logs, commands, heartbeats are partitioned/raw-SQL tables —
+# they are NOT ORM models and are created via op.execute() in the migration.

backend/database/pg_mqtt.py

@@ -0,0 +1,411 @@
+"""
+Phase 5 — MQTT live data functions backed by Postgres.
+
+device_logs is a partitioned table; heartbeats and commands are plain tables.
+All three are accessed via raw SQL (not ORM) because device_logs partitioning
+does not play well with SQLAlchemy's declarative ORM.
+
+device_alerts is an ORM model (devices/orm.py) and is handled here via raw SQL
+to keep a single consistent interface for callers that used to import from database.core.
+"""
+
+import asyncio
+import json
+import logging
+from datetime import date, datetime, timedelta, timezone
+
+from sqlalchemy import text
+from config import settings
+from database.postgres import AsyncSessionLocal
+
+logger = logging.getLogger("database.pg_mqtt")
+
+
+# ---------------------------------------------------------------------------
+# Insert operations
+# ---------------------------------------------------------------------------
+
+async def insert_log(device_serial: str, level: str, message: str,
+                     device_timestamp: int | None = None) -> int:
+    async with AsyncSessionLocal() as session:
+        result = await session.execute(
+            text("""
+                INSERT INTO device_logs (device_serial, level, message, device_timestamp, received_at)
+                VALUES (:serial, :level, :message, :ts, now())
+                RETURNING id
+            """),
+            {"serial": device_serial, "level": level, "message": message, "ts": device_timestamp},
+        )
+        row = result.fetchone()
+        await session.commit()
+        return row[0]
+
+
+async def insert_heartbeat(device_serial: str, device_id: str,
+                           firmware_version: str, ip_address: str,
+                           gateway: str, uptime_ms: int, uptime_display: str) -> int:
+    async with AsyncSessionLocal() as session:
+        result = await session.execute(
+            text("""
+                INSERT INTO heartbeats
+                    (device_serial, device_id, firmware_version, ip_address,
+                     gateway, uptime_ms, uptime_display, received_at)
+                VALUES
+                    (:serial, :device_id, :fw, :ip, :gw, :uptime_ms, :uptime_display, now())
+                RETURNING id
+            """),
+            {
+                "serial": device_serial,
+                "device_id": device_id,
+                "fw": firmware_version,
+                "ip": ip_address,
+                "gw": gateway,
+                "uptime_ms": uptime_ms,
+                "uptime_display": uptime_display,
+            },
+        )
+        row = result.fetchone()
+        await session.commit()
+        return row[0]
+
+
+async def insert_command(device_serial: str, command_name: str,
+                         command_payload: dict) -> int:
+    async with AsyncSessionLocal() as session:
+        result = await session.execute(
+            text("""
+                INSERT INTO commands (device_serial, command_name, command_payload, sent_at)
+                VALUES (:serial, :name, :payload, now())
+                RETURNING id
+            """),
+            {
+                "serial": device_serial,
+                "name": command_name,
+                "payload": json.dumps(command_payload),
+            },
+        )
+        row = result.fetchone()
+        await session.commit()
+        return row[0]
+
+
+async def update_command_response(command_id: int, status: str,
+                                  response_payload: dict | None = None):
+    async with AsyncSessionLocal() as session:
+        await session.execute(
+            text("""
+                UPDATE commands
+                SET status = :status,
+                    response_payload = :payload,
+                    responded_at = now()
+                WHERE id = :id
+            """),
+            {
+                "id": command_id,
+                "status": status,
+                "payload": json.dumps(response_payload) if response_payload else None,
+            },
+        )
+        await session.commit()
+
+
+# ---------------------------------------------------------------------------
+# Query operations
+# ---------------------------------------------------------------------------
+
+async def get_logs(device_serial: str, level: str | None = None,
+                   search: str | None = None,
+                   limit: int = 100, offset: int = 0) -> tuple[list, int]:
+    where = "device_serial = :serial"
+    params: dict = {"serial": device_serial, "limit": limit, "offset": offset}
+
+    if level:
+        where += " AND level = :level"
+        params["level"] = level
+    if search:
+        where += " AND message ILIKE :search"
+        params["search"] = f"%{search}%"
+
+    async with AsyncSessionLocal() as session:
+        count_result = await session.execute(
+            text(f"SELECT COUNT(*) FROM device_logs WHERE {where}"), params
+        )
+        total = count_result.scalar()
+
+        rows_result = await session.execute(
+            text(f"""
+                SELECT id, device_serial, level, message, device_timestamp,
+                       received_at AT TIME ZONE 'UTC' AS received_at
+                FROM device_logs
+                WHERE {where}
+                ORDER BY received_at DESC
+                LIMIT :limit OFFSET :offset
+            """),
+            params,
+        )
+        rows = rows_result.mappings().all()
+
+    return [_row_to_dict(r) for r in rows], total
+
+
+async def get_heartbeats(device_serial: str, limit: int = 100,
+                         offset: int = 0) -> tuple[list, int]:
+    async with AsyncSessionLocal() as session:
+        count_result = await session.execute(
+            text("SELECT COUNT(*) FROM heartbeats WHERE device_serial = :serial"),
+            {"serial": device_serial},
+        )
+        total = count_result.scalar()
+
+        rows_result = await session.execute(
+            text("""
+                SELECT id, device_serial, device_id, firmware_version, ip_address,
+                       gateway, uptime_ms, uptime_display,
+                       received_at AT TIME ZONE 'UTC' AS received_at
+                FROM heartbeats
+                WHERE device_serial = :serial
+                ORDER BY received_at DESC
+                LIMIT :limit OFFSET :offset
+            """),
+            {"serial": device_serial, "limit": limit, "offset": offset},
+        )
+        rows = rows_result.mappings().all()
+
+    return [_row_to_dict(r) for r in rows], total
+
+
+async def get_commands(device_serial: str, limit: int = 100,
+                       offset: int = 0) -> tuple[list, int]:
+    async with AsyncSessionLocal() as session:
+        count_result = await session.execute(
+            text("SELECT COUNT(*) FROM commands WHERE device_serial = :serial"),
+            {"serial": device_serial},
+        )
+        total = count_result.scalar()
+
+        rows_result = await session.execute(
+            text("""
+                SELECT id, device_serial, command_name, command_payload, status,
+                       response_payload,
+                       sent_at AT TIME ZONE 'UTC' AS sent_at,
+                       responded_at AT TIME ZONE 'UTC' AS responded_at
+                FROM commands
+                WHERE device_serial = :serial
+                ORDER BY sent_at DESC
+                LIMIT :limit OFFSET :offset
+            """),
+            {"serial": device_serial, "limit": limit, "offset": offset},
+        )
+        rows = rows_result.mappings().all()
+
+    return [_row_to_dict(r) for r in rows], total
+
+
+async def get_latest_heartbeats() -> list:
+    async with AsyncSessionLocal() as session:
+        rows_result = await session.execute(
+            text("""
+                SELECT DISTINCT ON (device_serial)
+                    id, device_serial, device_id, firmware_version, ip_address,
+                    gateway, uptime_ms, uptime_display,
+                    received_at AT TIME ZONE 'UTC' AS received_at
+                FROM heartbeats
+                ORDER BY device_serial, received_at DESC
+            """)
+        )
+        rows = rows_result.mappings().all()
+
+    return [_row_to_dict(r) for r in rows]
+
+
+async def get_pending_command(device_serial: str) -> dict | None:
+    async with AsyncSessionLocal() as session:
+        result = await session.execute(
+            text("""
+                SELECT id, device_serial, command_name, command_payload, status,
+                       response_payload,
+                       sent_at AT TIME ZONE 'UTC' AS sent_at,
+                       responded_at AT TIME ZONE 'UTC' AS responded_at
+                FROM commands
+                WHERE device_serial = :serial AND status = 'pending'
+                ORDER BY sent_at DESC
+                LIMIT 1
+            """),
+            {"serial": device_serial},
+        )
+        row = result.mappings().fetchone()
+
+    return _row_to_dict(row) if row else None
+
+
+# ---------------------------------------------------------------------------
+# Device alerts
+# ---------------------------------------------------------------------------
+
+async def upsert_alert(device_serial: str, subsystem: str, state: str,
+                       message: str | None = None):
+    async with AsyncSessionLocal() as session:
+        await session.execute(
+            text("""
+                INSERT INTO device_alerts (device_serial, subsystem, state, message, updated_at)
+                VALUES (:serial, :subsystem, :state, :message, now())
+                ON CONFLICT (device_serial, subsystem)
+                DO UPDATE SET
+                    state      = EXCLUDED.state,
+                    message    = EXCLUDED.message,
+                    updated_at = EXCLUDED.updated_at
+            """),
+            {"serial": device_serial, "subsystem": subsystem, "state": state, "message": message},
+        )
+        await session.commit()
+
+
+async def delete_alert(device_serial: str, subsystem: str):
+    async with AsyncSessionLocal() as session:
+        await session.execute(
+            text("DELETE FROM device_alerts WHERE device_serial = :serial AND subsystem = :subsystem"),
+            {"serial": device_serial, "subsystem": subsystem},
+        )
+        await session.commit()
+
+
+async def get_alerts(device_serial: str) -> list:
+    async with AsyncSessionLocal() as session:
+        result = await session.execute(
+            text("""
+                SELECT id, device_serial, subsystem, state, message,
+                       updated_at AT TIME ZONE 'UTC' AS updated_at
+                FROM device_alerts
+                WHERE device_serial = :serial
+                ORDER BY updated_at DESC
+            """),
+            {"serial": device_serial},
+        )
+        rows = result.mappings().all()
+
+    return [_row_to_dict(r) for r in rows]
+
+
+# ---------------------------------------------------------------------------
+# Partition management
+# ---------------------------------------------------------------------------
+
+def _add_months(d: date, months: int) -> date:
+    month = d.month - 1 + months
+    year = d.year + month // 12
+    month = month % 12 + 1
+    return d.replace(year=year, month=month, day=1)
+
+
+async def ensure_current_partitions():
+    """Create device_logs partitions for the current and next month if missing."""
+    async with AsyncSessionLocal() as session:
+        for month_offset in (0, 1):
+            d = _add_months(date.today().replace(day=1), month_offset)
+            partition_name = f"device_logs_{d.strftime('%Y_%m')}"
+            start = d.isoformat()
+            end = _add_months(d, 1).isoformat()
+            await session.execute(text(f"""
+                CREATE TABLE IF NOT EXISTS {partition_name}
+                PARTITION OF device_logs
+                FOR VALUES FROM ('{start}') TO ('{end}')
+            """))
+        await session.commit()
+    logger.info("Partition check complete")
+
+
+async def drop_old_partitions(keep_months: int = 6):
+    """Drop device_logs partitions older than keep_months."""
+    cutoff = _add_months(date.today().replace(day=1), -keep_months)
+    async with AsyncSessionLocal() as session:
+        result = await session.execute(text("""
+            SELECT tablename FROM pg_tables
+            WHERE schemaname = 'public'
+              AND tablename LIKE 'device_logs_%'
+        """))
+        partitions = [r[0] for r in result.fetchall()]
+
+    for name in partitions:
+        # name format: device_logs_YYYY_MM
+        parts = name.split("_")
+        if len(parts) != 4:
+            continue
+        try:
+            partition_date = date(int(parts[2]), int(parts[3]), 1)
+        except ValueError:
+            continue
+        if partition_date < cutoff:
+            async with AsyncSessionLocal() as session:
+                await session.execute(text(f"DROP TABLE IF EXISTS {name}"))
+                await session.commit()
+            logger.info(f"Dropped old partition: {name}")
+
+
+async def partition_manager_loop():
+    """Runs once on startup, then monthly thereafter."""
+    await ensure_current_partitions()
+    while True:
+        # Sleep ~30 days, wake up and ensure next month's partition exists
+        await asyncio.sleep(30 * 24 * 3600)
+        try:
+            await ensure_current_partitions()
+            await drop_old_partitions()
+        except Exception as e:
+            logger.error(f"Partition manager error: {e}")
+
+
+# ---------------------------------------------------------------------------
+# Cleanup (replaces SQLite purge_loop — now a no-op since Postgres uses
+# partition drops instead of row-by-row deletes for device_logs; heartbeats
+# and commands are still purged by row deletion)
+# ---------------------------------------------------------------------------
+
+async def purge_old_data(retention_days: int | None = None):
+    days = retention_days or settings.mqtt_data_retention_days
+    cutoff = datetime.now(timezone.utc) - timedelta(days=days)
+    async with AsyncSessionLocal() as session:
+        await session.execute(
+            text("DELETE FROM heartbeats WHERE received_at < :cutoff"),
+            {"cutoff": cutoff},
+        )
+        await session.execute(
+            text("DELETE FROM commands WHERE sent_at < :cutoff"),
+            {"cutoff": cutoff},
+        )
+        await session.commit()
+    logger.info(f"Purged heartbeats and commands older than {days} days")
+
+
+async def purge_loop():
+    while True:
+        await asyncio.sleep(86400)
+        try:
+            await purge_old_data()
+        except Exception as e:
+            logger.error(f"Purge failed: {e}")
+
+
+# ---------------------------------------------------------------------------
+# Stub — no longer needed but kept so nothing that imports init_db/close_db breaks
+# ---------------------------------------------------------------------------
+
+async def init_db():
+    """No-op: Postgres schema is managed by Alembic, not runtime init."""
+    logger.info("Postgres MQTT backend active — no SQLite init needed")
+
+
+async def close_db():
+    """No-op: SQLAlchemy engine lifecycle is managed by the process."""
+
+
+# ---------------------------------------------------------------------------
+# Helpers
+# ---------------------------------------------------------------------------
+
+def _row_to_dict(row) -> dict:
+    """Convert a SQLAlchemy RowMapping to a plain dict with ISO string timestamps."""
+    d = dict(row)
+    for key, val in d.items():
+        if isinstance(val, datetime):
+            d[key] = val.isoformat()
+    return d

backend/database/postgres.py

@@ -0,0 +1,16 @@
+from sqlalchemy.ext.asyncio import create_async_engine, AsyncSession, async_sessionmaker
+from sqlalchemy.orm import DeclarativeBase
+from config import settings
+
+engine = create_async_engine(settings.database_url, pool_size=10, echo=False)
+AsyncSessionLocal = async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
+
+
+class Base(DeclarativeBase):
+    pass
+
+
+async def get_pg_session() -> AsyncSession:
+    """FastAPI dependency — yields a DB session and closes it after the request."""
+    async with AsyncSessionLocal() as session:
+        yield session

backend/devices/orm.py

@@ -0,0 +1,31 @@
+from datetime import datetime, timezone
+from sqlalchemy import BigInteger, Column, DateTime, Index, String, Text, UniqueConstraint
+from sqlalchemy.dialects.postgresql import JSONB
+from database.postgres import Base
+
+
+def _now():
+    return datetime.now(timezone.utc)
+
+
+class DeviceAlert(Base):
+    """Current alert state per device+subsystem (upserted, not appended)."""
+    __tablename__ = "device_alerts"
+    __table_args__ = (
+        UniqueConstraint("device_serial", "subsystem"),
+        Index("idx_device_alerts_serial", "device_serial"),
+    )
+
+    id            = Column(BigInteger, primary_key=True, autoincrement=True)
+    device_serial = Column(String(128), nullable=False)
+    subsystem     = Column(String(128), nullable=False)
+    state         = Column(String(64), nullable=False)
+    message       = Column(Text)
+    updated_at    = Column(DateTime(timezone=True), nullable=False, default=_now, onupdate=_now)
+
+
+# NOTE: device_logs, commands, and heartbeats are NOT declared as ORM models here.
+# device_logs is a partitioned table — SQLAlchemy ORM does not support declarative
+# partitioned tables cleanly. All three tables are created via raw SQL in the
+# Alembic migration and accessed via raw queries in database/core.py (SQLite now)
+# and will be accessed via raw async SQL after Phase 5 cutover.

backend/devices/router.py

@@ -3,6 +3,7 @@ from datetime import datetime
 from fastapi import APIRouter, Depends, Query, HTTPException
 from typing import Optional, List
 from pydantic import BaseModel
+from sqlalchemy.ext.asyncio import AsyncSession
 from auth.models import TokenPayload
 from auth.dependencies import require_permission
 from devices.models import (
@@ -14,6 +15,8 @@ from devices import service
 import database as mqtt_db
 from mqtt.models import DeviceAlertEntry, DeviceAlertsResponse
 from shared.firebase import get_db as get_firestore
+from database.postgres import get_pg_session
+from shared.audit import log_action
 
 router = APIRouter(prefix="/api/devices", tags=["devices"])
 
@@ -58,8 +61,12 @@ async def get_device_users(
 async def create_device(
     body: DeviceCreate,
     _user: TokenPayload = Depends(require_permission("devices", "add")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
-    return service.create_device(body)
+    device = service.create_device(body)
+    await log_action(db, _user.sub, _user.name or _user.email, "CREATE", "device",
+                     device.device_id, device.device_name or device.device_id)
+    return device
 
 
 @router.put("/{device_id}", response_model=DeviceInDB)
@@ -67,16 +74,32 @@ async def update_device(
     device_id: str,
     body: DeviceUpdate,
     _user: TokenPayload = Depends(require_permission("devices", "edit")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
-    return service.update_device(device_id, body)
+    old = service.get_device(device_id)
+    device = service.update_device(device_id, body)
+    _SKIP = {"updated_at", "device_id", "tags", "user_list"}
+    changes = {
+        k: {"old": getattr(old, k, None), "new": getattr(device, k, None)}
+        for k in body.model_fields_set
+        if k not in _SKIP and getattr(old, k, None) != getattr(device, k, None)
+    }
+    if "tags" in body.model_fields_set and (old.tags or []) != (device.tags or []):
+        changes["tags"] = {"old": sorted(old.tags or []), "new": sorted(device.tags or [])}
+    await log_action(db, _user.sub, _user.name or _user.email, "UPDATE", "device",
+                     device_id, device.device_name or device_id, changes=changes or None)
+    return device
 
 
 @router.delete("/{device_id}", status_code=204)
 async def delete_device(
     device_id: str,
     _user: TokenPayload = Depends(require_permission("devices", "delete")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
     service.delete_device(device_id)
+    await log_action(db, _user.sub, _user.name or _user.email, "DELETE", "device",
+                     device_id, device_id)
 
 
 @router.get("/{device_id}/alerts", response_model=DeviceAlertsResponse)
@@ -100,16 +123,16 @@ async def list_device_notes(
 ):
     """List all notes for a device."""
     db = get_firestore()
-    docs = db.collection(NOTES_COLLECTION).where("device_id", "==", device_id).order_by("created_at").stream()
+    docs = db.collection(NOTES_COLLECTION).where("device_id", "==", device_id).stream()
     notes = []
     for doc in docs:
         note = doc.to_dict()
         note["id"] = doc.id
-        # Convert Firestore Timestamps to ISO strings
         for f in ("created_at", "updated_at"):
             if hasattr(note.get(f), "isoformat"):
                 note[f] = note[f].isoformat()
         notes.append(note)
+    notes.sort(key=lambda n: n.get("created_at") or "", reverse=False)
     return {"notes": notes, "total": len(notes)}
 
 
@@ -251,6 +274,7 @@ async def assign_device_to_customer(
     device_id: str,
     body: AssignCustomerBody,
     _user: TokenPayload = Depends(require_permission("devices", "edit")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
     """Assign a device to a customer.
 
@@ -290,6 +314,9 @@ async def assign_device_to_customer(
         })
         customer_ref.update({"owned_items": owned_items})
 
+    await log_action(db, _user.sub, _user.name or _user.email, "UPDATE", "device",
+                     device_id, device_id, meta={"action_detail": "assigned_to_customer",
+                                                  "customer_id": body.customer_id})
     return {"status": "assigned", "device_id": device_id, "customer_id": body.customer_id}
 
 
@@ -298,6 +325,7 @@ async def unassign_device_from_customer(
     device_id: str,
     customer_id: str = Query(...),
     _user: TokenPayload = Depends(require_permission("devices", "edit")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
     """Remove device assignment from a customer."""
     db = get_firestore()
@@ -317,6 +345,10 @@ async def unassign_device_from_customer(
         ]
         customer_ref.update({"owned_items": owned_items})
 
+    await log_action(db, _user.sub, _user.name or _user.email, "UPDATE", "device",
+                     device_id, device_id, meta={"action_detail": "unassigned_from_customer",
+                                                  "customer_id": customer_id})
+
 
 # ─────────────────────────────────────────────────────────────────────────────
 # Customer detail (for Owner display in fleet)
@@ -402,6 +434,7 @@ async def add_user_to_device(
     device_id: str,
     body: AddUserBody,
     _user: TokenPayload = Depends(require_permission("devices", "edit")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
     """Add a user reference to the device's user_list field."""
     db = get_firestore()
@@ -432,6 +465,9 @@ async def add_user_to_device(
         user_list.append(user_ref)
         device_ref.update({"user_list": user_list})
 
+    await log_action(db, _user.sub, _user.name or _user.email, "UPDATE", "device",
+                     device_id, device_id, meta={"action_detail": "user_added",
+                                                  "user_id": body.user_id})
     return {"status": "added", "user_id": body.user_id}
 
 
@@ -440,6 +476,7 @@ async def remove_user_from_device(
     device_id: str,
     user_id: str,
     _user: TokenPayload = Depends(require_permission("devices", "edit")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
     """Remove a user reference from the device's user_list field."""
     db = get_firestore()
@@ -451,11 +488,20 @@ async def remove_user_from_device(
     data = device_doc.to_dict() or {}
     user_list = data.get("user_list", []) or []
 
-    # Remove any entry that resolves to this user_id
+    from google.cloud.firestore_v1 import DocumentReference as DocRef
-    new_list = [
+
-        entry for entry in user_list
+    def resolves_to(entry, uid: str) -> bool:
-        if not (isinstance(entry, str) and entry.split("/")[-1] == user_id)
+        if isinstance(entry, DocRef):
-    ]
+            return entry.id == uid
+        if isinstance(entry, str):
+            return entry.split("/")[-1] == uid
+        return False
+
+    # Remove any entry that resolves to this user_id (handles both DocRef and string paths)
+    new_list = [entry for entry in user_list if not resolves_to(entry, user_id)]
     device_ref.update({"user_list": new_list})
 
+    await log_action(db, _user.sub, _user.name or _user.email, "UPDATE", "device",
+                     device_id, device_id, meta={"action_detail": "user_removed",
+                                                  "user_id": user_id})
     return {"status": "removed", "user_id": user_id}

backend/firmware/router.py

@@ -3,11 +3,14 @@ from fastapi.responses import FileResponse, PlainTextResponse
 from pydantic import BaseModel
 from typing import Optional
 import logging
+from sqlalchemy.ext.asyncio import AsyncSession
 
 from auth.models import TokenPayload
 from auth.dependencies import require_permission
 from firmware.models import FirmwareVersion, FirmwareListResponse, FirmwareMetadataResponse, UpdateType
 from firmware import service
+from database.postgres import get_pg_session
+from shared.audit import log_action
 
 logger = logging.getLogger(__name__)
 
@@ -27,9 +30,10 @@ async def upload_firmware(
     bespoke_uid: Optional[str] = Form(None),
     file: UploadFile = File(...),
     _user: TokenPayload = Depends(require_permission("manufacturing", "add")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
     file_bytes = await file.read()
-    return service.upload_firmware(
+    fw = service.upload_firmware(
         hw_type=hw_type,
         channel=channel,
         version=version,
@@ -40,6 +44,9 @@ async def upload_firmware(
         release_note=release_note,
         bespoke_uid=bespoke_uid,
     )
+    await log_action(db, _user.sub, _user.name or _user.email, "CREATE", "firmware",
+                     fw.id, f"{hw_type} v{version} ({channel})")
+    return fw
 
 
 @router.get("", response_model=FirmwareListResponse)
@@ -108,9 +115,10 @@ async def edit_firmware(
     bespoke_uid: Optional[str] = Form(None),
     file: Optional[UploadFile] = File(None),
     _user: TokenPayload = Depends(require_permission("manufacturing", "add")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
     file_bytes = await file.read() if file and file.filename else None
-    return service.edit_firmware(
+    fw = service.edit_firmware(
         doc_id=firmware_id,
         channel=channel,
         version=version,
@@ -121,14 +129,22 @@ async def edit_firmware(
         bespoke_uid=bespoke_uid,
         file_bytes=file_bytes,
     )
+    await log_action(db, _user.sub, _user.name or _user.email, "UPDATE", "firmware",
+                     firmware_id, f"{fw.hw_type} v{fw.version} ({fw.channel})" if fw else firmware_id)
+    return fw
 
 
 @router.delete("/{firmware_id}", status_code=204)
-def delete_firmware(
+async def delete_firmware(
     firmware_id: str,
     _user: TokenPayload = Depends(require_permission("manufacturing", "delete")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
+    fw = service.get_firmware(firmware_id) if hasattr(service, "get_firmware") else None
     service.delete_firmware(firmware_id)
+    label = f"{fw.hw_type} v{fw.version} ({fw.channel})" if fw else firmware_id
+    await log_action(db, _user.sub, _user.name or _user.email, "DELETE", "firmware",
+                     firmware_id, label)
 
 
 # ─────────────────────────────────────────────────────────────────────────────

backend/main.py

@@ -25,6 +25,10 @@ from crm.media_router import router as crm_media_router
 from crm.nextcloud_router import router as crm_nextcloud_router
 from crm.quotations_router import router as crm_quotations_router
 from public.router import router as public_router
+from notes.router import router as notes_router
+from tickets.router import router as tickets_router
+from audit.router import router as audit_router
+from search.router import router as search_router
 from crm.nextcloud import close_client as close_nextcloud_client, keepalive_ping as nextcloud_keepalive
 from crm.mail_accounts import get_mail_accounts
 from mqtt.client import mqtt_manager
@@ -70,6 +74,10 @@ app.include_router(crm_media_router)
 app.include_router(crm_nextcloud_router)
 app.include_router(crm_quotations_router)
 app.include_router(public_router)
+app.include_router(notes_router)
+app.include_router(tickets_router)
+app.include_router(audit_router)
+app.include_router(search_router)
 
 
 async def nextcloud_keepalive_loop():
@@ -102,9 +110,11 @@ async def crm_poll_loop():
 @app.on_event("startup")
 async def startup():
     init_firebase()
-    await db.init_db()
+    from database.core import init_db as sqlite_init_db
+    await sqlite_init_db()
     await melody_service.migrate_from_firestore()
     mqtt_manager.start(asyncio.get_event_loop())
+    asyncio.create_task(db.partition_manager_loop())
     asyncio.create_task(db.purge_loop())
     asyncio.create_task(nextcloud_keepalive_loop())
     asyncio.create_task(crm_poll_loop())
@@ -119,7 +129,8 @@ async def startup():
 @app.on_event("shutdown")
 async def shutdown():
     mqtt_manager.stop()
-    await db.close_db()
+    from database.core import close_db as sqlite_close_db
+    await sqlite_close_db()
     await close_nextcloud_client()
 
 

backend/manufacturing/orm.py

@@ -0,0 +1,22 @@
+from datetime import datetime, timezone
+from sqlalchemy import BigInteger, Column, DateTime, Index, String, Text
+from database.postgres import Base
+
+
+def _now():
+    return datetime.now(timezone.utc)
+
+
+class MfgAuditLog(Base):
+    __tablename__ = "mfg_audit_log"
+    __table_args__ = (
+        Index("idx_mfg_audit_time", "timestamp"),
+        Index("idx_mfg_audit_action", "action"),
+    )
+
+    id            = Column(BigInteger, primary_key=True, autoincrement=True)
+    timestamp     = Column(DateTime(timezone=True), nullable=False, default=_now)
+    admin_user    = Column(String(256), nullable=False)
+    action        = Column(String(128), nullable=False)
+    serial_number = Column(String(128))
+    detail        = Column(Text)

backend/manufacturing/router.py

@@ -3,6 +3,7 @@ from fastapi.responses import Response
 from fastapi.responses import RedirectResponse
 from typing import Optional
 from pydantic import BaseModel
+from sqlalchemy.ext.asyncio import AsyncSession
 
 from auth.models import TokenPayload
 from auth.dependencies import require_permission
@@ -13,9 +14,10 @@ from manufacturing.models import (
     ManufacturingStats,
 )
 from manufacturing import service
-from manufacturing import audit
+from shared.audit import log_action
 from shared.exceptions import NotFoundError
 from shared.firebase import get_db as get_firestore
+from database.postgres import get_pg_session
 
 
 class LifecycleEntryPatch(BaseModel):
@@ -43,26 +45,21 @@ def get_stats(
     return service.get_stats()
 
 
-@router.get("/audit-log")
-async def get_audit_log(
-    limit: int = Query(20, ge=1, le=100),
-    _user: TokenPayload = Depends(require_permission("manufacturing", "view")),
-):
-    entries = await audit.get_recent(limit=limit)
-    return {"entries": entries}
-
 
 @router.post("/batch", response_model=BatchResponse, status_code=201)
 async def create_batch(
     body: BatchCreate,
     user: TokenPayload = Depends(require_permission("manufacturing", "add")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
     result = service.create_batch(body)
-    await audit.log_action(
+    await log_action(
-        admin_user=user.email,
+        db, user.sub, user.email,
-        action="batch_created",
+        action="CREATE",
-        detail={
+        entity_type="device_batch",
-            "batch_id": result.batch_id,
+        entity_id=result.batch_id,
+        entity_label=f"Batch {result.batch_id} ({result.board_type}, qty {len(result.serial_numbers)})",
+        meta={
             "board_type": result.board_type,
             "board_version": result.board_version,
             "quantity": len(result.serial_numbers),
@@ -137,6 +134,7 @@ async def update_status(
     sn: str,
     body: DeviceStatusUpdate,
     user: TokenPayload = Depends(require_permission("manufacturing", "edit")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
     # Guard: claimed requires at least one user in user_list
     # (allow if explicitly force_claimed=true, which the mfg UI sets after adding a user manually)
@@ -167,11 +165,13 @@ async def update_status(
                 )
 
     result = service.update_device_status(sn, body, set_by=user.email)
-    await audit.log_action(
+    await log_action(
-        admin_user=user.email,
+        db, user.sub, user.email,
-        action="status_updated",
+        action="STATUS_CHANGE",
-        serial_number=sn,
+        entity_type="device",
-        detail={"status": body.status.value, "note": body.note},
+        entity_id=sn,
+        entity_label=sn,
+        meta={"status": body.status.value, "note": body.note},
     )
     return result
 
@@ -181,10 +181,11 @@ async def patch_lifecycle_entry(
     sn: str,
     body: LifecycleEntryPatch,
     user: TokenPayload = Depends(require_permission("manufacturing", "edit")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
     """Edit the date and/or note of a lifecycle history entry by index."""
-    db = get_firestore()
+    fs = get_firestore()
-    docs = list(db.collection("devices").where("serial_number", "==", sn).limit(1).stream())
+    docs = list(fs.collection("devices").where("serial_number", "==", sn).limit(1).stream())
     if not docs:
         raise HTTPException(status_code=404, detail="Device not found")
     doc_ref = docs[0].reference
@@ -198,7 +199,16 @@ async def patch_lifecycle_entry(
         history[body.index]["note"] = body.note
     doc_ref.update({"lifecycle_history": history})
     from manufacturing.service import _doc_to_inventory_item
-    return _doc_to_inventory_item(doc_ref.get())
+    result = _doc_to_inventory_item(doc_ref.get())
+    await log_action(
+        db, user.sub, user.email,
+        action="UPDATE",
+        entity_type="device",
+        entity_id=sn,
+        entity_label=sn,
+        meta={"lifecycle_index": body.index, "date": body.date, "note": body.note},
+    )
+    return result
 
 
 @router.post("/devices/{sn}/lifecycle", response_model=DeviceInventoryItem, status_code=200)
@@ -206,6 +216,7 @@ async def create_lifecycle_entry(
     sn: str,
     body: LifecycleEntryCreate,
     user: TokenPayload = Depends(require_permission("manufacturing", "edit")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
     """Upsert a lifecycle history entry for the given status_id.
 
@@ -214,8 +225,8 @@ async def create_lifecycle_entry(
     a status is visited more than once (max one entry per status).
     """
     from datetime import datetime, timezone
-    db = get_firestore()
+    fs = get_firestore()
-    docs = list(db.collection("devices").where("serial_number", "==", sn).limit(1).stream())
+    docs = list(fs.collection("devices").where("serial_number", "==", sn).limit(1).stream())
     if not docs:
         raise HTTPException(status_code=404, detail="Device not found")
     doc_ref = docs[0].reference
@@ -229,19 +240,28 @@ async def create_lifecycle_entry(
         "set_by": user.email,
     }
 
-    # Overwrite existing entry for this status if present, else append
     existing_idx = next(
         (i for i, e in enumerate(history) if e.get("status_id") == body.status_id),
         None,
     )
-    if existing_idx is not None:
+    is_update = existing_idx is not None
+    if is_update:
         history[existing_idx] = new_entry
     else:
         history.append(new_entry)
 
     doc_ref.update({"lifecycle_history": history})
     from manufacturing.service import _doc_to_inventory_item
-    return _doc_to_inventory_item(doc_ref.get())
+    result = _doc_to_inventory_item(doc_ref.get())
+    await log_action(
+        db, user.sub, user.email,
+        action="UPDATE" if is_update else "CREATE",
+        entity_type="device",
+        entity_id=sn,
+        entity_label=sn,
+        meta={"lifecycle_status": body.status_id, "date": new_entry["date"], "note": body.note},
+    )
+    return result
 
 
 @router.delete("/devices/{sn}/lifecycle/{index}", response_model=DeviceInventoryItem)
@@ -249,10 +269,11 @@ async def delete_lifecycle_entry(
     sn: str,
     index: int,
     user: TokenPayload = Depends(require_permission("manufacturing", "edit")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
     """Delete a lifecycle history entry by index. Cannot delete the entry for the current status."""
-    db = get_firestore()
+    fs = get_firestore()
-    docs = list(db.collection("devices").where("serial_number", "==", sn).limit(1).stream())
+    docs = list(fs.collection("devices").where("serial_number", "==", sn).limit(1).stream())
     if not docs:
         raise HTTPException(status_code=404, detail="Device not found")
     doc_ref = docs[0].reference
@@ -261,12 +282,22 @@ async def delete_lifecycle_entry(
     if index < 0 or index >= len(history):
         raise HTTPException(status_code=400, detail="Invalid lifecycle entry index")
     current_status = data.get("mfg_status", "")
-    if history[index].get("status_id") == current_status:
+    deleted_entry = history[index]
+    if deleted_entry.get("status_id") == current_status:
         raise HTTPException(status_code=400, detail="Cannot delete the entry for the current status. Change the status first.")
     history.pop(index)
     doc_ref.update({"lifecycle_history": history})
     from manufacturing.service import _doc_to_inventory_item
-    return _doc_to_inventory_item(doc_ref.get())
+    result = _doc_to_inventory_item(doc_ref.get())
+    await log_action(
+        db, user.sub, user.email,
+        action="DELETE",
+        entity_type="device",
+        entity_id=sn,
+        entity_label=sn,
+        meta={"lifecycle_status": deleted_entry.get("status_id"), "index": index},
+    )
+    return result
 
 
 @router.get("/devices/{sn}/nvs.bin")
@@ -274,13 +305,18 @@ async def download_nvs(
     sn: str,
     hw_type_override: Optional[str] = Query(None, description="Override hw_type written to NVS (for bespoke firmware)"),
     hw_revision_override: Optional[str] = Query(None, description="Override hw_revision written to NVS (for bespoke firmware)"),
+    nvs_schema: Optional[str] = Query(None, description="NVS schema to use: 'legacy' or 'new' (default)"),
     user: TokenPayload = Depends(require_permission("manufacturing", "view")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
-    binary = service.get_nvs_binary(sn, hw_type_override=hw_type_override, hw_revision_override=hw_revision_override)
+    binary = service.get_nvs_binary(sn, hw_type_override=hw_type_override, hw_revision_override=hw_revision_override, legacy=(nvs_schema == "legacy"))
-    await audit.log_action(
+    await log_action(
-        admin_user=user.email,
+        db, user.sub, user.email,
-        action="device_flashed",
+        action="COMMAND",
-        serial_number=sn,
+        entity_type="device",
+        entity_id=sn,
+        entity_label=sn,
+        meta={"command": "nvs_flash", "hw_type_override": hw_type_override, "nvs_schema": nvs_schema or "new"},
     )
     return Response(
         content=binary,
@@ -294,16 +330,19 @@ async def assign_device(
     sn: str,
     body: DeviceAssign,
     user: TokenPayload = Depends(require_permission("manufacturing", "edit")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
     try:
         result = service.assign_device(sn, body)
     except NotFoundError as e:
         raise HTTPException(status_code=404, detail=str(e))
-    await audit.log_action(
+    await log_action(
-        admin_user=user.email,
+        db, user.sub, user.email,
-        action="device_assigned",
+        action="UPDATE",
-        serial_number=sn,
+        entity_type="device",
-        detail={"customer_id": body.customer_id},
+        entity_id=sn,
+        entity_label=sn,
+        meta={"customer_id": body.customer_id},
     )
     return result
 
@@ -313,6 +352,7 @@ async def delete_device(
     sn: str,
     force: bool = Query(False, description="Required to delete sold/claimed devices"),
     user: TokenPayload = Depends(require_permission("manufacturing", "delete")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
     """Delete a device. Sold/claimed devices require force=true."""
     try:
@@ -321,11 +361,13 @@ async def delete_device(
         raise HTTPException(status_code=404, detail="Device not found")
     except PermissionError as e:
         raise HTTPException(status_code=403, detail=str(e))
-    await audit.log_action(
+    await log_action(
-        admin_user=user.email,
+        db, user.sub, user.email,
-        action="device_deleted",
+        action="DELETE",
-        serial_number=sn,
+        entity_type="device",
-        detail={"force": force},
+        entity_id=sn,
+        entity_label=sn,
+        meta={"force": force},
     )
 
 
@@ -333,6 +375,7 @@ async def delete_device(
 async def send_manufactured_email(
     sn: str,
     user: TokenPayload = Depends(require_permission("manufacturing", "edit")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
     """Send the 'device manufactured' notification to the assigned customer's email."""
     db = get_firestore()
@@ -360,11 +403,13 @@ async def send_manufactured_email(
         device_name=hw_family.replace("_", " ").title(),
         customer_name=customer_name,
     )
-    await audit.log_action(
+    await log_action(
-        admin_user=user.email,
+        db, user.sub, user.email,
-        action="email_manufactured_sent",
+        action="COMMAND",
-        serial_number=sn,
+        entity_type="device",
-        detail={"recipient": email},
+        entity_id=sn,
+        entity_label=sn,
+        meta={"command": "email_manufactured", "recipient": email},
     )
 
 
@@ -372,6 +417,7 @@ async def send_manufactured_email(
 async def send_assigned_email(
     sn: str,
     user: TokenPayload = Depends(require_permission("manufacturing", "edit")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
     """Send the 'device assigned / app instructions' email to the assigned user(s)."""
     db = get_firestore()
@@ -406,24 +452,30 @@ async def send_assigned_email(
             errors.append(str(exc))
     if errors:
         raise HTTPException(status_code=500, detail=f"Some emails failed: {'; '.join(errors)}")
-    await audit.log_action(
+    await log_action(
-        admin_user=user.email,
+        db, user.sub, user.email,
-        action="email_assigned_sent",
+        action="COMMAND",
-        serial_number=sn,
+        entity_type="device",
-        detail={"user_count": len(user_list)},
+        entity_id=sn,
+        entity_label=sn,
+        meta={"command": "email_assigned", "user_count": len(user_list)},
     )
 
 
 @router.delete("/devices", status_code=200)
 async def delete_unprovisioned(
     user: TokenPayload = Depends(require_permission("manufacturing", "delete")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
     """Delete all devices with status 'manufactured' (never provisioned)."""
     deleted = service.delete_unprovisioned_devices()
-    await audit.log_action(
+    await log_action(
-        admin_user=user.email,
+        db, user.sub, user.email,
-        action="bulk_delete_unprovisioned",
+        action="DELETE",
-        detail={"count": len(deleted), "serial_numbers": deleted},
+        entity_type="device_batch",
+        entity_id="bulk_unprovisioned",
+        entity_label=f"Bulk delete unprovisioned ({len(deleted)} devices)",
+        meta={"count": len(deleted), "serial_numbers": deleted},
     )
     return {"deleted": deleted, "count": len(deleted)}
 
@@ -465,6 +517,7 @@ async def delete_flash_asset(
     hw_type: str,
     asset: str,
     user: TokenPayload = Depends(require_permission("manufacturing", "delete")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
     """Delete a single flash asset file (bootloader.bin or partitions.bin)."""
     if asset not in VALID_FLASH_ASSETS:
@@ -473,10 +526,13 @@ async def delete_flash_asset(
         service.delete_flash_asset(hw_type, asset)
     except NotFoundError as e:
         raise HTTPException(status_code=404, detail=str(e))
-    await audit.log_action(
+    await log_action(
-        admin_user=user.email,
+        db, user.sub, user.email,
-        action="flash_asset_deleted",
+        action="DELETE",
-        detail={"hw_type": hw_type, "asset": asset},
+        entity_type="firmware",
+        entity_id=f"{hw_type}/{asset}",
+        entity_label=f"{hw_type} / {asset}",
+        meta={"hw_type": hw_type, "asset": asset},
     )
 
 
@@ -488,7 +544,8 @@ class FlashAssetNoteBody(BaseModel):
 async def set_flash_asset_note(
     hw_type: str,
     body: FlashAssetNoteBody,
-    _user: TokenPayload = Depends(require_permission("manufacturing", "edit")),
+    user: TokenPayload = Depends(require_permission("manufacturing", "edit")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
     """Save (or overwrite) the note for a hw_type's flash asset set.
 
@@ -496,6 +553,14 @@ async def set_flash_asset_note(
     Pass an empty string to clear the note.
     """
     service.set_flash_asset_note(hw_type, body.note)
+    await log_action(
+        db, user.sub, user.email,
+        action="UPDATE",
+        entity_type="firmware",
+        entity_id=hw_type,
+        entity_label=hw_type,
+        meta={"note": body.note},
+    )
 
 
 @router.post("/flash-assets/{hw_type}/{asset}", status_code=204)
@@ -503,7 +568,8 @@ async def upload_flash_asset(
     hw_type: str,
     asset: str,
     file: UploadFile = File(...),
-    _user: TokenPayload = Depends(require_permission("manufacturing", "add")),
+    user: TokenPayload = Depends(require_permission("manufacturing", "add")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
     """Upload a bootloader.bin or partitions.bin for a given hw_type.
 
@@ -511,13 +577,20 @@ async def upload_flash_asset(
     and .pio/build/{env}/partitions.bin). Upload them once per hw_type after
     each PlatformIO build that changes the partition layout.
     """
-    # hw_type can be a standard board type OR a bespoke UID (any non-empty slug)
     if not hw_type or len(hw_type) > 128:
         raise HTTPException(status_code=400, detail="Invalid hw_type/bespoke UID.")
     if asset not in VALID_FLASH_ASSETS:
         raise HTTPException(status_code=400, detail=f"Invalid asset. Must be one of: {', '.join(sorted(VALID_FLASH_ASSETS))}")
     data = await file.read()
     service.save_flash_asset(hw_type, asset, data)
+    await log_action(
+        db, user.sub, user.email,
+        action="CREATE",
+        entity_type="firmware",
+        entity_id=f"{hw_type}/{asset}",
+        entity_label=f"{hw_type} / {asset}",
+        meta={"hw_type": hw_type, "asset": asset, "size_bytes": len(data)},
+    )
 
 
 @router.get("/devices/{sn}/bootloader.bin")

backend/manufacturing/service.py

@@ -197,12 +197,13 @@ def update_device_status(sn: str, data: DeviceStatusUpdate, set_by: str | None =
     return _doc_to_inventory_item(doc_ref.get())
 
 
-def get_nvs_binary(sn: str, hw_type_override: str | None = None, hw_revision_override: str | None = None) -> bytes:
+def get_nvs_binary(sn: str, hw_type_override: str | None = None, hw_revision_override: str | None = None, legacy: bool = False) -> bytes:
     item = get_device_by_sn(sn)
     return generate_nvs_binary(
         serial_number=item.serial_number,
         hw_family=hw_type_override if hw_type_override else item.hw_type,
         hw_revision=hw_revision_override if hw_revision_override else item.hw_version,
+        legacy=legacy,
     )
 
 

backend/melodies/orm.py

@@ -0,0 +1,39 @@
+from datetime import datetime, timezone
+from sqlalchemy import Boolean, Column, DateTime, Index, String, Text
+from sqlalchemy.dialects.postgresql import JSONB
+from database.postgres import Base
+
+
+def _now():
+    return datetime.now(timezone.utc)
+
+
+class MelodyDraft(Base):
+    __tablename__ = "melody_drafts"
+    __table_args__ = (
+        Index("idx_melody_drafts_status", "status"),
+    )
+
+    id         = Column(String(128), primary_key=True)
+    status     = Column(String(32), nullable=False, default="draft")
+    # 'data' stores the full melody definition as JSON (was TEXT/JSON in SQLite)
+    data       = Column(JSONB, nullable=False)
+    created_at = Column(DateTime(timezone=True), nullable=False, default=_now)
+    updated_at = Column(DateTime(timezone=True), nullable=False, default=_now, onupdate=_now)
+
+
+class BuiltMelody(Base):
+    __tablename__ = "built_melodies"
+
+    id                  = Column(String(128), primary_key=True)
+    name                = Column(String(500), nullable=False)
+    pid                 = Column(String(128), nullable=False)
+    # 'steps' is a JSON array of step definitions
+    steps               = Column(JSONB, nullable=False)
+    binary_path         = Column(String(1000))
+    progmem_code        = Column(Text)
+    # JSON array of melody IDs this built melody is assigned to
+    assigned_melody_ids = Column(JSONB, nullable=False, default=list)
+    is_builtin          = Column(Boolean, nullable=False, default=False)
+    created_at          = Column(DateTime(timezone=True), nullable=False, default=_now)
+    updated_at          = Column(DateTime(timezone=True), nullable=False, default=_now, onupdate=_now)

backend/melodies/router.py

@@ -1,15 +1,35 @@
 from fastapi import APIRouter, Depends, UploadFile, File, Query, HTTPException, Response
+from fastapi.responses import FileResponse
 from typing import Optional
+from sqlalchemy.ext.asyncio import AsyncSession
 from auth.models import TokenPayload
 from auth.dependencies import require_permission
 from melodies.models import (
     MelodyCreate, MelodyUpdate, MelodyInDB, MelodyListResponse, MelodyInfo,
 )
 from melodies import service
+from database.postgres import get_pg_session
+from shared.audit import log_action
 
 router = APIRouter(prefix="/api/melodies", tags=["melodies"])
 
 
+@router.get("/download/{pid}")
+async def download_binary_by_pid(pid: str):
+    """Download a melody's .bsm binary by PID over plain HTTP.
+
+    No auth — ESP32 devices call this directly and can't afford a TLS client's
+    memory footprint, so this is served from a plain-HTTP-only host
+    (melodies.bellsystems.net) rather than the HTTPS console domain.
+    """
+    path = await service.get_binary_path_by_pid(pid)
+    return FileResponse(
+        path=str(path),
+        media_type="application/octet-stream",
+        filename=f"{pid}.bsm",
+    )
+
+
 @router.get("", response_model=MelodyListResponse)
 async def list_melodies(
     search: Optional[str] = Query(None),
@@ -42,8 +62,12 @@ async def create_melody(
     body: MelodyCreate,
     publish: bool = Query(False),
     _user: TokenPayload = Depends(require_permission("melodies", "add")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
-    return await service.create_melody(body, publish=publish, actor_name=_user.name)
+    melody = await service.create_melody(body, publish=publish, actor_name=_user.name)
+    await log_action(db, _user.sub, _user.name or _user.email, "CREATE", "melody",
+                     melody.id, melody.information.name if melody.information else melody.id)
+    return melody
 
 
 @router.put("/{melody_id}", response_model=MelodyInDB)
@@ -51,32 +75,61 @@ async def update_melody(
     melody_id: str,
     body: MelodyUpdate,
     _user: TokenPayload = Depends(require_permission("melodies", "edit")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
-    return await service.update_melody(melody_id, body, actor_name=_user.name)
+    old = await service.get_melody(melody_id)
+    melody = await service.update_melody(melody_id, body, actor_name=_user.name)
+    _SKIP = {"updated_at", "id", "metadata", "information", "noteAssignments"}
+    changes = {
+        k: {"old": getattr(old, k, None), "new": getattr(melody, k, None)}
+        for k in body.model_fields_set
+        if k not in _SKIP and getattr(old, k, None) != getattr(melody, k, None)
+    }
+    # Surface the name change from inside the information sub-object
+    old_name = old.information.name if old.information else None
+    new_name = melody.information.name if melody.information else None
+    if old_name != new_name:
+        changes["name"] = {"old": old_name, "new": new_name}
+    await log_action(db, _user.sub, _user.name or _user.email, "UPDATE", "melody",
+                     melody_id, new_name or melody_id, changes=changes or None)
+    return melody
 
 
 @router.delete("/{melody_id}", status_code=204)
 async def delete_melody(
     melody_id: str,
     _user: TokenPayload = Depends(require_permission("melodies", "delete")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
+    melody = await service.get_melody(melody_id)
+    label = melody.information.name if melody.information else melody_id
     await service.delete_melody(melody_id)
+    await log_action(db, _user.sub, _user.name or _user.email, "DELETE", "melody",
+                     melody_id, label)
 
 
 @router.post("/{melody_id}/publish", response_model=MelodyInDB)
 async def publish_melody(
     melody_id: str,
     _user: TokenPayload = Depends(require_permission("melodies", "edit")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
-    return await service.publish_melody(melody_id)
+    melody = await service.publish_melody(melody_id)
+    await log_action(db, _user.sub, _user.name or _user.email, "PUBLISH", "melody",
+                     melody_id, melody.information.name if melody.information else melody_id)
+    return melody
 
 
 @router.post("/{melody_id}/unpublish", response_model=MelodyInDB)
 async def unpublish_melody(
     melody_id: str,
     _user: TokenPayload = Depends(require_permission("melodies", "edit")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
-    return await service.unpublish_melody(melody_id)
+    melody = await service.unpublish_melody(melody_id)
+    await log_action(db, _user.sub, _user.name or _user.email, "UNPUBLISH", "melody",
+                     melody_id, melody.information.name if melody.information else melody_id)
+    return melody
 
 
 @router.post("/{melody_id}/upload/{file_type}")
@@ -98,7 +151,13 @@ async def upload_file(
 
     if file_type == "binary":
         content_type = "application/octet-stream"
-
+        if not melody.pid:
+            raise HTTPException(status_code=400, detail="Melody must have a PID before uploading a binary")
+        url = service.save_binary_for_melody(
+            pid=melody.pid,
+            file_bytes=contents,
+        )
+    else:
         url = service.upload_file_for_melody(
             melody_id=melody_id,
             melody_uid=melody.uid,
@@ -133,7 +192,7 @@ async def delete_file(
         raise HTTPException(status_code=400, detail="file_type must be 'binary' or 'preview'")
 
     melody = await service.get_melody(melody_id)
-    service.delete_file(melody_id, file_type, melody.uid)
+    await service.delete_file(melody_id, file_type, melody.uid, melody.pid)
 
 
 @router.get("/{melody_id}/files")
@@ -143,7 +202,7 @@ async def get_files(
 ):
     """Get storage file URLs for a melody."""
     melody = await service.get_melody(melody_id)
-    return service.get_storage_files(melody_id, melody.uid)
+    return service.get_storage_files(melody_id, melody.uid, melody.pid)
 
 
 @router.patch("/{melody_id}/set-outdated", response_model=MelodyInDB)
@@ -170,7 +229,7 @@ async def download_binary_file(
 ):
     """Download current melody binary with a PID-based filename."""
     melody = await service.get_melody(melody_id)
-    file_bytes, content_type = service.get_binary_file_bytes(melody_id, melody.uid)
+    file_bytes, content_type = service.get_binary_file_bytes(melody_id, melody.pid)
     filename = f"{(melody.pid or 'binary')}.bsm"
     headers = {"Content-Disposition": f'attachment; filename="{filename}"'}
     return Response(content=file_bytes, media_type=content_type, headers=headers)

backend/melodies/service.py

@@ -2,15 +2,39 @@ import json
 import uuid
 import logging
 from datetime import datetime
+from pathlib import Path
+
+from fastapi import HTTPException
 
 from shared.firebase import get_db as get_firestore, get_bucket
 from shared.exceptions import NotFoundError
 from melodies.models import MelodyCreate, MelodyUpdate, MelodyInDB
 from melodies import database as melody_db
+from config import settings
 
 COLLECTION = "melodies"
 logger = logging.getLogger("melodies.service")
 
+# Local disk storage for melody .bsm binaries — served over plain HTTP for ESP32 compatibility.
+# The audio preview file still goes to Firebase Storage (only ever fetched by the browser admin UI).
+BINARY_STORAGE_DIR = Path(settings.melody_binaries_storage_path)
+
+
+def _ensure_binary_storage_dir():
+    BINARY_STORAGE_DIR.mkdir(parents=True, exist_ok=True)
+
+
+def _binary_file_path(pid: str) -> Path:
+    """Path to the local .bsm file for a given archetype PID.
+
+    Keyed on pid, not melody uid: pid identifies the underlying archetype binary
+    (raw note sequence). Multiple melodies can legitimately share one pid — each
+    melody remaps those notes to actual bells via its own noteAssignments/speed/
+    duration settings — so several melodies writing/reading the same {pid}.bsm
+    file is expected, not a collision.
+    """
+    return BINARY_STORAGE_DIR / f"{pid}.bsm"
+
 
 def _parse_localized_string(value: str) -> dict:
     """Parse a JSON-encoded localized string into a dict. Returns {} on failure."""
@@ -232,12 +256,50 @@ async def delete_melody(melody_id: str) -> None:
             doc_ref.delete()
 
     # Delete storage files
-    _delete_storage_files(melody_id, row["data"].get("uid"))
+    await _delete_storage_files(melody_id, row["data"].get("uid"), row["data"].get("pid"))
 
     # Delete from SQLite
     await melody_db.delete_melody(melody_id)
 
 
+def save_binary_for_melody(pid: str, file_bytes: bytes) -> str:
+    """Save an archetype binary to local disk under its pid, replacing any previous
+    file for that pid. Multiple melodies can share one pid (they remap the same
+    underlying note sequence via their own noteAssignments/speed/duration), so
+    writing the same {pid}.bsm from a different melody is expected — not a collision.
+
+    Returns the plain-HTTP download URL devices use (served from melody_download_base_url,
+    not Firebase — ESP32 devices can't afford the RAM for a TLS client).
+    """
+    _ensure_binary_storage_dir()
+    path = _binary_file_path(pid)
+    path.write_bytes(file_bytes)
+    return f"{settings.melody_download_base_url}/{pid}"
+
+
+async def get_binary_path_by_pid(pid: str) -> Path:
+    """Resolve a pid to its local .bsm file path. Used by the unauthenticated
+    device-facing download route."""
+    path = _binary_file_path(pid)
+    if path.exists():
+        return path
+    raise NotFoundError("Binary file")
+
+
+def delete_local_binary(pid: str) -> None:
+    """Delete the local .bsm binary file for a pid, if present.
+
+    Only call this when no other melody still references this pid — since the
+    file may be shared by multiple melodies, deleting one melody shouldn't
+    reflexively delete a binary others still use.
+    """
+    if not pid:
+        return
+    path = _binary_file_path(pid)
+    if path.exists():
+        path.unlink()
+
+
 def upload_file(melody_id: str, file_bytes: bytes, filename: str, content_type: str) -> str:
     """Upload a file to Firebase Storage under melodies/{melody_id}/."""
     bucket = get_bucket()
@@ -334,32 +396,32 @@ def upload_file_for_melody(melody_id: str, melody_uid: str | None, melody_pid: s
     return blob.public_url
 
 
-def get_binary_file_bytes(melody_id: str, melody_uid: str | None = None) -> tuple[bytes, str]:
+async def _pid_used_by_other_melody(pid: str, exclude_melody_id: str) -> bool:
-    """Fetch current binary bytes for a melody from Firebase Storage."""
+    """Check whether any melody other than exclude_melody_id still references this pid."""
-    bucket = get_bucket()
+    if not pid:
-    if not bucket:
+        return False
-        raise RuntimeError("Firebase Storage not initialized")
+    rows = await melody_db.list_melodies()
+    return any(
+        row["id"] != exclude_melody_id and row["data"].get("pid") == pid
+        for row in rows
+    )
 
-    prefixes = _storage_prefixes(melody_id, melody_uid)
+
-    blobs = [b for b in _list_blobs_for_prefixes(bucket, prefixes) if _is_binary_blob_name(b.name)]
+def get_binary_file_bytes(melody_id: str, melody_pid: str | None = None) -> tuple[bytes, str]:
-    if not blobs:
+    """Fetch current binary bytes for a melody from local disk storage."""
+    path = _binary_file_path(melody_pid) if melody_pid else None
+    if not path or not path.exists():
         raise NotFoundError("Binary file")
-
+    return path.read_bytes(), "application/octet-stream"
-    # Prefer explicit binary.* naming, then newest.
-    blobs.sort(
-        key=lambda b: (
-            0 if "binary" in b.name.rsplit("/", 1)[-1].lower() else 1,
-            -(int(b.time_created.timestamp()) if getattr(b, "time_created", None) else 0),
-        )
-    )
-    chosen = blobs[0]
-    data = chosen.download_as_bytes()
-    content_type = chosen.content_type or "application/octet-stream"
-    return data, content_type
 
 
-def delete_file(melody_id: str, file_type: str, melody_uid: str | None = None) -> None:
+async def delete_file(melody_id: str, file_type: str, melody_uid: str | None = None, melody_pid: str | None = None) -> None:
     """Delete a specific file from storage. file_type is 'binary' or 'preview'."""
+    if file_type == "binary":
+        if melody_pid and not await _pid_used_by_other_melody(melody_pid, melody_id):
+            delete_local_binary(melody_pid)
+        return
+
     bucket = get_bucket()
     if not bucket:
         return
@@ -368,14 +430,19 @@ def delete_file(melody_id: str, file_type: str, melody_uid: str | None = None) -
     blobs = _list_blobs_for_prefixes(bucket, prefixes)
 
     for blob in blobs:
-        if file_type == "binary" and "binary" in blob.name:
+        if file_type == "preview" and "preview" in blob.name:
-            blob.delete()
-        elif file_type == "preview" and "preview" in blob.name:
             blob.delete()
 
 
-def _delete_storage_files(melody_id: str, melody_uid: str | None = None) -> None:
+async def _delete_storage_files(melody_id: str, melody_uid: str | None = None, melody_pid: str | None = None) -> None:
-    """Delete all storage files for a melody."""
+    """Delete all storage files for a melody (local binary + Firebase preview).
+
+    The local binary is only deleted if no other melody still references the same
+    pid — multiple melodies can share one archetype binary by design.
+    """
+    if melody_pid and not await _pid_used_by_other_melody(melody_pid, melody_id):
+        delete_local_binary(melody_pid)
+
     bucket = get_bucket()
     if not bucket:
         return
@@ -383,24 +450,29 @@ def _delete_storage_files(melody_id: str, melody_uid: str | None = None) -> None
     prefixes = _storage_prefixes(melody_id, melody_uid)
     blobs = _list_blobs_for_prefixes(bucket, prefixes)
     for blob in blobs:
-        blob.delete()
-
-
-def get_storage_files(melody_id: str, melody_uid: str | None = None) -> dict:
-    """List storage files for a melody, returning URLs."""
-    bucket = get_bucket()
-    if not bucket:
-        return {"binary_url": None, "preview_url": None}
-
-    prefixes = _storage_prefixes(melody_id, melody_uid)
-    blobs = _list_blobs_for_prefixes(bucket, prefixes)
-
-    result = {"binary_url": None, "preview_url": None}
-    for blob in blobs:
-        blob.make_public()
         if _is_binary_blob_name(blob.name):
-            result["binary_url"] = blob.public_url
+            continue  # legacy Firebase binaries, if any, are no longer authoritative
-        elif "preview" in blob.name:
+        blob.delete()
+
+
+def get_storage_files(melody_id: str, melody_uid: str | None = None, melody_pid: str | None = None) -> dict:
+    """List storage files for a melody, returning URLs. Binary comes from local disk,
+    preview still comes from Firebase Storage."""
+    result = {"binary_url": None, "preview_url": None}
+
+    if melody_pid and _binary_file_path(melody_pid).exists():
+        result["binary_url"] = f"{settings.melody_download_base_url}/{melody_pid}"
+
+    bucket = get_bucket()
+    if not bucket:
+        return result
+
+    prefixes = _storage_prefixes(melody_id, melody_uid)
+    blobs = _list_blobs_for_prefixes(bucket, prefixes)
+
+    for blob in blobs:
+        if "preview" in blob.name:
+            blob.make_public()
             result["preview_url"] = blob.public_url
 
     return result

backend/migration/migrate_built_melodies.py

@@ -0,0 +1,65 @@
+"""
+Phase 1 — Step 1.2: built_melodies  (SQLite → Postgres)
+
+Run on VPS:
+    docker compose exec backend python -m migration.migrate_built_melodies
+"""
+
+import asyncio
+import sys
+
+from sqlalchemy.dialects.postgresql import insert as pg_insert
+
+from melodies.orm import BuiltMelody
+from migration.utils import open_sqlite, AsyncPgSession, parse_dt, parse_json, log_run, pg_count
+
+SCRIPT = "migrate_built_melodies"
+
+
+async def run() -> None:
+    sqlite = await open_sqlite()
+    rows = await sqlite.execute_fetchall("SELECT * FROM built_melodies")
+    await sqlite.close()
+
+    source_count = len(rows)
+    print(f"Source (SQLite): {source_count} built_melodies rows")
+
+    if source_count == 0:
+        print("Nothing to migrate.")
+        await log_run(SCRIPT, 0, 0, notes="source empty")
+        return
+
+    records = []
+    for r in rows:
+        records.append({
+            "id":                  r["id"],
+            "name":                r["name"],
+            "pid":                 r["pid"],
+            "steps":               parse_json(r["steps"], default=[]),
+            "binary_path":         r["binary_path"],
+            "progmem_code":        r["progmem_code"],
+            "assigned_melody_ids": parse_json(r["assigned_melody_ids"], default=[]),
+            "is_builtin":          bool(r["is_builtin"]) if r["is_builtin"] is not None else False,
+            "created_at":          parse_dt(r["created_at"]),
+            "updated_at":          parse_dt(r["updated_at"]),
+        })
+
+    async with AsyncPgSession() as session:
+        async with session.begin():
+            stmt = pg_insert(BuiltMelody).values(records)
+            stmt = stmt.on_conflict_do_nothing(index_elements=["id"])
+            await session.execute(stmt)
+            dest_count = await pg_count(session, "built_melodies")
+
+    if dest_count < source_count:
+        msg = f"Count mismatch: source={source_count}  postgres={dest_count}"
+        print(f"ERROR: {msg}", file=sys.stderr)
+        await log_run(SCRIPT, source_count, dest_count, success=False, notes=msg)
+        sys.exit(1)
+
+    print(f"Postgres: {dest_count} rows ✓")
+    await log_run(SCRIPT, source_count, dest_count)
+
+
+if __name__ == "__main__":
+    asyncio.run(run())

backend/migration/migrate_commands.py

@@ -0,0 +1,73 @@
+"""
+Phase 1 — Step 1.10: commands  (SQLite → Postgres)
+
+commands is a raw-SQL table (no ORM model). BIGSERIAL PK — SQLite integer IDs
+are NOT preserved; rows are inserted in sent_at order.
+
+Run on VPS:
+    docker compose exec backend python -m migration.migrate_commands
+"""
+
+import asyncio
+import sys
+
+from sqlalchemy import text
+
+from migration.utils import open_sqlite, AsyncPgSession, parse_dt, log_run, pg_count
+
+SCRIPT = "migrate_commands"
+
+
+async def run() -> None:
+    sqlite = await open_sqlite()
+    rows = await sqlite.execute_fetchall("SELECT * FROM commands ORDER BY sent_at")
+    await sqlite.close()
+
+    source_count = len(rows)
+    print(f"Source (SQLite): {source_count} commands rows")
+
+    if source_count == 0:
+        print("Nothing to migrate.")
+        await log_run(SCRIPT, 0, 0, notes="source empty")
+        return
+
+    records = [
+        {
+            "device_serial":    r["device_serial"],
+            "command_name":     r["command_name"],
+            "command_payload":  r["command_payload"],
+            "status":           r["status"] or "pending",
+            "response_payload": r["response_payload"],
+            "sent_at":          parse_dt(r["sent_at"]),
+            "responded_at":     parse_dt(r["responded_at"]),
+        }
+        for r in rows
+    ]
+
+    async with AsyncPgSession() as session:
+        async with session.begin():
+            await session.execute(
+                text("""
+                    INSERT INTO commands
+                        (device_serial, command_name, command_payload, status,
+                         response_payload, sent_at, responded_at)
+                    VALUES
+                        (:device_serial, :command_name, :command_payload, :status,
+                         :response_payload, :sent_at, :responded_at)
+                """),
+                records,
+            )
+            dest_count = await pg_count(session, "commands")
+
+    if dest_count < source_count:
+        msg = f"Count mismatch: source={source_count}  postgres={dest_count}"
+        print(f"ERROR: {msg}", file=sys.stderr)
+        await log_run(SCRIPT, source_count, dest_count, success=False, notes=msg)
+        sys.exit(1)
+
+    print(f"Postgres: {dest_count} rows ✓")
+    await log_run(SCRIPT, source_count, dest_count)
+
+
+if __name__ == "__main__":
+    asyncio.run(run())

backend/migration/migrate_crm_comms_log.py

@@ -0,0 +1,84 @@
+"""
+Phase 1 — Step 1.9: crm_comms_log  (SQLite → Postgres)
+
+FK to crm_customers(id) (nullable, ON DELETE SET NULL) — FK enforcement
+suppressed until Phase 2 populates crm_customers.
+
+Run on VPS:
+    docker compose exec backend python -m migration.migrate_crm_comms_log
+"""
+
+import asyncio
+import sys
+
+from sqlalchemy import text
+from sqlalchemy.dialects.postgresql import insert as pg_insert
+
+from crm.orm import CrmCommsLog
+from migration.utils import open_sqlite, AsyncPgSession, parse_dt, parse_json, log_run, pg_count
+
+SCRIPT = "migrate_crm_comms_log"
+
+
+async def run() -> None:
+    sqlite = await open_sqlite()
+    rows = await sqlite.execute_fetchall("SELECT * FROM crm_comms_log ORDER BY occurred_at")
+    await sqlite.close()
+
+    source_count = len(rows)
+    print(f"Source (SQLite): {source_count} crm_comms_log rows")
+
+    if source_count == 0:
+        print("Nothing to migrate.")
+        await log_run(SCRIPT, 0, 0, notes="source empty")
+        return
+
+    records = []
+    for r in rows:
+        # attachments stored as JSON text in SQLite
+        attachments = parse_json(r["attachments"], default=[])
+
+        # is_important / is_read stored as INTEGER (0/1) in SQLite
+        is_important = bool(r["is_important"]) if r["is_important"] is not None else False
+        is_read      = bool(r["is_read"])      if r["is_read"]      is not None else True
+
+        records.append({
+            "id":             r["id"],
+            "customer_id":    r["customer_id"],
+            "type":           r["type"],
+            "mail_account":   r["mail_account"],
+            "direction":      r["direction"],
+            "subject":        r["subject"],
+            "body":           r["body"],
+            "body_html":      r["body_html"],
+            "attachments":    attachments,
+            "ext_message_id": r["ext_message_id"],
+            "from_addr":      r["from_addr"],
+            "to_addrs":       r["to_addrs"],
+            "logged_by":      r["logged_by"],
+            "is_important":   is_important,
+            "is_read":        is_read,
+            "occurred_at":    parse_dt(r["occurred_at"]),
+            "created_at":     parse_dt(r["created_at"]),
+        })
+
+    async with AsyncPgSession() as session:
+        async with session.begin():
+            await session.execute(text("SET LOCAL session_replication_role = replica"))
+            stmt = pg_insert(CrmCommsLog).values(records)
+            stmt = stmt.on_conflict_do_nothing(index_elements=["id"])
+            await session.execute(stmt)
+            dest_count = await pg_count(session, "crm_comms_log")
+
+    if dest_count < source_count:
+        msg = f"Count mismatch: source={source_count}  postgres={dest_count}"
+        print(f"ERROR: {msg}", file=sys.stderr)
+        await log_run(SCRIPT, source_count, dest_count, success=False, notes=msg)
+        sys.exit(1)
+
+    print(f"Postgres: {dest_count} rows ✓")
+    await log_run(SCRIPT, source_count, dest_count)
+
+
+if __name__ == "__main__":
+    asyncio.run(run())

backend/migration/migrate_crm_customers.py

@@ -0,0 +1,172 @@
+"""
+Phase 2 — Step 2.4: crm_customers  (Firestore → Postgres)
+
+Reads the 'crm_customers' Firestore collection.
+- Strips legacy fields: 'negotiating', 'has_problem'
+- Converts Firestore DatetimeWithNanoseconds → UTC datetime
+- Converts nested dicts/lists → JSONB-ready Python objects
+
+After this runs, the FK constraints on crm_quotations, crm_comms_log,
+crm_media, and crm_orders (all inserted in Phase 1 with FK enforcement
+suppressed) become valid.
+
+Run on VPS:
+    docker compose exec backend python -m migration.migrate_crm_customers
+"""
+
+import asyncio
+import sys
+from datetime import datetime, timezone
+
+from sqlalchemy.dialects.postgresql import insert as pg_insert
+
+from crm.orm import CrmCustomer
+from shared.firebase import init_firebase, get_db as get_firestore
+from migration.utils import AsyncPgSession, parse_dt, log_run, pg_count
+
+SCRIPT = "migrate_crm_customers"
+COLLECTION = "crm_customers"
+
+_LEGACY_FIELDS = {"negotiating", "has_problem"}
+
+_VALID_STATUSES = {
+    "lead", "active", "inactive", "archived",
+    "prospect", "churned", "vip",
+}
+
+
+def _now_utc() -> datetime:
+    return datetime.now(timezone.utc)
+
+
+def _coerce_dt(val) -> datetime | None:
+    """Handle both Firestore DatetimeWithNanoseconds and ISO strings."""
+    if val is None:
+        return None
+    if isinstance(val, datetime):
+        return val.replace(tzinfo=timezone.utc) if val.tzinfo is None else val
+    return parse_dt(str(val))
+
+
+def _coerce_list(val, default=None) -> list:
+    if isinstance(val, list):
+        return val
+    return default if default is not None else []
+
+
+async def run() -> None:
+    init_firebase()
+    fs = get_firestore()
+    if fs is None:
+        print("ERROR: Firebase not initialised.", file=sys.stderr)
+        sys.exit(1)
+
+    docs = list(fs.collection(COLLECTION).stream())
+    source_count = len(docs)
+    print(f"Source (Firestore): {source_count} crm_customers documents")
+
+    if source_count == 0:
+        print("Nothing to migrate.")
+        await log_run(SCRIPT, 0, 0, notes="source empty")
+        return
+
+    records = []
+    skipped = 0
+    for doc in docs:
+        d = doc.to_dict()
+
+        # Strip legacy fields
+        for f in _LEGACY_FIELDS:
+            d.pop(f, None)
+
+        # folder_id is NOT NULL UNIQUE — skip docs missing it
+        folder_id = d.get("folder_id") or ""
+        if not folder_id:
+            print(f"  WARNING: customer {doc.id} has no folder_id — skipping", file=sys.stderr)
+            skipped += 1
+            continue
+
+        # relationship_status — normalise unknown values to 'lead'
+        rel_status = d.get("relationship_status") or "lead"
+        if rel_status not in _VALID_STATUSES:
+            rel_status = "lead"
+
+        # contacts / notes — Firestore stores as list of maps
+        contacts = _coerce_list(d.get("contacts"))
+        # Serialise nested Pydantic-style objects to plain dicts
+        contacts = [c if isinstance(c, dict) else vars(c) for c in contacts]
+
+        notes = _coerce_list(d.get("notes"))
+        notes = [n if isinstance(n, dict) else vars(n) for n in notes]
+
+        # location — may be a map or None
+        location = d.get("location")
+        if location and not isinstance(location, dict):
+            location = vars(location)
+
+        tags = _coerce_list(d.get("tags"))
+        owned_items = _coerce_list(d.get("owned_items"))
+        owned_items = [o if isinstance(o, dict) else vars(o) for o in owned_items]
+
+        linked_user_ids = _coerce_list(d.get("linked_user_ids"))
+
+        technical_issues    = _coerce_list(d.get("technical_issues"))
+        install_support     = _coerce_list(d.get("install_support"))
+        transaction_history = _coerce_list(d.get("transaction_history"))
+
+        crm_summary = d.get("crm_summary")
+        if crm_summary and not isinstance(crm_summary, dict):
+            crm_summary = vars(crm_summary)
+
+        created_at = _coerce_dt(d.get("created_at")) or _now_utc()
+        updated_at = _coerce_dt(d.get("updated_at")) or _now_utc()
+
+        records.append({
+            "id":                  doc.id,
+            "firestore_id":        doc.id,
+            "title":               d.get("title"),
+            "name":                d.get("name") or "",
+            "surname":             d.get("surname"),
+            "organization":        d.get("organization"),
+            "religion":            d.get("religion"),
+            "language":            d.get("language") or "el",
+            "folder_id":           folder_id,
+            "relationship_status": rel_status,
+            "nextcloud_folder":    d.get("nextcloud_folder"),
+            "contacts":            contacts,
+            "notes":               notes,
+            "location":            location,
+            "tags":                tags,
+            "owned_items":         owned_items,
+            "linked_user_ids":     linked_user_ids,
+            "technical_issues":    technical_issues,
+            "install_support":     install_support,
+            "transaction_history": transaction_history,
+            "crm_summary":         crm_summary,
+            "created_at":          created_at,
+            "updated_at":          updated_at,
+        })
+
+    actual_source = source_count - skipped
+    print(f"  {skipped} skipped (missing folder_id), {actual_source} to insert")
+
+    async with AsyncPgSession() as session:
+        async with session.begin():
+            stmt = pg_insert(CrmCustomer).values(records)
+            stmt = stmt.on_conflict_do_nothing(index_elements=["id"])
+            await session.execute(stmt)
+            dest_count = await pg_count(session, "crm_customers")
+
+    if dest_count < actual_source:
+        msg = f"Count mismatch: expected>={actual_source}  postgres={dest_count}"
+        print(f"ERROR: {msg}", file=sys.stderr)
+        await log_run(SCRIPT, source_count, dest_count, success=False, notes=msg)
+        sys.exit(1)
+
+    print(f"Postgres: {dest_count} rows ✓")
+    await log_run(SCRIPT, source_count, dest_count,
+                  notes=f"{skipped} skipped (no folder_id)" if skipped else None)
+
+
+if __name__ == "__main__":
+    asyncio.run(run())

backend/migration/migrate_crm_media.py

@@ -0,0 +1,75 @@
+"""
+Phase 1 — Step 1.8: crm_media  (SQLite → Postgres)
+
+FK to crm_customers(id) (nullable) — FK enforcement suppressed until Phase 2
+populates crm_customers.
+
+Run on VPS:
+    docker compose exec backend python -m migration.migrate_crm_media
+"""
+
+import asyncio
+import sys
+
+from sqlalchemy import text
+from sqlalchemy.dialects.postgresql import insert as pg_insert
+
+from crm.orm import CrmMedia
+from migration.utils import open_sqlite, AsyncPgSession, parse_dt, parse_json, log_run, pg_count
+
+SCRIPT = "migrate_crm_media"
+
+
+async def run() -> None:
+    sqlite = await open_sqlite()
+    rows = await sqlite.execute_fetchall("SELECT * FROM crm_media ORDER BY created_at")
+    await sqlite.close()
+
+    source_count = len(rows)
+    print(f"Source (SQLite): {source_count} crm_media rows")
+
+    if source_count == 0:
+        print("Nothing to migrate.")
+        await log_run(SCRIPT, 0, 0, notes="source empty")
+        return
+
+    records = []
+    for r in rows:
+        # SQLite stores tags as JSON text; Postgres column is JSONB
+        tags_raw = r["tags"]
+        tags = parse_json(tags_raw, default=[])
+
+        records.append({
+            "id":             r["id"],
+            "customer_id":    r["customer_id"],
+            "order_id":       r["order_id"],
+            "filename":       r["filename"],
+            "nextcloud_path": r["nextcloud_path"],
+            "thumbnail_path": r["thumbnail_path"],
+            "mime_type":      r["mime_type"],
+            "direction":      r["direction"],
+            "tags":           tags,
+            "uploaded_by":    r["uploaded_by"],
+            "created_at":     parse_dt(r["created_at"]),
+        })
+
+    async with AsyncPgSession() as session:
+        async with session.begin():
+            await session.execute(text("SET LOCAL session_replication_role = replica"))
+            stmt = pg_insert(CrmMedia).values(records)
+            stmt = stmt.on_conflict_do_nothing(index_elements=["id"])
+            await session.execute(stmt)
+            dest_count = await pg_count(session, "crm_media")
+
+    if dest_count < source_count:
+        msg = f"Count mismatch: source={source_count}  postgres={dest_count}"
+        print(f"ERROR: {msg}", file=sys.stderr)
+        await log_run(SCRIPT, source_count, dest_count, success=False, notes=msg)
+        sys.exit(1)
+
+    print(f"Postgres: {dest_count} rows ✓")
+    await log_run(SCRIPT, source_count, dest_count)
+
+
+if __name__ == "__main__":
+    asyncio.run(run())

backend/migration/migrate_crm_orders.py

@@ -0,0 +1,156 @@
+"""
+Phase 2 — Step 2.5: crm_orders  (Firestore → Postgres)
+
+Orders are stored as a subcollection under each customer:
+    crm_customers/{customer_id}/orders/{order_id}
+
+Uses collection_group("orders") to fetch all orders in one pass,
+then inserts into crm_orders. crm_customers MUST already be in Postgres
+(step 2.4) so the FK constraint is satisfied.
+
+Run on VPS:
+    docker compose exec backend python -m migration.migrate_crm_orders
+"""
+
+import asyncio
+import sys
+from datetime import datetime, timezone
+
+from sqlalchemy.dialects.postgresql import insert as pg_insert
+
+from crm.orm import CrmOrder
+from shared.firebase import init_firebase, get_db as get_firestore
+from migration.utils import AsyncPgSession, parse_dt, log_run, pg_count
+
+SCRIPT = "migrate_crm_orders"
+
+
+def _now_utc() -> datetime:
+    return datetime.now(timezone.utc)
+
+
+def _coerce_dt(val) -> datetime | None:
+    if val is None:
+        return None
+    if isinstance(val, datetime):
+        return val.replace(tzinfo=timezone.utc) if val.tzinfo is None else val
+    return parse_dt(str(val))
+
+
+def _coerce_list(val) -> list:
+    return val if isinstance(val, list) else []
+
+
+def _coerce_dict(val) -> dict:
+    return val if isinstance(val, dict) else {}
+
+
+async def run() -> None:
+    init_firebase()
+    fs = get_firestore()
+    if fs is None:
+        print("ERROR: Firebase not initialised.", file=sys.stderr)
+        sys.exit(1)
+
+    # collection_group fetches from ALL customers' 'orders' subcollections
+    docs = list(fs.collection_group("orders").stream())
+    source_count = len(docs)
+    print(f"Source (Firestore): {source_count} order documents (via collection_group)")
+
+    if source_count == 0:
+        print("Nothing to migrate.")
+        await log_run(SCRIPT, 0, 0, notes="source empty")
+        return
+
+    # First, collect all customer IDs already in Postgres so we can skip
+    # orphaned orders whose customer didn't migrate (missing folder_id edge case)
+    async with AsyncPgSession() as session:
+        from sqlalchemy import text
+        result = await session.execute(text("SELECT id FROM crm_customers"))
+        valid_customer_ids = {row[0] for row in result.fetchall()}
+
+    records = []
+    skipped = 0
+    seen_order_numbers: set[str] = set()
+    for doc in docs:
+        d = doc.to_dict()
+
+        # Extract customer_id from the document path:
+        # crm_customers/{customer_id}/orders/{order_id}
+        path_parts = doc.reference.path.split("/")
+        # path: crm_customers / <cid> / orders / <oid>
+        try:
+            customer_id = path_parts[1]
+        except IndexError:
+            print(f"  WARNING: cannot parse customer_id from path {doc.reference.path} — skipping")
+            skipped += 1
+            continue
+
+        if customer_id not in valid_customer_ids:
+            print(f"  WARNING: order {doc.id} references unknown customer {customer_id} — skipping")
+            skipped += 1
+            continue
+
+        order_number = d.get("order_number") or f"ORD-LEGACY-{doc.id}"
+        # Deduplicate: if this order_number was already seen in this batch,
+        # make it unique by appending the doc ID suffix.
+        if order_number in seen_order_numbers:
+            order_number = f"{order_number}-{doc.id[:8]}"
+            print(f"  INFO: duplicate order_number — renamed to {order_number}")
+        seen_order_numbers.add(order_number)
+
+        created_at = _coerce_dt(d.get("created_at")) or _now_utc()
+        updated_at = _coerce_dt(d.get("updated_at")) or _now_utc()
+        status_updated_date = _coerce_dt(d.get("status_updated_date"))
+
+        records.append({
+            "id":                  doc.id,
+            "customer_id":         customer_id,
+            "order_number":        order_number,
+            "title":               d.get("title"),
+            "created_by":          d.get("created_by"),
+            "status":              d.get("status") or "negotiating",
+            "status_updated_date": status_updated_date,
+            "status_updated_by":   d.get("status_updated_by"),
+            "items":               _coerce_list(d.get("items")),
+            "subtotal":            float(d.get("subtotal") or 0),
+            "discount":            d.get("discount") if isinstance(d.get("discount"), dict) else None,
+            "total_price":         float(d.get("total_price") or 0),
+            "currency":            d.get("currency") or "EUR",
+            "shipping":            d.get("shipping") if isinstance(d.get("shipping"), dict) else None,
+            "payment_status":      _coerce_dict(d.get("payment_status")),
+            "invoice_path":        d.get("invoice_path"),
+            "notes":               d.get("notes") if isinstance(d.get("notes"), str) else None,
+            "timeline":            _coerce_list(d.get("timeline")),
+            "created_at":          created_at,
+            "updated_at":          updated_at,
+        })
+
+    actual_source = source_count - skipped
+    print(f"  {skipped} skipped (orphaned/bad path), {actual_source} to insert")
+
+    if not records:
+        print("Nothing valid to insert.")
+        await log_run(SCRIPT, source_count, 0, notes=f"{skipped} all skipped")
+        return
+
+    async with AsyncPgSession() as session:
+        async with session.begin():
+            stmt = pg_insert(CrmOrder).values(records)
+            stmt = stmt.on_conflict_do_nothing(index_elements=["id"])
+            await session.execute(stmt)
+            dest_count = await pg_count(session, "crm_orders")
+
+    if dest_count < actual_source:
+        msg = f"Count mismatch: expected>={actual_source}  postgres={dest_count}"
+        print(f"ERROR: {msg}", file=sys.stderr)
+        await log_run(SCRIPT, source_count, dest_count, success=False, notes=msg)
+        sys.exit(1)
+
+    print(f"Postgres: {dest_count} rows ✓")
+    await log_run(SCRIPT, source_count, dest_count,
+                  notes=f"{skipped} skipped" if skipped else None)
+
+
+if __name__ == "__main__":
+    asyncio.run(run())

backend/migration/migrate_crm_products.py

@@ -0,0 +1,102 @@
+"""
+Phase 2 — Step 2.3: crm_products  (Firestore → Postgres)
+
+Reads the 'crm_products' Firestore collection. The Firestore schema is richer
+than the Postgres target (has costs, stock, name_en, etc.) — we extract only
+what the Postgres ORM model covers. The rest stays in Firestore until the
+service is fully cut over.
+
+Run on VPS:
+    docker compose exec backend python -m migration.migrate_crm_products
+"""
+
+import asyncio
+import sys
+from datetime import datetime, timezone
+
+from sqlalchemy.dialects.postgresql import insert as pg_insert
+
+from crm.orm import CrmProduct
+from shared.firebase import init_firebase, get_db as get_firestore
+from migration.utils import AsyncPgSession, parse_dt, log_run, pg_count
+
+SCRIPT = "migrate_crm_products"
+COLLECTION = "crm_products"
+
+_LEGACY_STATUS_MAP = {
+    "active": True,
+    "discontinued": False,
+    "planned": True,
+}
+
+
+def _now_utc() -> datetime:
+    return datetime.now(timezone.utc)
+
+
+async def run() -> None:
+    init_firebase()
+    fs = get_firestore()
+    if fs is None:
+        print("ERROR: Firebase not initialised.", file=sys.stderr)
+        sys.exit(1)
+
+    docs = list(fs.collection(COLLECTION).stream())
+    source_count = len(docs)
+    print(f"Source (Firestore): {source_count} crm_products documents")
+
+    if source_count == 0:
+        print("Nothing to migrate.")
+        await log_run(SCRIPT, 0, 0, notes="source empty")
+        return
+
+    records = []
+    for doc in docs:
+        d = doc.to_dict()
+
+        # is_active: prefer 'active' bool field, fall back to 'status' string
+        if "active" in d:
+            is_active = bool(d["active"])
+        else:
+            is_active = _LEGACY_STATUS_MAP.get(d.get("status", "active"), True)
+
+        # unit_cost: Firestore uses 'price'
+        unit_cost = d.get("unit_cost") or d.get("price") or 0
+
+        created_at = parse_dt(d.get("created_at")) or _now_utc()
+        updated_at = parse_dt(d.get("updated_at")) or _now_utc()
+
+        records.append({
+            "id":           doc.id,
+            "firestore_id": doc.id,
+            "name":         d.get("name") or d.get("name_en") or "",
+            "sku":          d.get("sku"),
+            "category":     d.get("category"),
+            "description":  d.get("description") or d.get("description_en"),
+            "unit_cost":    unit_cost,
+            "currency":     d.get("currency") or "EUR",
+            "unit_type":    d.get("unit_type") or "pcs",
+            "is_active":    is_active,
+            "created_at":   created_at,
+            "updated_at":   updated_at,
+        })
+
+    async with AsyncPgSession() as session:
+        async with session.begin():
+            stmt = pg_insert(CrmProduct).values(records)
+            stmt = stmt.on_conflict_do_nothing(index_elements=["id"])
+            await session.execute(stmt)
+            dest_count = await pg_count(session, "crm_products")
+
+    if dest_count < source_count:
+        msg = f"Count mismatch: source={source_count}  postgres={dest_count}"
+        print(f"ERROR: {msg}", file=sys.stderr)
+        await log_run(SCRIPT, source_count, dest_count, success=False, notes=msg)
+        sys.exit(1)
+
+    print(f"Postgres: {dest_count} rows ✓")
+    await log_run(SCRIPT, source_count, dest_count)
+
+
+if __name__ == "__main__":
+    asyncio.run(run())

backend/migration/migrate_crm_quotation_items.py

@@ -0,0 +1,84 @@
+"""
+Phase 1 — Step 1.7: crm_quotation_items  (SQLite → Postgres)
+
+FK to crm_quotations(id) — quotations must be migrated first (step 1.6).
+FK enforcement suppressed via session_replication_role for the same reason
+as in migrate_crm_quotations (parent crm_customers not yet in PG).
+
+Run on VPS:
+    docker compose exec backend python -m migration.migrate_crm_quotation_items
+"""
+
+import asyncio
+import sys
+from decimal import Decimal
+
+from sqlalchemy import text
+from sqlalchemy.dialects.postgresql import insert as pg_insert
+
+from crm.orm import CrmQuotationItem
+from migration.utils import open_sqlite, AsyncPgSession, log_run, pg_count
+
+SCRIPT = "migrate_crm_quotation_items"
+
+
+def _dec(val, default="0") -> Decimal:
+    try:
+        return Decimal(str(val)) if val is not None else Decimal(default)
+    except Exception:
+        return Decimal(default)
+
+
+async def run() -> None:
+    sqlite = await open_sqlite()
+    rows = await sqlite.execute_fetchall(
+        "SELECT * FROM crm_quotation_items ORDER BY quotation_id, sort_order"
+    )
+    await sqlite.close()
+
+    source_count = len(rows)
+    print(f"Source (SQLite): {source_count} crm_quotation_items rows")
+
+    if source_count == 0:
+        print("Nothing to migrate.")
+        await log_run(SCRIPT, 0, 0, notes="source empty")
+        return
+
+    records = []
+    for r in rows:
+        records.append({
+            "id":               r["id"],
+            "quotation_id":     r["quotation_id"],
+            "product_id":       r["product_id"],
+            "description":      r["description"],
+            "description_en":   r["description_en"],
+            "description_gr":   r["description_gr"],
+            "unit_type":        r["unit_type"] or "pcs",
+            "unit_cost":        _dec(r["unit_cost"]),
+            "discount_percent": _dec(r["discount_percent"]),
+            "vat_percent":      _dec(r["vat_percent"], "24"),
+            "quantity":         _dec(r["quantity"], "1"),
+            "line_total":       _dec(r["line_total"]),
+            "sort_order":       int(r["sort_order"]) if r["sort_order"] is not None else 0,
+        })
+
+    async with AsyncPgSession() as session:
+        async with session.begin():
+            await session.execute(text("SET LOCAL session_replication_role = replica"))
+            stmt = pg_insert(CrmQuotationItem).values(records)
+            stmt = stmt.on_conflict_do_nothing(index_elements=["id"])
+            await session.execute(stmt)
+            dest_count = await pg_count(session, "crm_quotation_items")
+
+    if dest_count < source_count:
+        msg = f"Count mismatch: source={source_count}  postgres={dest_count}"
+        print(f"ERROR: {msg}", file=sys.stderr)
+        await log_run(SCRIPT, source_count, dest_count, success=False, notes=msg)
+        sys.exit(1)
+
+    print(f"Postgres: {dest_count} rows ✓")
+    await log_run(SCRIPT, source_count, dest_count)
+
+
+if __name__ == "__main__":
+    asyncio.run(run())

backend/migration/migrate_crm_quotations.py

@@ -0,0 +1,118 @@
+"""
+Phase 1 — Step 1.6: crm_quotations  (SQLite → Postgres)
+
+NOTE: crm_quotations has a FK to crm_customers(id).
+The customer rows DO NOT exist in Postgres yet (they migrate in Phase 2).
+To avoid FK violations, this script temporarily disables FK checks for the
+session using SET CONSTRAINTS ALL DEFERRED — but since customer_id is a real
+FK with ON DELETE CASCADE, we instead insert with the constraint deferred.
+
+Safer approach used here: insert with `customer_id` as-is and rely on the
+fact that crm_customers will be populated in Phase 2 before any service
+reads join across the two tables. The FK is not deferred — instead we disable
+the FK constraint enforcement for this transaction only via a session-level
+SET session_replication_role = replica; which suppresses FK checks in Postgres.
+We restore it immediately after the transaction.
+
+Run on VPS:
+    docker compose exec backend python -m migration.migrate_crm_quotations
+"""
+
+import asyncio
+import sys
+from decimal import Decimal
+
+from sqlalchemy import text
+from sqlalchemy.dialects.postgresql import insert as pg_insert
+
+from crm.orm import CrmQuotation
+from migration.utils import open_sqlite, AsyncPgSession, parse_dt, parse_json, log_run, pg_count
+
+SCRIPT = "migrate_crm_quotations"
+
+
+def _dec(val, default="0") -> Decimal:
+    try:
+        return Decimal(str(val)) if val is not None else Decimal(default)
+    except Exception:
+        return Decimal(default)
+
+
+async def run() -> None:
+    sqlite = await open_sqlite()
+    rows = await sqlite.execute_fetchall("SELECT * FROM crm_quotations ORDER BY created_at")
+    await sqlite.close()
+
+    source_count = len(rows)
+    print(f"Source (SQLite): {source_count} crm_quotations rows")
+
+    if source_count == 0:
+        print("Nothing to migrate.")
+        await log_run(SCRIPT, 0, 0, notes="source empty")
+        return
+
+    records = []
+    for r in rows:
+        records.append({
+            "id":                       r["id"],
+            "quotation_number":         r["quotation_number"],
+            "title":                    r["title"],
+            "subtitle":                 r["subtitle"],
+            "customer_id":              r["customer_id"],
+            "language":                 r["language"] or "en",
+            "status":                   r["status"] or "draft",
+            "order_type":               r["order_type"],
+            "shipping_method":          r["shipping_method"],
+            "estimated_shipping_date":  r["estimated_shipping_date"],
+            "global_discount_label":    r["global_discount_label"],
+            "global_discount_percent":  _dec(r["global_discount_percent"]),
+            "vat_percent":              _dec(r["vat_percent"], "24"),
+            "global_vat_percent":       _dec(r["global_vat_percent"], "24"),
+            "shipping_cost":            _dec(r["shipping_cost"]),
+            "shipping_cost_discount":   _dec(r["shipping_cost_discount"]),
+            "install_cost":             _dec(r["install_cost"]),
+            "install_cost_discount":    _dec(r["install_cost_discount"]),
+            "extras_label":             r["extras_label"],
+            "extras_cost":              _dec(r["extras_cost"]),
+            "comments":                 parse_json(r["comments"], default=[]),
+            "quick_notes":              parse_json(r["quick_notes"], default={}),
+            "subtotal_before_discount": _dec(r["subtotal_before_discount"]),
+            "global_discount_amount":   _dec(r["global_discount_amount"]),
+            "new_subtotal":             _dec(r["new_subtotal"]),
+            "vat_amount":               _dec(r["vat_amount"]),
+            "final_total":              _dec(r["final_total"]),
+            "nextcloud_pdf_path":       r["nextcloud_pdf_path"],
+            "nextcloud_pdf_url":        r["nextcloud_pdf_url"],
+            "client_org":               r["client_org"],
+            "client_name":              r["client_name"],
+            "client_location":          r["client_location"],
+            "client_phone":             r["client_phone"],
+            "client_email":             r["client_email"],
+            "is_legacy":                bool(r["is_legacy"]) if r["is_legacy"] is not None else False,
+            "legacy_date":              r["legacy_date"],
+            "legacy_pdf_path":          r["legacy_pdf_path"],
+            "created_at":               parse_dt(r["created_at"]),
+            "updated_at":               parse_dt(r["updated_at"]),
+        })
+
+    async with AsyncPgSession() as session:
+        async with session.begin():
+            # Disable FK enforcement so we can insert before crm_customers arrives in Phase 2.
+            await session.execute(text("SET LOCAL session_replication_role = replica"))
+            stmt = pg_insert(CrmQuotation).values(records)
+            stmt = stmt.on_conflict_do_nothing(index_elements=["id"])
+            await session.execute(stmt)
+            dest_count = await pg_count(session, "crm_quotations")
+
+    if dest_count < source_count:
+        msg = f"Count mismatch: source={source_count}  postgres={dest_count}"
+        print(f"ERROR: {msg}", file=sys.stderr)
+        await log_run(SCRIPT, source_count, dest_count, success=False, notes=msg)
+        sys.exit(1)
+
+    print(f"Postgres: {dest_count} rows ✓")
+    await log_run(SCRIPT, source_count, dest_count)
+
+
+if __name__ == "__main__":
+    asyncio.run(run())

backend/migration/migrate_crm_sync_state.py

@@ -0,0 +1,54 @@
+"""
+Phase 1 — Step 1.5: crm_sync_state  (SQLite → Postgres)
+
+Simple key/value table — small, no FK deps.
+
+Run on VPS:
+    docker compose exec backend python -m migration.migrate_crm_sync_state
+"""
+
+import asyncio
+import sys
+
+from sqlalchemy.dialects.postgresql import insert as pg_insert
+
+from crm.orm import CrmSyncState
+from migration.utils import open_sqlite, AsyncPgSession, log_run, pg_count
+
+SCRIPT = "migrate_crm_sync_state"
+
+
+async def run() -> None:
+    sqlite = await open_sqlite()
+    rows = await sqlite.execute_fetchall("SELECT * FROM crm_sync_state")
+    await sqlite.close()
+
+    source_count = len(rows)
+    print(f"Source (SQLite): {source_count} crm_sync_state rows")
+
+    if source_count == 0:
+        print("Nothing to migrate.")
+        await log_run(SCRIPT, 0, 0, notes="source empty")
+        return
+
+    records = [{"key": r["key"], "value": r["value"]} for r in rows]
+
+    async with AsyncPgSession() as session:
+        async with session.begin():
+            stmt = pg_insert(CrmSyncState).values(records)
+            stmt = stmt.on_conflict_do_nothing(index_elements=["key"])
+            await session.execute(stmt)
+            dest_count = await pg_count(session, "crm_sync_state")
+
+    if dest_count < source_count:
+        msg = f"Count mismatch: source={source_count}  postgres={dest_count}"
+        print(f"ERROR: {msg}", file=sys.stderr)
+        await log_run(SCRIPT, source_count, dest_count, success=False, notes=msg)
+        sys.exit(1)
+
+    print(f"Postgres: {dest_count} rows ✓")
+    await log_run(SCRIPT, source_count, dest_count)
+
+
+if __name__ == "__main__":
+    asyncio.run(run())

backend/migration/migrate_device_alerts.py

@@ -0,0 +1,69 @@
+"""
+Phase 1 — Step 1.4: device_alerts  (SQLite → Postgres)
+
+device_alerts is a "current state" table — one row per (device_serial, subsystem).
+The SQLite PK is (device_serial, subsystem); Postgres adds a BIGSERIAL surrogate PK
+with a unique constraint on the pair.
+
+Run on VPS:
+    docker compose exec backend python -m migration.migrate_device_alerts
+"""
+
+import asyncio
+import sys
+
+from sqlalchemy import text
+
+from migration.utils import open_sqlite, AsyncPgSession, parse_dt, log_run, pg_count
+
+SCRIPT = "migrate_device_alerts"
+
+
+async def run() -> None:
+    sqlite = await open_sqlite()
+    rows = await sqlite.execute_fetchall("SELECT * FROM device_alerts")
+    await sqlite.close()
+
+    source_count = len(rows)
+    print(f"Source (SQLite): {source_count} device_alerts rows")
+
+    if source_count == 0:
+        print("Nothing to migrate.")
+        await log_run(SCRIPT, 0, 0, notes="source empty")
+        return
+
+    records = [
+        {
+            "device_serial": r["device_serial"],
+            "subsystem":     r["subsystem"],
+            "state":         r["state"],
+            "message":       r["message"],
+            "updated_at":    parse_dt(r["updated_at"]),
+        }
+        for r in rows
+    ]
+
+    async with AsyncPgSession() as session:
+        async with session.begin():
+            await session.execute(
+                text("""
+                    INSERT INTO device_alerts (device_serial, subsystem, state, message, updated_at)
+                    VALUES (:device_serial, :subsystem, :state, :message, :updated_at)
+                    ON CONFLICT (device_serial, subsystem) DO NOTHING
+                """),
+                records,
+            )
+            dest_count = await pg_count(session, "device_alerts")
+
+    if dest_count < source_count:
+        msg = f"Count mismatch: source={source_count}  postgres={dest_count}"
+        print(f"ERROR: {msg}", file=sys.stderr)
+        await log_run(SCRIPT, source_count, dest_count, success=False, notes=msg)
+        sys.exit(1)
+
+    print(f"Postgres: {dest_count} rows ✓")
+    await log_run(SCRIPT, source_count, dest_count)
+
+
+if __name__ == "__main__":
+    asyncio.run(run())

backend/migration/migrate_device_logs.py

@@ -0,0 +1,93 @@
+"""
+Phase 1 — Step 1.12: device_logs  (SQLite → Postgres)
+
+Largest table — migrated in batches of 10,000 rows to avoid memory issues.
+device_logs is a partitioned table; rows route automatically to the correct
+monthly partition based on received_at.
+
+Run on VPS:
+    docker compose exec backend python -m migration.migrate_device_logs
+"""
+
+import asyncio
+import sys
+
+from sqlalchemy import text
+
+from migration.utils import open_sqlite, AsyncPgSession, parse_dt, log_run, pg_count
+
+SCRIPT = "migrate_device_logs"
+BATCH_SIZE = 10_000
+
+
+async def run() -> None:
+    sqlite = await open_sqlite()
+
+    # Total count first
+    count_row = await sqlite.execute_fetchall("SELECT COUNT(*) FROM device_logs")
+    source_count = count_row[0][0]
+    print(f"Source (SQLite): {source_count} device_logs rows")
+
+    if source_count == 0:
+        await sqlite.close()
+        print("Nothing to migrate.")
+        await log_run(SCRIPT, 0, 0, notes="source empty")
+        return
+
+    offset = 0
+    total_inserted = 0
+
+    while offset < source_count:
+        rows = await sqlite.execute_fetchall(
+            "SELECT * FROM device_logs ORDER BY received_at LIMIT ? OFFSET ?",
+            (BATCH_SIZE, offset),
+        )
+        if not rows:
+            break
+
+        records = [
+            {
+                "device_serial":    r["device_serial"],
+                "level":            r["level"],
+                "message":          r["message"],
+                "device_timestamp": r["device_timestamp"],
+                "received_at":      parse_dt(r["received_at"]),
+            }
+            for r in rows
+        ]
+
+        async with AsyncPgSession() as session:
+            async with session.begin():
+                await session.execute(
+                    text("""
+                        INSERT INTO device_logs
+                            (device_serial, level, message, device_timestamp, received_at)
+                        VALUES
+                            (:device_serial, :level, :message, :device_timestamp, :received_at)
+                    """),
+                    records,
+                )
+
+        total_inserted += len(records)
+        offset += BATCH_SIZE
+        pct = min(100, int(total_inserted / source_count * 100))
+        print(f"  {total_inserted}/{source_count} rows inserted ({pct}%)")
+
+    await sqlite.close()
+
+    # Final count verify
+    async with AsyncPgSession() as session:
+        dest_count = await pg_count(session, "device_logs")
+
+    if dest_count < source_count:
+        msg = f"Count mismatch: source={source_count}  postgres={dest_count}"
+        print(f"ERROR: {msg}", file=sys.stderr)
+        await log_run(SCRIPT, source_count, dest_count, success=False, notes=msg)
+        sys.exit(1)
+
+    print(f"Postgres: {dest_count} rows ✓")
+    await log_run(SCRIPT, source_count, dest_count)
+
+
+if __name__ == "__main__":
+    asyncio.run(run())

backend/migration/migrate_heartbeats.py

@@ -0,0 +1,73 @@
+"""
+Phase 1 — Step 1.11: heartbeats  (SQLite → Postgres)
+
+Raw-SQL table (no ORM model). BIGSERIAL PK — SQLite IDs not preserved.
+
+Run on VPS:
+    docker compose exec backend python -m migration.migrate_heartbeats
+"""
+
+import asyncio
+import sys
+
+from sqlalchemy import text
+
+from migration.utils import open_sqlite, AsyncPgSession, parse_dt, log_run, pg_count
+
+SCRIPT = "migrate_heartbeats"
+
+
+async def run() -> None:
+    sqlite = await open_sqlite()
+    rows = await sqlite.execute_fetchall("SELECT * FROM heartbeats ORDER BY received_at")
+    await sqlite.close()
+
+    source_count = len(rows)
+    print(f"Source (SQLite): {source_count} heartbeats rows")
+
+    if source_count == 0:
+        print("Nothing to migrate.")
+        await log_run(SCRIPT, 0, 0, notes="source empty")
+        return
+
+    records = [
+        {
+            "device_serial":    r["device_serial"],
+            "device_id":        r["device_id"],
+            "firmware_version": r["firmware_version"],
+            "ip_address":       r["ip_address"],
+            "gateway":          r["gateway"],
+            "uptime_ms":        r["uptime_ms"],
+            "uptime_display":   r["uptime_display"],
+            "received_at":      parse_dt(r["received_at"]),
+        }
+        for r in rows
+    ]
+
+    async with AsyncPgSession() as session:
+        async with session.begin():
+            await session.execute(
+                text("""
+                    INSERT INTO heartbeats
+                        (device_serial, device_id, firmware_version, ip_address,
+                         gateway, uptime_ms, uptime_display, received_at)
+                    VALUES
+                        (:device_serial, :device_id, :firmware_version, :ip_address,
+                         :gateway, :uptime_ms, :uptime_display, :received_at)
+                """),
+                records,
+            )
+            dest_count = await pg_count(session, "heartbeats")
+
+    if dest_count < source_count:
+        msg = f"Count mismatch: source={source_count}  postgres={dest_count}"
+        print(f"ERROR: {msg}", file=sys.stderr)
+        await log_run(SCRIPT, source_count, dest_count, success=False, notes=msg)
+        sys.exit(1)
+
+    print(f"Postgres: {dest_count} rows ✓")
+    await log_run(SCRIPT, source_count, dest_count)
+
+
+if __name__ == "__main__":
+    asyncio.run(run())

backend/migration/migrate_melody_drafts.py

@@ -0,0 +1,66 @@
+"""
+Phase 1 — Step 1.1: melody_drafts  (SQLite → Postgres)
+
+Run on VPS:
+    docker compose exec backend python -m migration.migrate_melody_drafts
+"""
+
+import asyncio
+import json
+import sys
+
+from sqlalchemy import text
+from sqlalchemy.dialects.postgresql import insert as pg_insert
+
+from melodies.orm import MelodyDraft
+from migration.utils import open_sqlite, AsyncPgSession, parse_dt, parse_json, log_run, pg_count
+
+SCRIPT = "migrate_melody_drafts"
+
+
+async def run() -> None:
+    sqlite = await open_sqlite()
+    rows = await sqlite.execute_fetchall("SELECT * FROM melody_drafts")
+    await sqlite.close()
+
+    source_count = len(rows)
+    print(f"Source (SQLite): {source_count} melody_drafts rows")
+
+    if source_count == 0:
+        print("Nothing to migrate.")
+        await log_run(SCRIPT, 0, 0, notes="source empty")
+        return
+
+    records = []
+    for r in rows:
+        data_raw = r["data"]
+        # SQLite stores data as JSON text; Postgres column is JSONB
+        data = parse_json(data_raw, default={})
+
+        records.append({
+            "id":         r["id"],
+            "status":     r["status"] or "draft",
+            "data":       data,
+            "created_at": parse_dt(r["created_at"]),
+            "updated_at": parse_dt(r["updated_at"]),
+        })
+
+    async with AsyncPgSession() as session:
+        async with session.begin():
+            stmt = pg_insert(MelodyDraft).values(records)
+            stmt = stmt.on_conflict_do_nothing(index_elements=["id"])
+            await session.execute(stmt)
+            dest_count = await pg_count(session, "melody_drafts")
+
+    if dest_count < source_count:
+        msg = f"Count mismatch: source={source_count}  postgres={dest_count}"
+        print(f"ERROR: {msg}", file=sys.stderr)
+        await log_run(SCRIPT, source_count, dest_count, success=False, notes=msg)
+        sys.exit(1)
+
+    print(f"Postgres: {dest_count} rows ✓")
+    await log_run(SCRIPT, source_count, dest_count)
+
+
+if __name__ == "__main__":
+    asyncio.run(run())

backend/migration/migrate_mfg_audit_log.py

@@ -0,0 +1,67 @@
+"""
+Phase 1 — Step 1.3: mfg_audit_log  (SQLite → Postgres)
+
+Run on VPS:
+    docker compose exec backend python -m migration.migrate_mfg_audit_log
+"""
+
+import asyncio
+import sys
+
+from sqlalchemy import text
+
+from migration.utils import open_sqlite, AsyncPgSession, parse_dt, log_run, pg_count
+
+SCRIPT = "migrate_mfg_audit_log"
+
+
+async def run() -> None:
+    sqlite = await open_sqlite()
+    rows = await sqlite.execute_fetchall("SELECT * FROM mfg_audit_log ORDER BY id")
+    await sqlite.close()
+
+    source_count = len(rows)
+    print(f"Source (SQLite): {source_count} mfg_audit_log rows")
+
+    if source_count == 0:
+        print("Nothing to migrate.")
+        await log_run(SCRIPT, 0, 0, notes="source empty")
+        return
+
+    # mfg_audit_log uses a BIGSERIAL PK — we don't preserve SQLite integer IDs
+    # because the Postgres sequence will assign new ones. We insert in the same
+    # timestamp order so the audit trail remains coherent.
+    records = [
+        {
+            "timestamp":     parse_dt(r["timestamp"]),
+            "admin_user":    r["admin_user"],
+            "action":        r["action"],
+            "serial_number": r["serial_number"],
+            "detail":        r["detail"],
+        }
+        for r in rows
+    ]
+
+    async with AsyncPgSession() as session:
+        async with session.begin():
+            await session.execute(
+                text("""
+                    INSERT INTO mfg_audit_log (timestamp, admin_user, action, serial_number, detail)
+                    VALUES (:timestamp, :admin_user, :action, :serial_number, :detail)
+                """),
+                records,
+            )
+            dest_count = await pg_count(session, "mfg_audit_log")
+
+    if dest_count < source_count:
+        msg = f"Count mismatch: source={source_count}  postgres={dest_count}"
+        print(f"ERROR: {msg}", file=sys.stderr)
+        await log_run(SCRIPT, source_count, dest_count, success=False, notes=msg)
+        sys.exit(1)
+
+    print(f"Postgres: {dest_count} rows ✓")
+    await log_run(SCRIPT, source_count, dest_count)
+
+
+if __name__ == "__main__":
+    asyncio.run(run())

backend/migration/migrate_public_features.py

@@ -0,0 +1,56 @@
+"""
+Phase 2 — Step 2.2: public_features  (Firestore → Postgres)
+
+Reads the single 'admin_settings/public_features' doc from Firestore and
+flattens each field into a key/value row in public_features.
+
+Run on VPS:
+    docker compose exec backend python -m migration.migrate_public_features
+"""
+
+import asyncio
+import sys
+
+from sqlalchemy.dialects.postgresql import insert as pg_insert
+
+from settings.orm import PublicFeature
+from shared.firebase import init_firebase, get_db as get_firestore
+from migration.utils import AsyncPgSession, log_run, pg_count
+
+SCRIPT = "migrate_public_features"
+COLLECTION = "admin_settings"
+DOC_ID = "public_features"
+
+
+async def run() -> None:
+    init_firebase()
+    fs = get_firestore()
+    if fs is None:
+        print("ERROR: Firebase not initialised — check service account path.", file=sys.stderr)
+        sys.exit(1)
+
+    doc = fs.collection(COLLECTION).document(DOC_ID).get()
+    if not doc.exists:
+        print("No public_features document found in Firestore — skipping.")
+        await log_run(SCRIPT, 0, 0, notes="source doc not found")
+        return
+
+    data = doc.to_dict()
+    source_count = len(data)
+    print(f"Source (Firestore): {source_count} fields in {COLLECTION}/{DOC_ID}")
+
+    records = [{"key": k, "value": v} for k, v in data.items()]
+
+    async with AsyncPgSession() as session:
+        async with session.begin():
+            stmt = pg_insert(PublicFeature).values(records)
+            stmt = stmt.on_conflict_do_nothing(index_elements=["key"])
+            await session.execute(stmt)
+            dest_count = await pg_count(session, "public_features")
+
+    print(f"Postgres public_features: {dest_count} rows ✓")
+    await log_run(SCRIPT, source_count, dest_count)
+
+
+if __name__ == "__main__":
+    asyncio.run(run())

backend/migration/migrate_settings.py

@@ -0,0 +1,56 @@
+"""
+Phase 2 — Step 2.1: console_settings  (Firestore → Postgres)
+
+Reads the single 'admin_settings/melody_settings' doc from Firestore and
+flattens each field into a key/value row in console_settings.
+
+Run on VPS:
+    docker compose exec backend python -m migration.migrate_settings
+"""
+
+import asyncio
+import sys
+
+from sqlalchemy.dialects.postgresql import insert as pg_insert
+
+from settings.orm import ConsoleSetting
+from shared.firebase import init_firebase, get_db as get_firestore
+from migration.utils import AsyncPgSession, log_run, pg_count
+
+SCRIPT = "migrate_settings"
+COLLECTION = "admin_settings"
+DOC_ID = "melody_settings"
+
+
+async def run() -> None:
+    init_firebase()
+    fs = get_firestore()
+    if fs is None:
+        print("ERROR: Firebase not initialised — check service account path.", file=sys.stderr)
+        sys.exit(1)
+
+    doc = fs.collection(COLLECTION).document(DOC_ID).get()
+    if not doc.exists:
+        print("No melody_settings document found in Firestore — skipping.")
+        await log_run(SCRIPT, 0, 0, notes="source doc not found")
+        return
+
+    data = doc.to_dict()
+    source_count = len(data)
+    print(f"Source (Firestore): {source_count} fields in {COLLECTION}/{DOC_ID}")
+
+    records = [{"key": k, "value": v} for k, v in data.items()]
+
+    async with AsyncPgSession() as session:
+        async with session.begin():
+            stmt = pg_insert(ConsoleSetting).values(records)
+            stmt = stmt.on_conflict_do_nothing(index_elements=["key"])
+            await session.execute(stmt)
+            dest_count = await pg_count(session, "console_settings")
+
+    print(f"Postgres console_settings: {dest_count} rows ✓")
+    await log_run(SCRIPT, source_count, dest_count)
+
+
+if __name__ == "__main__":
+    asyncio.run(run())

backend/migration/migrate_staff.py

@@ -0,0 +1,143 @@
+"""
+Phase 3 — Step 3.1: admin_users (Firestore → Postgres staff table)
+
+Reads every document in the 'admin_users' Firestore collection and inserts
+a matching row into the Postgres 'staff' table.
+
+Key transformations:
+  - Legacy role names mapped to canonical roles (superadmin→sysadmin, etc.)
+  - permissions=None stored as JSONB null (sysadmin/admin have no permission map)
+  - ui_prefs column NOT migrated (not part of the Postgres schema — dropped)
+  - Firestore doc ID preserved as staff.id and staff.firestore_id
+  - created_at/updated_at default to now() if missing from Firestore doc
+
+Run on VPS:
+    docker compose exec backend python -m migration.migrate_staff
+"""
+
+import asyncio
+import sys
+from datetime import datetime, timezone
+
+from sqlalchemy.dialects.postgresql import insert as pg_insert
+
+from staff.orm import Staff
+from shared.firebase import init_firebase, get_db as get_firestore
+from migration.utils import AsyncPgSession, parse_dt, log_run, pg_count
+
+SCRIPT = "migrate_staff"
+COLLECTION = "admin_users"
+
+_ROLE_MAP = {
+    "superadmin":    "sysadmin",
+    "melody_editor": "editor",
+    "device_manager": "editor",
+    "user_manager":  "editor",
+    "viewer":        "user",
+    # canonical roles pass through unchanged
+    "sysadmin": "sysadmin",
+    "admin":    "admin",
+    "editor":   "editor",
+    "user":     "user",
+    "staff":    "user",
+}
+
+
+def _now_utc() -> datetime:
+    return datetime.now(timezone.utc)
+
+
+def _coerce_dt(val) -> datetime | None:
+    if val is None:
+        return None
+    if isinstance(val, datetime):
+        return val.replace(tzinfo=timezone.utc) if val.tzinfo is None else val
+    return parse_dt(str(val))
+
+
+async def run() -> None:
+    init_firebase()
+    fs = get_firestore()
+    if fs is None:
+        print("ERROR: Firebase not initialised.", file=sys.stderr)
+        sys.exit(1)
+
+    docs = list(fs.collection(COLLECTION).stream())
+    source_count = len(docs)
+    print(f"Source (Firestore): {source_count} admin_users documents")
+
+    if source_count == 0:
+        print("Nothing to migrate.")
+        await log_run(SCRIPT, 0, 0, notes="source empty")
+        return
+
+    records = []
+    skipped = 0
+
+    for doc in docs:
+        d = doc.to_dict()
+
+        hashed_password = d.get("hashed_password") or ""
+        if not hashed_password:
+            print(f"  WARNING: {doc.id} ({d.get('email')}) has no hashed_password — skipping",
+                  file=sys.stderr)
+            skipped += 1
+            continue
+
+        email = d.get("email") or ""
+        if not email:
+            print(f"  WARNING: {doc.id} has no email — skipping", file=sys.stderr)
+            skipped += 1
+            continue
+
+        raw_role = d.get("role") or "user"
+        role = _ROLE_MAP.get(raw_role, "user")
+
+        # sysadmin/admin have no permission map
+        permissions = d.get("permissions")
+        if role in ("sysadmin", "admin"):
+            permissions = None
+
+        now = _now_utc()
+        records.append({
+            "id":              doc.id,
+            "firestore_id":    doc.id,
+            "email":           email,
+            "name":            d.get("name") or "",
+            "role":            role,
+            "permissions":     permissions,
+            "hashed_password": hashed_password,
+            "is_active":       bool(d.get("is_active", True)),
+            "created_at":      _coerce_dt(d.get("created_at")) or now,
+            "updated_at":      _coerce_dt(d.get("updated_at")) or now,
+        })
+
+    actual_source = source_count - skipped
+    print(f"  {skipped} skipped (missing email or password), {actual_source} to insert")
+
+    if not records:
+        print("Nothing to insert after filtering.")
+        await log_run(SCRIPT, source_count, 0, success=False,
+                      notes="all docs skipped — missing required fields")
+        sys.exit(1)
+
+    async with AsyncPgSession() as session:
+        async with session.begin():
+            stmt = pg_insert(Staff).values(records)
+            stmt = stmt.on_conflict_do_nothing(index_elements=["id"])
+            await session.execute(stmt)
+            dest_count = await pg_count(session, "staff")
+
+    if dest_count < actual_source:
+        msg = f"Count mismatch: expected>={actual_source}  postgres={dest_count}"
+        print(f"ERROR: {msg}", file=sys.stderr)
+        await log_run(SCRIPT, source_count, dest_count, success=False, notes=msg)
+        sys.exit(1)
+
+    print(f"Postgres: {dest_count} rows ✓")
+    note = f"{skipped} skipped (missing fields)" if skipped else None
+    await log_run(SCRIPT, source_count, dest_count, notes=note)
+
+
+if __name__ == "__main__":
+    asyncio.run(run())

backend/migration/utils.py

@@ -0,0 +1,116 @@
+"""
+Shared helpers for all Phase 1 SQLite → Postgres migration scripts.
+
+Usage in each script:
+    from migration.utils import open_sqlite, get_pg, log_run, parse_dt, parse_json
+"""
+
+import json
+import sys
+from datetime import datetime, timezone
+from pathlib import Path
+
+import aiosqlite
+from sqlalchemy import text
+from sqlalchemy.ext.asyncio import create_async_engine, async_sessionmaker, AsyncSession
+
+from config import settings
+
+
+# ── SQLite ────────────────────────────────────────────────────────────────────
+
+async def open_sqlite() -> aiosqlite.Connection:
+    """Open the SQLite database (read-only; no writes during migration)."""
+    db_path = Path(settings.sqlite_db_path)
+    if not db_path.exists():
+        print(f"ERROR: SQLite database not found at {db_path.resolve()}", file=sys.stderr)
+        sys.exit(1)
+    conn = await aiosqlite.connect(str(db_path))
+    conn.row_factory = aiosqlite.Row
+    return conn
+
+
+# ── Postgres ──────────────────────────────────────────────────────────────────
+
+def _make_pg_session() -> async_sessionmaker:
+    engine = create_async_engine(settings.database_url, pool_size=5, echo=False)
+    return async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
+
+
+AsyncPgSession = _make_pg_session()
+
+
+# ── Type helpers ──────────────────────────────────────────────────────────────
+
+def parse_dt(value: str | None) -> datetime | None:
+    """Parse a SQLite TEXT timestamp → timezone-aware datetime (UTC)."""
+    if not value:
+        return None
+    for fmt in (
+        "%Y-%m-%dT%H:%M:%S.%f",
+        "%Y-%m-%dT%H:%M:%S",
+        "%Y-%m-%d %H:%M:%S.%f",
+        "%Y-%m-%d %H:%M:%S",
+        "%Y-%m-%d",
+    ):
+        try:
+            dt = datetime.strptime(value, fmt)
+            return dt.replace(tzinfo=timezone.utc)
+        except ValueError:
+            continue
+    # ISO format with offset — let fromisoformat handle it
+    try:
+        dt = datetime.fromisoformat(value)
+        if dt.tzinfo is None:
+            dt = dt.replace(tzinfo=timezone.utc)
+        return dt
+    except ValueError:
+        pass
+    print(f"WARNING: could not parse timestamp {value!r} — using now()", file=sys.stderr)
+    return datetime.now(timezone.utc)
+
+
+def parse_json(value: str | None, default=None):
+    """Parse a SQLite TEXT JSON column → Python object."""
+    if value is None:
+        return default
+    try:
+        return json.loads(value)
+    except (json.JSONDecodeError, TypeError):
+        return default
+
+
+# ── Migration run log ─────────────────────────────────────────────────────────
+
+async def log_run(
+    script_name: str,
+    source_rows: int,
+    dest_rows: int,
+    success: bool = True,
+    notes: str | None = None,
+) -> None:
+    """Insert a row into _migration_runs recording this script's execution."""
+    async with AsyncPgSession() as session:
+        await session.execute(
+            text("""
+                INSERT INTO _migration_runs
+                    (script_name, ran_at, source_rows, dest_rows, success, notes)
+                VALUES
+                    (:script_name, now(), :source_rows, :dest_rows, :success, :notes)
+            """),
+            {
+                "script_name": script_name,
+                "source_rows": source_rows,
+                "dest_rows":   dest_rows,
+                "success":     "ok" if success else "error",
+                "notes":       notes,
+            },
+        )
+        await session.commit()
+
+
+# ── Count helper ──────────────────────────────────────────────────────────────
+
+async def pg_count(session: AsyncSession, table: str) -> int:
+    row = await session.execute(text(f"SELECT COUNT(*) FROM {table}"))
+    return row.scalar()

backend/mqtt/router.py

@@ -129,27 +129,29 @@ async def mqtt_websocket(websocket: WebSocket):
 
     try:
         from auth.utils import decode_access_token
-        from shared.firebase import get_db
+        from sqlalchemy import select
+        from database.postgres import AsyncSessionLocal
+        from staff.orm import Staff
+
         payload = decode_access_token(token)
         role = payload.get("role", "")
 
         # sysadmin and admin always have MQTT access
         if role not in ("sysadmin", "admin"):
-            # Check MQTT permission for editor/user
             user_sub = payload.get("sub", "")
-            db_inst = get_db()
+            async with AsyncSessionLocal() as session:
-            if db_inst:
+                result = await session.execute(
-                doc = db_inst.collection("admin_users").document(user_sub).get()
+                    select(Staff).where(Staff.id == user_sub).limit(1)
-                if doc.exists:
+                )
-                    perms = doc.to_dict().get("permissions", {})
+                staff = result.scalar_one_or_none()
-                    if not perms.get("mqtt", False):
+
-                        await websocket.close(code=4003, reason="MQTT access denied")
+            if staff is None:
-                        return
-                else:
                 await websocket.close(code=4003, reason="User not found")
                 return
-            else:
+
-                await websocket.close(code=4003, reason="Service unavailable")
+            perms = staff.permissions or {}
+            if not perms.get("mqtt", {}).get("access", False):
+                await websocket.close(code=4003, reason="MQTT access denied")
                 return
     except Exception:
         await websocket.close(code=4001, reason="Invalid token")

backend/notes/models.py

@@ -0,0 +1,100 @@
+from pydantic import BaseModel, Field, field_validator
+from typing import Optional, List
+from uuid import UUID
+from datetime import datetime
+
+VALID_TYPES      = {"note", "issue"}
+VALID_STATUSES   = {"open", "researching", "resolved"}
+VALID_SEVERITIES = {"low", "medium", "high", "critical"}
+VALID_CATEGORIES = {"technical", "install_support", "general"}
+VALID_ENTITIES   = {"device", "app_user", "customer"}
+
+
+class EntryLinkIn(BaseModel):
+    entity_type: str
+    entity_id: str
+
+    @field_validator("entity_type")
+    @classmethod
+    def check_entity_type(cls, v):
+        if v not in VALID_ENTITIES:
+            raise ValueError(f"entity_type must be one of {VALID_ENTITIES}")
+        return v
+
+
+class EntryCreate(BaseModel):
+    type: str
+    title: str = Field(..., max_length=500)
+    body: Optional[str] = None
+    status: Optional[str] = None
+    severity: Optional[str] = None
+    category: Optional[str] = None
+    links: List[EntryLinkIn] = []
+
+    @field_validator("type")
+    @classmethod
+    def check_type(cls, v):
+        if v not in VALID_TYPES:
+            raise ValueError(f"type must be one of {VALID_TYPES}")
+        return v
+
+    @field_validator("status")
+    @classmethod
+    def check_status(cls, v):
+        if v is not None and v not in VALID_STATUSES:
+            raise ValueError(f"status must be one of {VALID_STATUSES}")
+        return v
+
+    @field_validator("severity")
+    @classmethod
+    def check_severity(cls, v):
+        if v is not None and v not in VALID_SEVERITIES:
+            raise ValueError(f"severity must be one of {VALID_SEVERITIES}")
+        return v
+
+    @field_validator("category")
+    @classmethod
+    def check_category(cls, v):
+        if v is not None and v not in VALID_CATEGORIES:
+            raise ValueError(f"category must be one of {VALID_CATEGORIES}")
+        return v
+
+
+class EntryUpdate(BaseModel):
+    title: Optional[str] = Field(None, max_length=500)
+    body: Optional[str] = None
+    status: Optional[str] = None
+    severity: Optional[str] = None
+    category: Optional[str] = None
+
+
+class EntryLinkOut(BaseModel):
+    id: UUID
+    entity_type: str
+    entity_id: str
+    model_config = {"from_attributes": True}
+
+
+class EntryOut(BaseModel):
+    id: UUID
+    type: str
+    title: str
+    body: Optional[str]
+    status: Optional[str]
+    severity: Optional[str]
+    category: Optional[str]
+    author_id: str
+    author_name: Optional[str]
+    created_at: datetime
+    updated_at: datetime
+    links: List[EntryLinkOut] = []
+    model_config = {"from_attributes": True}
+
+
+class EntryListResponse(BaseModel):
+    data: List[EntryOut]
+    pagination: dict
+
+
+class LinksReplaceIn(BaseModel):
+    links: List[EntryLinkIn]

backend/notes/orm.py

@@ -0,0 +1,42 @@
+import uuid
+from datetime import datetime, timezone
+from sqlalchemy import Column, String, Text, DateTime, ForeignKey, UniqueConstraint
+from sqlalchemy.dialects.postgresql import UUID
+from sqlalchemy.orm import relationship
+from database.postgres import Base
+
+
+def _now():
+    return datetime.now(timezone.utc)
+
+
+class Entry(Base):
+    __tablename__ = "crm_entries"
+
+    id          = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
+    type        = Column(String(10), nullable=False)          # 'note' | 'issue'
+    title       = Column(String(500), nullable=False)
+    body        = Column(Text, nullable=True)
+    status      = Column(String(20), nullable=True)           # null for notes; open/researching/resolved for issues
+    severity    = Column(String(10), nullable=True)           # null | low | medium | high | critical
+    category    = Column(String(30), nullable=True)           # null for notes; technical | install_support | general
+    author_id   = Column(String(128), nullable=False)         # staff user ID from JWT
+    author_name = Column(String(255), nullable=True)          # denormalized for display
+    created_at  = Column(DateTime(timezone=True), nullable=False, default=_now)
+    updated_at  = Column(DateTime(timezone=True), nullable=False, default=_now, onupdate=_now)
+
+    links = relationship("EntryLink", back_populates="entry", cascade="all, delete-orphan", lazy="noload")
+
+
+class EntryLink(Base):
+    __tablename__ = "crm_entry_links"
+    __table_args__ = (
+        UniqueConstraint("entry_id", "entity_type", "entity_id"),
+    )
+
+    id          = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
+    entry_id    = Column(UUID(as_uuid=True), ForeignKey("crm_entries.id", ondelete="CASCADE"), nullable=False)
+    entity_type = Column(String(20), nullable=False)   # 'device' | 'app_user' | 'customer'
+    entity_id   = Column(String(128), nullable=False)  # Firestore ID or Postgres UUID as string
+
+    entry = relationship("Entry", back_populates="links")

backend/notes/router.py

@@ -0,0 +1,93 @@
+from fastapi import APIRouter, Depends, Query
+from uuid import UUID
+from sqlalchemy.ext.asyncio import AsyncSession
+from database.postgres import get_pg_session
+from auth.dependencies import require_permission
+from auth.models import TokenPayload
+from notes import service
+from notes.models import EntryCreate, EntryUpdate, EntryOut, EntryListResponse, LinksReplaceIn
+from shared.audit import log_action
+
+router = APIRouter(prefix="/api/notes", tags=["notes"])
+
+
+@router.get("", response_model=EntryListResponse)
+async def list_entries(
+    type:     str | None = Query(None),
+    status:   str | None = Query(None),
+    severity: str | None = Query(None),
+    category: str | None = Query(None),
+    page:     int = Query(1, ge=1),
+    limit:    int = Query(25, ge=1, le=100),
+    db:       AsyncSession = Depends(get_pg_session),
+    _user:    TokenPayload = Depends(require_permission("crm", "view")),
+):
+    rows, total = await service.list_entries(db, type, status, severity, category, page, limit)
+    return {"data": rows, "pagination": {"page": page, "limit": limit, "total": total}}
+
+
+@router.get("/by-entity/{entity_type}/{entity_id}", response_model=list[EntryOut])
+async def list_by_entity(
+    entity_type: str, entity_id: str,
+    db:    AsyncSession = Depends(get_pg_session),
+    _user: TokenPayload = Depends(require_permission("crm", "view")),
+):
+    return await service.list_entries_for_entity(db, entity_type, entity_id)
+
+
+@router.get("/{entry_id}", response_model=EntryOut)
+async def get_entry(
+    entry_id: UUID,
+    db:    AsyncSession = Depends(get_pg_session),
+    _user: TokenPayload = Depends(require_permission("crm", "view")),
+):
+    return await service.get_entry(db, entry_id)
+
+
+@router.post("", response_model=EntryOut, status_code=201)
+async def create_entry(
+    body:  EntryCreate,
+    db:    AsyncSession = Depends(get_pg_session),
+    _user: TokenPayload = Depends(require_permission("crm", "add")),
+):
+    entry = await service.create_entry(db, body, _user.sub, _user.name or _user.email)
+    await log_action(db, _user.sub, _user.name or _user.email, "CREATE", "note",
+                     str(entry.id), entry.title or entry.type)
+    return entry
+
+
+@router.patch("/{entry_id}", response_model=EntryOut)
+async def update_entry(
+    entry_id: UUID, body: EntryUpdate,
+    db:    AsyncSession = Depends(get_pg_session),
+    _user: TokenPayload = Depends(require_permission("crm", "edit")),
+):
+    entry = await service.update_entry(db, entry_id, body)
+    await log_action(db, _user.sub, _user.name or _user.email, "UPDATE", "note",
+                     str(entry_id), entry.title or entry.type)
+    return entry
+
+
+@router.patch("/{entry_id}/links", response_model=EntryOut)
+async def replace_links(
+    entry_id: UUID, body: LinksReplaceIn,
+    db:    AsyncSession = Depends(get_pg_session),
+    _user: TokenPayload = Depends(require_permission("crm", "edit")),
+):
+    entry = await service.replace_links(db, entry_id, body.links)
+    await log_action(db, _user.sub, _user.name or _user.email, "UPDATE", "note",
+                     str(entry_id), entry.title or entry.type,
+                     meta={"action_detail": "links_updated"})
+    return entry
+
+
+@router.delete("/{entry_id}", status_code=204)
+async def delete_entry(
+    entry_id: UUID,
+    db:    AsyncSession = Depends(get_pg_session),
+    _user: TokenPayload = Depends(require_permission("crm", "delete")),
+):
+    entry = await service.get_entry(db, entry_id)
+    await service.delete_entry(db, entry_id)
+    await log_action(db, _user.sub, _user.name or _user.email, "DELETE", "note",
+                     str(entry_id), entry.title or entry.type if entry else str(entry_id))

backend/notes/service.py

@@ -0,0 +1,93 @@
+import uuid
+from sqlalchemy.ext.asyncio import AsyncSession
+from sqlalchemy import select, delete, func
+from sqlalchemy.orm import selectinload
+from notes.orm import Entry, EntryLink
+from notes.models import EntryCreate, EntryUpdate, EntryLinkIn
+from shared.exceptions import NotFoundError
+
+
+async def create_entry(db: AsyncSession, data: EntryCreate, author_id: str, author_name: str) -> Entry:
+    entry = Entry(
+        type=data.type,
+        title=data.title,
+        body=data.body,
+        status=data.status if data.type == "issue" else None,
+        severity=data.severity if data.type == "issue" else None,
+        category=data.category if data.type == "issue" else None,
+        author_id=author_id,
+        author_name=author_name,
+    )
+    db.add(entry)
+    await db.flush()  # get the ID before inserting links
+
+    for link_in in data.links:
+        db.add(EntryLink(entry_id=entry.id, entity_type=link_in.entity_type, entity_id=link_in.entity_id))
+
+    await db.commit()
+    await db.refresh(entry)
+    return await _get_entry_with_links(db, entry.id)
+
+
+async def get_entry(db: AsyncSession, entry_id: uuid.UUID) -> Entry:
+    return await _get_entry_with_links(db, entry_id)
+
+
+async def list_entries(
+    db: AsyncSession,
+    type: str | None, status: str | None, severity: str | None, category: str | None,
+    page: int, limit: int,
+) -> tuple[list[Entry], int]:
+    limit  = min(100, max(1, limit))
+    offset = (max(1, page) - 1) * limit
+
+    q = select(Entry).options(selectinload(Entry.links))
+    if type:     q = q.where(Entry.type == type)
+    if status:   q = q.where(Entry.status == status)
+    if severity: q = q.where(Entry.severity == severity)
+    if category: q = q.where(Entry.category == category)
+
+    total_q = select(func.count()).select_from(q.subquery())
+    total   = (await db.execute(total_q)).scalar()
+    rows    = (await db.execute(q.order_by(Entry.created_at.desc()).limit(limit).offset(offset))).scalars().all()
+    return rows, total
+
+
+async def update_entry(db: AsyncSession, entry_id: uuid.UUID, data: EntryUpdate) -> Entry:
+    entry = await _get_entry_with_links(db, entry_id)
+    for field, value in data.model_dump(exclude_unset=True).items():
+        setattr(entry, field, value)
+    await db.commit()
+    return await _get_entry_with_links(db, entry_id)
+
+
+async def delete_entry(db: AsyncSession, entry_id: uuid.UUID):
+    entry = await _get_entry_with_links(db, entry_id)
+    await db.delete(entry)
+    await db.commit()
+
+
+async def replace_links(db: AsyncSession, entry_id: uuid.UUID, links: list[EntryLinkIn]) -> Entry:
+    await _get_entry_with_links(db, entry_id)  # raises 404 if not found
+    await db.execute(delete(EntryLink).where(EntryLink.entry_id == entry_id))
+    for link_in in links:
+        db.add(EntryLink(entry_id=entry_id, entity_type=link_in.entity_type, entity_id=link_in.entity_id))
+    await db.commit()
+    return await _get_entry_with_links(db, entry_id)
+
+
+async def list_entries_for_entity(db: AsyncSession, entity_type: str, entity_id: str) -> list[Entry]:
+    link_sq = select(EntryLink.entry_id).where(
+        EntryLink.entity_type == entity_type,
+        EntryLink.entity_id == entity_id,
+    ).subquery()
+    q = select(Entry).options(selectinload(Entry.links)).where(Entry.id.in_(select(link_sq)))
+    return (await db.execute(q.order_by(Entry.created_at.desc()))).scalars().all()
+
+
+async def _get_entry_with_links(db: AsyncSession, entry_id: uuid.UUID) -> Entry:
+    q = select(Entry).options(selectinload(Entry.links)).where(Entry.id == entry_id)
+    result = (await db.execute(q)).scalar_one_or_none()
+    if not result:
+        raise NotFoundError("Entry")
+    return result

backend/requirements.txt

@@ -15,3 +15,6 @@ weasyprint>=62.0
 jinja2>=3.1.0
 Pillow>=10.0.0
 pdf2image>=1.17.0
+asyncpg==0.30.0
+sqlalchemy[asyncio]==2.0.36
+alembic==1.14.0

backend/scripts/fix_storage_acl.py

@@ -0,0 +1,88 @@
+"""
+Fixup: call make_public() on every .bsm binary blob under melodies/ in Firebase Storage.
+Run this if blobs were uploaded but are returning AccessDenied.
+
+    docker exec -it bellsystems-backend python scripts/fix_storage_acl.py --dry-run
+    docker exec -it bellsystems-backend python scripts/fix_storage_acl.py
+"""
+
+import argparse
+import os
+import sys
+from pathlib import Path
+
+# ---------------------------------------------------------------------------
+# .env loader
+# ---------------------------------------------------------------------------
+
+def _load_env() -> dict:
+    search = Path(__file__).resolve().parent
+    for _ in range(4):
+        env_file = search / ".env"
+        if env_file.exists():
+            result = {}
+            for line in env_file.read_text(encoding="utf-8").splitlines():
+                line = line.strip()
+                if not line or line.startswith("#") or "=" not in line:
+                    continue
+                key, _, val = line.partition("=")
+                result[key.strip()] = val.strip().strip('"').strip("'")
+            print(f"[INFO] Loaded config from {env_file}")
+            return result
+        search = search.parent
+    return {}
+
+_env = _load_env()
+
+def _cfg(key: str, default: str = "") -> str:
+    return _env.get(key) or os.environ.get(key) or default
+
+
+# ---------------------------------------------------------------------------
+# Main
+# ---------------------------------------------------------------------------
+
+def run(dry_run: bool = False):
+    label = "[DRY-RUN]" if dry_run else "[LIVE]"
+
+    sa_path = _cfg("FIREBASE_SERVICE_ACCOUNT_PATH", "./firebase-service-account.json")
+    bucket_name = _cfg("FIREBASE_STORAGE_BUCKET")
+
+    if not bucket_name:
+        print("ERROR: FIREBASE_STORAGE_BUCKET not set in .env")
+        sys.exit(1)
+
+    import firebase_admin
+    from firebase_admin import credentials, storage as fb_storage
+
+    cred = credentials.Certificate(sa_path)
+    firebase_admin.initialize_app(cred, {"storageBucket": bucket_name})
+    bucket = fb_storage.bucket()
+
+    print(f"\n{label} Scanning melodies/ in bucket: {bucket_name}\n")
+
+    blobs = list(bucket.list_blobs(prefix="melodies/"))
+    bsm_blobs = [b for b in blobs if b.name.lower().endswith(".bsm")]
+
+    print(f"Found {len(bsm_blobs)} .bsm blob(s)\n")
+
+    fixed = 0
+    for blob in bsm_blobs:
+        print(f"  {'[skip] ' if dry_run else '[fix]  '}{blob.name}")
+        if not dry_run:
+            try:
+                blob.make_public()
+                fixed += 1
+            except Exception as e:
+                print(f"         ERROR: {e}")
+
+    print(f"\n{label} Done. {fixed if not dry_run else len(bsm_blobs)} blob(s) {'would be ' if dry_run else ''}made public.")
+    if dry_run:
+        print("Run without --dry-run to apply.")
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(description="Make all melody .bsm blobs public in Firebase Storage")
+    parser.add_argument("--dry-run", action="store_true")
+    args = parser.parse_args()
+    run(dry_run=args.dry_run)

backend/scripts/migrate_melody_binaries_to_local.py

@@ -0,0 +1,262 @@
+"""
+One-time migration: move every melody's .bsm binary off Firebase Storage and
+onto local disk, updating melody.url to point at our own plain-HTTP download
+endpoint instead of the Firebase public URL.
+
+Background: ESP32 devices can't afford the RAM for a TLS client, so melody
+binaries are now served from ./storage/melody_binaries via
+GET /api/melodies/download/{pid} (exposed publicly as
+melodies.bellsystems.net/download/{pid}) instead of Firebase Storage's
+HTTPS-only URLs. This script backfills existing melodies created before that
+change — melodies created/edited after the change already use the new path
+automatically (via "Select Archetype" / "Build on the Fly").
+
+What this script does, for every melody whose url points at Firebase Storage:
+  1. Downloads the .bsm bytes from the Firebase Storage URL
+  2. Writes them to ./storage/melody_binaries/{pid}.bsm
+  3. Updates melody.url -> http://melodies.bellsystems.net/download/{pid}
+  4. Updates both SQLite (melody_drafts) and Firestore (if published)
+
+Storage is keyed on pid, not melody uid: pid identifies the underlying
+archetype binary, and multiple melodies legitimately share one pid (each
+remaps the same note sequence to different bells/speed/duration). The script
+downloads each distinct pid's binary only once (from whichever melody it
+encounters first) and reuses it for every other melody sharing that pid.
+
+CAVEAT: some existing melodies share a pid despite pointing at *different*
+Firebase source files (observed in practice — e.g. voice-count variants like
+1N_/2N_/3N_/4N_-prefixed filenames all filed under one pid). This script does
+NOT attempt to detect or fix that — it flags it in the output (see "WARNING:
+also seen with a different source file") so you can review and re-assign the
+correct archetype per melody afterward using the in-app playback button. This
+is a pre-existing PID data issue, not something safe to silently resolve here.
+
+Melodies with no pid are skipped and reported, since the new URL scheme
+requires one (pid is the public lookup key for the download route).
+
+Run from the backend/ directory (or scripts/ — it searches upward for .env):
+
+    docker exec -it bellsystems-backend python scripts/migrate_melody_binaries_to_local.py --dry-run
+    docker exec -it bellsystems-backend python scripts/migrate_melody_binaries_to_local.py
+
+Requires: firebase-admin, requests (both already in the backend image).
+"""
+
+import argparse
+import json
+import os
+import sqlite3
+import sys
+from pathlib import Path
+
+import requests
+
+# Melody names may contain Greek/non-ASCII text — force UTF-8 stdout so this
+# doesn't crash on Windows consoles defaulting to cp1252.
+if hasattr(sys.stdout, "reconfigure"):
+    sys.stdout.reconfigure(encoding="utf-8", errors="replace")
+
+
+# ---------------------------------------------------------------------------
+# .env loader — searches upward from script location for a .env file
+# ---------------------------------------------------------------------------
+
+def _load_env() -> dict:
+    search = Path(__file__).resolve().parent
+    for _ in range(4):
+        env_file = search / ".env"
+        if env_file.exists():
+            result = {}
+            for line in env_file.read_text(encoding="utf-8").splitlines():
+                line = line.strip()
+                if not line or line.startswith("#") or "=" not in line:
+                    continue
+                key, _, val = line.partition("=")
+                result[key.strip()] = val.strip().strip('"').strip("'")
+            print(f"[INFO] Loaded config from {env_file}")
+            return result
+        search = search.parent
+    print("[WARN] No .env file found — relying on environment variables")
+    return {}
+
+
+_env = _load_env()
+
+
+def _cfg(key: str, default: str = "") -> str:
+    return _env.get(key) or os.environ.get(key) or default
+
+
+# ---------------------------------------------------------------------------
+# Firebase (only needed to read Firestore for published melodies)
+# ---------------------------------------------------------------------------
+try:
+    import firebase_admin
+    from firebase_admin import credentials, firestore
+
+    _fb_app = None
+
+    def _init_firebase(sa_path: str, bucket_name: str):
+        global _fb_app
+        if _fb_app is not None:
+            return
+        cred = credentials.Certificate(sa_path)
+        _fb_app = firebase_admin.initialize_app(cred, {"storageBucket": bucket_name})
+
+    def get_firestore(sa_path: str, bucket_name: str):
+        _init_firebase(sa_path, bucket_name)
+        return firestore.client()
+
+    FIREBASE_AVAILABLE = True
+except Exception as _fb_err:
+    print(f"[WARN] Firebase unavailable: {_fb_err}")
+    FIREBASE_AVAILABLE = False
+
+    def get_firestore(sa_path: str, bucket_name: str):
+        return None
+
+
+def _is_firebase_url(url: str) -> bool:
+    return bool(url) and ("firebasestorage" in url or "storage.googleapis.com" in url)
+
+
+# ---------------------------------------------------------------------------
+# Main migration
+# ---------------------------------------------------------------------------
+
+def run(dry_run: bool = False, db_path: str = "", storage_dir: str = "", base_url: str = ""):
+    label = "[DRY-RUN]" if dry_run else "[LIVE]"
+    db_path = db_path or _cfg("SQLITE_DB_PATH", "./data/database.db")
+    storage_dir = Path(storage_dir or _cfg("MELODY_BINARIES_STORAGE_PATH", "./storage/melody_binaries"))
+    base_url = base_url or _cfg("MELODY_DOWNLOAD_BASE_URL", "http://melodies.bellsystems.net/download")
+    sa_path = _cfg("FIREBASE_SERVICE_ACCOUNT_PATH", "./firebase-service-account.json")
+    bucket_name = _cfg("FIREBASE_STORAGE_BUCKET")
+
+    print(f"\n{label} Database: {db_path}")
+    print(f"{label} Local binary storage: {storage_dir}")
+    print(f"{label} New base URL: {base_url}\n")
+
+    firestore_db = get_firestore(sa_path, bucket_name) if FIREBASE_AVAILABLE and bucket_name else None
+
+    con = sqlite3.connect(db_path)
+    con.row_factory = sqlite3.Row
+
+    rows = con.execute("SELECT * FROM melody_drafts").fetchall()
+    candidates = []
+    for row in rows:
+        data = json.loads(row["data"])
+        if _is_firebase_url(data.get("url", "")):
+            candidates.append((dict(row), data))
+
+    if not candidates:
+        print("No melodies with a Firebase Storage url found. Nothing to do.")
+        con.close()
+        return
+
+    print(f"Found {len(candidates)} melody(ies) with a Firebase Storage binary url:\n")
+
+    if not dry_run:
+        storage_dir.mkdir(parents=True, exist_ok=True)
+
+    migrated = 0
+    skipped_no_pid = 0
+    failed = 0
+    downloaded_pids: dict[str, bool] = {}  # pid -> whether the local file is ready to use
+    pid_source_urls: dict[str, str] = {}   # pid -> first-seen Firebase source url, to detect mismatches
+    mismatch_warnings = []
+
+    for row, data in candidates:
+        melody_id = row["id"]
+        pid = data.get("pid")
+        old_url = data.get("url")
+        name = (data.get("information") or {}).get("name", "")
+
+        label_id = f"[{melody_id[:8]}]"
+
+        if not pid:
+            print(f"  {label_id} SKIPPED: no pid set — the new download route requires one (name: {name})")
+            skipped_no_pid += 1
+            continue
+
+        print(f"  {label_id} {name!r}  pid={pid}")
+        print(f"           {old_url}")
+
+        first_seen_url = pid_source_urls.get(pid)
+        if first_seen_url and first_seen_url != old_url:
+            warning = (f"pid '{pid}': melody {label_id} points at a DIFFERENT Firebase file "
+                       f"than the one already saved for this pid — only the first one encountered "
+                       f"is kept as {pid}.bsm. Verify with the playback button after migrating.")
+            print(f"           WARNING: also seen with a different source file! {warning}")
+            mismatch_warnings.append(warning)
+        else:
+            pid_source_urls[pid] = old_url
+
+        dest = storage_dir / f"{pid}.bsm"
+        new_url = f"{base_url}/{pid}"
+
+        if dry_run:
+            if pid in downloaded_pids:
+                print(f"           -> pid already handled by another melody in this run, would reuse {dest}")
+            else:
+                print(f"           -> would download and save to {dest}")
+            print(f"           -> would set url = {new_url}")
+            downloaded_pids[pid] = True
+            continue
+
+        if pid not in downloaded_pids:
+            try:
+                resp = requests.get(old_url, timeout=30)
+                resp.raise_for_status()
+            except Exception as e:
+                print(f"           ERROR downloading binary: {e}")
+                failed += 1
+                continue
+            dest.write_bytes(resp.content)
+            print(f"           saved {len(resp.content)} bytes -> {dest}")
+            downloaded_pids[pid] = True
+        else:
+            print(f"           reusing already-downloaded {dest} (shared pid)")
+
+        print(f"           url -> {new_url}")
+
+        data["url"] = new_url
+        con.execute(
+            "UPDATE melody_drafts SET data=? WHERE id=?",
+            (json.dumps(data), melody_id),
+        )
+        con.commit()
+
+        if row["status"] == "published":
+            if firestore_db:
+                try:
+                    firestore_db.collection("melodies").document(melody_id).update({"url": new_url})
+                    print(f"           Firestore updated")
+                except Exception as e:
+                    print(f"           ERROR updating Firestore: {e}")
+            else:
+                print(f"           WARNING: melody is published but Firestore unavailable!")
+
+        migrated += 1
+
+    con.close()
+    print(f"\n{'-'*60}")
+    print(f"{label} Done. Migrated: {migrated}, skipped (no pid): {skipped_no_pid}, failed: {failed}")
+    if mismatch_warnings:
+        print(f"\n{len(mismatch_warnings)} pid(s) had melodies pointing at different source files "
+              f"— please review these with the playback button:")
+        for w in mismatch_warnings:
+            print(f"  - {w}")
+    if dry_run:
+        print("\nThis was a dry run. No changes were made. Run without --dry-run to apply.")
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(
+        description="Migrate melody .bsm binaries from Firebase Storage to local disk"
+    )
+    parser.add_argument("--dry-run", action="store_true", help="Preview changes without writing anything")
+    parser.add_argument("--db", default="", help="Override SQLite database path (default: read from .env)")
+    parser.add_argument("--storage-dir", default="", help="Override local binary storage dir (default: read from .env)")
+    parser.add_argument("--base-url", default="", help="Override download base URL (default: read from .env)")
+    args = parser.parse_args()
+    run(dry_run=args.dry_run, db_path=args.db, storage_dir=args.storage_dir, base_url=args.base_url)

backend/scripts/migrate_pids_hyphens_to_underscores.py

@@ -0,0 +1,408 @@
+"""
+One-time migration: replace hyphens with underscores in archetype PIDs and all
+melody PIDs/URLs that reference them.
+
+What this script does:
+  1. Renames each archetype's PID in SQLite (built_melodies.pid)
+  2. Renames the local .bsm binary file on disk
+  3. Updates built_melodies.binary_path in SQLite
+  4. Regenerates built_melodies.progmem_code
+  5. For every melody assigned to that archetype:
+       a. Downloads the .bsm bytes from Firebase Storage
+       b. Deletes the old blob
+       c. Re-uploads under the new PID name  -> gets new public URL
+       d. Updates melody.pid  (if it matched old archetype PID)
+       e. Updates melody.url  -> new Firebase URL
+       f. Updates both SQLite AND Firestore (if melody is published)
+
+Run from the backend/ directory (or scripts/ — it searches upward for .env):
+
+    python scripts/migrate_pids_hyphens_to_underscores.py --dry-run
+    python scripts/migrate_pids_hyphens_to_underscores.py
+
+All config is auto-loaded from the project .env file. No extra arguments needed.
+Optional overrides:
+    --db        Override SQLite database path
+    --dry-run   Preview changes without writing anything
+
+Requires only: firebase-admin  (pip install firebase-admin)
+"""
+
+import argparse
+import json
+import os
+import shutil
+import sqlite3
+import sys
+from datetime import datetime
+from pathlib import Path
+
+
+# ---------------------------------------------------------------------------
+# .env loader — searches upward from script location for a .env file
+# ---------------------------------------------------------------------------
+
+def _load_env() -> dict:
+    """Parse key=value pairs from the nearest .env file up the directory tree."""
+    search = Path(__file__).resolve().parent
+    for _ in range(4):  # look up to 4 levels up
+        env_file = search / ".env"
+        if env_file.exists():
+            result = {}
+            for line in env_file.read_text(encoding="utf-8").splitlines():
+                line = line.strip()
+                if not line or line.startswith("#") or "=" not in line:
+                    continue
+                key, _, val = line.partition("=")
+                result[key.strip()] = val.strip().strip('"').strip("'")
+            print(f"[INFO] Loaded config from {env_file}")
+            return result
+        search = search.parent
+    print("[WARN] No .env file found — relying on environment variables")
+    return {}
+
+
+_env = _load_env()
+
+
+def _cfg(key: str, default: str = "") -> str:
+    """Get a config value: .env first, then os.environ, then default."""
+    return _env.get(key) or os.environ.get(key) or default
+
+# ---------------------------------------------------------------------------
+# Firebase (optional – skipped if not configured)
+# ---------------------------------------------------------------------------
+try:
+    import firebase_admin
+    from firebase_admin import credentials, firestore, storage as fb_storage
+
+    _fb_app = None
+
+    def _init_firebase(sa_path: str, bucket_name: str):
+        global _fb_app
+        if _fb_app is not None:
+            return
+        cred = credentials.Certificate(sa_path)
+        _fb_app = firebase_admin.initialize_app(cred, {
+            "storageBucket": bucket_name,
+        })
+
+    def get_firestore(sa_path: str, bucket_name: str):
+        _init_firebase(sa_path, bucket_name)
+        return firestore.client()
+
+    def get_bucket(sa_path: str, bucket_name: str):
+        _init_firebase(sa_path, bucket_name)
+        return fb_storage.bucket()
+
+    FIREBASE_AVAILABLE = True
+except Exception as _fb_err:
+    print(f"[WARN] Firebase unavailable: {_fb_err}")
+    FIREBASE_AVAILABLE = False
+
+    def get_firestore(sa_path: str, bucket_name: str):
+        return None
+
+    def get_bucket(sa_path: str, bucket_name: str):
+        return None
+
+
+# ---------------------------------------------------------------------------
+# Helpers
+# ---------------------------------------------------------------------------
+
+def fix_pid(pid: str) -> str:
+    """Replace hyphens with underscores in a PID string."""
+    return pid.replace("-", "_")
+
+
+def needs_fix(pid: str) -> bool:
+    return pid is not None and "-" in pid
+
+
+def _is_binary_blob(name: str) -> bool:
+    lower = (name or "").lower()
+    base = lower.rsplit("/", 1)[-1]
+    if "preview" in base:
+        return False
+    return ("binary" in base) or base.endswith(".bin") or base.endswith(".bsm")
+
+
+def _safe_seg(raw: str | None, fallback: str) -> str:
+    value = (raw or "").strip() or fallback
+    chars = []
+    for ch in value:
+        if ch.isalnum() or ch in ("-", "_", "."):
+            chars.append(ch)
+        else:
+            chars.append("_")
+    cleaned = "".join(chars).strip("._")
+    return cleaned or fallback
+
+
+def _storage_prefixes(melody_id: str, melody_uid: str | None) -> list[str]:
+    uid_seg = _safe_seg(melody_uid, melody_id)
+    id_seg = _safe_seg(melody_id, melody_id)
+    prefixes = [f"melodies/{uid_seg}/"]
+    if uid_seg != id_seg:
+        prefixes.append(f"melodies/{id_seg}/")
+    return prefixes
+
+
+def _progmem_array(name: str, values: list[int], vpl: int = 8) -> str:
+    array_name = f"melody_builtin_{name.lower()}"
+    lines = [f"const uint16_t PROGMEM {array_name}[] = {{"]
+    for i in range(0, len(values), vpl):
+        chunk = values[i: i + vpl]
+        hex_vals = [f"0x{v:04X}" for v in chunk]
+        suffix = "," if i + len(chunk) < len(values) else ""
+        lines.append("    " + ", ".join(hex_vals) + suffix)
+    lines.append("};")
+    return "\n".join(lines)
+
+
+def _parse_notation(token: str) -> int:
+    token = token.strip()
+    if not token or token == "0":
+        return 0
+    v = 0
+    for part in token.split("+"):
+        try:
+            n = int(part.strip())
+            if 1 <= n <= 16:
+                v |= 1 << (n - 1)
+        except ValueError:
+            pass
+    return v
+
+
+def _steps_to_values(steps: str) -> list[int]:
+    return [_parse_notation(s) for s in steps.split(",")]
+
+
+def _regenerate_progmem(name: str, pid: str, steps: str) -> str:
+    values = _steps_to_values(steps)
+    array_name = f"melody_builtin_{name.lower()}"
+    id_name = pid if pid else f"builtin_{name.lower()}"
+    display_name = name.replace("_", " ").title()
+    ts = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
+    parts = [
+        f"// Generated: {ts}",
+        f"// Melody: {display_name}  |  PID: {id_name}",
+        "",
+        _progmem_array(name, values),
+        "",
+        "// --- Add this entry to your MELODY_LIBRARY[] array: ---",
+        "// {",
+        f'//     "{display_name}",',
+        f'//     "{id_name}",',
+        f"//     {array_name},",
+        f"//     sizeof({array_name}) / sizeof(uint16_t)",
+        "// }",
+    ]
+    return "\n".join(parts)
+
+
+# ---------------------------------------------------------------------------
+# Main migration
+# ---------------------------------------------------------------------------
+
+def run(dry_run: bool = False, db_path: str = ""):
+    label = "[DRY-RUN]" if dry_run else "[LIVE]"
+    db_path = db_path or _cfg("SQLITE_DB_PATH", "./data/database.db")
+    sa_path = _cfg("FIREBASE_SERVICE_ACCOUNT_PATH", "./firebase-service-account.json")
+    bucket_name = _cfg("FIREBASE_STORAGE_BUCKET")
+    print(f"\n{label} Database: {db_path}")
+    print(f"{label} Firebase available: {FIREBASE_AVAILABLE}, bucket: {bucket_name or '(not set)'}\n")
+
+    con = sqlite3.connect(db_path)
+    con.row_factory = sqlite3.Row
+
+    # -----------------------------------------------------------------------
+    # Step 1: collect archetypes that need fixing
+    # -----------------------------------------------------------------------
+    archetypes = con.execute("SELECT * FROM built_melodies").fetchall()
+    to_fix = [dict(a) for a in archetypes if needs_fix(a["pid"])]
+
+    if not to_fix:
+        print("No archetypes with hyphens in PID found. Nothing to do.")
+        con.close()
+        return
+
+    print(f"Found {len(to_fix)} archetype(s) with hyphens in PID:\n")
+    for a in to_fix:
+        print(f"  [{a['id'][:8]}...]  '{a['pid']}'  →  '{fix_pid(a['pid'])}'  (name: {a['name']})")
+    print()
+
+    bucket = get_bucket(sa_path, bucket_name) if FIREBASE_AVAILABLE and bucket_name else None
+    firestore_db = get_firestore(sa_path, bucket_name) if FIREBASE_AVAILABLE and bucket_name else None
+
+    total_melodies_updated = 0
+
+    for archetype in to_fix:
+        old_pid = archetype["pid"]
+        new_pid = fix_pid(old_pid)
+        arch_id = archetype["id"]
+        arch_name = archetype["name"]
+        assigned_ids: list[str] = json.loads(archetype["assigned_melody_ids"] or "[]")
+
+        print(f"━━━ Archetype: {arch_name}  ({old_pid} → {new_pid}) ━━━")
+
+        # -------------------------------------------------------------------
+        # Step 2: rename local .bsm file
+        # -------------------------------------------------------------------
+        old_path = Path(archetype["binary_path"]) if archetype.get("binary_path") else None
+        new_path = None
+
+        if old_path and old_path.exists():
+            new_path = old_path.parent / f"{new_pid}.bsm"
+            print(f"  [BSM] {old_path.name}  →  {new_path.name}")
+            if not dry_run:
+                shutil.move(str(old_path), str(new_path))
+        elif old_path:
+            # File expected but missing — still derive new path so DB is correct
+            new_path = old_path.parent / f"{new_pid}.bsm"
+            print(f"  [BSM] WARNING: expected file not found: {old_path}")
+        else:
+            print(f"  [BSM] No binary_path recorded, skipping file rename")
+
+        # -------------------------------------------------------------------
+        # Step 3 & 4: update SQLite — pid, binary_path, progmem_code
+        # -------------------------------------------------------------------
+        new_progmem = _regenerate_progmem(arch_name, new_pid, archetype["steps"])
+        print(f"  [DB]  Updating archetype record in SQLite")
+        if not dry_run:
+            con.execute(
+                "UPDATE built_melodies SET pid=?, binary_path=?, progmem_code=?, updated_at=? WHERE id=?",
+                (new_pid, str(new_path) if new_path else archetype["binary_path"],
+                 new_progmem, datetime.utcnow().isoformat(), arch_id),
+            )
+            con.commit()
+
+        # -------------------------------------------------------------------
+        # Step 5–7: update each assigned melody
+        # -------------------------------------------------------------------
+        if not assigned_ids:
+            print(f"  [MELODIES] No assigned melodies, skipping.\n")
+            continue
+
+        print(f"  [MELODIES] Processing {len(assigned_ids)} assigned melody(ies)...")
+
+        for melody_id in assigned_ids:
+            row = con.execute("SELECT * FROM melody_drafts WHERE id=?", (melody_id,)).fetchone()
+            if not row:
+                print(f"    [{melody_id[:8]}] WARNING: melody not found in SQLite, skipping")
+                continue
+
+            row = dict(row)
+            melody_data: dict = json.loads(row["data"]) if isinstance(row["data"], str) else row["data"]
+            melody_uid = melody_data.get("uid")
+            melody_pid = melody_data.get("pid", "")
+            melody_url = melody_data.get("url", "")
+            status = row.get("status", "draft")
+
+            # Determine if this melody's pid also has hyphens matching old archetype pid
+            new_melody_pid = fix_pid(melody_pid) if melody_pid and "-" in melody_pid else melody_pid
+
+            new_url = melody_url  # will be updated if Firebase succeeds
+
+            # ---------------------------------------------------------------
+            # Firebase Storage: delete old blob, re-upload under new name
+            # ---------------------------------------------------------------
+            if bucket and melody_url:
+                try:
+                    prefixes = _storage_prefixes(melody_id, melody_uid)
+                    primary_prefix = prefixes[0]
+
+                    # Find and download the current binary blob
+                    all_blobs = []
+                    for prefix in prefixes:
+                        all_blobs.extend(list(bucket.list_blobs(prefix=prefix)))
+                    binary_blobs = [b for b in all_blobs if _is_binary_blob(b.name)]
+
+                    if binary_blobs:
+                        # Download bytes from the first (should only be one)
+                        src_blob = binary_blobs[0]
+                        binary_bytes = src_blob.download_as_bytes()
+
+                        new_storage_path = f"{primary_prefix}{new_pid}.bsm"
+                        print(f"    [{melody_id[:8]}] Storage: {src_blob.name.split('/')[-1]}  →  {new_pid}.bsm")
+
+                        if not dry_run:
+                            # Delete old blob(s)
+                            for b in binary_blobs:
+                                b.delete()
+
+                            # Upload under new name
+                            new_blob = bucket.blob(new_storage_path)
+                            new_blob.upload_from_string(binary_bytes, content_type="application/octet-stream")
+                            new_blob.make_public()
+                            new_url = new_blob.public_url
+                    else:
+                        print(f"    [{melody_id[:8]}] WARNING: no binary blob found in storage for this melody")
+
+                except Exception as e:
+                    print(f"    [{melody_id[:8]}] ERROR during Firebase Storage operation: {e}")
+            elif not bucket:
+                print(f"    [{melody_id[:8]}] Firebase not available, skipping storage rename")
+
+            # ---------------------------------------------------------------
+            # Update melody data
+            # ---------------------------------------------------------------
+            changed = False
+            if new_melody_pid != melody_pid:
+                print(f"    [{melody_id[:8]}] PID: '{melody_pid}'  →  '{new_melody_pid}'")
+                melody_data["pid"] = new_melody_pid
+                changed = True
+
+            if new_url != melody_url:
+                print(f"    [{melody_id[:8]}] URL updated")
+                melody_data["url"] = new_url
+                changed = True
+
+            if not changed and new_url == melody_url:
+                print(f"    [{melody_id[:8]}] No data changes needed")
+                continue
+
+            if not dry_run:
+                # Update SQLite
+                con.execute(
+                    "UPDATE melody_drafts SET data=? WHERE id=?",
+                    (json.dumps(melody_data), melody_id),
+                )
+                con.commit()
+
+                # Update Firestore if published
+                if status == "published" and firestore_db:
+                    try:
+                        doc_ref = firestore_db.collection("melodies").document(melody_id)
+                        update_fields = {}
+                        if new_melody_pid != melody_pid:
+                            update_fields["pid"] = new_melody_pid
+                        if new_url != melody_url:
+                            update_fields["url"] = new_url
+                        if update_fields:
+                            doc_ref.update(update_fields)
+                            print(f"    [{melody_id[:8]}] Firestore updated")
+                    except Exception as e:
+                        print(f"    [{melody_id[:8]}] ERROR updating Firestore: {e}")
+                elif status == "published" and not firestore_db:
+                    print(f"    [{melody_id[:8]}] WARNING: melody is published but Firestore unavailable!")
+
+            total_melodies_updated += 1
+
+        print()
+
+    con.close()
+    print(f"{'━'*60}")
+    print(f"{label} Done. Archetypes fixed: {len(to_fix)}, Melody records updated: {total_melodies_updated}")
+    if dry_run:
+        print("\nThis was a dry run. No changes were made. Run without --dry-run to apply.")
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(description="Migrate archetype/melody PIDs: replace hyphens with underscores")
+    parser.add_argument("--dry-run", action="store_true", help="Preview changes without writing anything")
+    parser.add_argument("--db", default="", help="Override SQLite database path (default: read from .env)")
+    args = parser.parse_args()
+    run(dry_run=args.dry_run, db_path=args.db)

backend/search/router.py

@@ -0,0 +1,163 @@
+import asyncio
+import json
+from fastapi import APIRouter, Depends, Query
+from auth.dependencies import get_current_user
+from auth.models import TokenPayload
+from devices import service as devices_service
+from users import service as users_service
+from crm import service as crm_service
+from melodies import service as melodies_service
+
+router = APIRouter(prefix="/api/search", tags=["search"])
+
+LIMIT = 5
+
+
+def _truncate(s: str, n: int = 48) -> str:
+    if not s:
+        return ""
+    return s if len(s) <= n else s[:n - 1] + "…"
+
+
+def _search_devices(q: str) -> list[dict]:
+    try:
+        results = devices_service.list_devices(search=q)
+    except Exception:
+        return []
+    out = []
+    for d in results[:LIMIT]:
+        label = d.device_name or d.serial_number or d.device_id or d.id
+        sublabel = d.serial_number if d.device_name else None
+        out.append({
+            "type": "device",
+            "id": d.id,
+            "label": _truncate(label),
+            "sublabel": _truncate(sublabel) if sublabel else None,
+            "url": f"/devices/{d.id}",
+        })
+    return out
+
+
+def _search_users(q: str) -> list[dict]:
+    try:
+        results = users_service.list_users(search=q)
+    except Exception:
+        return []
+    out = []
+    for u in results[:LIMIT]:
+        label = u.display_name or u.email or u.id
+        sublabel = u.email if u.display_name else None
+        out.append({
+            "type": "user",
+            "id": u.id,
+            "label": _truncate(label),
+            "sublabel": _truncate(sublabel) if sublabel else None,
+            "url": f"/users/{u.id}",
+        })
+    return out
+
+
+def _search_customers(q: str) -> list[dict]:
+    try:
+        results = crm_service.list_customers(search=q)
+    except Exception:
+        return []
+    out = []
+    for c in results[:LIMIT]:
+        name_parts = [c.name, c.surname]
+        label = " ".join(p for p in name_parts if p) or c.organization or c.id
+        sublabel_parts = []
+        if c.organization and (c.name or c.surname):
+            sublabel_parts.append(c.organization)
+        if c.location:
+            if c.location.city:
+                sublabel_parts.append(c.location.city)
+            if c.location.country:
+                sublabel_parts.append(c.location.country)
+        out.append({
+            "type": "customer",
+            "id": c.id,
+            "label": _truncate(label),
+            "sublabel": _truncate(" · ".join(sublabel_parts)) if sublabel_parts else None,
+            "url": f"/crm/customers/{c.id}",
+        })
+    return out
+
+
+def _search_products(q: str) -> list[dict]:
+    try:
+        results = crm_service.list_products(search=q)
+    except Exception:
+        return []
+    out = []
+    for p in results[:LIMIT]:
+        sublabel_parts = []
+        if p.category:
+            sublabel_parts.append(p.category.value.replace("_", " ").title())
+        if p.sku:
+            sublabel_parts.append(p.sku)
+        out.append({
+            "type": "product",
+            "id": p.id,
+            "label": _truncate(p.name or p.id),
+            "sublabel": _truncate(" · ".join(sublabel_parts)) if sublabel_parts else None,
+            "url": f"/crm/products/{p.id}",
+        })
+    return out
+
+
+async def _search_melodies(q: str) -> list[dict]:
+    try:
+        results = await melodies_service.list_melodies(search=q)
+    except Exception:
+        return []
+    out = []
+    for m in results[:LIMIT]:
+        try:
+            name_dict = json.loads(m.information.name) if m.information.name else {}
+            label = name_dict.get("en") or name_dict.get("gr") or next(iter(name_dict.values()), None) or m.id
+        except Exception:
+            label = m.information.name or m.id
+        sublabel_parts = []
+        if m.pid:
+            sublabel_parts.append(m.pid)
+        if m.information.melodyTone:
+            sublabel_parts.append(m.information.melodyTone.value.title())
+        if m.information.totalActiveBells:
+            sublabel_parts.append(f"{m.information.totalActiveBells} bells")
+        out.append({
+            "type": "melody",
+            "id": m.id,
+            "label": _truncate(label),
+            "sublabel": _truncate(" · ".join(sublabel_parts)) if sublabel_parts else None,
+            "url": f"/melodies/{m.id}",
+        })
+    return out
+
+
+@router.get("")
+async def global_search(
+    q: str = Query(..., min_length=1, max_length=100),
+    _user: TokenPayload = Depends(get_current_user),
+):
+    q = q.strip()
+    if not q:
+        return {"results": []}
+
+    # Run sync searches in a thread pool, melody search is already async
+    loop = asyncio.get_event_loop()
+    devices_fut = loop.run_in_executor(None, _search_devices, q)
+    users_fut = loop.run_in_executor(None, _search_users, q)
+    customers_fut = loop.run_in_executor(None, _search_customers, q)
+    products_fut = loop.run_in_executor(None, _search_products, q)
+    melodies_task = _search_melodies(q)
+
+    devices, users, customers, products, melodies = await asyncio.gather(
+        devices_fut, users_fut, customers_fut, products_fut, melodies_task
+    )
+
+    results = []
+    for group in (devices, users, customers, products, melodies):
+        results.extend(group)
+
+    return {"results": results}

backend/seed_admin_postgres.py

@@ -0,0 +1,66 @@
+"""
+Seed script to create the first sysadmin user directly in Postgres.
+Use this after Phase 3 cutover — do not use seed_admin.py (Firestore) anymore.
+
+Usage:
+    python seed_admin_postgres.py
+    python seed_admin_postgres.py --email admin@bellsystems.com --password secret --name "Admin"
+"""
+import argparse
+import asyncio
+import sys
+import uuid
+from datetime import datetime, timezone
+from getpass import getpass
+
+from sqlalchemy import select
+
+from database.postgres import AsyncSessionLocal
+from staff.orm import Staff
+from auth.utils import hash_password
+
+
+async def seed_superadmin(email: str, password: str, name: str) -> None:
+    async with AsyncSessionLocal() as db:
+        existing = await db.execute(select(Staff).where(Staff.email == email).limit(1))
+        if existing.scalar_one_or_none() is not None:
+            print(f"User with email '{email}' already exists. Aborting.")
+            sys.exit(1)
+
+        now = datetime.now(timezone.utc)
+        uid = str(uuid.uuid4())
+        staff = Staff(
+            id=uid,
+            firestore_id=None,
+            email=email,
+            name=name,
+            role="sysadmin",
+            hashed_password=hash_password(password),
+            is_active=True,
+            permissions=None,
+            ui_prefs={},
+            created_at=now,
+            updated_at=now,
+        )
+        db.add(staff)
+        await db.commit()
+
+    print("SysAdmin created successfully!")
+    print(f"  Email: {email}")
+    print(f"  Name:  {name}")
+    print(f"  Role:  sysadmin")
+    print(f"  ID:    {uid}")
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(description="Seed a sysadmin user in Postgres")
+    parser.add_argument("--email", default=None)
+    parser.add_argument("--password", default=None)
+    parser.add_argument("--name", default=None)
+    args = parser.parse_args()
+
+    email    = args.email    or input("Email: ")
+    name     = args.name     or input("Name: ")
+    password = args.password or getpass("Password: ")
+
+    asyncio.run(seed_superadmin(email, password, name))

backend/settings/orm.py

@@ -0,0 +1,26 @@
+from datetime import datetime, timezone
+from sqlalchemy import Column, DateTime, String, Text
+from sqlalchemy.dialects.postgresql import JSONB
+from database.postgres import Base
+
+
+def _now():
+    return datetime.now(timezone.utc)
+
+
+class ConsoleSetting(Base):
+    """Key/value store for console configuration (replaces Firestore 'settings' doc)."""
+    __tablename__ = "console_settings"
+
+    key        = Column(String(128), primary_key=True)
+    value      = Column(JSONB)                           # any JSON value
+    updated_at = Column(DateTime(timezone=True), nullable=False, default=_now, onupdate=_now)
+
+
+class PublicFeature(Base):
+    """Public-facing feature flags and configuration (replaces Firestore 'public_features' doc)."""
+    __tablename__ = "public_features"
+
+    key        = Column(String(128), primary_key=True)
+    value      = Column(JSONB)
+    updated_at = Column(DateTime(timezone=True), nullable=False, default=_now, onupdate=_now)

backend/shared/audit.py

@@ -0,0 +1,84 @@
+"""
+Audit log utility — all services call log_action() to record staff events.
+
+Usage:
+    from shared.audit import log_action
+
+    await log_action(
+        db, actor_id, actor_name,
+        action="CREATE", entity_type="customer", entity_id=cust_id,
+        entity_label="Church of St. George",
+    )
+
+    await log_action(
+        db, actor_id, actor_name,
+        action="UPDATE", entity_type="order", entity_id=order_id,
+        entity_label="ORD-0042",
+        changes={"status": {"old": "negotiating", "new": "confirmed"}},
+    )
+
+Never raises — a logging failure must never break the primary operation.
+The call is fire-and-forget safe: wrap in try/except internally.
+"""
+
+from datetime import datetime, timezone
+from typing import Any
+
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from shared.orm import AuditLog
+
+
+async def log_action(
+    db: AsyncSession,
+    actor_id: str,
+    actor_name: str,
+    action: str,
+    entity_type: str,
+    entity_id: str,
+    entity_label: str | None = None,
+    changes: dict[str, Any] | None = None,
+    meta: dict[str, Any] | None = None,
+) -> None:
+    """
+    Insert one row into audit_log. Never raises — failures are silently swallowed
+    so a logging error never disrupts the primary request.
+
+    Always commits its own mini-transaction. Callers that run inside a larger
+    transaction (e.g. staff service) should commit themselves after calling this;
+    the extra commit here is a no-op if the session is already clean.
+    """
+    try:
+        entry = AuditLog(
+            occurred_at=datetime.now(timezone.utc),
+            actor_id=actor_id,
+            actor_name=actor_name,
+            action=action,
+            entity_type=entity_type,
+            entity_id=entity_id,
+            entity_label=entity_label,
+            changes=changes,
+            meta=meta,
+        )
+        db.add(entry)
+        await db.commit()
+    except Exception:
+        await db.rollback()
+
+
+def diff(old: dict, new: dict) -> dict[str, dict]:
+    """
+    Build a changes dict from two flat dicts.
+    Only includes keys whose values actually changed.
+    Skip internal/unloggable keys (hashed_password, updated_at).
+
+    Usage:
+        changes = diff(old_record, new_record)
+        await log_action(..., changes=changes or None)
+    """
+    _SKIP = {"hashed_password", "updated_at", "firestore_id"}
+    return {
+        k: {"old": old.get(k), "new": new.get(k)}
+        for k in new
+        if k not in _SKIP and old.get(k) != new.get(k)
+    }

backend/shared/exceptions.py

@@ -18,3 +18,8 @@ class AuthorizationError(HTTPException):
 class NotFoundError(HTTPException):
     def __init__(self, resource: str = "Resource"):
         super().__init__(status_code=404, detail=f"{resource} not found")
+
+
+class ValidationError(HTTPException):
+    def __init__(self, detail: str = "Validation error"):
+        super().__init__(status_code=422, detail=detail)

backend/shared/orm.py

@@ -0,0 +1,46 @@
+from datetime import datetime, timezone
+from sqlalchemy import BigInteger, Column, DateTime, Index, String, Text
+from sqlalchemy.dialects.postgresql import JSONB
+from database.postgres import Base
+
+
+def _now():
+    return datetime.now(timezone.utc)
+
+
+class MigrationRun(Base):
+    """Tracks every migration script execution — what ran, when, row counts, success/failure."""
+    __tablename__ = "_migration_runs"
+
+    id          = Column(BigInteger, primary_key=True, autoincrement=True)
+    script_name = Column(String(256), nullable=False)
+    ran_at      = Column(DateTime(timezone=True), nullable=False, default=_now)
+    source_rows = Column(BigInteger, nullable=False, default=0)
+    dest_rows   = Column(BigInteger, nullable=False, default=0)
+    success     = Column(String(8), nullable=False, default="ok")  # 'ok' | 'error'
+    notes       = Column(Text)
+
+
+class AuditLog(Base):
+    """Staff action audit trail — all create/update/delete/command events."""
+    __tablename__ = "audit_log"
+    __table_args__ = (
+        Index("idx_audit_actor",    "actor_id",    "occurred_at"),
+        Index("idx_audit_entity",   "entity_type", "entity_id", "occurred_at"),
+        Index("idx_audit_action",   "action",      "occurred_at"),
+        Index("idx_audit_occurred", "occurred_at"),
+    )
+
+    id           = Column(BigInteger, primary_key=True, autoincrement=True)
+    occurred_at  = Column(DateTime(timezone=True), nullable=False, default=_now)
+    actor_id     = Column(String(128), nullable=False)
+    actor_name   = Column(String(255), nullable=False)
+    action       = Column(String(64), nullable=False)
+    # CREATE | UPDATE | DELETE | COMMAND | PUBLISH | UNPUBLISH |
+    # LOGIN | LOGOUT | PERMISSION_CHANGE | STATUS_CHANGE
+    entity_type  = Column(String(64), nullable=False)
+    # customer | order | device | melody | product | staff | ticket | note | quotation | ...
+    entity_id    = Column(String(128), nullable=False)
+    entity_label = Column(String(500))   # denormalised human name
+    changes      = Column(JSONB)         # {"field": {"old": x, "new": y}} — null for CREATE/DELETE
+    meta         = Column(JSONB)         # extra context: ip_address, command_name, etc.

backend/staff/models.py

@@ -1,5 +1,5 @@
 from pydantic import BaseModel
-from typing import Optional
+from typing import Any, Dict, Optional
 from auth.models import StaffPermissions
 
 
@@ -35,3 +35,7 @@ class StaffResponse(BaseModel):
 class StaffListResponse(BaseModel):
     staff: list[StaffResponse]
     total: int
+
+
+class PreferencesUpdate(BaseModel):
+    prefs: Dict[str, Any]

backend/staff/orm.py

@@ -0,0 +1,24 @@
+from datetime import datetime, timezone
+from sqlalchemy import Boolean, Column, DateTime, String
+from sqlalchemy.dialects.postgresql import JSONB
+from database.postgres import Base
+
+
+def _now():
+    return datetime.now(timezone.utc)
+
+
+class Staff(Base):
+    __tablename__ = "staff"
+
+    id              = Column(String(128), primary_key=True)   # Firestore doc ID during transition
+    firestore_id    = Column(String(128), unique=True)        # same as id during transition
+    email           = Column(String(256), unique=True, nullable=False)
+    name            = Column(String(255), nullable=False)
+    role            = Column(String(64), nullable=False, default="staff")
+    permissions     = Column(JSONB, nullable=True)
+    hashed_password = Column(String(256), nullable=False)
+    is_active       = Column(Boolean, nullable=False, default=True)
+    ui_prefs        = Column(JSONB, nullable=False, default=dict)
+    created_at      = Column(DateTime(timezone=True), nullable=False, default=_now)
+    updated_at      = Column(DateTime(timezone=True), nullable=False, default=_now, onupdate=_now)

backend/staff/router.py

@@ -1,18 +1,43 @@
 from fastapi import APIRouter, Depends, Query
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from database.postgres import get_pg_session
 from auth.dependencies import get_current_user, require_staff_management
 from auth.models import TokenPayload
 from staff import service
 from staff.models import (
     StaffCreate, StaffUpdate, StaffPasswordUpdate,
     StaffResponse, StaffListResponse,
+    PreferencesUpdate,
 )
 
 router = APIRouter(prefix="/api/staff", tags=["staff"])
 
 
 @router.get("/me", response_model=StaffResponse)
-async def get_current_staff(current_user: TokenPayload = Depends(get_current_user)):
+async def get_current_staff(
-    return await service.get_staff_me(current_user.sub)
+    current_user: TokenPayload = Depends(get_current_user),
+    db: AsyncSession = Depends(get_pg_session),
+):
+    return await service.get_staff_me(db, current_user.sub)
+
+
+@router.get("/me/preferences", response_model=dict)
+async def get_preferences(
+    current_user: TokenPayload = Depends(get_current_user),
+    db: AsyncSession = Depends(get_pg_session),
+):
+    return await service.get_preferences(db, current_user.sub)
+
+
+@router.patch("/me/preferences/{page_key}", response_model=dict)
+async def update_preferences(
+    page_key: str,
+    body: PreferencesUpdate,
+    current_user: TokenPayload = Depends(get_current_user),
+    db: AsyncSession = Depends(get_pg_session),
+):
+    return await service.update_preferences(db, current_user.sub, page_key, body.prefs)
 
 
 @router.get("", response_model=StaffListResponse)
@@ -20,26 +45,32 @@ async def list_staff(
     search: str = Query(None),
     role: str = Query(None),
     current_user: TokenPayload = Depends(require_staff_management),
+    db: AsyncSession = Depends(get_pg_session),
 ):
-    return await service.list_staff(search=search, role_filter=role)
+    return await service.list_staff(db, search=search, role_filter=role)
 
 
 @router.get("/{staff_id}", response_model=StaffResponse)
 async def get_staff(
     staff_id: str,
     current_user: TokenPayload = Depends(require_staff_management),
+    db: AsyncSession = Depends(get_pg_session),
 ):
-    return await service.get_staff(staff_id)
+    return await service.get_staff(db, staff_id)
 
 
 @router.post("", response_model=StaffResponse)
 async def create_staff(
     body: StaffCreate,
     current_user: TokenPayload = Depends(require_staff_management),
+    db: AsyncSession = Depends(get_pg_session),
 ):
     return await service.create_staff(
+        db,
         data=body.model_dump(),
         current_user_role=current_user.role,
+        actor_id=current_user.sub,
+        actor_name=current_user.name,
     )
 
 
@@ -48,12 +79,16 @@ async def update_staff(
     staff_id: str,
     body: StaffUpdate,
     current_user: TokenPayload = Depends(require_staff_management),
+    db: AsyncSession = Depends(get_pg_session),
 ):
     return await service.update_staff(
+        db,
         staff_id=staff_id,
         data=body.model_dump(exclude_unset=True),
         current_user_role=current_user.role,
         current_user_id=current_user.sub,
+        actor_id=current_user.sub,
+        actor_name=current_user.name,
     )
 
 
@@ -62,11 +97,15 @@ async def update_staff_password(
     staff_id: str,
     body: StaffPasswordUpdate,
     current_user: TokenPayload = Depends(require_staff_management),
+    db: AsyncSession = Depends(get_pg_session),
 ):
     return await service.update_staff_password(
+        db,
         staff_id=staff_id,
         new_password=body.new_password,
         current_user_role=current_user.role,
+        actor_id=current_user.sub,
+        actor_name=current_user.name,
     )
 
 
@@ -74,9 +113,13 @@ async def update_staff_password(
 async def delete_staff(
     staff_id: str,
     current_user: TokenPayload = Depends(require_staff_management),
+    db: AsyncSession = Depends(get_pg_session),
 ):
     return await service.delete_staff(
+        db,
         staff_id=staff_id,
         current_user_role=current_user.role,
         current_user_id=current_user.sub,
+        actor_id=current_user.sub,
+        actor_name=current_user.name,
     )

backend/staff/service.py

@@ -1,6 +1,12 @@
-from shared.firebase import get_db
+from datetime import datetime, timezone
+
+from sqlalchemy import select, func, or_
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from staff.orm import Staff
 from auth.utils import hash_password
 from auth.models import default_permissions_for_role
+from shared.audit import log_action, diff
 from shared.exceptions import NotFoundError, AuthorizationError
 import uuid
 
@@ -8,171 +14,255 @@ import uuid
 VALID_ROLES = ("sysadmin", "admin", "editor", "user")
 
 
-def _staff_doc_to_response(doc_id: str, data: dict) -> dict:
+def _now() -> datetime:
+    return datetime.now(timezone.utc)
+
+
+def _to_dict(staff: Staff) -> dict:
     return {
-        "id": doc_id,
+        "id":          staff.id,
-        "email": data.get("email", ""),
+        "email":       staff.email,
-        "name": data.get("name", ""),
+        "name":        staff.name,
-        "role": data.get("role", ""),
+        "role":        staff.role,
-        "is_active": data.get("is_active", True),
+        "is_active":   staff.is_active,
-        "permissions": data.get("permissions"),
+        "permissions": staff.permissions,
     }
 
 
-async def list_staff(search: str = None, role_filter: str = None) -> dict:
+def _to_response(staff: Staff) -> dict:
-    db = get_db()
+    return _to_dict(staff)
-    ref = db.collection("admin_users")
-    docs = ref.get()
 
-    staff = []
+
-    for doc in docs:
+async def list_staff(db: AsyncSession, search: str = None, role_filter: str = None) -> dict:
-        data = doc.to_dict()
+    stmt = select(Staff)
-        if search:
-            s = search.lower()
-            if s not in (data.get("name", "").lower()) and s not in (data.get("email", "").lower()):
-                continue
     if role_filter:
-            if data.get("role") != role_filter:
+        stmt = stmt.where(Staff.role == role_filter)
-                continue
+    if search:
-        staff.append(_staff_doc_to_response(doc.id, data))
+        s = f"%{search.lower()}%"
-
+        stmt = stmt.where(
-    return {"staff": staff, "total": len(staff)}
+            or_(
+                func.lower(Staff.name).like(s),
+                func.lower(Staff.email).like(s),
+            )
+        )
+    stmt = stmt.order_by(Staff.name)
+    result = await db.execute(stmt)
+    rows = result.scalars().all()
+    return {"staff": [_to_response(r) for r in rows], "total": len(rows)}
 
 
-async def get_staff(staff_id: str) -> dict:
+async def get_staff(db: AsyncSession, staff_id: str) -> dict:
-    db = get_db()
+    result = await db.execute(select(Staff).where(Staff.id == staff_id).limit(1))
-    doc = db.collection("admin_users").document(staff_id).get()
+    staff = result.scalar_one_or_none()
-    if not doc.exists:
+    if staff is None:
         raise NotFoundError("Staff member not found")
-    return _staff_doc_to_response(doc.id, doc.to_dict())
+    return _to_response(staff)
 
 
-async def get_staff_me(user_sub: str) -> dict:
+async def get_staff_me(db: AsyncSession, user_sub: str) -> dict:
-    db = get_db()
+    return await get_staff(db, user_sub)
-    doc = db.collection("admin_users").document(user_sub).get()
-    if not doc.exists:
-        raise NotFoundError("Staff member not found")
-    return _staff_doc_to_response(doc.id, doc.to_dict())
 
 
-async def create_staff(data: dict, current_user_role: str) -> dict:
+async def create_staff(
+    db: AsyncSession,
+    data: dict,
+    current_user_role: str,
+    actor_id: str,
+    actor_name: str,
+) -> dict:
     role = data.get("role", "user")
     if role not in VALID_ROLES:
         raise AuthorizationError(f"Invalid role: {role}")
-
-    # Admin cannot create sysadmin
     if current_user_role == "admin" and role == "sysadmin":
         raise AuthorizationError("Admin cannot create sysadmin accounts")
 
-    db = get_db()
+    existing = await db.execute(
-
+        select(Staff).where(Staff.email == data["email"]).limit(1)
-    # Check for duplicate email
+    )
-    existing = db.collection("admin_users").where("email", "==", data["email"]).limit(1).get()
+    if existing.scalar_one_or_none() is not None:
-    if existing:
         raise AuthorizationError("A staff member with this email already exists")
 
-    uid = str(uuid.uuid4())
-    hashed = hash_password(data["password"])
-
-    # Set default permissions for editor/user if not provided
     permissions = data.get("permissions")
     if permissions is None and role in ("editor", "user"):
         permissions = default_permissions_for_role(role)
 
-    doc_data = {
+    uid = str(uuid.uuid4())
-        "uid": uid,
+    now = _now()
-        "email": data["email"],
+    staff = Staff(
-        "hashed_password": hashed,
+        id=uid,
-        "name": data["name"],
+        firestore_id=None,
-        "role": role,
+        email=data["email"],
-        "is_active": True,
+        name=data["name"],
-        "permissions": permissions,
+        role=role,
-    }
+        hashed_password=hash_password(data["password"]),
+        is_active=True,
+        permissions=permissions,
+        ui_prefs={},
+        created_at=now,
+        updated_at=now,
+    )
+    db.add(staff)
+    await db.flush()
 
-    doc_ref = db.collection("admin_users").document(uid)
+    await log_action(
-    doc_ref.set(doc_data)
+        db,
-
+        actor_id=actor_id,
-    return _staff_doc_to_response(uid, doc_data)
+        actor_name=actor_name,
+        action="CREATE",
+        entity_type="staff",
+        entity_id=uid,
+        entity_label=data["email"],
+        meta={"role": role},
+    )
+    await db.commit()
+    await db.refresh(staff)
+    return _to_response(staff)
 
 
-async def update_staff(staff_id: str, data: dict, current_user_role: str, current_user_id: str) -> dict:
+async def update_staff(
-    db = get_db()
+    db: AsyncSession,
-    doc_ref = db.collection("admin_users").document(staff_id)
+    staff_id: str,
-    doc = doc_ref.get()
+    data: dict,
-    if not doc.exists:
+    current_user_role: str,
+    current_user_id: str,
+    actor_id: str,
+    actor_name: str,
+) -> dict:
+    result = await db.execute(select(Staff).where(Staff.id == staff_id).limit(1))
+    staff = result.scalar_one_or_none()
+    if staff is None:
         raise NotFoundError("Staff member not found")
 
-    existing = doc.to_dict()
+    if current_user_role == "admin" and staff.role == "sysadmin":
-
-    # Admin cannot edit sysadmin accounts
-    if current_user_role == "admin" and existing.get("role") == "sysadmin":
         raise AuthorizationError("Admin cannot modify sysadmin accounts")
-
-    # Admin cannot promote to sysadmin
     if current_user_role == "admin" and data.get("role") == "sysadmin":
         raise AuthorizationError("Admin cannot promote to sysadmin")
 
-    update_data = {}
+    old = _to_dict(staff)
+
     if data.get("email") is not None:
-        # Check for duplicate email
+        dup = await db.execute(
-        others = db.collection("admin_users").where("email", "==", data["email"]).limit(1).get()
+            select(Staff).where(Staff.email == data["email"], Staff.id != staff_id).limit(1)
-        for other in others:
+        )
-            if other.id != staff_id:
+        if dup.scalar_one_or_none() is not None:
             raise AuthorizationError("A staff member with this email already exists")
-        update_data["email"] = data["email"]
+        staff.email = data["email"]
+
     if data.get("name") is not None:
-        update_data["name"] = data["name"]
+        staff.name = data["name"]
     if data.get("role") is not None:
         if data["role"] not in VALID_ROLES:
             raise AuthorizationError(f"Invalid role: {data['role']}")
-        update_data["role"] = data["role"]
+        staff.role = data["role"]
     if data.get("is_active") is not None:
-        update_data["is_active"] = data["is_active"]
+        staff.is_active = data["is_active"]
     if "permissions" in data:
-        update_data["permissions"] = data["permissions"]
+        staff.permissions = data["permissions"]
 
-    if update_data:
+    staff.updated_at = _now()
-        doc_ref.update(update_data)
+    await db.flush()
 
-    updated = {**existing, **update_data}
+    changes = diff(old, _to_dict(staff))
-    return _staff_doc_to_response(staff_id, updated)
+    action = "PERMISSION_CHANGE" if "permissions" in data and len(changes) == 1 else "UPDATE"
+    await log_action(
+        db,
+        actor_id=actor_id,
+        actor_name=actor_name,
+        action=action,
+        entity_type="staff",
+        entity_id=staff_id,
+        entity_label=staff.email,
+        changes=changes or None,
+    )
+    await db.commit()
+    await db.refresh(staff)
+    return _to_response(staff)
 
 
-async def update_staff_password(staff_id: str, new_password: str, current_user_role: str) -> dict:
+async def update_staff_password(
-    db = get_db()
+    db: AsyncSession,
-    doc_ref = db.collection("admin_users").document(staff_id)
+    staff_id: str,
-    doc = doc_ref.get()
+    new_password: str,
-    if not doc.exists:
+    current_user_role: str,
+    actor_id: str,
+    actor_name: str,
+) -> dict:
+    result = await db.execute(select(Staff).where(Staff.id == staff_id).limit(1))
+    staff = result.scalar_one_or_none()
+    if staff is None:
         raise NotFoundError("Staff member not found")
-
+    if current_user_role == "admin" and staff.role == "sysadmin":
-    existing = doc.to_dict()
-
-    # Admin cannot change sysadmin password
-    if current_user_role == "admin" and existing.get("role") == "sysadmin":
         raise AuthorizationError("Admin cannot modify sysadmin accounts")
 
-    hashed = hash_password(new_password)
+    staff.hashed_password = hash_password(new_password)
-    doc_ref.update({"hashed_password": hashed})
+    staff.updated_at = _now()
+    await db.flush()
 
+    await log_action(
+        db,
+        actor_id=actor_id,
+        actor_name=actor_name,
+        action="UPDATE",
+        entity_type="staff",
+        entity_id=staff_id,
+        entity_label=staff.email,
+        meta={"detail": "password changed"},
+    )
+    await db.commit()
     return {"message": "Password updated successfully"}
 
 
-async def delete_staff(staff_id: str, current_user_role: str, current_user_id: str) -> dict:
+async def get_preferences(db: AsyncSession, staff_id: str) -> dict:
-    db = get_db()
+    result = await db.execute(select(Staff).where(Staff.id == staff_id).limit(1))
-    doc_ref = db.collection("admin_users").document(staff_id)
+    staff = result.scalar_one_or_none()
-    doc = doc_ref.get()
+    if staff is None:
-    if not doc.exists:
+        raise NotFoundError("Staff member not found")
+    return staff.ui_prefs or {}
+
+
+async def update_preferences(db: AsyncSession, staff_id: str, page_key: str, prefs: dict) -> dict:
+    result = await db.execute(select(Staff).where(Staff.id == staff_id).limit(1))
+    staff = result.scalar_one_or_none()
+    if staff is None:
         raise NotFoundError("Staff member not found")
 
-    existing = doc.to_dict()
+    current = dict(staff.ui_prefs or {})
+    current[page_key] = {**current.get(page_key, {}), **prefs}
+    staff.ui_prefs = current
+    staff.updated_at = _now()
+    await db.commit()
+    return current
 
-    # Cannot delete self
+
+async def delete_staff(
+    db: AsyncSession,
+    staff_id: str,
+    current_user_role: str,
+    current_user_id: str,
+    actor_id: str,
+    actor_name: str,
+) -> dict:
     if staff_id == current_user_id:
         raise AuthorizationError("Cannot delete your own account")
 
-    # Admin cannot delete sysadmin
+    result = await db.execute(select(Staff).where(Staff.id == staff_id).limit(1))
-    if current_user_role == "admin" and existing.get("role") == "sysadmin":
+    staff = result.scalar_one_or_none()
+    if staff is None:
+        raise NotFoundError("Staff member not found")
+
+    if current_user_role == "admin" and staff.role == "sysadmin":
         raise AuthorizationError("Admin cannot delete sysadmin accounts")
 
-    doc_ref.delete()
+    label = staff.email
+    await log_action(
+        db,
+        actor_id=actor_id,
+        actor_name=actor_name,
+        action="DELETE",
+        entity_type="staff",
+        entity_id=staff_id,
+        entity_label=label,
+    )
+    await db.delete(staff)
+    await db.commit()
     return {"message": "Staff member deleted"}

backend/templates/quotation.html

@@ -370,12 +370,11 @@
   {% set L_DISC           = "Έκπτ." %}
   {% set L_QTY            = "Ποσ." %}
   {% set L_UNIT           = "Μον." %}
-  {% set L_VAT_COL        = "Φ.Π.Α." %}
   {% set L_TOTAL          = "Σύνολο" %}
   {% set L_SUBTOTAL       = "Υποσύνολο" %}
   {% set L_GLOBAL_DISC    = quotation.global_discount_label or "Έκπτωση" %}
   {% set L_NEW_SUBTOTAL   = "Νέο Υποσύνολο" %}
-  {% set L_VAT            = "ΣΥΝΟΛΟ Φ.Π.Α." %}
+  {% set L_VAT            = "ΣΥΝΟΛΟ Φ.Π.Α. " ~ (quotation.global_vat_percent | int) ~ "%" %}
   {% set L_SHIPPING_COST  = "Μεταφορικά / Shipping" %}
   {% set L_INSTALL_COST   = "Εγκατάσταση / Installation" %}
   {% set L_EXTRAS         = quotation.extras_label or "Άλλα" %}
@@ -403,12 +402,11 @@
   {% set L_DISC           = "Disc." %}
   {% set L_QTY            = "Qty" %}
   {% set L_UNIT           = "Unit" %}
-  {% set L_VAT_COL        = "VAT" %}
   {% set L_TOTAL          = "Total" %}
   {% set L_SUBTOTAL       = "Subtotal" %}
   {% set L_GLOBAL_DISC    = quotation.global_discount_label or "Discount" %}
   {% set L_NEW_SUBTOTAL   = "New Subtotal" %}
-  {% set L_VAT            = "Total VAT" %}
+  {% set L_VAT            = "Total VAT " ~ (quotation.global_vat_percent | int) ~ "%" %}
   {% set L_SHIPPING_COST  = "Shipping / Transport" %}
   {% set L_INSTALL_COST   = "Installation" %}
   {% set L_EXTRAS         = quotation.extras_label or "Extras" %}
@@ -469,7 +467,7 @@
 
   <div class="order-block">
     <div class="block-title">{{ L_ORDER_META }}</div>
-    <table class="fields"><tbody>{% if quotation.order_type %}<tr><td class="lbl">{{ L_ORDER_TYPE }}</td><td class="val">{{ quotation.order_type }}</td></tr>{% endif %}{% if quotation.shipping_method %}<tr><td class="lbl">{{ L_SHIP_METHOD }}</td><td class="val">{{ quotation.shipping_method }}</td></tr>{% endif %}{% if quotation.estimated_shipping_date %}<tr><td class="lbl">{{ L_SHIP_DATE }}</td><td class="val">{{ quotation.estimated_shipping_date }}</td></tr>{% else %}<tr><td class="lbl">{{ L_SHIP_DATE }}</td><td class="val text-muted">—</td></tr>{% endif %}</tbody></table>
+    <table class="fields"><tbody>{% if quotation.order_type %}<tr><td class="lbl">{{ L_ORDER_TYPE }}</td><td class="val">{{ quotation.order_type }}</td></tr>{% endif %}{% if quotation.shipping_method %}<tr><td class="lbl">{{ L_SHIP_METHOD }}</td><td class="val">{{ quotation.shipping_method }}</td></tr>{% endif %}{% if quotation.estimated_shipping_date %}{% set _dp = quotation.estimated_shipping_date.split('-') %}{% set _dfmt = _dp[2] + '/' + _dp[1] + '/' + _dp[0] if _dp | length == 3 else quotation.estimated_shipping_date %}<tr><td class="lbl">{{ L_SHIP_DATE }}</td><td class="val">{{ _dfmt }}</td></tr>{% else %}<tr><td class="lbl">{{ L_SHIP_DATE }}</td><td class="val text-muted">—</td></tr>{% endif %}</tbody></table>
   </div>
 
 </div>
@@ -478,13 +476,12 @@
 <table class="items-table">
   <thead>
     <tr>
-      <th style="width:38%">{{ L_DESC }}</th>
+      <th style="width:44%">{{ L_DESC }}</th>
-      <th class="right" style="width:11%">{{ L_UNIT_COST }}</th>
+      <th class="right" style="width:13%">{{ L_UNIT_COST }}</th>
-      <th class="center" style="width:7%">{{ L_DISC }}</th>
+      <th class="center" style="width:8%">{{ L_DISC }}</th>
-      <th class="center" style="width:7%">{{ L_QTY }}</th>
+      <th class="center" style="width:8%">{{ L_QTY }}</th>
-      <th class="center" style="width:7%">{{ L_UNIT }}</th>
+      <th class="center" style="width:8%">{{ L_UNIT }}</th>
-      <th class="center" style="width:6%">{{ L_VAT_COL }}</th>
+      <th class="right" style="width:14%">{{ L_TOTAL }}</th>
-      <th class="right" style="width:12%">{{ L_TOTAL }}</th>
     </tr>
   </thead>
   <tbody>
@@ -501,26 +498,19 @@
       </td>
       <td class="center">{{ item.quantity | int if item.quantity == (item.quantity | int) else item.quantity }}</td>
       <td class="center muted">{{ item.unit_type }}</td>
-      <td class="center">
-        {% if item.vat_percent and item.vat_percent > 0 %}
-          {{ item.vat_percent | int }}%
-        {% else %}
-          <span class="dash">—</span>
-        {% endif %}
-      </td>
       <td class="right">{{ item.line_total | format_money }}</td>
     </tr>
     {% endfor %}
     {% if quotation.items | length == 0 %}
     <tr>
-      <td colspan="7" class="text-muted" style="text-align:center; padding: 12px;">—</td>
+      <td colspan="6" class="text-muted" style="text-align:center; padding: 12px;">—</td>
     </tr>
     {% endif %}
 
     {# ── Shipping / Install as special rows ── #}
     {% set has_special = (quotation.shipping_cost and quotation.shipping_cost > 0) or (quotation.install_cost and quotation.install_cost > 0) %}
     {% if has_special %}
-    <tr class="special-spacer"><td colspan="7"></td></tr>
+    <tr class="special-spacer"><td colspan="6"></td></tr>
     {% endif %}
 
     {% if quotation.shipping_cost and quotation.shipping_cost > 0 %}
@@ -531,7 +521,6 @@
       <td class="center"><span class="dash">—</span></td>
       <td class="center">1</td>
       <td class="center muted">—</td>
-      <td class="center"><span class="dash">—</span></td>
       <td class="right">{{ ship_net | format_money }}</td>
     </tr>
     {% endif %}
@@ -544,7 +533,6 @@
       <td class="center"><span class="dash">—</span></td>
       <td class="center">1</td>
       <td class="center muted">—</td>
-      <td class="center"><span class="dash">—</span></td>
       <td class="right">{{ install_net | format_money }}</td>
     </tr>
     {% endif %}

backend/tickets/models.py

@@ -0,0 +1,92 @@
+from pydantic import BaseModel, Field, field_validator
+from typing import Optional, List
+from uuid import UUID
+from datetime import datetime
+
+VALID_STATUSES   = {"open", "waiting_on_customer", "waiting_on_staff", "resolved", "closed"}
+VALID_PRIORITIES = {"low", "medium", "high", "urgent"}
+VALID_OPENED_VIA = {"app", "email", "phone", "staff"}
+VALID_SENDERS    = {"staff", "customer"}
+
+
+class TicketCreate(BaseModel):
+    customer_id:   str
+    customer_name: Optional[str] = None
+    subject:       str = Field(..., max_length=500)
+    device_id:     Optional[str] = None
+    device_serial: Optional[str] = None
+    opened_via:    Optional[str] = None
+    priority:      Optional[str] = None
+
+    @field_validator("priority")
+    @classmethod
+    def check_priority(cls, v):
+        if v is not None and v not in VALID_PRIORITIES:
+            raise ValueError(f"priority must be one of {VALID_PRIORITIES}")
+        return v
+
+
+class TicketUpdate(BaseModel):
+    status:        Optional[str] = None
+    priority:      Optional[str] = None
+    device_id:     Optional[str] = None
+    device_serial: Optional[str] = None
+
+    @field_validator("status")
+    @classmethod
+    def check_status(cls, v):
+        if v is not None and v not in VALID_STATUSES:
+            raise ValueError(f"status must be one of {VALID_STATUSES}")
+        return v
+
+
+class MessageCreate(BaseModel):
+    sender_type: str
+    sender_id:   str
+    sender_name: Optional[str] = None
+    body:        str
+    is_internal: bool = False
+
+    @field_validator("sender_type")
+    @classmethod
+    def check_sender_type(cls, v):
+        if v not in VALID_SENDERS:
+            raise ValueError(f"sender_type must be one of {VALID_SENDERS}")
+        return v
+
+
+class EscalateIn(BaseModel):
+    entry_id: UUID
+
+
+class MessageOut(BaseModel):
+    id:          UUID
+    sender_type: str
+    sender_id:   str
+    sender_name: Optional[str]
+    body:        str
+    is_internal: bool
+    created_at:  datetime
+    model_config = {"from_attributes": True}
+
+
+class TicketOut(BaseModel):
+    id:              UUID
+    customer_id:     str
+    customer_name:   Optional[str]
+    device_id:       Optional[str]
+    device_serial:   Optional[str]
+    subject:         str
+    status:          str
+    priority:        Optional[str]
+    opened_via:      Optional[str]
+    linked_entry_id: Optional[UUID]
+    created_at:      datetime
+    updated_at:      datetime
+    messages:        List[MessageOut] = []
+    model_config = {"from_attributes": True}
+
+
+class TicketListResponse(BaseModel):
+    data: List[TicketOut]
+    pagination: dict

backend/tickets/orm.py

@@ -0,0 +1,46 @@
+import uuid
+from datetime import datetime, timezone
+from sqlalchemy import Column, String, Text, DateTime, Boolean, ForeignKey
+from sqlalchemy.dialects.postgresql import UUID
+from sqlalchemy.orm import relationship
+from database.postgres import Base
+
+
+def _now():
+    return datetime.now(timezone.utc)
+
+
+class SupportTicket(Base):
+    __tablename__ = "support_tickets"
+
+    id              = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
+    customer_id     = Column(String(128), nullable=False)    # Firestore ID (moves to UUID when customers migrate)
+    customer_name   = Column(String(255), nullable=True)     # denormalized snapshot
+    device_id       = Column(String(128), nullable=True)     # Firestore ID
+    device_serial   = Column(String(64), nullable=True)      # denormalized snapshot
+    subject         = Column(String(500), nullable=False)
+    status          = Column(String(30), nullable=False, default="open")
+    # open | waiting_on_customer | waiting_on_staff | resolved | closed
+    priority        = Column(String(10), nullable=True)      # low | medium | high | urgent
+    opened_via      = Column(String(20), nullable=True)      # app | email | phone | staff
+    linked_entry_id = Column(UUID(as_uuid=True), ForeignKey("crm_entries.id", ondelete="SET NULL"), nullable=True)
+    created_at      = Column(DateTime(timezone=True), nullable=False, default=_now)
+    updated_at      = Column(DateTime(timezone=True), nullable=False, default=_now, onupdate=_now)
+
+    messages = relationship("TicketMessage", back_populates="ticket",
+                            cascade="all, delete-orphan", order_by="TicketMessage.created_at", lazy="noload")
+
+
+class TicketMessage(Base):
+    __tablename__ = "ticket_messages"
+
+    id          = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
+    ticket_id   = Column(UUID(as_uuid=True), ForeignKey("support_tickets.id", ondelete="CASCADE"), nullable=False)
+    sender_type = Column(String(10), nullable=False)    # 'staff' | 'customer'
+    sender_id   = Column(String(128), nullable=False)
+    sender_name = Column(String(255), nullable=True)
+    body        = Column(Text, nullable=False)
+    is_internal = Column(Boolean, nullable=False, default=False)
+    created_at  = Column(DateTime(timezone=True), nullable=False, default=_now)
+
+    ticket = relationship("SupportTicket", back_populates="messages")

backend/tickets/router.py

@@ -0,0 +1,101 @@
+from fastapi import APIRouter, Depends, Query
+from uuid import UUID
+from sqlalchemy.ext.asyncio import AsyncSession
+from database.postgres import get_pg_session
+from auth.dependencies import require_permission
+from auth.models import TokenPayload
+from tickets import service
+from tickets.models import TicketCreate, TicketUpdate, MessageCreate, EscalateIn, TicketOut, TicketListResponse
+from shared.audit import log_action
+
+router = APIRouter(prefix="/api/tickets", tags=["tickets"])
+
+
+@router.get("", response_model=TicketListResponse)
+async def list_tickets(
+    status:      str | None = Query(None),
+    priority:    str | None = Query(None),
+    customer_id: str | None = Query(None),
+    page:  int = Query(1, ge=1),
+    limit: int = Query(25, ge=1, le=100),
+    db:    AsyncSession = Depends(get_pg_session),
+    _user: TokenPayload = Depends(require_permission("crm", "view")),
+):
+    rows, total = await service.list_tickets(db, status, priority, customer_id, page, limit)
+    return {"data": rows, "pagination": {"page": page, "limit": limit, "total": total}}
+
+
+@router.get("/by-customer/{customer_id}", response_model=list[TicketOut])
+async def list_by_customer(
+    customer_id: str,
+    db:    AsyncSession = Depends(get_pg_session),
+    _user: TokenPayload = Depends(require_permission("crm", "view")),
+):
+    return await service.list_by_customer(db, customer_id)
+
+
+@router.get("/by-device/{device_id}", response_model=list[TicketOut])
+async def list_by_device(
+    device_id: str,
+    db:    AsyncSession = Depends(get_pg_session),
+    _user: TokenPayload = Depends(require_permission("crm", "view")),
+):
+    return await service.list_by_device(db, device_id)
+
+
+@router.get("/{ticket_id}", response_model=TicketOut)
+async def get_ticket(
+    ticket_id: UUID,
+    db:    AsyncSession = Depends(get_pg_session),
+    _user: TokenPayload = Depends(require_permission("crm", "view")),
+):
+    return await service.get_ticket(db, ticket_id)
+
+
+@router.post("", response_model=TicketOut, status_code=201)
+async def create_ticket(
+    body:  TicketCreate,
+    db:    AsyncSession = Depends(get_pg_session),
+    _user: TokenPayload = Depends(require_permission("crm", "add")),
+):
+    ticket = await service.create_ticket(db, body)
+    await log_action(db, _user.sub, _user.name or _user.email, "CREATE", "ticket",
+                     str(ticket.id), ticket.subject)
+    return ticket
+
+
+@router.patch("/{ticket_id}", response_model=TicketOut)
+async def update_ticket(
+    ticket_id: UUID, body: TicketUpdate,
+    db:    AsyncSession = Depends(get_pg_session),
+    _user: TokenPayload = Depends(require_permission("crm", "edit")),
+):
+    ticket = await service.update_ticket(db, ticket_id, body)
+    action = "STATUS_CHANGE" if body.status is not None else "UPDATE"
+    await log_action(db, _user.sub, _user.name or _user.email, action, "ticket",
+                     str(ticket_id), ticket.subject)
+    return ticket
+
+
+@router.post("/{ticket_id}/messages", response_model=TicketOut)
+async def add_message(
+    ticket_id: UUID, body: MessageCreate,
+    db:    AsyncSession = Depends(get_pg_session),
+    _user: TokenPayload = Depends(require_permission("crm", "edit")),
+):
+    ticket = await service.add_message(db, ticket_id, body)
+    await log_action(db, _user.sub, _user.name or _user.email, "UPDATE", "ticket",
+                     str(ticket_id), ticket.subject, meta={"action_detail": "message_added"})
+    return ticket
+
+
+@router.post("/{ticket_id}/escalate", response_model=TicketOut)
+async def escalate(
+    ticket_id: UUID, body: EscalateIn,
+    db:    AsyncSession = Depends(get_pg_session),
+    _user: TokenPayload = Depends(require_permission("crm", "edit")),
+):
+    ticket = await service.escalate_to_issue(db, ticket_id, body.entry_id)
+    await log_action(db, _user.sub, _user.name or _user.email, "STATUS_CHANGE", "ticket",
+                     str(ticket_id), ticket.subject, meta={"action_detail": "escalated_to_issue"})
+    return ticket

backend/tickets/service.py

@@ -0,0 +1,91 @@
+import uuid
+from sqlalchemy.ext.asyncio import AsyncSession
+from sqlalchemy import select, func
+from sqlalchemy.orm import selectinload
+from tickets.orm import SupportTicket, TicketMessage
+from tickets.models import TicketCreate, TicketUpdate, MessageCreate
+from shared.exceptions import NotFoundError
+
+
+async def create_ticket(db: AsyncSession, data: TicketCreate) -> SupportTicket:
+    ticket = SupportTicket(**data.model_dump())
+    db.add(ticket)
+    await db.commit()
+    return await _get_ticket(db, ticket.id, include_internal=True)
+
+
+async def get_ticket(db: AsyncSession, ticket_id: uuid.UUID, include_internal: bool = True) -> SupportTicket:
+    return await _get_ticket(db, ticket_id, include_internal)
+
+
+async def list_tickets(
+    db: AsyncSession,
+    status: str | None, priority: str | None, customer_id: str | None,
+    page: int, limit: int,
+) -> tuple[list[SupportTicket], int]:
+    limit  = min(100, max(1, limit))
+    offset = (max(1, page) - 1) * limit
+
+    q = select(SupportTicket).options(selectinload(SupportTicket.messages))
+    if status:      q = q.where(SupportTicket.status == status)
+    if priority:    q = q.where(SupportTicket.priority == priority)
+    if customer_id: q = q.where(SupportTicket.customer_id == customer_id)
+
+    total = (await db.execute(select(func.count()).select_from(q.subquery()))).scalar()
+    rows  = (await db.execute(q.order_by(SupportTicket.created_at.desc()).limit(limit).offset(offset))).scalars().all()
+    return rows, total
+
+
+async def update_ticket(db: AsyncSession, ticket_id: uuid.UUID, data: TicketUpdate) -> SupportTicket:
+    ticket = await _get_ticket(db, ticket_id)
+    for field, value in data.model_dump(exclude_unset=True).items():
+        setattr(ticket, field, value)
+    await db.commit()
+    return await _get_ticket(db, ticket_id)
+
+
+async def add_message(db: AsyncSession, ticket_id: uuid.UUID, data: MessageCreate) -> SupportTicket:
+    ticket = await _get_ticket(db, ticket_id)
+    msg = TicketMessage(ticket_id=ticket_id, **data.model_dump())
+    db.add(msg)
+
+    # Auto-advance ticket status based on who replied (skip if already resolved/closed)
+    if ticket.status not in ("resolved", "closed"):
+        if data.sender_type == "staff" and not data.is_internal:
+            ticket.status = "waiting_on_customer"
+        elif data.sender_type == "customer":
+            ticket.status = "waiting_on_staff"
+
+    await db.commit()
+    return await _get_ticket(db, ticket_id)
+
+
+async def escalate_to_issue(db: AsyncSession, ticket_id: uuid.UUID, entry_id: uuid.UUID) -> SupportTicket:
+    ticket = await _get_ticket(db, ticket_id)
+    ticket.linked_entry_id = entry_id
+    await db.commit()
+    return await _get_ticket(db, ticket_id)
+
+
+async def list_by_customer(db: AsyncSession, customer_id: str) -> list[SupportTicket]:
+    q = select(SupportTicket).options(selectinload(SupportTicket.messages)).where(
+        SupportTicket.customer_id == customer_id
+    ).order_by(SupportTicket.created_at.desc())
+    return (await db.execute(q)).scalars().all()
+
+
+async def list_by_device(db: AsyncSession, device_id: str) -> list[SupportTicket]:
+    q = select(SupportTicket).options(selectinload(SupportTicket.messages)).where(
+        SupportTicket.device_id == device_id
+    ).order_by(SupportTicket.created_at.desc())
+    return (await db.execute(q)).scalars().all()
+
+
+async def _get_ticket(db: AsyncSession, ticket_id: uuid.UUID, include_internal: bool = True) -> SupportTicket:
+    q = select(SupportTicket).options(selectinload(SupportTicket.messages)).where(SupportTicket.id == ticket_id)
+    result = (await db.execute(q)).scalar_one_or_none()
+    if not result:
+        raise NotFoundError("Ticket")
+    if not include_internal:
+        result.messages = [m for m in result.messages if not m.is_internal]
+    return result

backend/users/models.py

@@ -41,3 +41,11 @@ class UserInDB(UserCreate):
 class UserListResponse(BaseModel):
     users: List[UserInDB]
     total: int
+
+
+class SetPasswordRequest(BaseModel):
+    password: str
+
+
+class ResetPasswordRequest(BaseModel):
+    new_password: str = "Bell1234!"  # default reset value

backend/users/router.py

@@ -1,11 +1,15 @@
 from fastapi import APIRouter, Depends, Query, UploadFile, File
 from typing import Optional, List
+from sqlalchemy.ext.asyncio import AsyncSession
 from auth.models import TokenPayload
 from auth.dependencies import require_permission
 from users.models import (
     UserCreate, UserUpdate, UserInDB, UserListResponse,
+    SetPasswordRequest, ResetPasswordRequest,
 )
 from users import service
+from database.postgres import get_pg_session
+from shared.audit import log_action
 
 router = APIRouter(prefix="/api/users", tags=["users"])
 
@@ -32,8 +36,12 @@ async def get_user(
 async def create_user(
     body: UserCreate,
     _user: TokenPayload = Depends(require_permission("app_users", "add")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
-    return service.create_user(body)
+    app_user = service.create_user(body)
+    await log_action(db, _user.sub, _user.name or _user.email, "CREATE", "app_user",
+                     app_user.id, app_user.display_name or app_user.email or app_user.id)
+    return app_user
 
 
 @router.put("/{user_id}", response_model=UserInDB)
@@ -41,32 +49,57 @@ async def update_user(
     user_id: str,
     body: UserUpdate,
     _user: TokenPayload = Depends(require_permission("app_users", "edit")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
-    return service.update_user(user_id, body)
+    old = service.get_user(user_id)
+    app_user = service.update_user(user_id, body)
+    _SKIP = {"updated_at", "id", "photo_url"}
+    changes = {
+        k: {"old": getattr(old, k, None), "new": getattr(app_user, k, None)}
+        for k in body.model_fields_set
+        if k not in _SKIP and getattr(old, k, None) != getattr(app_user, k, None)
+    }
+    await log_action(db, _user.sub, _user.name or _user.email, "UPDATE", "app_user",
+                     user_id, app_user.display_name or app_user.email or user_id,
+                     changes=changes or None)
+    return app_user
 
 
 @router.delete("/{user_id}", status_code=204)
 async def delete_user(
     user_id: str,
     _user: TokenPayload = Depends(require_permission("app_users", "delete")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
     service.delete_user(user_id)
+    await log_action(db, _user.sub, _user.name or _user.email, "DELETE", "app_user",
+                     user_id, user_id)
 
 
 @router.post("/{user_id}/block", response_model=UserInDB)
 async def block_user(
     user_id: str,
     _user: TokenPayload = Depends(require_permission("app_users", "edit")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
-    return service.block_user(user_id)
+    app_user = service.block_user(user_id)
+    await log_action(db, _user.sub, _user.name or _user.email, "STATUS_CHANGE", "app_user",
+                     user_id, app_user.display_name or app_user.email or user_id,
+                     meta={"status": "blocked"})
+    return app_user
 
 
 @router.post("/{user_id}/unblock", response_model=UserInDB)
 async def unblock_user(
     user_id: str,
     _user: TokenPayload = Depends(require_permission("app_users", "edit")),
+    db: AsyncSession = Depends(get_pg_session),
 ):
-    return service.unblock_user(user_id)
+    app_user = service.unblock_user(user_id)
+    await log_action(db, _user.sub, _user.name or _user.email, "STATUS_CHANGE", "app_user",
+                     user_id, app_user.display_name or app_user.email or user_id,
+                     meta={"status": "unblocked"})
+    return app_user
 
 
 @router.get("/{user_id}/devices", response_model=List[dict])
@@ -95,6 +128,26 @@ async def unassign_device(
     return service.unassign_device(user_id, device_id)
 
 
+@router.post("/{user_id}/set-password", status_code=204)
+async def set_password(
+    user_id: str,
+    body: SetPasswordRequest,
+    _user: TokenPayload = Depends(require_permission("app_users", "full_edit")),
+):
+    """Set a new password for the user via Firebase Auth (requires uid on the user doc)."""
+    service.set_password(user_id, body.password)
+
+
+@router.post("/{user_id}/reset-password", status_code=204)
+async def reset_password(
+    user_id: str,
+    body: ResetPasswordRequest,
+    _user: TokenPayload = Depends(require_permission("app_users", "full_edit")),
+):
+    """Reset a user's password to the supplied value (default: Bell1234!)."""
+    service.set_password(user_id, body.new_password)
+
+
 @router.post("/{user_id}/photo")
 async def upload_photo(
     user_id: str,

backend/users/service.py

@@ -2,8 +2,9 @@ from datetime import datetime
 
 from google.cloud.firestore_v1 import DocumentReference
 
+from firebase_admin import auth as firebase_auth
 from shared.firebase import get_db, get_bucket
-from shared.exceptions import NotFoundError
+from shared.exceptions import NotFoundError, ValidationError
 from users.models import UserCreate, UserUpdate, UserInDB
 
 COLLECTION = "users"
@@ -252,6 +253,31 @@ def get_user_devices(user_doc_id: str) -> list[dict]:
     return devices
 
 
+def set_password(user_doc_id: str, new_password: str) -> None:
+    """Set a Firebase Auth password for a user via their Firestore document ID.
+
+    Requires the user document to have a non-empty `uid` field — populated
+    automatically for users who registered via the Flutter app.
+    """
+    if not new_password or len(new_password) < 6:
+        raise ValidationError("Password must be at least 6 characters.")
+
+    db = get_db()
+    doc_ref = db.collection(COLLECTION).document(user_doc_id)
+    doc = doc_ref.get()
+    if not doc.exists:
+        raise NotFoundError("User")
+
+    uid = doc.to_dict().get("uid", "")
+    if not uid:
+        raise ValidationError("This user has no Firebase Auth UID — they may not have signed up via the app yet.")
+
+    try:
+        firebase_auth.update_user(uid, password=new_password)
+    except Exception as e:
+        raise RuntimeError(f"Firebase Auth error: {e}")
+
+
 def upload_photo(user_doc_id: str, file_bytes: bytes, filename: str, content_type: str) -> str:
     """Upload a profile photo to Firebase Storage and update the user's photo_url."""
     db = get_db()

docker-compose.yml

@@ -5,34 +5,57 @@ services:
     env_file: .env
     volumes:
       - ./backend:/app
-      # Persistent data - lives outside the container
       - ./data:/app/data
       - ./data/built_melodies:/app/storage/built_melodies
       - ./data/firmware:/app/storage/firmware
       - ./data/flash_assets:/app/storage/flash_assets
+      - ./data/melody_binaries:/app/storage/melody_binaries
       - ./data/firebase-service-account.json:/app/firebase-service-account.json:ro
-      # Auto-deploy: project root so container can write the trigger file
-      - /home/bellsystems/bellsystems-cp:/home/bellsystems/bellsystems-cp
     ports:
       - "8000:8000"
-    depends_on: []
+    depends_on:
+      postgres:
+        condition: service_healthy
+    networks:
+      - internal
 
   frontend:
     build: ./frontend
     container_name: bellsystems-frontend
-    volumes:
+    networks:
-      - ./frontend:/app
+      - internal
-      - /app/node_modules
-    ports:
-      - "5173:5173"
 
   nginx:
     image: nginx:alpine
     container_name: bellsystems-nginx
     ports:
-      - "${NGINX_PORT:-80}:80"
+      - "90:80"     # access v2 on localhost:8001
     volumes:
       - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
     depends_on:
       - backend
       - frontend
+    networks:
+      - internal
+
+  postgres:
+    image: postgres:16-alpine
+    container_name: bellsystems-postgres
+    restart: unless-stopped
+    environment:
+      POSTGRES_DB: ${POSTGRES_DB}
+      POSTGRES_USER: ${POSTGRES_USER}
+      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
+    volumes:
+      - ./data/postgres:/var/lib/postgresql/data
+    networks:
+      - internal
+    healthcheck:
+      test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER} -d ${POSTGRES_DB}"]
+      interval: 10s
+      timeout: 5s
+      retries: 5
+
+networks:
+  internal:
+    driver: bridge

docs/melody-binary-serving.md

@@ -0,0 +1,119 @@
+# Melody Binary Serving (Plain HTTP)
+
+## Why this exists
+
+ESP32 devices download `.bsm` melody files directly, using a URL supplied through the
+Android app. The download previously pointed at a Firebase Storage public URL (HTTPS
+only). ESP32's TLS client needs 40KB+ of RAM to open an HTTPS connection, which these
+devices can't reliably spare. To fix this, melody `.bsm` binaries are now stored and
+served by our own backend over plain HTTP, instead of Firebase Storage.
+
+This does **not** affect the audio preview file (`information.previewURL`) — that's
+only ever fetched by the browser admin UI, which is already HTTPS, so it stays on
+Firebase Storage unchanged.
+
+## PID vs UID — why storage is keyed on pid, not uid
+
+- **`pid`** ("Playback ID") identifies the underlying **archetype binary** — the raw
+  note sequence (A, B, C...) baked into a `.bsm` file. Multiple melodies can
+  legitimately share one `pid`: each melody remaps those notes to actual bells via its
+  own `noteAssignments`, speed, and duration settings. One binary can back many
+  different "remix" melodies. **Duplicate PIDs across melodies are correct and
+  expected, not a bug.**
+- **`uid`** is meant to be each melody's own unique database identity. It is *not*
+  currently populated anywhere in `MelodyForm.jsx` — this is a pre-existing, separate
+  bug, deferred as its own task. Because of this, local binary storage is keyed on
+  `pid`, not `uid`.
+
+## What changed
+
+- **Storage**: `.bsm` binaries are written to `./data/melody_binaries/{pid}.bsm` on
+  the host (mounted into the backend container at `/app/storage/melody_binaries`,
+  same pattern as `./data/firmware` and `./data/built_melodies`). Because storage is
+  keyed on `pid`, multiple melodies sharing a `pid` share one file — saving from any
+  of them overwrites the same file, which is expected.
+- **Upload path**: `SelectArchetypeModal` / `BuildOnTheFlyModal` still call
+  `POST /api/melodies/{melodyId}/upload/binary` exactly as before. The backend
+  (`backend/melodies/service.py::save_binary_for_melody`) now writes the bytes to
+  local disk instead of uploading to Firebase Storage, and returns a URL like
+  `http://melodies.bellsystems.net/download/{pid}`. The melody must have a `pid` set
+  before uploading a binary (the endpoint returns 400 if not).
+- **Stored URL**: the melody's `url` field (Firestore + SQLite) now holds that
+  plain-HTTP URL instead of a Firebase public URL. No schema change — `url` was
+  already a plain string field.
+- **Download route**: `GET /api/melodies/download/{pid}` (`backend/melodies/router.py`)
+  is unauthenticated (devices have no login token) and resolves `pid` directly to
+  `{pid}.bsm` on disk, same pattern as the existing firmware download route
+  (`backend/firmware/router.py::download_firmware`).
+- **Deletion is share-aware**: deleting a melody or its binary
+  (`service.delete_file` / `service._delete_storage_files`) only removes the local
+  `.bsm` file if no *other* melody still references the same `pid`
+  (`service._pid_used_by_other_melody`). This prevents one melody's deletion from
+  breaking playback for other melodies sharing its archetype binary.
+
+## Migrating existing melodies
+
+Melodies created before this change still have a Firebase Storage URL in their `url`
+field — deploying this code does not touch existing data. They keep working exactly
+as before (still HTTPS to Firebase) until migrated.
+
+Two ways to move a melody to the new plain-HTTP URL:
+
+1. **Per-melody, via the UI**: open the melody and re-run "Select Archetype" or
+   "Build on the Fly" — this naturally re-uploads through the same endpoint, which now
+   writes to local disk and updates `url`.
+2. **Bulk, via script**: `backend/scripts/migrate_melody_binaries_to_local.py`
+   downloads each melody's current Firebase binary and writes it to local disk under
+   its `pid`, then updates `url` (SQLite + Firestore if published). Run with
+   `--dry-run` first:
+
+   ```
+   docker exec -it bellsystems-backend python scripts/migrate_melody_binaries_to_local.py --dry-run
+   docker exec -it bellsystems-backend python scripts/migrate_melody_binaries_to_local.py
+   ```
+
+   **Known caveat**: some existing melodies share a `pid` despite their Firebase URLs
+   pointing at *different* source files (observed on real data — e.g. voice-count
+   variant filenames like `1N_`/`2N_`/`3N_`/`4N_` all filed under one `pid`). The
+   script downloads only the first melody's file for each `pid` and reuses it for
+   every other melody sharing that `pid` — it does not attempt to detect which file is
+   "correct". It prints a warning list of every `pid` where this happened; use each
+   melody's playback button afterward to verify the right archetype plays, and
+   manually re-assign the correct one via "Select Archetype" if it's wrong.
+
+## Infrastructure (outside this repo)
+
+The plain-HTTP requirement is handled by keeping `melodies.bellsystems.net` on a
+**separate** hostname from `console.bellsystems.net`, so the console's NPM proxy host
+can stay HTTPS-only with no per-path exceptions.
+
+Required, one-time setup outside this repository:
+
+1. **DNS**: add an A/CNAME record for `melodies.bellsystems.net` pointing at the same
+   host as `console.bellsystems.net`.
+2. **NPM (Nginx Proxy Manager) proxy host**: create a new proxy host for
+   `melodies.bellsystems.net` forwarding to the `nginx` container's exposed port
+   (`90` per `docker-compose.yml`). **Do not force SSL / do not enable the "Force
+   SSL" redirect** on this proxy host — it must remain reachable over plain HTTP.
+
+The in-repo `nginx/nginx.conf` already has a dedicated `server_name
+melodies.bellsystems.net` block that maps `/download/{pid}` to the backend's
+`/api/melodies/download/{pid}` route, so no further nginx changes are needed once
+the NPM proxy host exists.
+
+## Verification
+
+1. Build/select an archetype for a melody with a `pid` set → confirm a file appears
+   at `./data/melody_binaries/{pid}.bsm` and the melody's `url` becomes
+   `http://melodies.bellsystems.net/download/{pid}`.
+2. `curl http://localhost:8000/api/melodies/download/{pid}` (direct to backend,
+   bypassing nginx) → returns the `.bsm` bytes, no auth header needed.
+3. `curl http://localhost:90/api/melodies/download/{pid}` (through nginx on the
+   console server block) → same result.
+4. Once NPM is configured, `curl http://melodies.bellsystems.net/download/{pid}`
+   → same result, over plain HTTP.
+5. Delete a melody whose `pid` is *not* shared with any other melody → confirm the
+   local `.bsm` file is removed. Delete one of two melodies sharing a `pid` → confirm
+   the file survives until the last one is deleted.
+6. Upload a preview audio file → confirm it still lands in Firebase Storage and
+   `previewURL` still populates (regression check).

frontend/Dockerfile

@@ -1,4 +1,5 @@
-FROM node:20-alpine
+# Stage 1: build
+FROM node:20-alpine AS builder
 
 WORKDIR /app
 
@@ -6,5 +7,12 @@ COPY package.json package-lock.json ./
 RUN npm ci
 
 COPY . .
+RUN npm run build
 
-CMD ["npm", "run", "dev"]
+# Stage 2: serve with nginx
+FROM nginx:alpine
+
+COPY --from=builder /app/dist /usr/share/nginx/html
+COPY nginx.prod.conf /etc/nginx/conf.d/default.conf
+
+EXPOSE 80

frontend/nginx.prod.conf

@@ -0,0 +1,17 @@
+server {
+    listen 80;
+
+    root /usr/share/nginx/html;
+    index index.html;
+
+    # SPA fallback — all unknown routes serve index.html
+    location / {
+        try_files $uri $uri/ /index.html;
+    }
+
+    # Cache static assets
+    location ~* \.(js|css|png|svg|ico|woff2?)$ {
+        expires 1y;
+        add_header Cache-Control "public, immutable";
+    }
+}

frontend/src/_archive/App.jsx

@@ -1,6 +1,8 @@
 import { Routes, Route, Navigate } from "react-router-dom";
 import { useAuth } from "./auth/AuthContext";
 import CloudFlashPage from "./cloudflash/CloudFlashPage";
+import SerialMonitorPage from "./serial/SerialMonitorPage";
+import SerialLogViewer from "./serial/SerialLogViewer";
 import PublicFeaturesSettings from "./settings/PublicFeaturesSettings";
 import LoginPage from "./auth/LoginPage";
 import MainLayout from "./layout/MainLayout";
@@ -35,7 +37,7 @@ import DashboardPage from "./dashboard/DashboardPage";
 import ApiReferencePage from "./developer/ApiReferencePage";
 import { ProductList, ProductForm } from "./crm/products";
 import { CustomerList, CustomerForm, CustomerDetail } from "./crm/customers";
-import { OrderList, OrderForm, OrderDetail } from "./crm/orders";
+import { OrderList } from "./crm/orders";
 import { QuotationForm, AllQuotationsList } from "./crm/quotations";
 import CommsPage from "./crm/inbox/CommsPage";
 import MailPage from "./crm/mail/MailPage";
@@ -110,6 +112,7 @@ export default function App() {
     <Routes>
       {/* Public routes — no login required */}
       <Route path="/cloudflash" element={<CloudFlashPage />} />
+      <Route path="/serial-monitor" element={<SerialMonitorPage />} />
 
       <Route path="/login" element={<LoginPage />} />
       <Route
@@ -176,9 +179,6 @@ export default function App() {
         <Route path="crm/customers/:id" element={<PermissionGate section="crm"><CustomerDetail /></PermissionGate>} />
         <Route path="crm/customers/:id/edit" element={<PermissionGate section="crm" action="edit"><CustomerForm /></PermissionGate>} />
         <Route path="crm/orders" element={<PermissionGate section="crm"><OrderList /></PermissionGate>} />
-        <Route path="crm/orders/new" element={<PermissionGate section="crm" action="edit"><OrderForm /></PermissionGate>} />
-        <Route path="crm/orders/:id" element={<PermissionGate section="crm"><OrderDetail /></PermissionGate>} />
-        <Route path="crm/orders/:id/edit" element={<PermissionGate section="crm" action="edit"><OrderForm /></PermissionGate>} />
         <Route path="crm/quotations" element={<PermissionGate section="crm"><AllQuotationsList /></PermissionGate>} />
         <Route path="crm/quotations/new" element={<PermissionGate section="crm" action="edit"><QuotationForm /></PermissionGate>} />
         <Route path="crm/quotations/:id" element={<PermissionGate section="crm" action="edit"><QuotationForm /></PermissionGate>} />
@@ -196,6 +196,9 @@ export default function App() {
         {/* Settings - Public Features */}
         <Route path="settings/public-features" element={<RoleGate roles={["sysadmin", "admin"]}><PublicFeaturesSettings /></RoleGate>} />
 
+        {/* Settings - Serial Log Viewer */}
+        <Route path="settings/serial-logs" element={<RoleGate roles={["sysadmin", "admin"]}><SerialLogViewer /></RoleGate>} />
+
         <Route path="*" element={<Navigate to="/" replace />} />
       </Route>
     </Routes>

frontend/src/_archive/assets/comms/call.svg

@@ -0,0 +1,25 @@
+<?xml version="1.0" encoding="utf-8"?>
+
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
+<!-- Uploaded to: SVG Repo, www.svgrepo.com, Generator: SVG Repo Mixer Tools -->
+<svg height="800px" width="800px" version="1.1" id="_x32_" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" 
+	 viewBox="0 0 512 512"  xml:space="preserve">
+<style type="text/css">
+	.st0{fill:#000000;}
+</style>
+<g>
+	<path class="st0" d="M500.177,55.798c0,0-21.735-7.434-39.551-11.967C411.686,31.369,308.824,24.727,256,24.727
+		S100.314,31.369,51.374,43.831c-17.816,4.534-39.551,11.967-39.551,11.967c-7.542,2.28-12.444,9.524-11.76,17.374l8.507,97.835
+		c0.757,8.596,7.957,15.201,16.581,15.201h84.787c8.506,0,15.643-6.416,16.553-14.878l4.28-39.973
+		c0.847-7.93,7.2-14.138,15.148-14.815c0,0,68.484-6.182,110.081-6.182c41.586,0,110.08,6.182,110.08,6.182
+		c7.949,0.676,14.302,6.885,15.148,14.815l4.29,39.973c0.9,8.462,8.038,14.878,16.545,14.878h84.777
+		c8.632,0,15.832-6.605,16.589-15.201l8.507-97.835C512.621,65.322,507.72,58.078,500.177,55.798z"/>
+	<path class="st0" d="M357.503,136.629h-55.365v46.137h-92.275v-46.137h-55.365c0,0-9.228,119.957-119.957,207.618
+		c0,32.296,0,129.95,0,129.95c0,7.218,5.857,13.076,13.075,13.076h416.768c7.218,0,13.076-5.858,13.076-13.076
+		c0,0,0-97.654,0-129.95C366.73,256.586,357.503,136.629,357.503,136.629z M338.768,391.42v37.406h-37.396V391.42H338.768z
+		 M338.768,332.27v37.406h-37.396V332.27H338.768z M301.372,310.518v-37.396h37.396v37.396H301.372z M274.698,391.42v37.406h-37.396
+		V391.42H274.698z M274.698,332.27v37.406h-37.396V332.27H274.698z M274.698,273.122v37.396h-37.396v-37.396H274.698z
+		 M210.629,391.42v37.406h-37.397V391.42H210.629z M210.629,332.27v37.406h-37.397V332.27H210.629z M210.629,273.122v37.396h-37.397
+		v-37.396H210.629z"/>
+</g>
+</svg>