docs(12-01): complete schema + delta-only matching plan

2026-07-22 09:07:06 +02:00
parent b45047f0b3
commit 4823c245ee
4 changed files with 180 additions and 23 deletions
@@ -0,0 +1,154 @@
+---
+phase: 12-tender-notifications
+plan: 01
+subsystem: api
+tags: [prisma, nestjs, tender-radar, notifications, matching]
+
+requires:
+  - phase: 11-tender-saved-searches
+    provides: "TenderSavedSearch (filters Json, userId/tenantId scoping) + buildTenderWhere(dto) filter compiler"
+  - phase: 10-tender-radar-ingestion
+    provides: "TenderIngestionService.pollDueSources poll-tick + Tender catalog (dedupKey upsert)"
+provides:
+  - "TenderMatch model — matched-vs-notified state (single nullable notifiedAt gate)"
+  - "TenderNotificationPref model — per-user digest interval preference"
+  - "TenderSavedSearch.instantAlert — per-profile instant alert flag"
+  - "TenderMatchingService.matchDelta(newTenderIds) — delta-only matching engine"
+  - "pollDueSources now collects genuinely-new tender IDs and triggers matchDelta"
+affects: [12-02-tender-digest-send, 12-03-tender-instant-alerts, 12-04-tender-notification-settings-ui]
+
+tech-stack:
+  added: []
+  patterns:
+    - "Delta-only matching: matchDelta only ever sees id IN newTenderIds — no historical rescan, structural backfill-flood prevention (D-07)"
+    - "Idempotent match upsert on @@unique([tenderId,savedSearchId]) with update:{} preserves notifiedAt on re-match (D-06)"
+    - "Indexed dedupKey pre-check (findUnique before upsert) to detect genuinely-new rows, since Prisma upsert doesn't report create-vs-update"
+
+key-files:
+  created:
+    - apps/api/prisma/migrations/20260722100000_add_tender_notifications/migration.sql
+    - apps/api/src/tenders/tender-matching.service.ts
+    - apps/api/src/tenders/tender-matching.service.spec.ts
+  modified:
+    - apps/api/prisma/schema.prisma
+    - apps/api/src/tenders/tender-ingestion.service.ts
+    - apps/api/src/tenders/tender-ingestion.service.spec.ts
+    - apps/api/src/tenders/tenders.module.ts
+
+key-decisions:
+  - "One notifiedAt field (not two per-channel fields) — the eligibility gate is a single WHERE notifiedAt IS NULL predicate (D-06)"
+  - "Delta-only matching instead of a backfill/suppression table — a new profile collects zero matches against the existing ~2188-row catalog by construction (D-07)"
+  - "matchDelta wraps each profile's matching in its own try/catch — one broken profile's filters JSON never aborts matching for the rest (matches pollDueSources' existing catch-and-log convention)"
+
+patterns-established:
+  - "matchDelta reuses buildTenderWhere(profile.filters as TenderQueryDto) — no new filter logic duplicated"
+
+requirements-completed: [NOTIFY-03]
+
+coverage:
+  - id: D1
+    description: "TenderMatch/TenderNotificationPref schema + TenderSavedSearch.instantAlert; migration applied to local dev DB"
+    requirement: "NOTIFY-03"
+    verification:
+      - kind: integration
+        ref: "npx prisma validate && npx prisma generate (manual) + psql to_regclass check"
+        status: pass
+    human_judgment: false
+  - id: D2
+    description: "TenderMatchingService.matchDelta — delta-only matching, match creation, idempotent upsert preserving notifiedAt, empty-delta no-op"
+    requirement: "NOTIFY-03"
+    verification:
+      - kind: unit
+        ref: "apps/api/src/tenders/tender-matching.service.spec.ts (6 tests)"
+        status: pass
+    human_judgment: false
+  - id: D3
+    description: "pollDueSources collects genuinely-new tender IDs (indexed dedupKey pre-check) and calls matchDelta only with those; TenderMatchingService registered in TendersModule"
+    requirement: "NOTIFY-03"
+    verification:
+      - kind: unit
+        ref: "apps/api/src/tenders/tender-ingestion.service.spec.ts (9 tests, incl. 2 new delta-only tests)"
+        status: pass
+    human_judgment: false
+
+duration: 35min
+completed: 2026-07-22
+status: complete
+---
+
+# Phase 12 Plan 01: Schema + Delta-Only Matching Engine Summary
+
+**TenderMatch/TenderNotificationPref schema (single notifiedAt eligibility gate) plus a delta-only TenderMatchingService wired into pollDueSources — structurally prevents any backfill-flood on new saved-search profiles.**
+
+## Performance
+
+- **Duration:** ~35 min
+- **Completed:** 2026-07-22T09:06:00Z
+- **Tasks:** 3 completed (Task 2 followed RED→GREEN TDD)
+- **Files modified:** 7 (3 created, 4 modified)
+
+## Accomplishments
+- Added `TenderMatch` (one row per tender × savedSearch pair, `@@unique([tenderId, savedSearchId])`, single nullable `notifiedAt` as the matched-vs-notified invariant gate) and `TenderNotificationPref` (per-user digest interval, default `daily`) models; added `TenderSavedSearch.instantAlert Boolean @default(false)`.
+- Hand-written migration `20260722100000_add_tender_notifications` applied to the local dev DB (`tessera` database, container `tessera-ctl-db-1`) and recorded in `_prisma_migrations`; `prisma generate` run so the client knows the new models.
+- Implemented `TenderMatchingService.matchDelta(newTenderIds)`: for every active `TenderSavedSearch`, reuses `buildTenderWhere(profile.filters)` AND-ed with `{ id: { in: newTenderIds } }`, and upserts `TenderMatch` idempotently (`update: {}` preserves any already-set `notifiedAt`).
+- Extended `TenderIngestionService.pollDueSources`: an indexed `dedupKey` pre-check now distinguishes genuinely-new tender rows from changed/re-seen ones; only the new IDs are handed to `matchDelta` at the end of the tick (delta-only boundary, D-07 — no historical rescan, no backfill/suppression table).
+- Registered `TenderMatchingService` as a provider in `TendersModule` and injected it into `TenderIngestionService`.
+
+## Task Commits
+
+Each task was committed atomically:
+
+1. **Task 1: Schema models + hand-written migration + local DB apply** - `6c3e110` (feat)
+2. **Task 2: TenderMatchingService.matchDelta (TDD)** - `57d22a0` (test, RED) → `92d4c96` (feat, GREEN)
+3. **Task 3: Wire matchDelta into pollDueSources + provider registration** - `b45047f` (feat)
+
+**Plan metadata:** commit pending (this SUMMARY + STATE/ROADMAP update)
+
+## Files Created/Modified
+- `apps/api/prisma/schema.prisma` - Added `TenderMatch`, `TenderNotificationPref` models; `TenderSavedSearch.instantAlert` + `matches` relation; `Tender.matches` relation
+- `apps/api/prisma/migrations/20260722100000_add_tender_notifications/migration.sql` - Hand-written migration, applied to local dev DB
+- `apps/api/src/tenders/tender-matching.service.ts` - `matchDelta(newTenderIds)`: delta-only matching engine
+- `apps/api/src/tenders/tender-matching.service.spec.ts` - 6 unit tests (delta-only, match creation, idempotency, empty-delta)
+- `apps/api/src/tenders/tender-ingestion.service.ts` - Collects genuinely-new tender IDs; calls `matchDelta` at tick end
+- `apps/api/src/tenders/tender-ingestion.service.spec.ts` - 2 new tests asserting delta-only invocation of `matchDelta`
+- `apps/api/src/tenders/tenders.module.ts` - `TenderMatchingService` registered as provider
+
+## Decisions Made
+- Single `notifiedAt` field (not two per-channel timestamps) — matches RESEARCH.md's explicit recommendation; simpler invariant, no risk of implying double-send is allowed per channel.
+- Delta-only matching (no backfill/suppression table) as the structural mechanism for D-07 — verified: `matchDelta([])` performs zero DB access, and a large simulated pre-existing catalog (2188 IDs) that would all match a profile's filters still yields zero matches when none of those IDs are passed as `newTenderIds`.
+- Added per-profile try/catch inside `matchDelta` (Rule 2 — matches the established `pollDueSources` catch-and-log convention; not explicitly specified in the plan's action text, but the plan's own `<done>` criteria and RESEARCH.md's "Don't Hand-Roll"/pitfalls make single-profile robustness an implicit correctness requirement, since a malformed `filters` JSON on one profile must not silently prevent matching for all other profiles in the same tick).
+
+## Deviations from Plan
+
+### Auto-fixed Issues
+
+**1. [Rule 1 - Bug] Plan's verify command referenced the wrong local DB name**
+- **Found during:** Task 1 verification
+- **Issue:** The plan's `<verify>` block and `12-VALIDATION.md` reference `docker compose exec ... -d tessera_dev`, but `docker-compose.yml` actually provisions `POSTGRES_DB: tessera` (confirmed via `docker exec tessera-ctl-db-1 psql -U tessera -d tessera -c "\dt"`, which lists all existing Tender/TenderSavedSearch/TenderTriage tables). `tessera_dev` is the DB **password**, not the DB name.
+- **Fix:** Applied the migration and ran all verification (`to_regclass`, `\d`, `prisma validate`/`generate`) against `-d tessera` instead. No schema/code change — purely a corrected command target.
+- **Verification:** `SELECT to_regclass('public."TenderMatch"'), to_regclass('public."TenderNotificationPref"')` returned both table OIDs (non-null); `\d "TenderSavedSearch"` shows `instantAlert boolean not null default false`.
+- **Committed in:** 6c3e110 (Task 1 commit)
+
+---
+
+**Total deviations:** 1 auto-fixed (1 bug — wrong DB name in verify command, not a code defect)
+**Impact on plan:** No scope creep; purely a corrected verification target. All three tasks executed exactly as specified otherwise.
+
+## Issues Encountered
+None beyond the DB-name correction above. `gen_random_uuid()` was available on the DB for the manual `_prisma_migrations` bookkeeping insert (Postgres 16 built-in via `pgcrypto`/native function).
+
+## User Setup Required
+None - no external service configuration required. Migration was applied directly to the local dev DB per the environment constraints (no Docker rebuild, no test/prod server touched).
+
+## Next Phase Readiness
+- `TenderMatch` rows are now created (with `notifiedAt=NULL`) for every genuinely-new tender that matches an active saved search — ready for Plan 12-02 (digest send) and 12-03 (instant alerts) to consume via `WHERE notifiedAt IS NULL`.
+- `TenderNotificationPref` and `TenderSavedSearch.instantAlert` are in place for Plan 12-04's settings UI.
+- No blockers. `pnpm --filter api test` (full suite) is green: 172/172 tests across 15 files. `npx tsc --noEmit` clean.
+
+---
+*Phase: 12-tender-notifications*
+*Completed: 2026-07-22*
+
+## Self-Check: PASSED
+
+All created files verified present on disk; all 4 task commit hashes (6c3e110, 57d22a0, 92d4c96, b45047f) verified present in git log.