docs(12-01): complete schema + delta-only matching plan
This commit is contained in:
@@ -46,7 +46,7 @@
|
||||
|
||||
- [ ] **NOTIFY-01**: Nutzer erhält einen periodischen E-Mail-Digest passender Treffer; Intervall im Webinterface konfigurierbar.
|
||||
- [ ] **NOTIFY-02**: Nutzer erhält optional eine Sofort-E-Mail bei einem neuen Treffer eines aktiven Suchprofils; im Webinterface aktivierbar.
|
||||
- [ ] **NOTIFY-03**: Das System unterscheidet „getroffen" von „benachrichtigt" (kein Doppelversand Digest+Sofort; kein Rückstau-Massenversand beim Anlegen eines Suchprofils).
|
||||
- [x] **NOTIFY-03**: Das System unterscheidet „getroffen" von „benachrichtigt" (kein Doppelversand Digest+Sofort; kein Rückstau-Massenversand beim Anlegen eines Suchprofils).
|
||||
- [ ] **NOTIFY-04**: Der E-Mail-Versand nutzt die mandantenspezifische SMTP-Konfiguration (bestehendes DKV-Mail-Muster).
|
||||
|
||||
### CONFIG — Modul & Verwaltung
|
||||
@@ -94,7 +94,7 @@
|
||||
| UI-05 | Phase 11 | Complete |
|
||||
| NOTIFY-01 | Phase 12 | Pending |
|
||||
| NOTIFY-02 | Phase 12 | Pending |
|
||||
| NOTIFY-03 | Phase 12 | Pending |
|
||||
| NOTIFY-03 | Phase 12 | Complete |
|
||||
| NOTIFY-04 | Phase 12 | Pending |
|
||||
| INGEST-02 | Phase 13 | Pending |
|
||||
| INGEST-03 | Phase 13 | Pending |
|
||||
|
||||
@@ -410,12 +410,12 @@ Plans:
|
||||
3. Activating a new search profile with many pre-existing historical matches does not flood the user with individual emails -- backfill is suppressed/batched on first activation, and a match already notified once (digest or instant) is never notified again for the same tender+profile pair (explicit matched-vs-notified state)
|
||||
4. Notification emails are sent through the tenant's own SMTP configuration (reusing the DKV mail pattern), not a shared/global system mailer
|
||||
|
||||
**Plans**: 4 plans
|
||||
**Plans**: 1/4 plans executed
|
||||
**UI hint**: yes
|
||||
|
||||
**Wave 1**
|
||||
|
||||
- [ ] 12-01-PLAN.md — Schema (TenderMatch/NotificationPref/instantAlert) + delta-only Matching in den Poll-Tick (NOTIFY-03)
|
||||
- [x] 12-01-PLAN.md — Schema (TenderMatch/NotificationPref/instantAlert) + delta-only Matching in den Poll-Tick (NOTIFY-03)
|
||||
|
||||
**Wave 2** *(blocked on Wave 1)*
|
||||
|
||||
@@ -476,6 +476,6 @@ Phases execute in numeric order: 1 -> 2 -> 3 -> 4 -> 5 -> 6 -> 7 -> 8 -> 9 -> 10
|
||||
| 9. Cert Manager Module | 6/6 | Complete | 2026-07-02 |
|
||||
| 10. Ausschreibungs-Radar Foundation & DÖE Ingestion | 6/6 | Complete | 2026-07-21 |
|
||||
| 11. Filter Engine, Results UI & Saved Searches | 6/6 | In Progress| |
|
||||
| 12. Tender Notifications | 0/TBD | Not started | - |
|
||||
| 12. Tender Notifications | 1/4 | In Progress| |
|
||||
| 13. Scraping Adapters & Cross-Source Deduplication | 0/TBD | Not started | - |
|
||||
| 14. RSS, Email-Alert Ingestion & Module Rollout | 0/TBD | Not started | - |
|
||||
|
||||
+21
-18
@@ -2,18 +2,18 @@
|
||||
gsd_state_version: 1.0
|
||||
milestone: v1.1
|
||||
milestone_name: Ausschreibungs-Radar
|
||||
current_phase: 11
|
||||
current_phase_name: filter-engine-results-ui-saved-searches
|
||||
status: verified
|
||||
stopped_at: Phase 11 verified — live browser UAT passed (all 5 criteria)
|
||||
last_updated: "2026-07-21T14:52:59.205Z"
|
||||
last_activity: 2026-07-21
|
||||
last_activity_desc: Phase 11 execution started
|
||||
current_phase: 12
|
||||
current_phase_name: tender-notifications
|
||||
status: executing
|
||||
stopped_at: Completed 12-01-PLAN.md
|
||||
last_updated: "2026-07-22T07:06:59.374Z"
|
||||
last_activity: 2026-07-22
|
||||
last_activity_desc: Phase 12 execution started
|
||||
progress:
|
||||
total_phases: 11
|
||||
total_phases: 12
|
||||
completed_phases: 10
|
||||
total_plans: 52
|
||||
completed_plans: 51
|
||||
total_plans: 56
|
||||
completed_plans: 52
|
||||
---
|
||||
|
||||
# Project State
|
||||
@@ -23,16 +23,16 @@ progress:
|
||||
See: .planning/PROJECT.md (updated 2026-07-17)
|
||||
|
||||
**Core value:** Eine zentrale Plattform, in der beliebige Workflow-Tools als Module lizenziert, aktiviert und genutzt werden koennen -- ohne zwischen verschiedenen Anwendungen wechseln zu muessen.
|
||||
**Current focus:** Phase 11 — filter-engine-results-ui-saved-searches
|
||||
**Current focus:** Phase 12 — tender-notifications
|
||||
|
||||
## Current Position
|
||||
|
||||
Phase: 11 (filter-engine-results-ui-saved-searches) — EXECUTING
|
||||
Plan: 6 of 6
|
||||
Status: Phase complete — ready for verification
|
||||
Last activity: 2026-07-21 — Phase 11 execution started
|
||||
Phase: 12 (tender-notifications) — EXECUTING
|
||||
Plan: 2 of 4
|
||||
Status: Ready to execute
|
||||
Last activity: 2026-07-22 — Phase 12 execution started
|
||||
|
||||
Progress: [██████████] 98%
|
||||
Progress: [█████████░] 93%
|
||||
|
||||
## Performance Metrics
|
||||
|
||||
@@ -86,6 +86,7 @@ Progress: [██████████] 98%
|
||||
| Phase 11 P04 | 12min | 2 tasks | 5 files |
|
||||
| Phase 11 P05 | 24min | 3 tasks | 15 files |
|
||||
| Phase 11 P06 | 35min | 3 tasks | 12 files |
|
||||
| Phase 12 P01 | 35min | 3 tasks | 7 files |
|
||||
|
||||
## Accumulated Context
|
||||
|
||||
@@ -181,6 +182,8 @@ Recent decisions affecting current work:
|
||||
- [Phase ?]: TenderTriage (neu, per-user, kein forTenant/RLS) + favOnly-Sentinel-ID '__none__' für garantierten Zero-Match statt versehentlich ungefilterter Liste
|
||||
- [Phase ?]: TenderSavedSearch hat keinen Tender-FK — speichert nur Filterkriterien, unkritisch bei Retention.
|
||||
- [Phase ?]: page/tender-URL-Params bewusst aus dem Suchprofil-Payload ausgeschlossen (Navigations-/View-State, kein Filter-State).
|
||||
- [Phase ?]: Single notifiedAt field (not two per-channel timestamps) as the matched-vs-notified eligibility gate (12-01, D-06)
|
||||
- [Phase ?]: Delta-only matching (no backfill/suppression table) structurally prevents backfill-flood for new saved-search profiles (12-01, D-07)
|
||||
|
||||
### Pending Todos
|
||||
|
||||
@@ -218,7 +221,7 @@ Items acknowledged and carried forward from previous milestone close:
|
||||
|
||||
## Session Continuity
|
||||
|
||||
Last session: 2026-07-21T14:52:59.192Z
|
||||
Stopped at: Completed 11-06-PLAN.md
|
||||
Last session: 2026-07-22T07:06:59.360Z
|
||||
Stopped at: Completed 12-01-PLAN.md
|
||||
Resume file: None
|
||||
Last activity: 2026-07-14 - Built LDAP per-user exclude/denylist filter (9d1323f), migration applied on live DB, verified via Playwright: sync deactivated 4 excluded service accounts (administrator/krbtgt/guest/dns-ldap), 2 real LDAP users stay active, 0 wrongly created
|
||||
|
||||
@@ -0,0 +1,154 @@
|
||||
---
|
||||
phase: 12-tender-notifications
|
||||
plan: 01
|
||||
subsystem: api
|
||||
tags: [prisma, nestjs, tender-radar, notifications, matching]
|
||||
|
||||
requires:
|
||||
- phase: 11-tender-saved-searches
|
||||
provides: "TenderSavedSearch (filters Json, userId/tenantId scoping) + buildTenderWhere(dto) filter compiler"
|
||||
- phase: 10-tender-radar-ingestion
|
||||
provides: "TenderIngestionService.pollDueSources poll-tick + Tender catalog (dedupKey upsert)"
|
||||
provides:
|
||||
- "TenderMatch model — matched-vs-notified state (single nullable notifiedAt gate)"
|
||||
- "TenderNotificationPref model — per-user digest interval preference"
|
||||
- "TenderSavedSearch.instantAlert — per-profile instant alert flag"
|
||||
- "TenderMatchingService.matchDelta(newTenderIds) — delta-only matching engine"
|
||||
- "pollDueSources now collects genuinely-new tender IDs and triggers matchDelta"
|
||||
affects: [12-02-tender-digest-send, 12-03-tender-instant-alerts, 12-04-tender-notification-settings-ui]
|
||||
|
||||
tech-stack:
|
||||
added: []
|
||||
patterns:
|
||||
- "Delta-only matching: matchDelta only ever sees id IN newTenderIds — no historical rescan, structural backfill-flood prevention (D-07)"
|
||||
- "Idempotent match upsert on @@unique([tenderId,savedSearchId]) with update:{} preserves notifiedAt on re-match (D-06)"
|
||||
- "Indexed dedupKey pre-check (findUnique before upsert) to detect genuinely-new rows, since Prisma upsert doesn't report create-vs-update"
|
||||
|
||||
key-files:
|
||||
created:
|
||||
- apps/api/prisma/migrations/20260722100000_add_tender_notifications/migration.sql
|
||||
- apps/api/src/tenders/tender-matching.service.ts
|
||||
- apps/api/src/tenders/tender-matching.service.spec.ts
|
||||
modified:
|
||||
- apps/api/prisma/schema.prisma
|
||||
- apps/api/src/tenders/tender-ingestion.service.ts
|
||||
- apps/api/src/tenders/tender-ingestion.service.spec.ts
|
||||
- apps/api/src/tenders/tenders.module.ts
|
||||
|
||||
key-decisions:
|
||||
- "One notifiedAt field (not two per-channel fields) — the eligibility gate is a single WHERE notifiedAt IS NULL predicate (D-06)"
|
||||
- "Delta-only matching instead of a backfill/suppression table — a new profile collects zero matches against the existing ~2188-row catalog by construction (D-07)"
|
||||
- "matchDelta wraps each profile's matching in its own try/catch — one broken profile's filters JSON never aborts matching for the rest (matches pollDueSources' existing catch-and-log convention)"
|
||||
|
||||
patterns-established:
|
||||
- "matchDelta reuses buildTenderWhere(profile.filters as TenderQueryDto) — no new filter logic duplicated"
|
||||
|
||||
requirements-completed: [NOTIFY-03]
|
||||
|
||||
coverage:
|
||||
- id: D1
|
||||
description: "TenderMatch/TenderNotificationPref schema + TenderSavedSearch.instantAlert; migration applied to local dev DB"
|
||||
requirement: "NOTIFY-03"
|
||||
verification:
|
||||
- kind: integration
|
||||
ref: "npx prisma validate && npx prisma generate (manual) + psql to_regclass check"
|
||||
status: pass
|
||||
human_judgment: false
|
||||
- id: D2
|
||||
description: "TenderMatchingService.matchDelta — delta-only matching, match creation, idempotent upsert preserving notifiedAt, empty-delta no-op"
|
||||
requirement: "NOTIFY-03"
|
||||
verification:
|
||||
- kind: unit
|
||||
ref: "apps/api/src/tenders/tender-matching.service.spec.ts (6 tests)"
|
||||
status: pass
|
||||
human_judgment: false
|
||||
- id: D3
|
||||
description: "pollDueSources collects genuinely-new tender IDs (indexed dedupKey pre-check) and calls matchDelta only with those; TenderMatchingService registered in TendersModule"
|
||||
requirement: "NOTIFY-03"
|
||||
verification:
|
||||
- kind: unit
|
||||
ref: "apps/api/src/tenders/tender-ingestion.service.spec.ts (9 tests, incl. 2 new delta-only tests)"
|
||||
status: pass
|
||||
human_judgment: false
|
||||
|
||||
duration: 35min
|
||||
completed: 2026-07-22
|
||||
status: complete
|
||||
---
|
||||
|
||||
# Phase 12 Plan 01: Schema + Delta-Only Matching Engine Summary
|
||||
|
||||
**TenderMatch/TenderNotificationPref schema (single notifiedAt eligibility gate) plus a delta-only TenderMatchingService wired into pollDueSources — structurally prevents any backfill-flood on new saved-search profiles.**
|
||||
|
||||
## Performance
|
||||
|
||||
- **Duration:** ~35 min
|
||||
- **Completed:** 2026-07-22T09:06:00Z
|
||||
- **Tasks:** 3 completed (Task 2 followed RED→GREEN TDD)
|
||||
- **Files modified:** 7 (3 created, 4 modified)
|
||||
|
||||
## Accomplishments
|
||||
- Added `TenderMatch` (one row per tender × savedSearch pair, `@@unique([tenderId, savedSearchId])`, single nullable `notifiedAt` as the matched-vs-notified invariant gate) and `TenderNotificationPref` (per-user digest interval, default `daily`) models; added `TenderSavedSearch.instantAlert Boolean @default(false)`.
|
||||
- Hand-written migration `20260722100000_add_tender_notifications` applied to the local dev DB (`tessera` database, container `tessera-ctl-db-1`) and recorded in `_prisma_migrations`; `prisma generate` run so the client knows the new models.
|
||||
- Implemented `TenderMatchingService.matchDelta(newTenderIds)`: for every active `TenderSavedSearch`, reuses `buildTenderWhere(profile.filters)` AND-ed with `{ id: { in: newTenderIds } }`, and upserts `TenderMatch` idempotently (`update: {}` preserves any already-set `notifiedAt`).
|
||||
- Extended `TenderIngestionService.pollDueSources`: an indexed `dedupKey` pre-check now distinguishes genuinely-new tender rows from changed/re-seen ones; only the new IDs are handed to `matchDelta` at the end of the tick (delta-only boundary, D-07 — no historical rescan, no backfill/suppression table).
|
||||
- Registered `TenderMatchingService` as a provider in `TendersModule` and injected it into `TenderIngestionService`.
|
||||
|
||||
## Task Commits
|
||||
|
||||
Each task was committed atomically:
|
||||
|
||||
1. **Task 1: Schema models + hand-written migration + local DB apply** - `6c3e110` (feat)
|
||||
2. **Task 2: TenderMatchingService.matchDelta (TDD)** - `57d22a0` (test, RED) → `92d4c96` (feat, GREEN)
|
||||
3. **Task 3: Wire matchDelta into pollDueSources + provider registration** - `b45047f` (feat)
|
||||
|
||||
**Plan metadata:** commit pending (this SUMMARY + STATE/ROADMAP update)
|
||||
|
||||
## Files Created/Modified
|
||||
- `apps/api/prisma/schema.prisma` - Added `TenderMatch`, `TenderNotificationPref` models; `TenderSavedSearch.instantAlert` + `matches` relation; `Tender.matches` relation
|
||||
- `apps/api/prisma/migrations/20260722100000_add_tender_notifications/migration.sql` - Hand-written migration, applied to local dev DB
|
||||
- `apps/api/src/tenders/tender-matching.service.ts` - `matchDelta(newTenderIds)`: delta-only matching engine
|
||||
- `apps/api/src/tenders/tender-matching.service.spec.ts` - 6 unit tests (delta-only, match creation, idempotency, empty-delta)
|
||||
- `apps/api/src/tenders/tender-ingestion.service.ts` - Collects genuinely-new tender IDs; calls `matchDelta` at tick end
|
||||
- `apps/api/src/tenders/tender-ingestion.service.spec.ts` - 2 new tests asserting delta-only invocation of `matchDelta`
|
||||
- `apps/api/src/tenders/tenders.module.ts` - `TenderMatchingService` registered as provider
|
||||
|
||||
## Decisions Made
|
||||
- Single `notifiedAt` field (not two per-channel timestamps) — matches RESEARCH.md's explicit recommendation; simpler invariant, no risk of implying double-send is allowed per channel.
|
||||
- Delta-only matching (no backfill/suppression table) as the structural mechanism for D-07 — verified: `matchDelta([])` performs zero DB access, and a large simulated pre-existing catalog (2188 IDs) that would all match a profile's filters still yields zero matches when none of those IDs are passed as `newTenderIds`.
|
||||
- Added per-profile try/catch inside `matchDelta` (Rule 2 — matches the established `pollDueSources` catch-and-log convention; not explicitly specified in the plan's action text, but the plan's own `<done>` criteria and RESEARCH.md's "Don't Hand-Roll"/pitfalls make single-profile robustness an implicit correctness requirement, since a malformed `filters` JSON on one profile must not silently prevent matching for all other profiles in the same tick).
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
### Auto-fixed Issues
|
||||
|
||||
**1. [Rule 1 - Bug] Plan's verify command referenced the wrong local DB name**
|
||||
- **Found during:** Task 1 verification
|
||||
- **Issue:** The plan's `<verify>` block and `12-VALIDATION.md` reference `docker compose exec ... -d tessera_dev`, but `docker-compose.yml` actually provisions `POSTGRES_DB: tessera` (confirmed via `docker exec tessera-ctl-db-1 psql -U tessera -d tessera -c "\dt"`, which lists all existing Tender/TenderSavedSearch/TenderTriage tables). `tessera_dev` is the DB **password**, not the DB name.
|
||||
- **Fix:** Applied the migration and ran all verification (`to_regclass`, `\d`, `prisma validate`/`generate`) against `-d tessera` instead. No schema/code change — purely a corrected command target.
|
||||
- **Verification:** `SELECT to_regclass('public."TenderMatch"'), to_regclass('public."TenderNotificationPref"')` returned both table OIDs (non-null); `\d "TenderSavedSearch"` shows `instantAlert boolean not null default false`.
|
||||
- **Committed in:** 6c3e110 (Task 1 commit)
|
||||
|
||||
---
|
||||
|
||||
**Total deviations:** 1 auto-fixed (1 bug — wrong DB name in verify command, not a code defect)
|
||||
**Impact on plan:** No scope creep; purely a corrected verification target. All three tasks executed exactly as specified otherwise.
|
||||
|
||||
## Issues Encountered
|
||||
None beyond the DB-name correction above. `gen_random_uuid()` was available on the DB for the manual `_prisma_migrations` bookkeeping insert (Postgres 16 built-in via `pgcrypto`/native function).
|
||||
|
||||
## User Setup Required
|
||||
None - no external service configuration required. Migration was applied directly to the local dev DB per the environment constraints (no Docker rebuild, no test/prod server touched).
|
||||
|
||||
## Next Phase Readiness
|
||||
- `TenderMatch` rows are now created (with `notifiedAt=NULL`) for every genuinely-new tender that matches an active saved search — ready for Plan 12-02 (digest send) and 12-03 (instant alerts) to consume via `WHERE notifiedAt IS NULL`.
|
||||
- `TenderNotificationPref` and `TenderSavedSearch.instantAlert` are in place for Plan 12-04's settings UI.
|
||||
- No blockers. `pnpm --filter api test` (full suite) is green: 172/172 tests across 15 files. `npx tsc --noEmit` clean.
|
||||
|
||||
---
|
||||
*Phase: 12-tender-notifications*
|
||||
*Completed: 2026-07-22*
|
||||
|
||||
## Self-Check: PASSED
|
||||
|
||||
All created files verified present on disk; all 4 task commit hashes (6c3e110, 57d22a0, 92d4c96, b45047f) verified present in git log.
|
||||
Reference in New Issue
Block a user