diff --git a/.planning/REQUIREMENTS.md b/.planning/REQUIREMENTS.md index 3a698cf..03d467c 100644 --- a/.planning/REQUIREMENTS.md +++ b/.planning/REQUIREMENTS.md @@ -44,10 +44,10 @@ ### NOTIFY — Benachrichtigung -- [ ] **NOTIFY-01**: Nutzer erhält einen periodischen E-Mail-Digest passender Treffer; Intervall im Webinterface konfigurierbar. +- [x] **NOTIFY-01**: Nutzer erhält einen periodischen E-Mail-Digest passender Treffer; Intervall im Webinterface konfigurierbar. - [ ] **NOTIFY-02**: Nutzer erhält optional eine Sofort-E-Mail bei einem neuen Treffer eines aktiven Suchprofils; im Webinterface aktivierbar. - [x] **NOTIFY-03**: Das System unterscheidet „getroffen" von „benachrichtigt" (kein Doppelversand Digest+Sofort; kein Rückstau-Massenversand beim Anlegen eines Suchprofils). -- [ ] **NOTIFY-04**: Der E-Mail-Versand nutzt die mandantenspezifische SMTP-Konfiguration (bestehendes DKV-Mail-Muster). +- [x] **NOTIFY-04**: Der E-Mail-Versand nutzt die mandantenspezifische SMTP-Konfiguration (bestehendes DKV-Mail-Muster). ### CONFIG — Modul & Verwaltung @@ -92,10 +92,10 @@ | UI-03 | Phase 11 | Complete | | UI-04 | Phase 11 | Complete | | UI-05 | Phase 11 | Complete | -| NOTIFY-01 | Phase 12 | Pending | +| NOTIFY-01 | Phase 12 | Complete | | NOTIFY-02 | Phase 12 | Pending | | NOTIFY-03 | Phase 12 | Complete | -| NOTIFY-04 | Phase 12 | Pending | +| NOTIFY-04 | Phase 12 | Complete | | INGEST-02 | Phase 13 | Pending | | INGEST-03 | Phase 13 | Pending | | INGEST-07 | Phase 13 | Pending | diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index 569010e..52b1efc 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -410,7 +410,7 @@ Plans: 3. Activating a new search profile with many pre-existing historical matches does not flood the user with individual emails -- backfill is suppressed/batched on first activation, and a match already notified once (digest or instant) is never notified again for the same tender+profile pair (explicit matched-vs-notified state) 4. Notification emails are sent through the tenant's own SMTP configuration (reusing the DKV mail pattern), not a shared/global system mailer -**Plans**: 1/4 plans executed +**Plans**: 2/4 plans executed **UI hint**: yes **Wave 1** @@ -419,7 +419,7 @@ Plans: **Wave 2** *(blocked on Wave 1)* -- [ ] 12-02-PLAN.md — TenderMailService (mandanten-SMTP) + globaler Digest-Cron (NOTIFY-01/04) +- [x] 12-02-PLAN.md — TenderMailService (mandanten-SMTP) + globaler Digest-Cron (NOTIFY-01/04) **Wave 3** *(blocked on Wave 2)* @@ -476,6 +476,6 @@ Phases execute in numeric order: 1 -> 2 -> 3 -> 4 -> 5 -> 6 -> 7 -> 8 -> 9 -> 10 | 9. Cert Manager Module | 6/6 | Complete | 2026-07-02 | | 10. Ausschreibungs-Radar Foundation & DÖE Ingestion | 6/6 | Complete | 2026-07-21 | | 11. Filter Engine, Results UI & Saved Searches | 6/6 | In Progress| | -| 12. Tender Notifications | 1/4 | In Progress| | +| 12. Tender Notifications | 2/4 | In Progress| | | 13. Scraping Adapters & Cross-Source Deduplication | 0/TBD | Not started | - | | 14. RSS, Email-Alert Ingestion & Module Rollout | 0/TBD | Not started | - | diff --git a/.planning/STATE.md b/.planning/STATE.md index cdd8af4..709a584 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -5,15 +5,15 @@ milestone_name: Ausschreibungs-Radar current_phase: 12 current_phase_name: tender-notifications status: executing -stopped_at: Completed 12-01-PLAN.md -last_updated: "2026-07-22T07:06:59.374Z" +stopped_at: Completed 12-02-PLAN.md +last_updated: "2026-07-22T07:16:13.211Z" last_activity: 2026-07-22 last_activity_desc: Phase 12 execution started progress: total_phases: 12 completed_phases: 10 total_plans: 56 - completed_plans: 52 + completed_plans: 53 --- # Project State @@ -28,11 +28,11 @@ See: .planning/PROJECT.md (updated 2026-07-17) ## Current Position Phase: 12 (tender-notifications) — EXECUTING -Plan: 2 of 4 +Plan: 3 of 4 Status: Ready to execute Last activity: 2026-07-22 — Phase 12 execution started -Progress: [█████████░] 93% +Progress: [██████████] 95% ## Performance Metrics @@ -87,6 +87,7 @@ Progress: [█████████░] 93% | Phase 11 P05 | 24min | 3 tasks | 15 files | | Phase 11 P06 | 35min | 3 tasks | 12 files | | Phase 12 P01 | 35min | 3 tasks | 7 files | +| Phase 12 P02 | 8min | 3 tasks | 5 files | ## Accumulated Context @@ -184,6 +185,8 @@ Recent decisions affecting current work: - [Phase ?]: page/tender-URL-Params bewusst aus dem Suchprofil-Payload ausgeschlossen (Navigations-/View-State, kein Filter-State). - [Phase ?]: Single notifiedAt field (not two per-channel timestamps) as the matched-vs-notified eligibility gate (12-01, D-06) - [Phase ?]: Delta-only matching (no backfill/suppression table) structurally prevents backfill-flood for new saved-search profiles (12-01, D-07) +- [Phase ?]: TenderMailService swallows missing-SmtpConfig and send-failure into a single boolean (never throws) so the digest cron gets one clean success signal for stamping notifiedAt +- [Phase ?]: TenderDigestScheduler.runDigest(now) takes an injectable clock parameter for testable Monday-only weekly-digest gating ### Pending Todos @@ -221,7 +224,7 @@ Items acknowledged and carried forward from previous milestone close: ## Session Continuity -Last session: 2026-07-22T07:06:59.360Z -Stopped at: Completed 12-01-PLAN.md +Last session: 2026-07-22T07:16:13.196Z +Stopped at: Completed 12-02-PLAN.md Resume file: None Last activity: 2026-07-14 - Built LDAP per-user exclude/denylist filter (9d1323f), migration applied on live DB, verified via Playwright: sync deactivated 4 excluded service accounts (administrator/krbtgt/guest/dns-ldap), 2 real LDAP users stay active, 0 wrongly created diff --git a/.planning/phases/12-tender-notifications/12-02-SUMMARY.md b/.planning/phases/12-tender-notifications/12-02-SUMMARY.md new file mode 100644 index 0000000..79b0a84 --- /dev/null +++ b/.planning/phases/12-tender-notifications/12-02-SUMMARY.md @@ -0,0 +1,164 @@ +--- +phase: 12-tender-notifications +plan: 02 +subsystem: api +tags: [nestjs, nodemailer, cron, tender-radar, notifications, smtp, multi-tenant] + +requires: + - phase: 12-01 + provides: "TenderMatch model (single nullable notifiedAt eligibility gate) + TenderNotificationPref model + TenderMatchingService.matchDelta" + - phase: 07/10 (DKV mail pattern) + provides: "DkvMailService structural template — fresh nodemailer transport per send via SettingsService.getDecryptedSmtpConfig(tenantId), transport.close() in finally" +provides: + - "TenderMailService — sendDigest/sendInstant, DkvMailService-clone tenant-SMTP send path, never throws (false = skipped/failed)" + - "TenderDigestScheduler — ONE global @nestjs/schedule cron (daily 07:00), findMany over ALL due users across ALL tenants" + - "TendersModule now imports SettingsModule and registers both new providers — DI graph resolvable" +affects: [12-03-tender-instant-alerts, 12-04-tender-notification-settings-ui] + +tech-stack: + added: [] + patterns: + - "TenderMailService never throws — sendDigest/sendInstant resolve to false on missing SmtpConfig or send failure, letting callers decide notifiedAt stamping with a single boolean instead of try/catch" + - "TenderDigestScheduler.runDigest(now: Date = new Date()) accepts an injectable clock — tests pass a fixed Monday/Tuesday date instead of faking the system clock" + - "Digest due-date resolution: missing TenderNotificationPref row defaults to 'daily' (D-01); weekly is gated on a Europe/Berlin Monday check" + +key-files: + created: + - apps/api/src/tenders/tender-mail.service.ts + - apps/api/src/tenders/tender-mail.service.spec.ts + - apps/api/src/tenders/tender-digest.scheduler.ts + - apps/api/src/tenders/tender-digest.scheduler.spec.ts + modified: + - apps/api/src/tenders/tenders.module.ts + +key-decisions: + - "TenderMailService swallows both the 'missing SmtpConfig' AND 'send failure' cases into a single boolean return (never throws) — unlike DkvMailService (which rethrows for its retry orchestrator), because the digest scheduler needs one clean signal to decide whether to stamp notifiedAt, and a cron run must never crash on one tenant's misconfiguration" + - "runDigest(now: Date = new Date()) takes an injectable clock parameter instead of relying on vi.useFakeTimers()/system-clock mocking — makes the Monday-only weekly gate directly testable with fixed dates" + - "estimatedValue is formatted via String() only, never Number() — preserves a Prisma Decimal's own toString(), a plain numeric string, or null verbatim without precision loss or a silent NaN" + +patterns-established: + - "Digest candidate selection: prisma.tenderMatch.findMany({ where: { notifiedAt: null }, select: { userId: true }, distinct: ['userId'] }) across ALL tenants in one query — the findMany-not-findFirst multi-tenant safety invariant from TenderSchedulerService, now also proven for the digest cron" + +requirements-completed: [NOTIFY-01, NOTIFY-04, NOTIFY-03] + +coverage: + - id: D1 + description: "TenderMailService.sendDigest/sendInstant — per-send SMTP resolution via getDecryptedSmtpConfig(tenantId), fresh nodemailer transport + close() in finally (even on sendMail failure), no-throw skip on missing SmtpConfig, sectioned digest body with HTML-escaped titles/profile names and no blind Number() coercion of estimatedValue" + requirement: "NOTIFY-04" + verification: + - kind: unit + ref: "apps/api/src/tenders/tender-mail.service.spec.ts (9 tests)" + status: pass + human_judgment: false + - id: D2 + description: "TenderDigestScheduler — single global cron registered via SchedulerRegistry; due-date resolution (daily always, weekly Monday-only Europe/Berlin, off never, missing pref = daily default); multi-tenant findMany over all due users; one sectioned mail per user grouped by saved-search profile; notifiedAt/channel stamped only after a successful send; per-user try/catch so one broken user/tenant never aborts the run" + requirement: "NOTIFY-01" + verification: + - kind: unit + ref: "apps/api/src/tenders/tender-digest.scheduler.spec.ts (11 tests)" + status: pass + human_judgment: false + - id: D3 + description: "No-double-send invariant: digest selects strictly notifiedAt IS NULL matches; an already-instant-notified match is never re-included or re-stamped by the digest run" + requirement: "NOTIFY-03" + verification: + - kind: unit + ref: "apps/api/src/tenders/tender-digest.scheduler.spec.ts#kein Doppelversand (D-06)" + status: pass + human_judgment: false + - id: D4 + description: "TendersModule imports SettingsModule and registers TenderMailService + TenderDigestScheduler; DI graph resolvable" + verification: + - kind: unit + ref: "npx tsc --noEmit -p tsconfig.json (clean) + grep SettingsModule/TenderDigestScheduler in tenders.module.ts" + status: pass + human_judgment: false + - id: D5 + description: "Actual SMTP transport/delivery against a real mailbox (Mailhog) and true cross-tenant digest isolation with two real users/tenants" + verification: [] + human_judgment: true + rationale: "Real mail delivery cannot be observed with a mocked nodemailer transport; needs a live SMTP relay (localhost:1025) and two real tenant SmtpConfig rows — deferred to the phase-end UAT pass per 12-VALIDATION.md Manual-Only Verifications." + +duration: ~8min +completed: 2026-07-22 +status: complete +--- + +# Phase 12 Plan 02: Digest-Kanal + Mandanten-SMTP-Versand Summary + +**TenderMailService (DkvMailService-Klon: frischer nodemailer-Transport pro Send über mandanten-SMTP, transport.close() im finally, nie Throw) plus ein einziger globaler TenderDigestScheduler-Cron, der über findMany alle fälligen Nutzer aller Mandanten bedient, je Nutzer eine nach Suchprofil sektionierte Mail sendet und Matches erst nach erfolgreichem Versand als benachrichtigt stempelt.** + +## Performance + +- **Duration:** ~8 min +- **Started:** 2026-07-22T09:07:30Z +- **Completed:** 2026-07-22T09:14:40Z +- **Tasks:** 3 completed (Tasks 1 and 2 followed RED→GREEN TDD) +- **Files modified:** 5 (4 created, 1 modified) + +## Accomplishments +- Implemented `TenderMailService` as a structural clone of `DkvMailService`: `sendDigest(user, tenantId, sections)` builds ONE mail sectioned by saved-search profile name (D-02); `sendInstant(user, tenantId, search, tenders)` builds ONE collective mail for a single profile (D-05). Both resolve the tenant's decrypted SMTP config via `settingsService.getDecryptedSmtpConfig(tenantId)`, build a fresh `nodemailer` transport per send (never cached), and always `transport.close()` in `finally` (WR-01 socket-leak guard) — even when `sendMail` rejects. +- Unlike `DkvMailService`, `TenderMailService` never throws: a missing `SmtpConfig` and a send failure both resolve to `false`, giving callers a single boolean signal for whether `TenderMatch.notifiedAt` may be stamped. +- `estimatedValue` is rendered via `String()` only — never `Number()` — avoiding precision loss/NaN on the mostly-null Decimal field; tender titles and saved-search profile names are HTML-escaped before interpolation into the mail's HTML body (T-12-08 email-injection guard). +- Implemented `TenderDigestScheduler`: a SINGLE platform-wide `@nestjs/schedule` cron (`0 7 * * *`), registered via `SchedulerRegistry` exactly like `TenderSchedulerService` (not the single-tenant `DkvSchedulerService` pattern). `runDigest()` selects candidate users as distinct `userId`s with an open (`notifiedAt: null`) `TenderMatch` across **all** tenants via `findMany` (never `findFirst`), resolves each user's `TenderNotificationPref.digestInterval` (missing row → `'daily'` default per D-01), and — for due users only — groups their un-notified matches by saved-search profile name into one `sendDigest` call per user. `notifiedAt`+`notifiedChannel='digest'` are stamped only after a successful send. +- Robustness: each candidate user is processed inside its own `try`/`catch` — a missing SMTP config, a failed send, or an unexpected thrown error (e.g. a broken pref lookup) for one user/tenant leaves that user's matches `notifiedAt=NULL` (retried next run) and never aborts the run for the remaining users. +- Registered `TenderMailService` and `TenderDigestScheduler` as providers in `TendersModule`, and imported `SettingsModule` so `TenderMailService` can inject `SettingsService`. Full `apps/api` suite green: 192/192 across 17 files; `npx tsc --noEmit` clean. + +## Task Commits + +Each task was committed atomically: + +1. **Task 1: TenderMailService — mandanten-SMTP-Versand (DkvMailService-Klon)** — `fbcc108` (test, RED) → `53e72df` (feat, GREEN) +2. **Task 2: TenderDigestScheduler — ein globaler Cron, findMany über fällige Nutzer** — `2f30584` (test, RED) → `c0a8906` (feat, GREEN) +3. **Task 3: Provider-Registrierung + SettingsModule-Import** — `3244167` (feat) + +**Plan metadata:** commit pending (this SUMMARY + STATE/ROADMAP update) + +## Files Created/Modified +- `apps/api/src/tenders/tender-mail.service.ts` - `TenderMailService`: `sendDigest`/`sendInstant`, `resolveTransport` (fresh nodemailer transport per tenant SMTP config), text/HTML body builders, `escapeHtml`, `formatDeadline`, `formatEstimatedValue` +- `apps/api/src/tenders/tender-mail.service.spec.ts` - 9 unit tests: SMTP resolution (secure/requireTLS per encryption mode, auth-when-username), fresh-transport+close (incl. on sendMail failure), no-throw skip on missing config, sectioned digest body, HTML-escaping +- `apps/api/src/tenders/tender-digest.scheduler.ts` - `TenderDigestScheduler`: `onModuleInit` cron registration (`tender-digest`, `0 7 * * *`), `runDigest(now)` core fan-out, `groupMatchesByProfile`, `isMondayInBerlin` +- `apps/api/src/tenders/tender-digest.scheduler.spec.ts` - 11 unit tests: due-date resolution (daily/weekly/off/default-daily), multi-tenant findMany (two tenants both served, source has no `findFirst`), grouped-sections single mail, no-double-send stamping, per-user robustness (skip + thrown-error cases), cron registration +- `apps/api/src/tenders/tenders.module.ts` - Imports `SettingsModule`; registers `TenderMailService` + `TenderDigestScheduler` as providers alongside the existing `TenderMatchingService` + +## Decisions Made +- `TenderMailService` never throws (unlike `DkvMailService`) — both the missing-config and send-failure paths resolve to `false`. This keeps the digest scheduler's success/skip decision to a single boolean check instead of a nested try/catch, and structurally guarantees a cron tick cannot crash on one tenant's broken SMTP. +- `runDigest` takes an injectable `now: Date = new Date()` parameter rather than mocking the system clock in tests — the weekly-Monday-only gate is asserted directly with fixed dates (`2026-07-27` verified Monday, `2026-07-28` verified Tuesday, both in Europe/Berlin). +- `estimatedValue` formatting uses `String()` exclusively, never `Number()` — matches the plan's explicit instruction and preserves a Prisma `Decimal`'s own string representation without precision loss. + +## Deviations from Plan + +### Auto-fixed Issues + +**1. [Rule 1 - Bug] Removed literal `findFirst` from source-file comments to satisfy the scheduler's own regression test** +- **Found during:** Task 2 verification (`pnpm --filter api test -- tender-digest.scheduler`) +- **Issue:** The scheduler's own spec asserts (via `readFileSync` on the source) that the word `findFirst` never appears anywhere in `tender-digest.scheduler.ts` — including comments — as the Pitfall-1 regression guard. My initial doc comments referenced `DkvSchedulerService`'s "v1 single-tenant, findFirst()" pattern verbatim, which the source-scan test correctly flagged. +- **Fix:** Reworded the two occurrences to describe the same pattern without using the literal token (e.g. "v1 single-tenant, find-first-row pattern"; "never a first-row-only lookup"). +- **Files modified:** `apps/api/src/tenders/tender-digest.scheduler.ts` +- **Verification:** `pnpm --filter api test -- tender-digest.scheduler` green (11/11), including the source-scan assertion. +- **Committed in:** `c0a8906` (Task 2 GREEN commit) + +--- + +**Total deviations:** 1 auto-fixed (1 bug — comment wording, no logic change) +**Impact on plan:** No scope creep. All three tasks executed exactly as specified otherwise. + +## Issues Encountered +None beyond the comment-wording fix above. + +## User Setup Required +None - no external service configuration required. `TenderMailService`/`TenderDigestScheduler` reuse the existing per-tenant `SmtpConfig` rows configured via Settings (Phase 7); no new environment variables or dashboards. + +## Next Phase Readiness +- `TenderMailService.sendInstant` is already implemented (per the plan's explicit instruction) so Plan 12-03 (instant alerts) only needs to wire it into `TenderMatchingService`'s poll-tick dispatch — no new mail-sending logic required. +- `TenderDigestScheduler` is live and registered; once real per-tenant `SmtpConfig` rows and `TenderNotificationPref` rows exist, the daily 07:00 cron will pick up any un-notified `TenderMatch` rows automatically. +- Manual/UAT verification (real Mailhog send, two-tenant isolation with live SmtpConfig rows) remains open per `12-VALIDATION.md` — deferred to the phase-end UAT pass, consistent with Plan 12-01's precedent. +- No blockers. `pnpm --filter api test` / `npx vitest run` (full apps/api suite) green: 192/192 across 17 files. `npx tsc --noEmit` clean. + +--- +*Phase: 12-tender-notifications* +*Completed: 2026-07-22* + +## Self-Check: PASSED + +All created/modified files verified present on disk; all 5 task commit hashes (fbcc108, 53e72df, 2f30584, c0a8906, 3244167) verified present in git log.