--- phase: 14-rss-email-alert-ingestion-module-rollout plan: 03 subsystem: api,web tags: [nestjs, prisma, cheerio, imap, exchange-ews, aes-256-gcm, multi-tenant, nextjs] requires: - phase: 14-01 provides: "shared inbox/ module (InboxProvider.fetchMessages, ImapProvider/ExchangeInboxProvider)" - phase: 14-02 provides: "SourceType union widening pattern, normalizeBag() generic bag dispatch, pollGranularity='tick' mechanism" provides: - "EmailAlertAdapter: generic link/subject extraction + real per-tenant fan-out over active TenderEmailConfig rows" - "TenderEmailConfig Prisma model (per-tenant, encrypted creds) + TenderEmailConfigService admin CRUD" - "Tender.ownerTenantId nullable column (D-13) threaded write-side through dedup CREATE" - "D-13 read-side visibility filter in buildTenderWhere/listTenders/getTender" - "GET/PUT /modules/tender-radar/email-config routes + EmailAlertConfigForm UI" affects: [14-04, 14-05, tender-radar-verification] tech-stack: added: [] patterns: - "Per-tenant internal fan-out inside a single adapter (findMany({isActive:true}) across ALL tenants, deliberate cross-tenant platform-scheduler read, catch-per-tenant)" - "D-13 private-source visibility: nullable ownerTenantId on a platform-global model, OR[null,mine] read filter, CREATE-only write, fail-closed on unresolved requester" key-files: created: - apps/api/src/tenders/adapters/email-alert.adapter.ts - apps/api/src/tenders/adapters/email-alert.adapter.spec.ts - apps/api/src/tenders/tender-email-config.service.ts - apps/api/src/tenders/tender-email-config.service.spec.ts - apps/api/src/tenders/dto/tender-email-config.dto.ts - apps/api/prisma/migrations/20260723113917_tender_email_config_owner_tenant_id/migration.sql - apps/web/src/app/(portal)/modules/tender-radar/settings/components/EmailAlertConfigForm.tsx - apps/web/src/app/(portal)/modules/tender-radar/settings/components/EmailAlertConfigForm.test.tsx modified: - apps/api/prisma/schema.prisma - apps/api/src/tenders/tender.types.ts - apps/api/src/tenders/tender-normalizer.service.ts - apps/api/src/tenders/tender-normalizer.service.spec.ts - apps/api/src/tenders/tender-dedup.service.ts - apps/api/src/tenders/tender-dedup.service.spec.ts - apps/api/src/tenders/tender-query.builder.ts - apps/api/src/tenders/tender-query.builder.spec.ts - apps/api/src/tenders/tenders.module.ts - apps/api/src/tenders/tenders.controller.ts - apps/api/src/tenders/tenders.controller.spec.ts - apps/web/src/lib/tender-radar-api.ts - apps/web/src/app/(portal)/modules/tender-radar/settings/page.tsx key-decisions: - "D-13 read filter fails CLOSED: an unresolved requesting tenant (no auth context) sees ONLY global (ownerTenantId=null) tenders, never an accidental private-tender leak — this is a Rule 2 (missing critical functionality) addition beyond the plan's literal OR-clause spec, since the plan didn't specify the no-context case" - "getTender's D-13 guard throws the SAME NotFoundException as a missing id (not a distinct Forbidden), per plan's explicit no-detail-leak requirement" - "email-alert TenderSourcePollConfig seeded isActive=false (unlike rss's isActive=true) — no safe default mailbox exists to activate; framework-ready-activation-deferred stance, same as ai-netserver/cosinex-dtvp" - "EmailAlertConfigForm omits a Test-Connection button and an Abrufintervall field — no test-connection endpoint was built for this config, and poll cadence is governed by the shared platform-wide TenderSourcePollConfig row, not a per-tenant setting" patterns-established: - "TDD RED/GREEN split for Task 1: spec committed first against a moved-away adapter.ts (confirmed failing), then adapter.ts restored (confirmed passing) — two separate commits" requirements-completed: [] coverage: [] duration: ~70min completed: 2026-07-23 status: incomplete --- # Phase 14 Plan 03: E-Mail-Alert Ingestion (Tasks 1-3) Summary **Per-tenant encrypted portal-alert mailbox config + EmailAlertAdapter with real IMAP/EWS fan-out, tagging every extracted tender with a private `ownerTenantId` (D-13) that's enforced on both write (dedup CREATE-only) and read (OR[global,mine] filter, fail-closed) — Task 4's live-mailbox human-verify remains open.** ## Status: INCOMPLETE — Task 4 (human-verify) is the only remaining item This plan's Tasks 1-3 are fully implemented, tested, and committed. **Task 4 — a `checkpoint:human-verify` requiring a real portal-alert mailbox (including the Exchange/EWS live path) — was deliberately NOT executed**, per explicit scope instructions. No code changes are needed for Task 4; it is a pure operator verification step (see PLAN.md Task 4 for the exact steps) that must be performed once a real mailbox is available. ## Performance - **Tasks completed:** 3 of 4 (Task 4 open) - **Files modified:** 21 (8 created, 13 modified) - **Migration:** `20260723113917_tender_email_config_owner_tenant_id` (applied locally) ## Accomplishments - **Task 1 (TDD RED/GREEN):** `EmailAlertAdapter`'s three pure helpers — `extractCandidateLinks` (cheerio `a[href]` extraction with footer-noise filtering and a `MAX_LINKS_PER_EMAIL=5` cap, plaintext-regex fallback when no HTML body), `titleFromEmail` (subject → first body line → fallback), `sourceNoticeIdFor` (sha256 link hash) — with zero portal/sender-domain branching (D-04 restraint, verified by a source-scan test). `SourceType` gained `'email-alert'`; `TenderNormalizerService.normalize()` routes it through the existing `normalizeBag()` path unchanged. - **Task 2:** Real per-tenant fan-out in `EmailAlertAdapter.fetchTenders()` — one `prisma.tenderEmailConfig.findMany({where:{isActive:true}})` across ALL tenants (documented, deliberate cross-tenant platform-scheduler read, never `forTenant()`/RLS), per-config credential decryption via `CalendarCryptoService`, protocol-based provider routing (imap/exchange), catch-per-tenant so one broken mailbox never blocks others. New `TenderEmailConfig` Prisma model (per-tenant, `tenantId @unique`, encrypted creds) mirrors `DkvModuleConfig`. New `Tender.ownerTenantId` nullable column (D-13): `RawTenderRecord`/`NormalizedTenderFields` gained an optional `ownerTenantId`, threaded unchanged through `TenderNormalizerService.assemble()`, and written ONLY by `TenderDedupService`'s CREATE branch (never UPDATE — a tender later also seen on a public source is never retroactively hidden). - **Task 3:** D-13 read-side visibility — `buildTenderWhere` gained an optional `ownerTenantId` param producing `OR[{ownerTenantId:null},{ownerTenantId:}]`; `listTenders` resolves the requesting tenant leniently from the auth context (never throws — degrades to global-only when absent); `getTender` 404s (same exception as a missing id) when a tender's non-null `ownerTenantId` doesn't match the requester. New `GET`/`PUT /modules/tender-radar/email-config` routes (Roles ADMIN/SUPER_ADMIN, tenantId from auth context, declared before `@Get(':id')`). Web: `EmailAlertConfig` type + `fetchEmailConfig`/`saveEmailConfig` client functions, `EmailAlertConfigForm.tsx` (password blank-on-load, T-07-12) added as a new "E-Mail-Alerts" section on the tender-radar settings page. ## Task Commits 1. **Task 1 RED — failing spec** - `4d6fbb1` (test) — email-alert.adapter.spec.ts committed against a temporarily-removed adapter.ts, confirmed failing (module-not-found). 2. **Task 1 GREEN — implementation** - `8983231` (feat) — email-alert.adapter.ts + SourceType/normalizer dispatch; spec confirmed passing (14 tests). 3. **Task 2** - `1be6b15` (feat) — TenderEmailConfig model/migration + TenderEmailConfigService + ownerTenantId write-side (tender.types.ts, tender-normalizer.service.ts, tender-dedup.service.ts) + real EmailAlertAdapter fan-out + tenders.module.ts wiring. 4. **Task 3** - `48e1252` (feat) — tender-query.builder.ts D-13 filter, tenders.controller.ts email-config routes + visibility guards, web EmailAlertConfigForm + settings page section. **Plan metadata:** *(this SUMMARY's own commit — see final commit below)* _Task 1 followed the TDD RED→GREEN gate sequence exactly (`test(14-03): ...` then `feat(14-03): ...`); Tasks 2-3 are single `feat` commits per the plan's `type="auto"` (non-TDD) declaration._ ## Files Created/Modified - `apps/api/src/tenders/adapters/email-alert.adapter.ts` - Pure extraction helpers + real per-tenant fan-out `fetchTenders()` - `apps/api/src/tenders/adapters/email-alert.adapter.spec.ts` - 21 tests: pure functions + fan-out (routing, decrypt, catch-per-tenant, ownerTenantId tagging) - `apps/api/src/tenders/tender-email-config.service.ts` - Safe-select admin CRUD (encrypt-preserve-empty semantics, mirrors DkvService) - `apps/api/src/tenders/tender-email-config.service.spec.ts` - 6 tests: round-trip, safe-select, credential-preserve-on-partial-update - `apps/api/src/tenders/dto/tender-email-config.dto.ts` - class-validator DTO mirroring DkvConfigDto - `apps/api/prisma/schema.prisma` - `TenderEmailConfig` model + `Tender.ownerTenantId` nullable column + index - `apps/api/prisma/migrations/20260723113917_tender_email_config_owner_tenant_id/` - Applied locally against `172.19.0.2` - `apps/api/src/tenders/tender.types.ts` - `SourceType` +`'email-alert'`; `RawTenderRecord`/`NormalizedTenderFields` + optional `ownerTenantId` - `apps/api/src/tenders/tender-normalizer.service.ts` - `'email-alert'` case added to `normalizeBag()` dispatch; `assemble()` passes `ownerTenantId` through - `apps/api/src/tenders/tender-dedup.service.ts` - CREATE writes `ownerTenantId ?? null`; UPDATE branch untouched - `apps/api/src/tenders/tender-query.builder.ts` - `buildTenderWhere(dto, favIds, ownerTenantId)` — D-13 OR-clause, fail-closed default - `apps/api/src/tenders/tenders.module.ts` - Imports `CalendarModule`/`InboxModule`; registers `EmailAlertAdapter`/`TenderEmailConfigService`; seeds `email-alert` poll config (inactive, tick) - `apps/api/src/tenders/tenders.controller.ts` - `GET`/`PUT /email-config` routes; `listTenders`/`getTender` D-13 visibility - `apps/api/src/tenders/tenders.controller.spec.ts` - +10 tests: email-config wiring + D-13 list/detail visibility proofs - `apps/web/src/lib/tender-radar-api.ts` - `EmailAlertConfig` type + `fetchEmailConfig`/`saveEmailConfig` - `apps/web/src/app/(portal)/modules/tender-radar/settings/components/EmailAlertConfigForm.tsx` - Per-tenant mailbox config form - `apps/web/src/app/(portal)/modules/tender-radar/settings/components/EmailAlertConfigForm.test.tsx` - 5 tests: load/save/password-blank/protocol-switch - `apps/web/src/app/(portal)/modules/tender-radar/settings/page.tsx` - New "E-Mail-Alerts" section ## Decisions Made - D-13 read filter fails CLOSED when the requesting tenant can't be resolved (no auth context available) — only global tenders are ever returned to an unidentified requester. The plan specified the OR-clause shape for a resolved tenant but was silent on the no-context case; fail-closed is the only safe default for a security-critical visibility boundary (Rule 2). - `getTender`'s D-13 guard reuses the exact same `NotFoundException` as a genuinely-missing id — confirmed via test that a mismatched-tenant request gets the identical error, so no existence-leak. - `email-alert` poll config seeded `isActive: false` (unlike `rss`'s `isActive: true`) since there's no safe universal default mailbox — matches the "framework ready, activation deferred" stance already used for `ai-netserver`/`cosinex-dtvp`. - No Test-Connection button or Abrufintervall field in `EmailAlertConfigForm` — neither exists on the backend for this per-tenant config (poll cadence is the shared platform-wide `TenderSourcePollConfig` row); adding either would have been scope creep beyond what Task 3 specified. ## Deviations from Plan ### Auto-fixed Issues **1. [Rule 2 - Missing Critical] Added D-13 fail-closed default for unresolved tenant context** - **Found during:** Task 3 (`buildTenderWhere`/`listTenders` D-13 filter) - **Issue:** The plan specifies the `OR[{ownerTenantId:null},{ownerTenantId:}]` shape for a resolved requester, but doesn't address what `buildTenderWhere` should do when no tenant context is resolvable (e.g. a request the auth layer somehow let through without full context). - **Fix:** Added an explicit `else` branch: `{ ownerTenantId: null }` only — no private tenders are ever visible to an unidentified requester. - **Files modified:** `apps/api/src/tenders/tender-query.builder.ts` - **Verification:** `tender-query.builder.spec.ts` — "an unresolved ownerTenantId (no auth context) fails closed to global-only tenders"; `tenders.controller.spec.ts` — "GET / with no resolvable tenant context (no req) fails closed to only the global tender", "GET /:id 404s for an unauthenticated request (no req) to a private tender" - **Committed in:** `48e1252` (Task 3 commit) **2. [Rule 2 - Missing Critical] Added D-13 test coverage in `tender-dedup.service.spec.ts` and web `EmailAlertConfigForm.test.tsx`** - **Found during:** Task 2 (dedup ownerTenantId write-side) and Task 3 (form component) - **Issue:** The plan lists `tender-dedup.service.ts` and `EmailAlertConfigForm.tsx` as files to modify/create, but doesn't explicitly list their spec files among Task 2/3's ``. Given the acceptance criteria explicitly require proving "dedup create writes ownerTenantId; update branch does NOT reference ownerTenantId" and "EmailAlertConfigForm renders in settings", dedicated tests were added to make these claims verifiable rather than asserted only in prose. - **Files modified:** `apps/api/src/tenders/tender-dedup.service.spec.ts`, `apps/web/.../EmailAlertConfigForm.test.tsx` (new) - **Verification:** Both spec files pass (3 new dedup tests, 5 new form tests). - **Committed in:** `1be6b15` (dedup spec), `48e1252` (form spec) --- **Total deviations:** 2 auto-fixed (both Rule 2 — missing critical functionality: a security-relevant default and missing verification coverage for explicit acceptance criteria). **Impact on plan:** Both auto-fixes are narrowly scoped, additive, and directly required by either D-13's security intent or the plan's own stated acceptance criteria. No scope creep — no test-connection endpoint, no Abrufintervall field, and no i18n conversion were added (all explicitly out of scope per the plan/D-09/CONFIG-03). ## Issues Encountered None beyond the deviations above. Local Prisma migration applied cleanly against the running `tessera-ctl-db-1` container (`172.19.0.2`) on the first attempt; no Docker service restart was performed (per project convention). ## User Setup Required **Task 4 (live EWS/mailbox verification) requires manual operator action — see PLAN.md's Task 4 for exact steps.** In summary: an operator must configure a real portal-alert mailbox (try the Exchange/EWS path specifically) via Tessera → Ausschreibungs-Radar → Einstellungen → E-Mail-Alerts, activate the `email-alert` poll config, and confirm (a) a real alert email produces a tenant-private tender, (b) a different tenant does NOT see it, (c) the EWS body returns non-empty/non-garbled content. No environment variables or dashboard configuration beyond that form are required — the `CALENDAR_ENCRYPTION_KEY` env var (shared with DKV/SMTP) already exists in this environment. ## Next Phase Readiness - **NOT ready to close this plan.** Task 4 (`checkpoint:human-verify`, gate=`blocking`) is the sole remaining item — it requires a real mailbox the operator will provide separately, per the calling instructions. Do not mark INGEST-05's Exchange path production-ready, and do not advance `.planning/STATE.md`'s plan counter, until Task 4 is completed by a follow-up execution against this same PLAN.md. - All other acceptance criteria for Tasks 1-3 are met and automated-test-proven: full API suite 387/387 green (up from 343/343 at plan start), full web suite 144/144 green (up from 139/139), both `tsc --noEmit` clean. - Migration `20260723113917_tender_email_config_owner_tenant_id` is applied on the local dev DB; existing `Tender` rows retain `ownerTenantId = NULL` (verified via `\d "Tender"` — no backfill, no visibility change for any existing global tender). --- *Phase: 14-rss-email-alert-ingestion-module-rollout* *Completed: 2026-07-23 (Tasks 1-3 only — Task 4 OPEN)* ## Self-Check: PASSED All 8 created/key files confirmed present on disk; all 4 task commits (4d6fbb1, 8983231, 1be6b15, 48e1252) confirmed present in git log.