schalli/tessera-ctl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 14 Wiki Activity
Files
f7b5df4db830f3cafbf511de00941a8ce4258e20
tessera-ctl/.planning/phases/14-rss-email-alert-ingestion-module-rollout/14-03-SUMMARY.md
T

16 KiB
Raw Blame History

phase, plan, subsystem, tags, requires, provides, affects, tech-stack, key-files, key-decisions, patterns-established, requirements-completed, coverage, duration, completed, status
phase plan subsystem tags requires provides affects tech-stack key-files key-decisions patterns-established requirements-completed coverage duration completed status
14-rss-email-alert-ingestion-module-rollout 03 api,web
nestjs
prisma
cheerio
imap
exchange-ews
aes-256-gcm
multi-tenant
nextjs
phase provides
14-01 shared inbox/ module (InboxProvider.fetchMessages, ImapProvider/ExchangeInboxProvider)
phase provides
14-02 SourceType union widening pattern, normalizeBag() generic bag dispatch, pollGranularity='tick' mechanism
EmailAlertAdapter: generic link/subject extraction + real per-tenant fan-out over active TenderEmailConfig rows
TenderEmailConfig Prisma model (per-tenant, encrypted creds) + TenderEmailConfigService admin CRUD
Tender.ownerTenantId nullable column (D-13) threaded write-side through dedup CREATE
D-13 read-side visibility filter in buildTenderWhere/listTenders/getTender
GET/PUT /modules/tender-radar/email-config routes + EmailAlertConfigForm UI
14-04
14-05
tender-radar-verification
added patterns
Per-tenant internal fan-out inside a single adapter (findMany({isActive:true}) across ALL tenants, deliberate cross-tenant platform-scheduler read, catch-per-tenant)
D-13 private-source visibility: nullable ownerTenantId on a platform-global model, OR[null,mine] read filter, CREATE-only write, fail-closed on unresolved requester
created modified
apps/api/src/tenders/adapters/email-alert.adapter.ts
apps/api/src/tenders/adapters/email-alert.adapter.spec.ts
apps/api/src/tenders/tender-email-config.service.ts
apps/api/src/tenders/tender-email-config.service.spec.ts
apps/api/src/tenders/dto/tender-email-config.dto.ts
apps/api/prisma/migrations/20260723113917_tender_email_config_owner_tenant_id/migration.sql
apps/web/src/app/(portal)/modules/tender-radar/settings/components/EmailAlertConfigForm.tsx
apps/web/src/app/(portal)/modules/tender-radar/settings/components/EmailAlertConfigForm.test.tsx
apps/api/prisma/schema.prisma
apps/api/src/tenders/tender.types.ts
apps/api/src/tenders/tender-normalizer.service.ts
apps/api/src/tenders/tender-normalizer.service.spec.ts
apps/api/src/tenders/tender-dedup.service.ts
apps/api/src/tenders/tender-dedup.service.spec.ts
apps/api/src/tenders/tender-query.builder.ts
apps/api/src/tenders/tender-query.builder.spec.ts
apps/api/src/tenders/tenders.module.ts
apps/api/src/tenders/tenders.controller.ts
apps/api/src/tenders/tenders.controller.spec.ts
apps/web/src/lib/tender-radar-api.ts
apps/web/src/app/(portal)/modules/tender-radar/settings/page.tsx
D-13 read filter fails CLOSED: an unresolved requesting tenant (no auth context) sees ONLY global (ownerTenantId=null) tenders, never an accidental private-tender leak — this is a Rule 2 (missing critical functionality) addition beyond the plan's literal OR-clause spec, since the plan didn't specify the no-context case
getTender's D-13 guard throws the SAME NotFoundException as a missing id (not a distinct Forbidden), per plan's explicit no-detail-leak requirement
email-alert TenderSourcePollConfig seeded isActive=false (unlike rss's isActive=true) — no safe default mailbox exists to activate; framework-ready-activation-deferred stance, same as ai-netserver/cosinex-dtvp
EmailAlertConfigForm omits a Test-Connection button and an Abrufintervall field — no test-connection endpoint was built for this config, and poll cadence is governed by the shared platform-wide TenderSourcePollConfig row, not a per-tenant setting
TDD RED/GREEN split for Task 1: spec committed first against a moved-away adapter.ts (confirmed failing), then adapter.ts restored (confirmed passing) — two separate commits
~70min 2026-07-23 incomplete

Phase 14 Plan 03: E-Mail-Alert Ingestion (Tasks 1-3) Summary

Per-tenant encrypted portal-alert mailbox config + EmailAlertAdapter with real IMAP/EWS fan-out, tagging every extracted tender with a private ownerTenantId (D-13) that's enforced on both write (dedup CREATE-only) and read (OR[global,mine] filter, fail-closed) — Task 4's live-mailbox human-verify remains open.

Status: INCOMPLETE — Task 4 (human-verify) is the only remaining item

This plan's Tasks 1-3 are fully implemented, tested, and committed. Task 4 — a checkpoint:human-verify requiring a real portal-alert mailbox (including the Exchange/EWS live path) — was deliberately NOT executed, per explicit scope instructions. No code changes are needed for Task 4; it is a pure operator verification step (see PLAN.md Task 4 for the exact steps) that must be performed once a real mailbox is available.

Performance

  • Tasks completed: 3 of 4 (Task 4 open)
  • Files modified: 21 (8 created, 13 modified)
  • Migration: 20260723113917_tender_email_config_owner_tenant_id (applied locally)

Accomplishments

  • Task 1 (TDD RED/GREEN): EmailAlertAdapter's three pure helpers — extractCandidateLinks (cheerio a[href] extraction with footer-noise filtering and a MAX_LINKS_PER_EMAIL=5 cap, plaintext-regex fallback when no HTML body), titleFromEmail (subject → first body line → fallback), sourceNoticeIdFor (sha256 link hash) — with zero portal/sender-domain branching (D-04 restraint, verified by a source-scan test). SourceType gained 'email-alert'; TenderNormalizerService.normalize() routes it through the existing normalizeBag() path unchanged.
  • Task 2: Real per-tenant fan-out in EmailAlertAdapter.fetchTenders() — one prisma.tenderEmailConfig.findMany({where:{isActive:true}}) across ALL tenants (documented, deliberate cross-tenant platform-scheduler read, never forTenant()/RLS), per-config credential decryption via CalendarCryptoService, protocol-based provider routing (imap/exchange), catch-per-tenant so one broken mailbox never blocks others. New TenderEmailConfig Prisma model (per-tenant, tenantId @unique, encrypted creds) mirrors DkvModuleConfig. New Tender.ownerTenantId nullable column (D-13): RawTenderRecord/NormalizedTenderFields gained an optional ownerTenantId, threaded unchanged through TenderNormalizerService.assemble(), and written ONLY by TenderDedupService's CREATE branch (never UPDATE — a tender later also seen on a public source is never retroactively hidden).
  • Task 3: D-13 read-side visibility — buildTenderWhere gained an optional ownerTenantId param producing OR[{ownerTenantId:null},{ownerTenantId:<tenant>}]; listTenders resolves the requesting tenant leniently from the auth context (never throws — degrades to global-only when absent); getTender 404s (same exception as a missing id) when a tender's non-null ownerTenantId doesn't match the requester. New GET/PUT /modules/tender-radar/email-config routes (Roles ADMIN/SUPER_ADMIN, tenantId from auth context, declared before @Get(':id')). Web: EmailAlertConfig type + fetchEmailConfig/saveEmailConfig client functions, EmailAlertConfigForm.tsx (password blank-on-load, T-07-12) added as a new "E-Mail-Alerts" section on the tender-radar settings page.

Task Commits

  1. Task 1 RED — failing spec - 4d6fbb1 (test) — email-alert.adapter.spec.ts committed against a temporarily-removed adapter.ts, confirmed failing (module-not-found).
  2. Task 1 GREEN — implementation - 8983231 (feat) — email-alert.adapter.ts + SourceType/normalizer dispatch; spec confirmed passing (14 tests).
  3. Task 2 - 1be6b15 (feat) — TenderEmailConfig model/migration + TenderEmailConfigService + ownerTenantId write-side (tender.types.ts, tender-normalizer.service.ts, tender-dedup.service.ts) + real EmailAlertAdapter fan-out + tenders.module.ts wiring.
  4. Task 3 - 48e1252 (feat) — tender-query.builder.ts D-13 filter, tenders.controller.ts email-config routes + visibility guards, web EmailAlertConfigForm + settings page section.

Plan metadata: (this SUMMARY's own commit — see final commit below)

Task 1 followed the TDD RED→GREEN gate sequence exactly (test(14-03): ... then feat(14-03): ...); Tasks 2-3 are single feat commits per the plan's type="auto" (non-TDD) declaration.

Files Created/Modified

  • apps/api/src/tenders/adapters/email-alert.adapter.ts - Pure extraction helpers + real per-tenant fan-out fetchTenders()
  • apps/api/src/tenders/adapters/email-alert.adapter.spec.ts - 21 tests: pure functions + fan-out (routing, decrypt, catch-per-tenant, ownerTenantId tagging)
  • apps/api/src/tenders/tender-email-config.service.ts - Safe-select admin CRUD (encrypt-preserve-empty semantics, mirrors DkvService)
  • apps/api/src/tenders/tender-email-config.service.spec.ts - 6 tests: round-trip, safe-select, credential-preserve-on-partial-update
  • apps/api/src/tenders/dto/tender-email-config.dto.ts - class-validator DTO mirroring DkvConfigDto
  • apps/api/prisma/schema.prisma - TenderEmailConfig model + Tender.ownerTenantId nullable column + index
  • apps/api/prisma/migrations/20260723113917_tender_email_config_owner_tenant_id/ - Applied locally against 172.19.0.2
  • apps/api/src/tenders/tender.types.ts - SourceType +'email-alert'; RawTenderRecord/NormalizedTenderFields + optional ownerTenantId
  • apps/api/src/tenders/tender-normalizer.service.ts - 'email-alert' case added to normalizeBag() dispatch; assemble() passes ownerTenantId through
  • apps/api/src/tenders/tender-dedup.service.ts - CREATE writes ownerTenantId ?? null; UPDATE branch untouched
  • apps/api/src/tenders/tender-query.builder.ts - buildTenderWhere(dto, favIds, ownerTenantId) — D-13 OR-clause, fail-closed default
  • apps/api/src/tenders/tenders.module.ts - Imports CalendarModule/InboxModule; registers EmailAlertAdapter/TenderEmailConfigService; seeds email-alert poll config (inactive, tick)
  • apps/api/src/tenders/tenders.controller.ts - GET/PUT /email-config routes; listTenders/getTender D-13 visibility
  • apps/api/src/tenders/tenders.controller.spec.ts - +10 tests: email-config wiring + D-13 list/detail visibility proofs
  • apps/web/src/lib/tender-radar-api.ts - EmailAlertConfig type + fetchEmailConfig/saveEmailConfig
  • apps/web/src/app/(portal)/modules/tender-radar/settings/components/EmailAlertConfigForm.tsx - Per-tenant mailbox config form
  • apps/web/src/app/(portal)/modules/tender-radar/settings/components/EmailAlertConfigForm.test.tsx - 5 tests: load/save/password-blank/protocol-switch
  • apps/web/src/app/(portal)/modules/tender-radar/settings/page.tsx - New "E-Mail-Alerts" section

Decisions Made

  • D-13 read filter fails CLOSED when the requesting tenant can't be resolved (no auth context available) — only global tenders are ever returned to an unidentified requester. The plan specified the OR-clause shape for a resolved tenant but was silent on the no-context case; fail-closed is the only safe default for a security-critical visibility boundary (Rule 2).
  • getTender's D-13 guard reuses the exact same NotFoundException as a genuinely-missing id — confirmed via test that a mismatched-tenant request gets the identical error, so no existence-leak.
  • email-alert poll config seeded isActive: false (unlike rss's isActive: true) since there's no safe universal default mailbox — matches the "framework ready, activation deferred" stance already used for ai-netserver/cosinex-dtvp.
  • No Test-Connection button or Abrufintervall field in EmailAlertConfigForm — neither exists on the backend for this per-tenant config (poll cadence is the shared platform-wide TenderSourcePollConfig row); adding either would have been scope creep beyond what Task 3 specified.

Deviations from Plan

Auto-fixed Issues

1. [Rule 2 - Missing Critical] Added D-13 fail-closed default for unresolved tenant context

  • Found during: Task 3 (buildTenderWhere/listTenders D-13 filter)
  • Issue: The plan specifies the OR[{ownerTenantId:null},{ownerTenantId:<tenant>}] shape for a resolved requester, but doesn't address what buildTenderWhere should do when no tenant context is resolvable (e.g. a request the auth layer somehow let through without full context).
  • Fix: Added an explicit else branch: { ownerTenantId: null } only — no private tenders are ever visible to an unidentified requester.
  • Files modified: apps/api/src/tenders/tender-query.builder.ts
  • Verification: tender-query.builder.spec.ts — "an unresolved ownerTenantId (no auth context) fails closed to global-only tenders"; tenders.controller.spec.ts — "GET / with no resolvable tenant context (no req) fails closed to only the global tender", "GET /:id 404s for an unauthenticated request (no req) to a private tender"
  • Committed in: 48e1252 (Task 3 commit)

2. [Rule 2 - Missing Critical] Added D-13 test coverage in tender-dedup.service.spec.ts and web EmailAlertConfigForm.test.tsx

  • Found during: Task 2 (dedup ownerTenantId write-side) and Task 3 (form component)
  • Issue: The plan lists tender-dedup.service.ts and EmailAlertConfigForm.tsx as files to modify/create, but doesn't explicitly list their spec files among Task 2/3's <files>. Given the acceptance criteria explicitly require proving "dedup create writes ownerTenantId; update branch does NOT reference ownerTenantId" and "EmailAlertConfigForm renders in settings", dedicated tests were added to make these claims verifiable rather than asserted only in prose.
  • Files modified: apps/api/src/tenders/tender-dedup.service.spec.ts, apps/web/.../EmailAlertConfigForm.test.tsx (new)
  • Verification: Both spec files pass (3 new dedup tests, 5 new form tests).
  • Committed in: 1be6b15 (dedup spec), 48e1252 (form spec)

Total deviations: 2 auto-fixed (both Rule 2 — missing critical functionality: a security-relevant default and missing verification coverage for explicit acceptance criteria). Impact on plan: Both auto-fixes are narrowly scoped, additive, and directly required by either D-13's security intent or the plan's own stated acceptance criteria. No scope creep — no test-connection endpoint, no Abrufintervall field, and no i18n conversion were added (all explicitly out of scope per the plan/D-09/CONFIG-03).

Issues Encountered

None beyond the deviations above. Local Prisma migration applied cleanly against the running tessera-ctl-db-1 container (172.19.0.2) on the first attempt; no Docker service restart was performed (per project convention).

User Setup Required

Task 4 (live EWS/mailbox verification) requires manual operator action — see PLAN.md's Task 4 for exact steps. In summary: an operator must configure a real portal-alert mailbox (try the Exchange/EWS path specifically) via Tessera → Ausschreibungs-Radar → Einstellungen → E-Mail-Alerts, activate the email-alert poll config, and confirm (a) a real alert email produces a tenant-private tender, (b) a different tenant does NOT see it, (c) the EWS body returns non-empty/non-garbled content. No environment variables or dashboard configuration beyond that form are required — the CALENDAR_ENCRYPTION_KEY env var (shared with DKV/SMTP) already exists in this environment.

Next Phase Readiness

  • NOT ready to close this plan. Task 4 (checkpoint:human-verify, gate=blocking) is the sole remaining item — it requires a real mailbox the operator will provide separately, per the calling instructions. Do not mark INGEST-05's Exchange path production-ready, and do not advance .planning/STATE.md's plan counter, until Task 4 is completed by a follow-up execution against this same PLAN.md.
  • All other acceptance criteria for Tasks 1-3 are met and automated-test-proven: full API suite 387/387 green (up from 343/343 at plan start), full web suite 144/144 green (up from 139/139), both tsc --noEmit clean.
  • Migration 20260723113917_tender_email_config_owner_tenant_id is applied on the local dev DB; existing Tender rows retain ownerTenantId = NULL (verified via \d "Tender" — no backfill, no visibility change for any existing global tender).

Phase: 14-rss-email-alert-ingestion-module-rollout Completed: 2026-07-23 (Tasks 1-3 only — Task 4 OPEN)

Self-Check: PASSED

All 8 created/key files confirmed present on disk; all 4 task commits (4d6fbb1, 8983231, 1be6b15, 48e1252) confirmed present in git log.

Reference in New Issue View Git Blame Copy Permalink