tessera-ctl/.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-04-SUMMARY.md

phase, plan, subsystem, tags, requires, provides, affects, tech-stack, key-files, key-decisions, requirements-completed, coverage, duration, completed, status

phase	plan	subsystem	tags	requires	provides	affects	tech-stack	key-files	key-decisions	requirements-completed	coverage	duration	completed	status
10-ausschreibungs-radar-foundation-d-e-ingestion	04	api	scheduler ingestion multi-tenant-safety cron scheduler-registry tdd poll-once-fan-out-many	phase provides 10-01 (foundation & dependencies) Tender / TenderSourcePollConfig Prisma models (dedupKey @unique, sourceType @unique) phase provides 10-02 (marketplace registration) TendersModule skeleton, singleton doe-opendata poll config seeded on boot phase provides 10-03 (adapter & normalizer) DoeOpenDataAdapter.fetchTenders(dayCursor), TenderNormalizerService.normalize()	TenderIngestionService — pollDueSources() (day-cursor gate, SCHEMA-02 upsert change-detection, catch-up loop) + pruneExpiredTenders() (D-05 retention) TenderSchedulerService — single global cron job 'tender-doe-poll' (poll-once-fan-out-many, INGEST-06) Automated proof of the phase's headline acceptance criterion (Success Criteria 4 & 5) 2nd-tenant activation triggers zero additional DÖE calls/cron jobs/Tender rows	10-05 controller/DTOs (consumes setInterval()/stopJob() on admin config save) phase 11 saved searches/filter UI (consumes the ingested Tender delta)	added patterns Poll-once-fan-out-many scheduler: single named cron job, no tenant parameter anywhere in the class — the DKV findFirst()/activeTenantId single-tenant framing is explicitly NOT reused (Pitfall D) Day-cursor gate lives in the ingestion service, not the scheduler — cron-tick frequency (D-04, admin-configurable) is decoupled from actual-fetch frequency (day-granularity, Pitfall A) Singleton platform-wide config loaded via findUnique on a fixed slug (sourceType: 'doe-opendata'), never findFirst — self-documenting against future copy-paste into a per-tenant source prisma.tender.upsert({ where: { dedupKey } }) as the SCHEMA-02 change-detection seam — identical notice never duplicates, changed contentHash updates in place D-05 retention as two-phase updateMany/deleteMany with deadlineAt:{lt:...} filters — null-deadline rows are structurally excluded (SQL NULL comparisons are always false), not just by convention In-memory prisma-shaped fake (Maps) for service-level tests — matches this repo's established convention (ldap.service.spec.ts) rather than a live DB connection	created modified apps/api/src/tenders/tender-ingestion.service.ts apps/api/src/tenders/tender-ingestion.service.spec.ts apps/api/src/tenders/tender-scheduler.service.ts apps/api/src/tenders/tender-scheduler.service.spec.ts apps/api/src/tenders/tenders.module.ts	Comment wording avoiding literal 'forTenant'/'activeTenantId'/'findFirst' tokens in the two new service files — both grep-gate assertions (task-level and spec-level) check for the literal string, and an explanatory comment mentioning the anti-pattern by name would itself trip the assertion (same class of issue documented as a Rule 1 fix in Plan 10-01) TenderIngestionService.politeDelayMs is a protected, test-overridable field (not a hardcoded sleep) so the catch-up-loop unit test doesn't sleep for real while production still gets the ~1.5s politeness delay between successive day-fetches Task 3's two-tenant test drives the REAL (unmocked) ModuleRegistryService against a fake prisma, not a stand-in activation function — this exercises the genuine activateForTenant() call path, making the 'zero additional calls/jobs/rows' assertion a true integration proof rather than a tautology pruneExpiredTenders() is called once per successful pollDueSources() tick (i.e., only when at least one day was actually fetched) — not on the day-cursor no-op branch, so a no-op tick stays a true no-op with zero DB writes beyond the read	SCHEMA-02 INGEST-06	id description requirement verification human_judgment D1 A changed DÖE notice (same dedupKey, new contentHash) updates the existing Tender row instead of creating a duplicate (SCHEMA-02) SCHEMA-02 kind ref status unit tender-ingestion.service.spec.ts — 'updates the existing row in place when the same dedupKey reappears with a changed contentHash' — asserts store size stays 1, contentHash/deadlineAt reflect the new version pass kind ref status unit tender-ingestion.service.spec.ts — 'inserts a fresh notice, and does NOT duplicate when the identical notice ... reappears' — asserts store size stays 1 across 2 catch-up-day fetches of the identical record pass false id description requirement verification human_judgment D2 DÖE is polled once on a shared global schedule regardless of tenant count — poll-once-fan-out-many, never per-tenant (INGEST-06) INGEST-06 kind ref status integration tender-scheduler.service.spec.ts — 2nd-tenant activation asserts addCronJob still called exactly once, pollDueSources never called by activation, tender.count() stays 0 pass kind ref status automated_ui grep -c activeTenantId src/tenders/tender-scheduler.service.ts == 0; grep -c findFirst == 0; grep -c findUnique >= 1 pass false id description verification human_judgment D3 The poll tick is day-cursor gated: no upstream fetch when dayCursor >= today Europe/Berlin (D-01 from-now, no historical backfill) kind ref status unit tender-ingestion.service.spec.ts — 'makes NO adapter call and returns when the day-cursor is not strictly before Berlin-today' + 'does nothing when ... config is not active' pass false id description verification human_judgment D4 Tenders past their deadline are marked expired and pruned after 90 days; deadline-less rows are never auto-expired (D-05) kind ref status unit tender-ingestion.service.spec.ts — pruneExpiredTenders test: active+past-deadline -> expired, expired+91d-old -> deleted, expired+30d-old -> retained, null-deadline -> untouched, active+future-deadline -> untouched pass false id description verification human_judgment D5 Service uses the plain PrismaService — no tenant RLS extension on Tender/TenderSourcePollConfig (D-03, T-10-09) kind ref status automated_ui tender-ingestion.service.spec.ts source-inspection test: grep-equivalent expect(source).not.toMatch(/forTenant/) on tender-ingestion.service.ts pass false	~25min	2026-07-21	complete

Phase 10 Plan 04: Ingestion Orchestration & Multi-Tenant-Safe Scheduler Summary

TenderIngestionService (day-cursor gate, SCHEMA-02 upsert-based change detection, D-05 retention) and TenderSchedulerService (single global cron job, zero tenant dimension) built test-first, with an automated two-tenant integration test proving the phase's headline acceptance criterion: activating the module for a 2nd tenant triggers zero additional DÖE polls, cron jobs, or Tender rows.

Performance

• Duration: ~25 min
• Started: 2026-07-21
• Completed: 2026-07-21
• Tasks: 3 (Task 1 TDD RED->GREEN, Task 2 auto, Task 3 TDD spec-only — passed immediately against Task 2's already-correct implementation)
• Files modified: 5 (4 created, 1 modified)

Accomplishments

• TenderIngestionService.pollDueSources(): loads the singleton doe-opendata config via findUnique on the fixed slug, applies the day-cursor gate (nextDayToFetch(), no-op when nothing new — Pitfall A), catches up across missed days with a politeness delay between fetches, and upserts each normalized notice by dedupKey — the SCHEMA-02 change-detection seam (identical notice never duplicates; changed contentHash updates the row in place).
• pruneExpiredTenders() (D-05): marks past-deadline active rows expired, deletes expired rows older than 90 days, and structurally never touches deadlineAt IS NULL rows.
• TenderSchedulerService: reuses DkvSchedulerService's CronJob/SchedulerRegistry mechanics verbatim, but drops the per-tenant activeTenantId framing entirely — exactly one cron job (tender-doe-poll) for the whole platform, setInterval() takes no tenant argument.
• Two-tenant safety integration test (tender-scheduler.service.spec.ts) drives the real ModuleRegistryService.activateForTenant() against a fake prisma and asserts, after a 2nd tenant activates the module: zero additional addCronJob calls, zero additional pollDueSources() invocations, zero additional Tender rows — the absence of tenant-scaled behavior, not the presence of correct per-tenant iteration.
• Full API test suite green (69/69), tsc --noEmit clean, all grep gates (activeTenantId, forTenant, findFirst) return 0 in both new service files.

Task Commits

Each task committed atomically, RED before GREEN where TDD applies:

1. Task 1a: Failing spec — day-cursor gate, SCHEMA-02, D-05 (RED) — f9e52ab (test)
2. Task 1b: TenderIngestionService implementation (GREEN) — 6fb734b (feat)
3. Task 2: TenderSchedulerService — single global cron (poll-once-fan-out-many) — 0ba7410 (feat)
4. Task 3: Two-tenant safety integration test (Success Criteria 4 & 5) — 444c68b (test)

Plan metadata: see final docs(10-04) commit.

TDD Gate Compliance

• Task 1 (tdd="true"): genuine RED->GREEN. f9e52ab (test(10-04): ...) failed at module-resolution time (tender-ingestion.service.ts did not exist yet), confirmed via pnpm exec vitest run -- tender-ingestion before any implementation existed. 6fb734b (feat(10-04): ...) brought all 7 tests to green.
• Task 3 (tdd="true"): the spec (444c68b, test(10-04): ...) passed on first run, immediately after being written — this is expected, not an unexpected-pass violation of the RED-GREEN gate. Task 2's feat commit already implemented the poll-once-fan-out-many invariant correctly (no tenant dimension exists anywhere in TenderSchedulerService to begin with); Task 3's test locks in and regression-proofs that already-correct architecture via a genuine two-tenant activation exercise, rather than driving new production code. The plan's own task sequencing (implement scheduler in Task 2, then prove the property in Task 3) anticipates this — it is not a full-plan type: tdd gate (this plan's frontmatter is type: execute with per-task tdd="true" flags), so the strict "investigate a passing RED" rule does not apply at the plan level here.
• No REFACTOR commit was needed for either task — both implementations passed cleanly, no post-green cleanup required.

Files Created/Modified

• apps/api/src/tenders/tender-ingestion.service.ts — pollDueSources() (day-cursor gate, catch-up loop, upsert change-detection) + pruneExpiredTenders() (D-05); exports nextDayToFetch() for direct testing
• apps/api/src/tenders/tender-ingestion.service.spec.ts — 7 tests: 2 day-cursor gate, 2 SCHEMA-02 change-detection, 1 catch-up cursor advance, 1 D-05 retention (5 sub-assertions), 1 no-forTenant source check
• apps/api/src/tenders/tender-scheduler.service.ts — single global cron job (tender-doe-poll), setInterval()/stopJob(), onModuleInit() via findUnique on fixed slug
• apps/api/src/tenders/tender-scheduler.service.spec.ts — 3 tests: two-tenant safety integration proof, single-job/no-tenant-param check, source-level anti-pattern grep checks
• apps/api/src/tenders/tenders.module.ts — TenderIngestionService + TenderSchedulerService added to providers

Decisions Made

• Comment wording avoids the literal forTenant/activeTenantId/findFirst tokens in both new service files — the grep-gate assertions (task-level bash grep and spec-level source-inspection tests) check for the literal string, and an explanatory comment naming the anti-pattern directly would itself trip the assertion. This is the same class of issue Plan 10-01 documented as a Rule 1 fix; caught and fixed inline during this plan (see Deviations).
• politeDelayMs is a protected, test-overridable instance field, not a hardcoded setTimeout call — production gets the ~1.5s politeness delay between successive catch-up day-fetches (RESEARCH Open Question 1), while the catch-up-loop unit test zeroes it out ((service as any).politeDelayMs = 0) to run instantly.
• Task 3's test drives the real, unmocked ModuleRegistryService against a fake prisma rather than stubbing activateForTenant() directly — this makes the "zero additional calls/jobs/rows on 2nd activation" assertion a genuine integration proof of the actual activation call path, not a tautological check against a hand-written stand-in.
• pruneExpiredTenders() runs once per successful tick (only when the day-cursor loop actually processed at least one day), not on the no-op branch — keeps a true no-op tick free of any DB write beyond the initial config read.
• In-memory prisma-shaped fakes (Maps), not a live DB connection, for both new spec files — matches this repo's established test convention (ldap.service.spec.ts) rather than introducing a new DB-integration-test pattern; the local Docker Postgres stack was available (per environment context) but no existing vitest precedent uses it, so the repo-consistent approach was chosen.

Deviations from Plan

Auto-fixed Issues

1. [Rule 1 - Bug] Explanatory comment in tender-ingestion.service.ts contained the literal forTenant token

• Found during: Task 1 GREEN verification (first pnpm exec vitest run -- tender-ingestion after implementation)
• Issue: The class-level doc comment explained the multi-tenant-safety rationale using the phrase "Never wraps ... in forTenant()" — this literal substring made the spec's expect(source).not.toMatch(/forTenant/) assertion fail, a false positive against the actual D-03 invariant (the code itself never calls forTenant(); only the comment mentioned it by name).
• Fix: Reworded the comment to describe the same rationale without the literal token ("uses the plain, non-tenant-scoped PrismaService... Never wraps ... queries in the tenant RLS extension").
• Files modified: apps/api/src/tenders/tender-ingestion.service.ts
• Verification: grep -c forTenant → 0; tsc --noEmit exit 0; full spec re-run green.
• Committed in: 6fb734b (Task 1 GREEN commit, folded in before commit since caught during the same verification pass)

2. [Rule 1 - Bug] Explanatory comments in tender-scheduler.service.ts contained the literal activeTenantId and findFirst tokens

• Found during: Task 2 verification (grep -c activeTenantId returned 1, not the required 0)
• Issue: The class-level doc comment listed the anti-patterns being deliberately avoided using their literal names ("No activeTenantId field", "never findFirst()") — both tripped their respective grep-gate assertions the same way as Deviation 1, and would additionally have broken Task 3's planned source-inspection test if left uncorrected.
• Fix: Reworded both mentions to describe the same design intent without the literal tokens ("No per-tenant 'active tenant id' instance field at all", "a fixed-slug lookup, not an unfiltered/ordered 'first match' query").
• Files modified: apps/api/src/tenders/tender-scheduler.service.ts
• Verification: grep -c activeTenantId → 0; grep -c findFirst → 0; grep -c findUnique → 2; tsc --noEmit exit 0.
• Committed in: 0ba7410 (Task 2 commit, caught and fixed before commit)

---

Total deviations: 2 auto-fixed (2 Bugs, both comment-wording false-positives against grep-gate assertions — no production logic was ever incorrect) Impact on plan: Zero scope creep, zero behavior change. Both fixes are pure documentation-wording corrections needed to satisfy the plan's own literal-string acceptance criteria.

Issues Encountered

None beyond the two documented deviations above. The local Docker Postgres stack (per the environment context) was not needed for this plan's tests — both new spec files use in-memory prisma-shaped fakes, matching this repo's established test convention, so no live-DB dependency was introduced.

User Setup Required

None — no external service configuration required. The local Docker stack (tessera-ctl-api-1/-db-1) was left running unmodified; restarting/rebuilding it to pick up these code changes (so the scheduler actually starts polling against the live oeffentlichevergabe.de API) is left to the user per project convention.

Next Phase Readiness

• TenderIngestionService and TenderSchedulerService are both implemented, tested, and registered in TendersModule.providers — the full ingestion pipeline (fetch -> normalize -> upsert -> schedule) is wired end-to-end for the first time this phase.
• Plan 05 (controller/DTOs) can now wire POST /tenders/source-config to call tenderScheduler.setInterval()/stopJob() after an admin updates pollIntervalMin/isActive — both methods already exist with the exact signature the PATTERNS.md controller sketch expects.
• No open threat-model items from this plan carry forward unmitigated: T-10-09 (RLS-exempt global-table access) and T-10-10 (per-tenant scheduler scaling) are both mitigated and proven by automated tests; T-10-11 (URL tampering via the day-cursor) and T-10-12 (catch-up-loop DoS against DÖE) are both structurally addressed (internally-computed cursor, politeness delay) with no admin-facing input path introduced.
• No blockers for Plan 10-05.

Self-Check: PASSED

• FOUND: apps/api/src/tenders/tender-ingestion.service.ts
• FOUND: apps/api/src/tenders/tender-ingestion.service.spec.ts
• FOUND: apps/api/src/tenders/tender-scheduler.service.ts
• FOUND: apps/api/src/tenders/tender-scheduler.service.spec.ts
• FOUND: apps/api/src/tenders/tenders.module.ts
• FOUND commit: f9e52ab
• FOUND commit: 6fb734b
• FOUND commit: 0ba7410
• FOUND commit: 444c68b

---

Phase: 10-ausschreibungs-radar-foundation-d-e-ingestion Completed: 2026-07-21