--- phase: 13-scraping-adapters-cross-source-dedup plan: 02 subsystem: backend tags: [nestjs, di, security-gate, adapter-pattern] requires: - phase: 13-scraping-adapters-cross-source-dedup plan: 01 provides: SourceType open union, TenderSourceAdapter interface (Phase 10 foundation) provides: - "TenderSourceAdapter.portals: readonly string[] — adapters declare all portals they serve" - "SourceRegistry (Injectable) mapping SourceType -> TenderSourceAdapter, with get()/activeAdapters()" - "DeniedPortalError + DENYLISTED_PORTALS = ['vergabe24', 'aumass'] — hard code-level registration refusal" affects: [13-03-dedup-resolver-fan-out, 13-04-netserver-adapter, 13-05-cosinex-adapter] tech-stack: added: [] patterns: - "Denylist enforced as a code exception at DI-registration time (SourceRegistry.register), not documentation-only — checked per-portal, so a mixed portals array is rejected wholesale if any single entry is denylisted" key-files: created: - apps/api/src/tenders/source-registry.ts - apps/api/src/tenders/source-registry.spec.ts modified: - apps/api/src/tenders/adapters/tender-source-adapter.interface.ts - apps/api/src/tenders/adapters/doe-opendata.adapter.ts key-decisions: - "portals is a readonly string[] on the adapter interface (not a single sourcePortal) exactly per 13-RESEARCH Pattern 1/2 — lets one NetServer adapter (Plan 13-04) serve tender24/lhs-vpbw/vergabe.landbw while the registry still gates each portal individually" - "Denylist check iterates ALL adapter.portals before registering any — one denylisted portal in a mixed array rejects the entire adapter, no partial registration (T-13-02-02)" - "Fake adapter stub used in the spec (no real scraping/HTTP dependency) — keeps the registry test suite dependency-free per D-01" patterns-established: - "SourceRegistry.get()/activeAdapters() is the seam Plan 13-03's pollDueSources fan-out will use instead of a hardwired single-adapter injection" requirements-completed: [INGEST-07] coverage: - id: D1 description: "Registering an adapter whose portals include 'vergabe24' or 'aumass' throws DeniedPortalError as a code-level exception (not documentation-only), including when the denylisted portal is mixed into an otherwise-legitimate portals array" requirement: INGEST-07 verification: - kind: unit ref: "apps/api/src/tenders/source-registry.spec.ts — 'throws DeniedPortalError when registering an adapter serving vergabe24', 'throws DeniedPortalError when registering an adapter serving aumass', 'rejects a mixed portals array wholesale when one entry is denylisted' (6/6 tests pass)" status: pass human_judgment: false - id: D2 description: "Legitimate adapters register and are retrievable via get(sourceType)/activeAdapters(); get() of an unregistered sourceType returns undefined without throwing" requirement: INGEST-07 verification: - kind: unit ref: "apps/api/src/tenders/source-registry.spec.ts — 'registers a legitimate adapter...', 'activeAdapters() returns all registered adapters', 'get() returns undefined for an unregistered sourceType (no throw)'" status: pass human_judgment: false - id: D3 description: "TenderSourceAdapter interface generalized with portals[]; DoeOpenDataAdapter declares portals = ['doe-opendata'] without behavior change; project-wide typecheck and full existing tenders test slice remain green" requirement: INGEST-07 verification: - kind: unit ref: "cd apps/api && npx tsc --noEmit -p tsconfig.json (clean)" status: pass - kind: unit ref: "cd apps/api && npx vitest run src/tenders (18 files, 177/177 tests pass, includes unchanged doe-opendata.adapter.spec.ts 6/6)" status: pass human_judgment: false duration: 20min completed: 2026-07-23 status: complete --- # Phase 13 Plan 02: SourceRegistry + Denylist Gate Summary **Injectable `SourceRegistry` that throws `DeniedPortalError` in code — not just documentation — the instant an adapter declares `vergabe24` or `aumass` among its `portals`, plus the generalized `TenderSourceAdapter.portals[]` contract that lets one adapter serve multiple portals.** ## Performance - **Duration:** 20 min - **Started:** 2026-07-23T08:41:00Z - **Completed:** 2026-07-23T08:44:30Z - **Tasks:** 2 - **Files modified:** 4 (2 created, 2 modified) ## Accomplishments - `TenderSourceAdapter` interface gained `readonly portals: readonly string[]`; `DoeOpenDataAdapter` declares `portals = ['doe-opendata'] as const` — additive, no behavior change, existing spec (6/6) still green. - New `SourceRegistry` (`@Injectable`): `register()` iterates every `adapter.portals` entry and throws `DeniedPortalError` (German AGB message) if any match `DENYLISTED_PORTALS = ['vergabe24', 'aumass']`; `get(sourceType)` returns `undefined` (not a throw) for unregistered types; `activeAdapters()` returns all registered adapters for Plan 13-03's fan-out scheduler. - **Denylist refusal proof (Erfolgskriterium 4):** `source-registry.spec.ts` — RED-first TDD — proves `register()` throws for `portals: ['vergabe24']`, throws for `portals: ['aumass']`, and rejects a **mixed** array `['tender24', 'aumass']` wholesale (one denylisted portal is enough — no partial registration, T-13-02-02). Legitimate registration + `get()`/`activeAdapters()` also covered. 6/6 tests pass. - `npx tsc --noEmit` clean and full `src/tenders` slice (18 files, 177/177 tests) green — no Prisma/scraping import introduced. ## Task Commits Each task was committed atomically: 1. **Task 1: Adapter interface generalized with portals[]** - `1b11ada` (feat) 2. **Task 2a: RED — failing SourceRegistry denylist-gate spec** - `78b17ef` (test) 3. **Task 2b: GREEN — SourceRegistry implementation** - `0fa9567` (feat) _TDD task (Task 2) produced two commits (test → feat) per protocol; no refactor commit was needed — the first implementation passed all 6 tests cleanly._ ## Files Created/Modified - `apps/api/src/tenders/adapters/tender-source-adapter.interface.ts` - added `portals: readonly string[]` field + doc explaining the denylist-gate hook - `apps/api/src/tenders/adapters/doe-opendata.adapter.ts` - declares `readonly portals = ['doe-opendata'] as const` - `apps/api/src/tenders/source-registry.ts` - `DENYLISTED_PORTALS`, `DeniedPortalError`, `@Injectable() SourceRegistry` (register/get/activeAdapters) - `apps/api/src/tenders/source-registry.spec.ts` - 6 unit tests (denylist refusal x2, mixed-array refusal, legitimate register/get, activeAdapters, unregistered get) ## Decisions Made - **`portals` as `readonly string[]` on the interface, exactly per 13-RESEARCH Pattern 1/2** — decouples "one adapter" from "one portal" so the future NetServer adapter (Plan 13-04) can serve `tender24`/`lhs-vpbw`/`vergabe.landbw` from one class while the registry still checks each portal individually. - **Denylist check iterates the full `portals` array before any mutation of the adapters Map** — a mixed portals array containing even one denylisted entry is rejected entirely; the registry never partially registers an adapter. - **Fake adapter stub in the spec** (no `doe-opendata.adapter.ts` reuse, no HTTP/Prisma) — keeps `source-registry.spec.ts` fully dependency-free, consistent with D-01 ("robust core first, independent of live scrapability"). ## Deviations from Plan None - plan executed exactly as written. No Rule 1-4 auto-fixes were needed; the plan's design (Pattern 1 + Pattern 2 from 13-RESEARCH.md) was followed verbatim. ## Issues Encountered None. ## User Setup Required None - no external service configuration, no migration, no environment changes. Pure TypeScript/DI addition. ## Next Phase Readiness - `SourceRegistry` is ready to be wired into the Nest module (Plan 13-03 Task 3: register `DoeOpenDataAdapter` at boot) and consumed by `pollDueSources`'s fan-out (`activeAdapters()`/`get()`), replacing the current hardwired single-adapter injection. - The `portals[]` contract is the exact shape Plan 13-04's config-driven `NetServerAdapter` (3 portals) and Plan 13-05's `CosinexAdapter` will implement — no further interface changes needed. - INGEST-07 is now structurally satisfied at the registry layer; Plan 13-04/13-05 adapters will never be able to accidentally register `vergabe24`/`aumass` since the gate lives in `register()`, not in adapter-author discipline. - No blockers. --- *Phase: 13-scraping-adapters-cross-source-dedup* *Completed: 2026-07-23* ## Self-Check: PASSED All created/modified files verified present on disk; all 3 task commit hashes verified in git log.