docs(13-02): complete source-registry-denylist plan

.planning/REQUIREMENTS.md

@@ -16,7 +16,7 @@
 - [ ] **INGEST-04**: Das System importiert Ausschreibungen aus RSS-Feeds (subreport-elvis, service.bund.de).
 - [ ] **INGEST-05**: Das System liest Portal-Benachrichtigungs-E-Mails aus einem konfigurierten Postfach ein (nutzt bestehende DKV-Inbox-Infrastruktur) und extrahiert daraus Ausschreibungen.
 - [x] **INGEST-06**: Jede Quelle wird einmal zentral pro Zeitplan abgefragt (poll-once-fan-out-many), Intervall pro Quelle im Admin-Bereich konfigurierbar; das Ergebnis wird an alle passenden Mandanten-Suchprofile verteilt.
-- [ ] **INGEST-07**: vergabe24 und aumass sind als harte Denylist hinterlegt und können nicht als automatische Scraping-Quelle registriert werden (AGB-Verbot).
+- [x] **INGEST-07**: vergabe24 und aumass sind als harte Denylist hinterlegt und können nicht als automatische Scraping-Quelle registriert werden (AGB-Verbot).
 
 ### SCHEMA — Normalisierung & Deduplizierung
 
@@ -98,7 +98,7 @@
 | NOTIFY-04 | Phase 12 | Complete |
 | INGEST-02 | Phase 13 | Pending |
 | INGEST-03 | Phase 13 | Pending |
-| INGEST-07 | Phase 13 | Pending |
+| INGEST-07 | Phase 13 | Complete |
 | SCHEMA-03 | Phase 13 | Complete |
 | INGEST-04 | Phase 14 | Pending |
 | INGEST-05 | Phase 14 | Pending |

.planning/ROADMAP.md

@@ -439,12 +439,12 @@ Plans:
   3. A tender that appears via both DÖE and a scraping adapter shows up once in the results list (fuzzy fingerprint dedup on buyer+title+CPV+deadline+value), with links to all of its source portals -- dedup logic only activates once a second source is live
   4. Attempting to register vergabe24 or aumass as a poll source is refused by the system itself (adapter registry denylist enforced in code), not just documented as forbidden
 
-**Plans**: 1/6 plans executed
+**Plans**: 2/6 plans executed
 
 Plans:
 
 - [x] 13-01-PLAN.md — TenderSource-Schema + Backfill + NULL-tolerante Fingerprint-Fn + SourceType-Union (SCHEMA-03 Datenschicht)
-- [ ] 13-02-PLAN.md — SourceRegistry + harte Denylist-Gate (vergabe24/aumass) + Adapter-Interface-Generalisierung (INGEST-07)
+- [x] 13-02-PLAN.md — SourceRegistry + harte Denylist-Gate (vergabe24/aumass) + Adapter-Interface-Generalisierung (INGEST-07)
 - [ ] 13-03-PLAN.md — 3-Stufen-Dedup-Resolver + pollDueSources Fan-out (catch-per-source) + Modul-Wiring (SCHEMA-03)
 - [ ] 13-04-PLAN.md — NetServer-Adapter (config-getrieben, 3 Portale) + Package-Legitimacy-Checkpoint (INGEST-02)
 - [ ] 13-05-PLAN.md — cosinex/DTVP-Adapter (separat, best-effort) (INGEST-03)
@@ -486,5 +486,5 @@ Phases execute in numeric order: 1 -> 2 -> 3 -> 4 -> 5 -> 6 -> 7 -> 8 -> 9 -> 10
 | 10. Ausschreibungs-Radar Foundation & DÖE Ingestion | 6/6 | Complete   | 2026-07-21 |
 | 11. Filter Engine, Results UI & Saved Searches | 6/6 | In Progress|  |
 | 12. Tender Notifications | 4/4 | In Progress|  |
-| 13. Scraping Adapters & Cross-Source Deduplication | 1/6 | In Progress|  |
+| 13. Scraping Adapters & Cross-Source Deduplication | 2/6 | In Progress|  |
 | 14. RSS, Email-Alert Ingestion & Module Rollout | 0/TBD | Not started | - |

.planning/STATE.md

@@ -5,15 +5,15 @@ milestone_name: Ausschreibungs-Radar
 current_phase: 13
 current_phase_name: scraping-adapters-cross-source-dedup
 status: executing
-stopped_at: Completed 13-01-PLAN.md
+stopped_at: Completed 13-02-PLAN.md
-last_updated: "2026-07-23T06:40:39.176Z"
+last_updated: "2026-07-23T06:44:34.483Z"
 last_activity: 2026-07-23
 last_activity_desc: Phase 13 execution started
 progress:
   total_phases: 13
   completed_phases: 11
   total_plans: 62
-  completed_plans: 56
+  completed_plans: 57
 ---
 
 # Project State
@@ -28,11 +28,11 @@ See: .planning/PROJECT.md (updated 2026-07-17)
 ## Current Position
 
 Phase: 13 (scraping-adapters-cross-source-dedup) — EXECUTING
-Plan: 2 of 6
+Plan: 3 of 6
 Status: Ready to execute
 Last activity: 2026-07-23 — Phase 13 execution started
 
-Progress: [█████████░] 90%
+Progress: [█████████░] 92%
 
 ## Performance Metrics
 
@@ -91,6 +91,7 @@ Progress: [█████████░] 90%
 | Phase 12 P03 | 15min | 2 tasks | 3 files |
 | Phase 12 P04 | 12min | 3 tasks | 10 files |
 | Phase 13 P01 | 35min | 3 tasks | 6 files |
+| Phase 13 P02 | 20min | 2 tasks | 4 files |
 
 ## Accumulated Context
 
@@ -194,6 +195,7 @@ Recent decisions affecting current work:
 - [Phase ?]: Digest-interval selector inline in settings/page.tsx (already 'use client'); instantAlert toggle uses plain checkbox for chip-based SavedSearchBar UI
 - [Phase ?]: SCHEMA-03 fingerprint: title+buyer dominant, CPV division, value-bucket, deadline-day, sha256; dedupKey untouched, fingerprint additive
 - [Phase ?]: One-time TS backfill scripts run via compiled dist/ output (not raw .ts execution) to keep tsc --noEmit clean
+- [Phase ?]: SourceRegistry.register() throws DeniedPortalError for any portal in DENYLISTED_PORTALS (vergabe24, aumass), enforced at DI-registration time not just documented (INGEST-07)
 
 ### Pending Todos
 
@@ -231,7 +233,7 @@ Items acknowledged and carried forward from previous milestone close:
 
 ## Session Continuity
 
-Last session: 2026-07-23T06:40:39.162Z
+Last session: 2026-07-23T06:44:34.470Z
-Stopped at: Completed 13-01-PLAN.md
+Stopped at: Completed 13-02-PLAN.md
 Resume file: None
 Last activity: 2026-07-14 - Built LDAP per-user exclude/denylist filter (9d1323f), migration applied on live DB, verified via Playwright: sync deactivated 4 excluded service accounts (administrator/krbtgt/guest/dns-ldap), 2 real LDAP users stay active, 0 wrongly created

.planning/phases/13-scraping-adapters-cross-source-dedup/13-02-SUMMARY.md

@@ -0,0 +1,137 @@
+---
+phase: 13-scraping-adapters-cross-source-dedup
+plan: 02
+subsystem: backend
+tags: [nestjs, di, security-gate, adapter-pattern]
+
+requires:
+  - phase: 13-scraping-adapters-cross-source-dedup
+    plan: 01
+    provides: SourceType open union, TenderSourceAdapter interface (Phase 10 foundation)
+provides:
+  - "TenderSourceAdapter.portals: readonly string[] — adapters declare all portals they serve"
+  - "SourceRegistry (Injectable) mapping SourceType -> TenderSourceAdapter, with get()/activeAdapters()"
+  - "DeniedPortalError + DENYLISTED_PORTALS = ['vergabe24', 'aumass'] — hard code-level registration refusal"
+affects: [13-03-dedup-resolver-fan-out, 13-04-netserver-adapter, 13-05-cosinex-adapter]
+
+tech-stack:
+  added: []
+  patterns:
+    - "Denylist enforced as a code exception at DI-registration time (SourceRegistry.register), not documentation-only — checked per-portal, so a mixed portals array is rejected wholesale if any single entry is denylisted"
+
+key-files:
+  created:
+    - apps/api/src/tenders/source-registry.ts
+    - apps/api/src/tenders/source-registry.spec.ts
+  modified:
+    - apps/api/src/tenders/adapters/tender-source-adapter.interface.ts
+    - apps/api/src/tenders/adapters/doe-opendata.adapter.ts
+
+key-decisions:
+  - "portals is a readonly string[] on the adapter interface (not a single sourcePortal) exactly per 13-RESEARCH Pattern 1/2 — lets one NetServer adapter (Plan 13-04) serve tender24/lhs-vpbw/vergabe.landbw while the registry still gates each portal individually"
+  - "Denylist check iterates ALL adapter.portals before registering any — one denylisted portal in a mixed array rejects the entire adapter, no partial registration (T-13-02-02)"
+  - "Fake adapter stub used in the spec (no real scraping/HTTP dependency) — keeps the registry test suite dependency-free per D-01"
+
+patterns-established:
+  - "SourceRegistry.get()/activeAdapters() is the seam Plan 13-03's pollDueSources fan-out will use instead of a hardwired single-adapter injection"
+
+requirements-completed: [INGEST-07]
+
+coverage:
+  - id: D1
+    description: "Registering an adapter whose portals include 'vergabe24' or 'aumass' throws DeniedPortalError as a code-level exception (not documentation-only), including when the denylisted portal is mixed into an otherwise-legitimate portals array"
+    requirement: INGEST-07
+    verification:
+      - kind: unit
+        ref: "apps/api/src/tenders/source-registry.spec.ts — 'throws DeniedPortalError when registering an adapter serving vergabe24', 'throws DeniedPortalError when registering an adapter serving aumass', 'rejects a mixed portals array wholesale when one entry is denylisted' (6/6 tests pass)"
+        status: pass
+    human_judgment: false
+  - id: D2
+    description: "Legitimate adapters register and are retrievable via get(sourceType)/activeAdapters(); get() of an unregistered sourceType returns undefined without throwing"
+    requirement: INGEST-07
+    verification:
+      - kind: unit
+        ref: "apps/api/src/tenders/source-registry.spec.ts — 'registers a legitimate adapter...', 'activeAdapters() returns all registered adapters', 'get() returns undefined for an unregistered sourceType (no throw)'"
+        status: pass
+    human_judgment: false
+  - id: D3
+    description: "TenderSourceAdapter interface generalized with portals[]; DoeOpenDataAdapter declares portals = ['doe-opendata'] without behavior change; project-wide typecheck and full existing tenders test slice remain green"
+    requirement: INGEST-07
+    verification:
+      - kind: unit
+        ref: "cd apps/api && npx tsc --noEmit -p tsconfig.json (clean)"
+        status: pass
+      - kind: unit
+        ref: "cd apps/api && npx vitest run src/tenders (18 files, 177/177 tests pass, includes unchanged doe-opendata.adapter.spec.ts 6/6)"
+        status: pass
+    human_judgment: false
+
+duration: 20min
+completed: 2026-07-23
+status: complete
+---
+
+# Phase 13 Plan 02: SourceRegistry + Denylist Gate Summary
+
+**Injectable `SourceRegistry` that throws `DeniedPortalError` in code — not just documentation — the instant an adapter declares `vergabe24` or `aumass` among its `portals`, plus the generalized `TenderSourceAdapter.portals[]` contract that lets one adapter serve multiple portals.**
+
+## Performance
+
+- **Duration:** 20 min
+- **Started:** 2026-07-23T08:41:00Z
+- **Completed:** 2026-07-23T08:44:30Z
+- **Tasks:** 2
+- **Files modified:** 4 (2 created, 2 modified)
+
+## Accomplishments
+- `TenderSourceAdapter` interface gained `readonly portals: readonly string[]`; `DoeOpenDataAdapter` declares `portals = ['doe-opendata'] as const` — additive, no behavior change, existing spec (6/6) still green.
+- New `SourceRegistry` (`@Injectable`): `register()` iterates every `adapter.portals` entry and throws `DeniedPortalError` (German AGB message) if any match `DENYLISTED_PORTALS = ['vergabe24', 'aumass']`; `get(sourceType)` returns `undefined` (not a throw) for unregistered types; `activeAdapters()` returns all registered adapters for Plan 13-03's fan-out scheduler.
+- **Denylist refusal proof (Erfolgskriterium 4):** `source-registry.spec.ts` — RED-first TDD — proves `register()` throws for `portals: ['vergabe24']`, throws for `portals: ['aumass']`, and rejects a **mixed** array `['tender24', 'aumass']` wholesale (one denylisted portal is enough — no partial registration, T-13-02-02). Legitimate registration + `get()`/`activeAdapters()` also covered. 6/6 tests pass.
+- `npx tsc --noEmit` clean and full `src/tenders` slice (18 files, 177/177 tests) green — no Prisma/scraping import introduced.
+
+## Task Commits
+
+Each task was committed atomically:
+
+1. **Task 1: Adapter interface generalized with portals[]** - `1b11ada` (feat)
+2. **Task 2a: RED — failing SourceRegistry denylist-gate spec** - `78b17ef` (test)
+3. **Task 2b: GREEN — SourceRegistry implementation** - `0fa9567` (feat)
+
+_TDD task (Task 2) produced two commits (test → feat) per protocol; no refactor commit was needed — the first implementation passed all 6 tests cleanly._
+
+## Files Created/Modified
+- `apps/api/src/tenders/adapters/tender-source-adapter.interface.ts` - added `portals: readonly string[]` field + doc explaining the denylist-gate hook
+- `apps/api/src/tenders/adapters/doe-opendata.adapter.ts` - declares `readonly portals = ['doe-opendata'] as const`
+- `apps/api/src/tenders/source-registry.ts` - `DENYLISTED_PORTALS`, `DeniedPortalError`, `@Injectable() SourceRegistry` (register/get/activeAdapters)
+- `apps/api/src/tenders/source-registry.spec.ts` - 6 unit tests (denylist refusal x2, mixed-array refusal, legitimate register/get, activeAdapters, unregistered get)
+
+## Decisions Made
+- **`portals` as `readonly string[]` on the interface, exactly per 13-RESEARCH Pattern 1/2** — decouples "one adapter" from "one portal" so the future NetServer adapter (Plan 13-04) can serve `tender24`/`lhs-vpbw`/`vergabe.landbw` from one class while the registry still checks each portal individually.
+- **Denylist check iterates the full `portals` array before any mutation of the adapters Map** — a mixed portals array containing even one denylisted entry is rejected entirely; the registry never partially registers an adapter.
+- **Fake adapter stub in the spec** (no `doe-opendata.adapter.ts` reuse, no HTTP/Prisma) — keeps `source-registry.spec.ts` fully dependency-free, consistent with D-01 ("robust core first, independent of live scrapability").
+
+## Deviations from Plan
+
+None - plan executed exactly as written. No Rule 1-4 auto-fixes were needed; the plan's design (Pattern 1 + Pattern 2 from 13-RESEARCH.md) was followed verbatim.
+
+## Issues Encountered
+
+None.
+
+## User Setup Required
+
+None - no external service configuration, no migration, no environment changes. Pure TypeScript/DI addition.
+
+## Next Phase Readiness
+- `SourceRegistry` is ready to be wired into the Nest module (Plan 13-03 Task 3: register `DoeOpenDataAdapter` at boot) and consumed by `pollDueSources`'s fan-out (`activeAdapters()`/`get()`), replacing the current hardwired single-adapter injection.
+- The `portals[]` contract is the exact shape Plan 13-04's config-driven `NetServerAdapter` (3 portals) and Plan 13-05's `CosinexAdapter` will implement — no further interface changes needed.
+- INGEST-07 is now structurally satisfied at the registry layer; Plan 13-04/13-05 adapters will never be able to accidentally register `vergabe24`/`aumass` since the gate lives in `register()`, not in adapter-author discipline.
+- No blockers.
+
+---
+*Phase: 13-scraping-adapters-cross-source-dedup*
+*Completed: 2026-07-23*
+
+## Self-Check: PASSED
+
+All created/modified files verified present on disk; all 3 task commit hashes verified in git log.