docs(13-02): complete source-registry-denylist plan
This commit is contained in:
@@ -0,0 +1,137 @@
|
||||
---
|
||||
phase: 13-scraping-adapters-cross-source-dedup
|
||||
plan: 02
|
||||
subsystem: backend
|
||||
tags: [nestjs, di, security-gate, adapter-pattern]
|
||||
|
||||
requires:
|
||||
- phase: 13-scraping-adapters-cross-source-dedup
|
||||
plan: 01
|
||||
provides: SourceType open union, TenderSourceAdapter interface (Phase 10 foundation)
|
||||
provides:
|
||||
- "TenderSourceAdapter.portals: readonly string[] — adapters declare all portals they serve"
|
||||
- "SourceRegistry (Injectable) mapping SourceType -> TenderSourceAdapter, with get()/activeAdapters()"
|
||||
- "DeniedPortalError + DENYLISTED_PORTALS = ['vergabe24', 'aumass'] — hard code-level registration refusal"
|
||||
affects: [13-03-dedup-resolver-fan-out, 13-04-netserver-adapter, 13-05-cosinex-adapter]
|
||||
|
||||
tech-stack:
|
||||
added: []
|
||||
patterns:
|
||||
- "Denylist enforced as a code exception at DI-registration time (SourceRegistry.register), not documentation-only — checked per-portal, so a mixed portals array is rejected wholesale if any single entry is denylisted"
|
||||
|
||||
key-files:
|
||||
created:
|
||||
- apps/api/src/tenders/source-registry.ts
|
||||
- apps/api/src/tenders/source-registry.spec.ts
|
||||
modified:
|
||||
- apps/api/src/tenders/adapters/tender-source-adapter.interface.ts
|
||||
- apps/api/src/tenders/adapters/doe-opendata.adapter.ts
|
||||
|
||||
key-decisions:
|
||||
- "portals is a readonly string[] on the adapter interface (not a single sourcePortal) exactly per 13-RESEARCH Pattern 1/2 — lets one NetServer adapter (Plan 13-04) serve tender24/lhs-vpbw/vergabe.landbw while the registry still gates each portal individually"
|
||||
- "Denylist check iterates ALL adapter.portals before registering any — one denylisted portal in a mixed array rejects the entire adapter, no partial registration (T-13-02-02)"
|
||||
- "Fake adapter stub used in the spec (no real scraping/HTTP dependency) — keeps the registry test suite dependency-free per D-01"
|
||||
|
||||
patterns-established:
|
||||
- "SourceRegistry.get()/activeAdapters() is the seam Plan 13-03's pollDueSources fan-out will use instead of a hardwired single-adapter injection"
|
||||
|
||||
requirements-completed: [INGEST-07]
|
||||
|
||||
coverage:
|
||||
- id: D1
|
||||
description: "Registering an adapter whose portals include 'vergabe24' or 'aumass' throws DeniedPortalError as a code-level exception (not documentation-only), including when the denylisted portal is mixed into an otherwise-legitimate portals array"
|
||||
requirement: INGEST-07
|
||||
verification:
|
||||
- kind: unit
|
||||
ref: "apps/api/src/tenders/source-registry.spec.ts — 'throws DeniedPortalError when registering an adapter serving vergabe24', 'throws DeniedPortalError when registering an adapter serving aumass', 'rejects a mixed portals array wholesale when one entry is denylisted' (6/6 tests pass)"
|
||||
status: pass
|
||||
human_judgment: false
|
||||
- id: D2
|
||||
description: "Legitimate adapters register and are retrievable via get(sourceType)/activeAdapters(); get() of an unregistered sourceType returns undefined without throwing"
|
||||
requirement: INGEST-07
|
||||
verification:
|
||||
- kind: unit
|
||||
ref: "apps/api/src/tenders/source-registry.spec.ts — 'registers a legitimate adapter...', 'activeAdapters() returns all registered adapters', 'get() returns undefined for an unregistered sourceType (no throw)'"
|
||||
status: pass
|
||||
human_judgment: false
|
||||
- id: D3
|
||||
description: "TenderSourceAdapter interface generalized with portals[]; DoeOpenDataAdapter declares portals = ['doe-opendata'] without behavior change; project-wide typecheck and full existing tenders test slice remain green"
|
||||
requirement: INGEST-07
|
||||
verification:
|
||||
- kind: unit
|
||||
ref: "cd apps/api && npx tsc --noEmit -p tsconfig.json (clean)"
|
||||
status: pass
|
||||
- kind: unit
|
||||
ref: "cd apps/api && npx vitest run src/tenders (18 files, 177/177 tests pass, includes unchanged doe-opendata.adapter.spec.ts 6/6)"
|
||||
status: pass
|
||||
human_judgment: false
|
||||
|
||||
duration: 20min
|
||||
completed: 2026-07-23
|
||||
status: complete
|
||||
---
|
||||
|
||||
# Phase 13 Plan 02: SourceRegistry + Denylist Gate Summary
|
||||
|
||||
**Injectable `SourceRegistry` that throws `DeniedPortalError` in code — not just documentation — the instant an adapter declares `vergabe24` or `aumass` among its `portals`, plus the generalized `TenderSourceAdapter.portals[]` contract that lets one adapter serve multiple portals.**
|
||||
|
||||
## Performance
|
||||
|
||||
- **Duration:** 20 min
|
||||
- **Started:** 2026-07-23T08:41:00Z
|
||||
- **Completed:** 2026-07-23T08:44:30Z
|
||||
- **Tasks:** 2
|
||||
- **Files modified:** 4 (2 created, 2 modified)
|
||||
|
||||
## Accomplishments
|
||||
- `TenderSourceAdapter` interface gained `readonly portals: readonly string[]`; `DoeOpenDataAdapter` declares `portals = ['doe-opendata'] as const` — additive, no behavior change, existing spec (6/6) still green.
|
||||
- New `SourceRegistry` (`@Injectable`): `register()` iterates every `adapter.portals` entry and throws `DeniedPortalError` (German AGB message) if any match `DENYLISTED_PORTALS = ['vergabe24', 'aumass']`; `get(sourceType)` returns `undefined` (not a throw) for unregistered types; `activeAdapters()` returns all registered adapters for Plan 13-03's fan-out scheduler.
|
||||
- **Denylist refusal proof (Erfolgskriterium 4):** `source-registry.spec.ts` — RED-first TDD — proves `register()` throws for `portals: ['vergabe24']`, throws for `portals: ['aumass']`, and rejects a **mixed** array `['tender24', 'aumass']` wholesale (one denylisted portal is enough — no partial registration, T-13-02-02). Legitimate registration + `get()`/`activeAdapters()` also covered. 6/6 tests pass.
|
||||
- `npx tsc --noEmit` clean and full `src/tenders` slice (18 files, 177/177 tests) green — no Prisma/scraping import introduced.
|
||||
|
||||
## Task Commits
|
||||
|
||||
Each task was committed atomically:
|
||||
|
||||
1. **Task 1: Adapter interface generalized with portals[]** - `1b11ada` (feat)
|
||||
2. **Task 2a: RED — failing SourceRegistry denylist-gate spec** - `78b17ef` (test)
|
||||
3. **Task 2b: GREEN — SourceRegistry implementation** - `0fa9567` (feat)
|
||||
|
||||
_TDD task (Task 2) produced two commits (test → feat) per protocol; no refactor commit was needed — the first implementation passed all 6 tests cleanly._
|
||||
|
||||
## Files Created/Modified
|
||||
- `apps/api/src/tenders/adapters/tender-source-adapter.interface.ts` - added `portals: readonly string[]` field + doc explaining the denylist-gate hook
|
||||
- `apps/api/src/tenders/adapters/doe-opendata.adapter.ts` - declares `readonly portals = ['doe-opendata'] as const`
|
||||
- `apps/api/src/tenders/source-registry.ts` - `DENYLISTED_PORTALS`, `DeniedPortalError`, `@Injectable() SourceRegistry` (register/get/activeAdapters)
|
||||
- `apps/api/src/tenders/source-registry.spec.ts` - 6 unit tests (denylist refusal x2, mixed-array refusal, legitimate register/get, activeAdapters, unregistered get)
|
||||
|
||||
## Decisions Made
|
||||
- **`portals` as `readonly string[]` on the interface, exactly per 13-RESEARCH Pattern 1/2** — decouples "one adapter" from "one portal" so the future NetServer adapter (Plan 13-04) can serve `tender24`/`lhs-vpbw`/`vergabe.landbw` from one class while the registry still checks each portal individually.
|
||||
- **Denylist check iterates the full `portals` array before any mutation of the adapters Map** — a mixed portals array containing even one denylisted entry is rejected entirely; the registry never partially registers an adapter.
|
||||
- **Fake adapter stub in the spec** (no `doe-opendata.adapter.ts` reuse, no HTTP/Prisma) — keeps `source-registry.spec.ts` fully dependency-free, consistent with D-01 ("robust core first, independent of live scrapability").
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
None - plan executed exactly as written. No Rule 1-4 auto-fixes were needed; the plan's design (Pattern 1 + Pattern 2 from 13-RESEARCH.md) was followed verbatim.
|
||||
|
||||
## Issues Encountered
|
||||
|
||||
None.
|
||||
|
||||
## User Setup Required
|
||||
|
||||
None - no external service configuration, no migration, no environment changes. Pure TypeScript/DI addition.
|
||||
|
||||
## Next Phase Readiness
|
||||
- `SourceRegistry` is ready to be wired into the Nest module (Plan 13-03 Task 3: register `DoeOpenDataAdapter` at boot) and consumed by `pollDueSources`'s fan-out (`activeAdapters()`/`get()`), replacing the current hardwired single-adapter injection.
|
||||
- The `portals[]` contract is the exact shape Plan 13-04's config-driven `NetServerAdapter` (3 portals) and Plan 13-05's `CosinexAdapter` will implement — no further interface changes needed.
|
||||
- INGEST-07 is now structurally satisfied at the registry layer; Plan 13-04/13-05 adapters will never be able to accidentally register `vergabe24`/`aumass` since the gate lives in `register()`, not in adapter-author discipline.
|
||||
- No blockers.
|
||||
|
||||
---
|
||||
*Phase: 13-scraping-adapters-cross-source-dedup*
|
||||
*Completed: 2026-07-23*
|
||||
|
||||
## Self-Check: PASSED
|
||||
|
||||
All created/modified files verified present on disk; all 3 task commit hashes verified in git log.
|
||||
Reference in New Issue
Block a user