docs(13-04): complete NetServer adapter plan
Tessera CI/CD / Lint & Type Check (push) Successful in 48s
Tessera CI/CD / Tests (push) Successful in 45s
Tessera CI/CD / Build & Publish Images (push) Successful in 3m43s

This commit is contained in:
2026-07-23 09:16:04 +02:00
parent 88572f5694
commit 775ed153b7
4 changed files with 192 additions and 12 deletions
+2 -2
View File
@@ -11,7 +11,7 @@
### INGEST — Datenquellen ### INGEST — Datenquellen
- [x] **INGEST-01**: Das System ruft Ausschreibungen zeitgesteuert über die DÖE OpenData-API (oeffentlichevergabe.de, eForms/OCDS, auth-frei) ab. - [x] **INGEST-01**: Das System ruft Ausschreibungen zeitgesteuert über die DÖE OpenData-API (oeffentlichevergabe.de, eForms/OCDS, auth-frei) ab.
- [ ] **INGEST-02**: Das System importiert Ausschreibungen von Administration-Intelligence-NetServer-Portalen (lhs-vpbw, tender24, vergabe.landbw) über einen konfigurierbaren Adapter. - [x] **INGEST-02**: Das System importiert Ausschreibungen von Administration-Intelligence-NetServer-Portalen (lhs-vpbw, tender24, vergabe.landbw) über einen konfigurierbaren Adapter.
- [ ] **INGEST-03**: Das System importiert Ausschreibungen vom cosinex-Vergabemarktplatz (DTVP) über einen Adapter. - [ ] **INGEST-03**: Das System importiert Ausschreibungen vom cosinex-Vergabemarktplatz (DTVP) über einen Adapter.
- [ ] **INGEST-04**: Das System importiert Ausschreibungen aus RSS-Feeds (subreport-elvis, service.bund.de). - [ ] **INGEST-04**: Das System importiert Ausschreibungen aus RSS-Feeds (subreport-elvis, service.bund.de).
- [ ] **INGEST-05**: Das System liest Portal-Benachrichtigungs-E-Mails aus einem konfigurierten Postfach ein (nutzt bestehende DKV-Inbox-Infrastruktur) und extrahiert daraus Ausschreibungen. - [ ] **INGEST-05**: Das System liest Portal-Benachrichtigungs-E-Mails aus einem konfigurierten Postfach ein (nutzt bestehende DKV-Inbox-Infrastruktur) und extrahiert daraus Ausschreibungen.
@@ -96,7 +96,7 @@
| NOTIFY-02 | Phase 12 | Complete | | NOTIFY-02 | Phase 12 | Complete |
| NOTIFY-03 | Phase 12 | Complete | | NOTIFY-03 | Phase 12 | Complete |
| NOTIFY-04 | Phase 12 | Complete | | NOTIFY-04 | Phase 12 | Complete |
| INGEST-02 | Phase 13 | Pending | | INGEST-02 | Phase 13 | Complete |
| INGEST-03 | Phase 13 | Pending | | INGEST-03 | Phase 13 | Pending |
| INGEST-07 | Phase 13 | Complete | | INGEST-07 | Phase 13 | Complete |
| SCHEMA-03 | Phase 13 | Complete | | SCHEMA-03 | Phase 13 | Complete |
+3 -3
View File
@@ -439,14 +439,14 @@ Plans:
3. A tender that appears via both DÖE and a scraping adapter shows up once in the results list (fuzzy fingerprint dedup on buyer+title+CPV+deadline+value), with links to all of its source portals -- dedup logic only activates once a second source is live 3. A tender that appears via both DÖE and a scraping adapter shows up once in the results list (fuzzy fingerprint dedup on buyer+title+CPV+deadline+value), with links to all of its source portals -- dedup logic only activates once a second source is live
4. Attempting to register vergabe24 or aumass as a poll source is refused by the system itself (adapter registry denylist enforced in code), not just documented as forbidden 4. Attempting to register vergabe24 or aumass as a poll source is refused by the system itself (adapter registry denylist enforced in code), not just documented as forbidden
**Plans**: 4/6 plans executed **Plans**: 5/6 plans executed
Plans: Plans:
- [x] 13-01-PLAN.md — TenderSource-Schema + Backfill + NULL-tolerante Fingerprint-Fn + SourceType-Union (SCHEMA-03 Datenschicht) - [x] 13-01-PLAN.md — TenderSource-Schema + Backfill + NULL-tolerante Fingerprint-Fn + SourceType-Union (SCHEMA-03 Datenschicht)
- [x] 13-02-PLAN.md — SourceRegistry + harte Denylist-Gate (vergabe24/aumass) + Adapter-Interface-Generalisierung (INGEST-07) - [x] 13-02-PLAN.md — SourceRegistry + harte Denylist-Gate (vergabe24/aumass) + Adapter-Interface-Generalisierung (INGEST-07)
- [x] 13-03-PLAN.md — 3-Stufen-Dedup-Resolver + pollDueSources Fan-out (catch-per-source) + Modul-Wiring (SCHEMA-03) - [x] 13-03-PLAN.md — 3-Stufen-Dedup-Resolver + pollDueSources Fan-out (catch-per-source) + Modul-Wiring (SCHEMA-03)
- [ ] 13-04-PLAN.md — NetServer-Adapter (config-getrieben, 3 Portale) + Package-Legitimacy-Checkpoint (INGEST-02) - [x] 13-04-PLAN.md — NetServer-Adapter (config-getrieben, 3 Portale) + Package-Legitimacy-Checkpoint (INGEST-02)
- [ ] 13-05-PLAN.md — cosinex/DTVP-Adapter (separat, best-effort) (INGEST-03) - [ ] 13-05-PLAN.md — cosinex/DTVP-Adapter (separat, best-effort) (INGEST-03)
- [x] 13-06-PLAN.md — Read-Endpoint include sources[] + TenderDetail Multi-Source-Links (SCHEMA-03 Anzeige) - [x] 13-06-PLAN.md — Read-Endpoint include sources[] + TenderDetail Multi-Source-Links (SCHEMA-03 Anzeige)
@@ -486,5 +486,5 @@ Phases execute in numeric order: 1 -> 2 -> 3 -> 4 -> 5 -> 6 -> 7 -> 8 -> 9 -> 10
| 10. Ausschreibungs-Radar Foundation & DÖE Ingestion | 6/6 | Complete | 2026-07-21 | | 10. Ausschreibungs-Radar Foundation & DÖE Ingestion | 6/6 | Complete | 2026-07-21 |
| 11. Filter Engine, Results UI & Saved Searches | 6/6 | In Progress| | | 11. Filter Engine, Results UI & Saved Searches | 6/6 | In Progress| |
| 12. Tender Notifications | 4/4 | In Progress| | | 12. Tender Notifications | 4/4 | In Progress| |
| 13. Scraping Adapters & Cross-Source Deduplication | 4/6 | In Progress| | | 13. Scraping Adapters & Cross-Source Deduplication | 5/6 | In Progress| |
| 14. RSS, Email-Alert Ingestion & Module Rollout | 0/TBD | Not started | - | | 14. RSS, Email-Alert Ingestion & Module Rollout | 0/TBD | Not started | - |
+9 -7
View File
@@ -5,15 +5,15 @@ milestone_name: Ausschreibungs-Radar
current_phase: 13 current_phase: 13
current_phase_name: scraping-adapters-cross-source-dedup current_phase_name: scraping-adapters-cross-source-dedup
status: executing status: executing
stopped_at: Completed 13-06-PLAN.md stopped_at: Completed 13-04-PLAN.md
last_updated: "2026-07-23T06:57:20.973Z" last_updated: "2026-07-23T07:15:59.330Z"
last_activity: 2026-07-23 last_activity: 2026-07-23
last_activity_desc: Phase 13 execution started last_activity_desc: Phase 13 execution started
progress: progress:
total_phases: 13 total_phases: 13
completed_phases: 11 completed_phases: 11
total_plans: 62 total_plans: 62
completed_plans: 59 completed_plans: 60
--- ---
# Project State # Project State
@@ -28,11 +28,11 @@ See: .planning/PROJECT.md (updated 2026-07-17)
## Current Position ## Current Position
Phase: 13 (scraping-adapters-cross-source-dedup) — EXECUTING Phase: 13 (scraping-adapters-cross-source-dedup) — EXECUTING
Plan: 5 of 6 Plan: 6 of 6
Status: Ready to execute Status: Ready to execute
Last activity: 2026-07-23 — Phase 13 execution started Last activity: 2026-07-23 — Phase 13 execution started
Progress: [██████████] 95% Progress: [██████████] 97%
## Performance Metrics ## Performance Metrics
@@ -94,6 +94,7 @@ Progress: [██████████] 95%
| Phase 13 P02 | 20min | 2 tasks | 4 files | | Phase 13 P02 | 20min | 2 tasks | 4 files |
| Phase 13 P03 | 45min | 3 tasks | 5 files | | Phase 13 P03 | 45min | 3 tasks | 5 files |
| Phase 13 P06 | 15min | 2 tasks | 5 files | | Phase 13 P06 | 15min | 2 tasks | 5 files |
| Phase 13 P04 | 55min | 3 tasks | 6 files |
## Accumulated Context ## Accumulated Context
@@ -201,6 +202,7 @@ Recent decisions affecting current work:
- [Phase ?]: TenderDedupService tier-2 (source:noticeId) match is unconditional (not gated by dedupActive) — idempotent same-source re-poll must work even with a single active source - [Phase ?]: TenderDedupService tier-2 (source:noticeId) match is unconditional (not gated by dedupActive) — idempotent same-source re-poll must work even with a single active source
- [Phase ?]: Fingerprint always computed on tender.create regardless of dedupActive, so DÖE-only tenders become fingerprint-matchable the instant a 2nd source activates without further backfill - [Phase ?]: Fingerprint always computed on tender.create regardless of dedupActive, so DÖE-only tenders become fingerprint-matchable the instant a 2nd source activates without further backfill
- [Phase ?]: 13-06: portalLabel()-Map bleibt lokal in TenderDetail.tsx (nicht geteilt) — einziger Consumer bisher; Fallback auf sourceUrl-Block bei fehlendem/leerem sources[]. - [Phase ?]: 13-06: portalLabel()-Map bleibt lokal in TenderDetail.tsx (nicht geteilt) — einziger Consumer bisher; Fallback auf sourceUrl-Block bei fehlendem/leerem sources[].
- [Phase ?]: 13-04: cheerio (nicht node-html-parser) fuer NetServer-HTML-Parsing gewaehlt — Legitimacy-Gate flaggte node-html-parser als 'too-new' (Fehlmessung: Latest-Version-Datum statt Package-Alter); cheerio bestand das Gate sauber, Human-Approval eingeholt
### Pending Todos ### Pending Todos
@@ -238,7 +240,7 @@ Items acknowledged and carried forward from previous milestone close:
## Session Continuity ## Session Continuity
Last session: 2026-07-23T06:57:20.960Z Last session: 2026-07-23T07:15:59.317Z
Stopped at: Completed 13-06-PLAN.md Stopped at: Completed 13-04-PLAN.md
Resume file: None Resume file: None
Last activity: 2026-07-14 - Built LDAP per-user exclude/denylist filter (9d1323f), migration applied on live DB, verified via Playwright: sync deactivated 4 excluded service accounts (administrator/krbtgt/guest/dns-ldap), 2 real LDAP users stay active, 0 wrongly created Last activity: 2026-07-14 - Built LDAP per-user exclude/denylist filter (9d1323f), migration applied on live DB, verified via Playwright: sync deactivated 4 excluded service accounts (administrator/krbtgt/guest/dns-ldap), 2 real LDAP users stay active, 0 wrongly created
@@ -0,0 +1,178 @@
---
phase: 13-scraping-adapters-cross-source-dedup
plan: 04
subsystem: backend
tags: [cheerio, html-scraping, adapter-pattern, netserver, package-legitimacy-gate]
requires:
- phase: 13-scraping-adapters-cross-source-dedup
plan: 01
provides: SourceType open union (includes 'ai-netserver'), RawTenderRecord contract
- phase: 13-scraping-adapters-cross-source-dedup
plan: 02
provides: TenderSourceAdapter.portals[], SourceRegistry with DeniedPortalError denylist gate
- phase: 13-scraping-adapters-cross-source-dedup
plan: 03
provides: pollDueSources fan-out + TenderDedupService, TendersModule onModuleInit registration hook
provides:
- "NetServerAdapter — ONE config-driven adapter serving all 3 AI-AG Vergabe@Net portals (tender24, lhs-vpbw, vergabe.landbw)"
- "cheerio (^1.2.0) as the project's HTML-parsing dependency (package-legitimacy-gated, human-approved)"
- "Live-captured netserver-search.html fixture for stable, network-free adapter tests"
- "NetServerAdapter registered as a TendersModule provider + at DI boot via SourceRegistry.register(); ai-netserver TenderSourcePollConfig seeded (isActive: false)"
affects: [13-05-cosinex-adapter, phase-14-admin-source-activation-ui]
tech-stack:
added:
- "cheerio ^1.2.0 (HTML parsing, package-legitimacy-gated: human approved over node-html-parser after gate flagged the latter's most-recent-version-publish-date as a 'too-new' false positive)"
patterns:
- "Config-driven multi-portal adapter: one class + a hardcoded portal-to-baseUrl Record, iterated per fetchTenders() call, each portal wrapped in its own try/catch so one blocked/broken portal never prevents the other two from being polled in the same tick"
- "Per-row try/catch inside cheerio .each() — a single malformed table row is skipped (logger.warn) rather than aborting the whole portal's parse; cheerio.load() itself is also wrapped, so a totally unparsable page returns [] instead of throwing"
- "RawTenderRecord.ocdsPayload used as a generic field bag (title/buyerName/procedureType/legalFramework/deadlineAt) for a non-OCDS source — deliberately NOT reshaping the raw eForms/OCDS-shaped RawTenderRecord interface; the shared TenderNormalizerService does not yet understand this shape (out of scope for this plan, see Known Stubs)"
key-files:
created:
- apps/api/src/tenders/adapters/netserver.adapter.ts
- apps/api/src/tenders/adapters/netserver.adapter.spec.ts
- apps/api/src/tenders/__fixtures__/netserver-search.html
modified:
- apps/api/package.json
- pnpm-lock.yaml
- apps/api/src/tenders/tenders.module.ts
key-decisions:
- "cheerio chosen over node-html-parser at the blocking-human package-legitimacy checkpoint — the automated gate flagged node-html-parser 'SUS'/'too-new' purely because it measures the latest-*version*-publish date (2026-07-06), not true package age (first published 2017-06-14, 129 versions, v9.0.0); cheerio passed the gate cleanly (OK, 27M weekly downloads, published since 2011) so the human approver picked the unambiguous option"
- "sourceUrl falls back to the portal's search-results URL, NOT a per-notice deep link — live inspection of the captured HTML found NO <a href> per result row; navigation to a notice's detail page is client-side-JS-only (data-oid + an external click handler script), with no statically-derivable detail URL anywhere in the search response. This resolves 13-RESEARCH.md Open Question 2 with a documented limitation rather than a guessed/unverified URL pattern (avoiding a fabricated, unverifiable servlet path)."
- "sourceNoticeId = the row's data-oid attribute (e.g. '54321-NetTender-19f800ce3b1-...') — stable and unique per notice, confirmed unique across all rows in the live fixture"
- "dayCursor parameter is accepted (interface conformance) but not used as a query filter — the NetServer search servlet has no documented incremental/date-range parameter (unlike DÖE's day-batch export); first-page-only fetch is accepted as sufficient for the MVP proof (13-RESEARCH.md Open Question 2 recommendation), pagination is out of scope for this plan"
- "NetServer's '24:00' deadline convention (meaning midnight at the END of the stated day) is normalized to 00:00 of the next day, rather than left as an invalid Date"
- "TenderSourcePollConfig row for 'ai-netserver' is seeded with isActive: false — this plan builds, registers, and tests the adapter; it does not force-activate live polling of the 3 real portals (D-02: activation is a later admin/seed decision, Phase 14 UI)"
requirements-completed: [INGEST-02]
coverage:
- id: D1
description: "ONE config-driven NetServerAdapter declares portals=['tender24','lhs-vpbw','vergabe.landbw'] and a hardcoded per-portal baseUrl map; fetchTenders() fetches and parses all 3 portals independently, sourcePortal set correctly per parsed row"
requirement: INGEST-02
verification:
- kind: unit
ref: "apps/api/src/tenders/adapters/netserver.adapter.spec.ts — 'declares sourceType ai-netserver and the 3 AI-AG NetServer portals', 'fetches all 3 portals and aggregates their parsed records' (14/14 tests pass)"
status: pass
human_judgment: false
- id: D2
description: "Adapter parses the live-captured NetServer <table> results into RawTenderRecord[] with correct sourceNoticeId (data-oid), umlaut/&amp;-entity-decoded buyer names, and deadline mapped to a Date or null (never throwing on empty deadlines)"
requirement: INGEST-02
verification:
- kind: unit
ref: "apps/api/src/tenders/adapters/netserver.adapter.spec.ts — 'decodes an umlaut + &amp;-entity buyer name correctly', 'uses the row data-oid attribute as sourceNoticeId', 'maps a populated deadline row to a Date and an empty-deadline row to null'"
status: pass
human_judgment: false
- id: D3
description: "Parse-Totalausfall (empty HTML, no matching rows, malformed row, portal fetch throwing, portal responding non-2xx) returns [] instead of throwing, per portal and in aggregate (D-01 error tolerance)"
requirement: INGEST-02
verification:
- kind: unit
ref: "apps/api/src/tenders/adapters/netserver.adapter.spec.ts — 'returns [] for empty HTML', 'returns [] for HTML with no matching result rows', 'skips a row missing data-oid without throwing', 'returns [] for a portal whose fetch throws, without aborting the other portals', 'returns [] for every portal when fetch always throws', 'returns [] without throwing when a portal responds non-2xx'"
status: pass
human_judgment: false
- id: D4
description: "NetServerAdapter is a TendersModule provider and is registered with SourceRegistry at DI boot alongside DoeOpenDataAdapter; the INGEST-07 denylist gate structurally applies (tender24/lhs-vpbw/vergabe.landbw are not denylisted, so registration succeeds); project-wide typecheck and the full tenders test slice remain green"
requirement: INGEST-02
verification:
- kind: unit
ref: "cd apps/api && npx tsc --noEmit -p tsconfig.json (clean)"
status: pass
- kind: unit
ref: "cd apps/api && npx vitest run src/tenders (20 files, 205/205 pass, includes source-registry.spec.ts 6/6 unchanged and netserver.adapter.spec.ts 14/14 new)"
status: pass
human_judgment: false
- id: D5
description: "Package-legitimacy checkpoint (blocking-human) ran BEFORE any install; cheerio was human-approved and installed only after explicit confirmation, never auto-approved"
requirement: INGEST-02
verification:
- kind: manual_gate
ref: "gsd-tools query package-legitimacy check --ecosystem npm node-html-parser cheerio (run pre-install) + npm view node-html-parser time.created / npm view cheerio time.created (cross-checked); human approved 'cheerio' via explicit resume instruction"
status: pass
human_judgment: true
duration: 55min
completed: 2026-07-23
status: complete
---
# Phase 13 Plan 04: NetServer-Adapter (tender24/lhs-vpbw/vergabe.landbw) Summary
**ONE config-driven `NetServerAdapter` (cheerio-based, human-approved at a blocking package-legitimacy checkpoint) now serves all three AI-AG "Vergabe@Net" portals from a single class, parsing the live public `PublicationSearchControllerServlet` results table into `RawTenderRecord[]` with full row/portal error isolation — the second real ingestion source that makes the Phase-13 dedup core (13-03) live-observable (`activePortalCount >= 2` once activated).**
## Performance
- **Duration:** ~55 min (including the blocking package-legitimacy checkpoint pause)
- **Tasks:** 3/3 completed
- **Files modified:** 6 (3 created, 3 modified)
## Accomplishments
- **Blocking package-legitimacy checkpoint honored correctly**: paused BEFORE any install, ran `gsd-tools query package-legitimacy check` against both `node-html-parser` and `cheerio`, cross-verified the flagged `node-html-parser` "too-new" verdict against `npm view ... time.created` (found it to be a false positive — the gate measures latest-version-publish date, not package age; the package is actually 9+ years old). Reported both findings and waited for explicit human approval. Human selected `cheerio` — installed only after that confirmation.
- `NetServerAdapter`: ONE class, `portals = ['tender24', 'lhs-vpbw', 'vergabe.landbw']`, hardcoded per-portal `baseUrl` constants (SSRF guard, T-13-04-01), fetches each portal's `PublicationSearchControllerServlet?function=SearchPublications` via native `fetch` + `AbortController` 15s timeout + `redirect: 'follow'`, parses the `<table>` results with cheerio.
- Fault tolerance exactly per D-01/Pitfall 2: try/catch per row (skip + `logger.warn`), try/catch per portal (one blocked/broken portal never prevents the other two from being polled), `cheerio.load()` itself wrapped so a totally unparsable page returns `[]` for that portal instead of throwing.
- Live-captured fixture (`__fixtures__/netserver-search.html`, 6 real rows from a live `tender24.de` GET, 2026-07-23) exercises umlaut + `&amp;`-entity buyer decoding, a populated deadline, and multiple empty-deadline rows.
- 14 spec tests, all green: sourceType/portals declaration, full-fixture parse with correct `sourcePortal`/`sourceNoticeId`/`sourceUrl`, umlaut/entity decoding, `data-oid`-uniqueness, deadline-to-Date-or-null mapping (including NetServer's `24:00` = next-day-midnight convention), empty/broken-HTML `[]`-fallback, missing-`data-oid`-row skip, per-portal fetch-fail isolation, total-fetch-failure `[]`-fallback, non-2xx `[]`-fallback, no-axios guard, no-input-interpolated-URL guard.
- `NetServerAdapter` registered as a `TendersModule` provider and at DI boot via `SourceRegistry.register()` (denylist gate structurally applies — none of the 3 portals are denylisted); `ai-netserver` `TenderSourcePollConfig` seeded with `isActive: false` (activation deferred to admin/Phase 14, per D-02).
- `npx tsc --noEmit` clean; `src/tenders` slice: 20 files, 205/205 tests pass (191 pre-existing + 14 new).
## Task Commits
Each task was committed atomically:
1. **Task 1: Package-legitimacy checkpoint (blocking-human)** — no commit (gate/decision only, per protocol); cheerio approved by the coordinator/human
2. **Task 2: cheerio install + NetServerAdapter + fixture + spec** - `043ada9` (feat)
3. **Task 3: NetServerAdapter registered in TendersModule** - `88572f5` (feat)
## Files Created/Modified
- `apps/api/src/tenders/adapters/netserver.adapter.ts` - config-driven multi-portal `NetServerAdapter` (`fetchTenders`, `parseSearchResults`, `parseNetServerDate`/`parseNetServerDeadline` helpers)
- `apps/api/src/tenders/adapters/netserver.adapter.spec.ts` - 14 unit tests against the fixture + mocked fetch
- `apps/api/src/tenders/__fixtures__/netserver-search.html` - live-captured NetServer results table (6 rows)
- `apps/api/package.json` / `pnpm-lock.yaml` - `cheerio ^1.2.0` dependency added
- `apps/api/src/tenders/tenders.module.ts` - `NetServerAdapter` provider + `onModuleInit` registration + `ai-netserver` poll-config seed (`isActive: false`)
## Decisions Made
- **cheerio over node-html-parser** — see `key-decisions` in frontmatter. The automated legitimacy gate's "too-new" flag on `node-html-parser` was investigated and found to be a measurement artifact (latest-version-publish date, not package founding date); both packages are legitimate, but `cheerio` passed the gate cleanly with zero flags, so it was the unambiguous choice for the human approver.
- **`sourceUrl` = search-results URL, not a per-notice deep link** — live HTML inspection found no `<a href>` per row (JS-only navigation via `data-oid`); rather than guess/invent an unverified detail-servlet URL pattern, this is documented as a best-effort limitation (Open Question 2).
- **`ocdsPayload` used as a generic NetServer field bag** rather than reshaping `RawTenderRecord`'s DÖE-shaped interface — keeps the shared adapter contract stable; full normalizer support for this shape is explicitly out of this plan's scope (see Known Stubs below).
- **`isActive: false` on the seeded poll config** — this plan proves the adapter works against a live-captured fixture; it deliberately does not flip the platform-global scheduler into actually polling 3 real portals every tick without an explicit admin/Phase-14 decision.
## Deviations from Plan
### Auto-fixed Issues
None — Rule 1-3 auto-fixes were not needed; implementation followed the plan and 13-RESEARCH.md's Pattern 1 design directly.
### Checkpoint Handling (not a deviation, documented per protocol)
**Package selection changed from the plan's primary suggestion.** 13-RESEARCH.md's `[ASSUMED]` recommendation was `node-html-parser` (with `cheerio` as the stated alternative). The blocking-human checkpoint's own automated gate flagged `node-html-parser` as `SUS`/"too-new" (a false positive on version-publish-date vs. package age, verified via `npm view ... time.created`), while `cheerio` passed cleanly. The coordinator/human explicitly approved `cheerio` after reviewing both findings. This is exactly the checkpoint's intended function — no auto-approval occurred, and both plan-sanctioned options were legitimate; the final selection was a human decision, not an executor deviation.
**Total:** 0 auto-fixed deviations. 1 human-directed package selection (checkpoint outcome, not a deviation).
## Known Stubs
- **`ocdsPayload` generic bag is not yet consumed by `TenderNormalizerService`.** The normalizer (`tender-normalizer.service.ts`) is currently hardcoded to the DÖE eForms/OCDS shape (`getOcdsTender`, `getEformsRoot`, etc.) and does not know how to map `NetServerAdapter`'s `{title, buyerName, procedureType, legalFramework, deadlineAt}` bag into `NormalizedTenderFields`. This means that even once `ai-netserver`'s poll config is activated, ingested NetServer records would currently normalize incorrectly (falling through to DÖE-shaped field extraction, which would find nothing and default most fields to null/'Unbenannte Ausschreibung'). **This is explicitly out of this plan's `files_modified` scope** (only `netserver.adapter.ts`/`.spec.ts`, the fixture, and `tenders.module.ts` were planned) — normalizer extension for non-DÖE-shaped payloads is a natural follow-up (likely alongside or after 13-05's cosinex adapter, which will face the identical problem) and should be a deliberate next plan, not silently patched in here. `activePortalCount >= 2` (the dedup-gate signal) is unaffected by this — it counts active poll configs, not correctly-normalized records — but real cross-source dedup value from NetServer will only materialize once the normalizer is extended AND the poll config is activated.
## Issues Encountered
None beyond the expected checkpoint pause. Live network access was available in this environment, which let the fixture be captured genuinely live (not synthesized) — matching the plan's "live-gecapturte HTML-Fixture" requirement exactly.
## User Setup Required
None. `ai-netserver` remains `isActive: false` — no live polling will start until an admin explicitly activates it (Phase 14 UI) or a manual DB update is made. No test/prod server was touched (local dev API only, host-side Vitest).
## Next Phase Readiness
- `SourceRegistry`, `TenderDedupService`, and `pollDueSources`' fan-out (13-02/13-03) now have a second real, tested adapter to exercise — 13-05 (cosinex/DTVP) can follow the identical pattern (config-driven-per-portal is N/A there since it's one portal, but the fetch/parse/error-isolation/registration shape repeats).
- **Follow-up needed before NetServer data is usably normalized end-to-end:** extend `TenderNormalizerService` (or introduce a per-sourceType normalization strategy) to understand the `ocdsPayload` generic bag this adapter produces — flagged above under Known Stubs, not blocking for INGEST-02's own success criteria (adapter parses to `RawTenderRecord[]` correctly) but blocking for a live end-to-end NetServer ingest to produce correct `Tender` rows.
- Manual/live verification (13-VALIDATION.md) — an optional live poll against `tender24.de` to confirm real records ingest — remains a manual step, not executed here (no `ai-netserver` activation was performed; the adapter's live-fetch path was exercised for the fixture capture itself, confirming reachability, but not through the full poll/ingest pipeline).
- No blockers for 13-05.
---
*Phase: 13-scraping-adapters-cross-source-dedup*
*Completed: 2026-07-23*
## Self-Check: PASSED
All created files verified present on disk; both task commit hashes (043ada9, 88572f5) verified in git log.