docs(quick-260723-e7i): complete normalizer-gap-phase-13-schliessen-tende quick task

2026-07-23 10:21:18 +02:00
parent 988100576f
commit d5128d5759
2 changed files with 83 additions and 4 deletions
@@ -0,0 +1,77 @@
+---
+phase: quick-260723-e7i
+plan: 01
+status: complete
+subsystem: tenders
+tags: [normalizer, cross-source-dedup, gap-closure, phase-13]
+requirements:
+  - INGEST-02
+  - INGEST-03
+  - SCHEMA-03
+dependency-graph:
+  requires:
+    - apps/api/src/tenders/tender.types.ts
+    - apps/api/src/tenders/adapters/netserver.adapter.ts
+    - apps/api/src/tenders/adapters/cosinex.adapter.ts
+  provides:
+    - TenderNormalizerService per-sourceType dispatch (ai-netserver/cosinex-dtvp bag mapping)
+  affects:
+    - TenderFingerprint cross-source dedup (13-01) — NetServer/Cosinex records now carry real title/buyer/deadline instead of defaults
+tech-stack:
+  added: []
+  patterns:
+    - "Shared assemble() tail extracted from normalize() so status/dedupKey/contentHash/publishedAt are computed identically across all sourceTypes"
+key-files:
+  created: []
+  modified:
+    - apps/api/src/tenders/tender-normalizer.service.ts
+    - apps/api/src/tenders/tender-normalizer.service.spec.ts
+decisions:
+  - "normalize() dispatches on raw.sourceType via switch; default case falls through to the DOE path (never throws) so future additive SourceType union members can't break existing callers"
+  - "legalFramework from the scraper bag is deliberately not mapped — NormalizedTenderFields has no target field for it"
+  - "CoreTenderFields (Omit type) defines the 10 source-specific fields each extraction path must produce; assemble() owns everything else"
+metrics:
+  duration: "~25 min"
+  completed: "2026-07-23"
+  tasks_completed: 3
+  tasks_total: 3
+  files_modified: 2
+---
+
+# Quick Task 260723-e7i: Normalizer-Gap Phase 13 schliessen (NetServer/Cosinex-Bag-Dispatch) Summary
+
+Fixed `TenderNormalizerService.normalize()` to dispatch per `sourceType`, so `ai-netserver` and `cosinex-dtvp` records (which carry a flat `ocdsPayload` bag instead of eForms/OCDS structure) map their real `title`/`buyerName`/`procedureType`/`deadlineAt` instead of falling through to defaults/nulls — unblocking cross-source fingerprint dedup between DÖE and the scraper adapters.
+
+## What Was Built
+
+**Task 1 — Per-sourceType dispatch + shared `assemble()`:**
+- Extracted a private `assemble(raw, core)` method that computes `status`, `dedupKey`, `contentHash` (via the existing `computeContentHash`, not duplicated), and `publishedAt`, plus the pass-through identity fields (`sourcePortal`, `sourceNoticeId`, `ocid`, `sourceUrl`).
+- Moved the existing DÖE eForms/OCDS extraction logic into `normalizeDoe(raw)` — byte-identical behavior, confirmed by the full pre-existing test suite staying green.
+- Added `normalizeBag(raw)` for the flat bag shape shared by NetServer and cosinex/DTVP: reads `raw.ocdsPayload` defensively, maps `title` (fallback `'Unbenannte Ausschreibung'` when empty/missing), `buyerName`/`procedureType` (null-tolerant, no invented fallback strings), `deadlineAt` via the existing `parseOcdsDate()`. `cpvCodes`/`cpvDivisions` = `[]`, `region`/`plz`/`bundesland`/`estimatedValue` = `null`. `legalFramework` is read but intentionally not mapped (no target field).
+- `normalize()` now does `switch (raw.sourceType)`: `ai-netserver`/`cosinex-dtvp` → `normalizeBag`; `doe-opendata`/`default` → `normalizeDoe` (default never throws, protecting the additive `SourceType` union).
+
+**Task 2 — Unit tests:**
+- Added `bagRecord(sourceType, bag)` helper to construct inline `RawTenderRecord`s for the bag sources (no fixture ZIPs exist for these).
+- New tests: full-bag mapping for `ai-netserver` and `cosinex-dtvp` (identical shape → identical mapping), null/empty-field fallback (`buyerName`/`procedureType`/`deadlineAt` stay `null`, empty `title` falls back), and a `contentHash` sha256-format assertion proving `assemble()` reuse.
+- Existing DÖE fixture-based regression tests (10 pre-existing tests) untouched and still green.
+
+**Task 3 — Non-regression gate:**
+- `npx tsc --noEmit -p tsconfig.json` — clean.
+- `npx vitest run` — full API suite: **288/288 tests passed** across 25 test files (up from the pre-existing baseline + 4 new bag tests), including `tender-dedup.service.spec.ts`, `tender-ingestion.service.spec.ts`, and `tender-fingerprint.spec.ts`. No code change needed in this task.
+
+## Deviations from Plan
+
+None — plan executed exactly as written.
+
+## Test Results
+
+- `npx vitest run src/tenders/tender-normalizer.service.spec.ts` — **14/14 passed** (10 pre-existing DÖE tests + 4 new bag tests)
+- `npx tsc --noEmit -p tsconfig.json` — clean, no errors
+- `npx vitest run` (full API suite) — **288/288 passed**, 25 test files, 3.06s
+
+## Self-Check: PASSED
+
+- FOUND: apps/api/src/tenders/tender-normalizer.service.ts
+- FOUND: apps/api/src/tenders/tender-normalizer.service.spec.ts
+- FOUND commit 82c9a48 (Task 1)
+- FOUND commit 9881005 (Task 2)