From 2c286053974da501966f9a46f8c128b00687b6aa Mon Sep 17 00:00:00 2001 From: Schalli Date: Thu, 23 Jul 2026 08:57:25 +0200 Subject: [PATCH] docs(13-06): complete multi-source read surface plan --- .planning/ROADMAP.md | 6 +- .planning/STATE.md | 16 ++- .../13-06-SUMMARY.md | 125 ++++++++++++++++++ 3 files changed, 137 insertions(+), 10 deletions(-) create mode 100644 .planning/phases/13-scraping-adapters-cross-source-dedup/13-06-SUMMARY.md diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index 4bf15fc..72b0e35 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -439,7 +439,7 @@ Plans: 3. A tender that appears via both DÖE and a scraping adapter shows up once in the results list (fuzzy fingerprint dedup on buyer+title+CPV+deadline+value), with links to all of its source portals -- dedup logic only activates once a second source is live 4. Attempting to register vergabe24 or aumass as a poll source is refused by the system itself (adapter registry denylist enforced in code), not just documented as forbidden -**Plans**: 3/6 plans executed +**Plans**: 4/6 plans executed Plans: @@ -448,7 +448,7 @@ Plans: - [x] 13-03-PLAN.md — 3-Stufen-Dedup-Resolver + pollDueSources Fan-out (catch-per-source) + Modul-Wiring (SCHEMA-03) - [ ] 13-04-PLAN.md — NetServer-Adapter (config-getrieben, 3 Portale) + Package-Legitimacy-Checkpoint (INGEST-02) - [ ] 13-05-PLAN.md — cosinex/DTVP-Adapter (separat, best-effort) (INGEST-03) -- [ ] 13-06-PLAN.md — Read-Endpoint include sources[] + TenderDetail Multi-Source-Links (SCHEMA-03 Anzeige) +- [x] 13-06-PLAN.md — Read-Endpoint include sources[] + TenderDetail Multi-Source-Links (SCHEMA-03 Anzeige) ### Phase 14: RSS, Email-Alert Ingestion & Module Rollout @@ -486,5 +486,5 @@ Phases execute in numeric order: 1 -> 2 -> 3 -> 4 -> 5 -> 6 -> 7 -> 8 -> 9 -> 10 | 10. Ausschreibungs-Radar Foundation & DÖE Ingestion | 6/6 | Complete | 2026-07-21 | | 11. Filter Engine, Results UI & Saved Searches | 6/6 | In Progress| | | 12. Tender Notifications | 4/4 | In Progress| | -| 13. Scraping Adapters & Cross-Source Deduplication | 3/6 | In Progress| | +| 13. Scraping Adapters & Cross-Source Deduplication | 4/6 | In Progress| | | 14. RSS, Email-Alert Ingestion & Module Rollout | 0/TBD | Not started | - | diff --git a/.planning/STATE.md b/.planning/STATE.md index 3ceea34..caca99f 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -5,15 +5,15 @@ milestone_name: Ausschreibungs-Radar current_phase: 13 current_phase_name: scraping-adapters-cross-source-dedup status: executing -stopped_at: Completed 13-03-PLAN.md -last_updated: "2026-07-23T06:53:28.370Z" +stopped_at: Completed 13-06-PLAN.md +last_updated: "2026-07-23T06:57:20.973Z" last_activity: 2026-07-23 last_activity_desc: Phase 13 execution started progress: total_phases: 13 completed_phases: 11 total_plans: 62 - completed_plans: 58 + completed_plans: 59 --- # Project State @@ -28,11 +28,11 @@ See: .planning/PROJECT.md (updated 2026-07-17) ## Current Position Phase: 13 (scraping-adapters-cross-source-dedup) — EXECUTING -Plan: 4 of 6 +Plan: 5 of 6 Status: Ready to execute Last activity: 2026-07-23 — Phase 13 execution started -Progress: [█████████░] 94% +Progress: [██████████] 95% ## Performance Metrics @@ -93,6 +93,7 @@ Progress: [█████████░] 94% | Phase 13 P01 | 35min | 3 tasks | 6 files | | Phase 13 P02 | 20min | 2 tasks | 4 files | | Phase 13 P03 | 45min | 3 tasks | 5 files | +| Phase 13 P06 | 15min | 2 tasks | 5 files | ## Accumulated Context @@ -199,6 +200,7 @@ Recent decisions affecting current work: - [Phase ?]: SourceRegistry.register() throws DeniedPortalError for any portal in DENYLISTED_PORTALS (vergabe24, aumass), enforced at DI-registration time not just documented (INGEST-07) - [Phase ?]: TenderDedupService tier-2 (source:noticeId) match is unconditional (not gated by dedupActive) — idempotent same-source re-poll must work even with a single active source - [Phase ?]: Fingerprint always computed on tender.create regardless of dedupActive, so DÖE-only tenders become fingerprint-matchable the instant a 2nd source activates without further backfill +- [Phase ?]: 13-06: portalLabel()-Map bleibt lokal in TenderDetail.tsx (nicht geteilt) — einziger Consumer bisher; Fallback auf sourceUrl-Block bei fehlendem/leerem sources[]. ### Pending Todos @@ -236,7 +238,7 @@ Items acknowledged and carried forward from previous milestone close: ## Session Continuity -Last session: 2026-07-23T06:53:22.726Z -Stopped at: Completed 13-03-PLAN.md +Last session: 2026-07-23T06:57:20.960Z +Stopped at: Completed 13-06-PLAN.md Resume file: None Last activity: 2026-07-14 - Built LDAP per-user exclude/denylist filter (9d1323f), migration applied on live DB, verified via Playwright: sync deactivated 4 excluded service accounts (administrator/krbtgt/guest/dns-ldap), 2 real LDAP users stay active, 0 wrongly created diff --git a/.planning/phases/13-scraping-adapters-cross-source-dedup/13-06-SUMMARY.md b/.planning/phases/13-scraping-adapters-cross-source-dedup/13-06-SUMMARY.md new file mode 100644 index 0000000..d30cb11 --- /dev/null +++ b/.planning/phases/13-scraping-adapters-cross-source-dedup/13-06-SUMMARY.md @@ -0,0 +1,125 @@ +--- +phase: 13-scraping-adapters-cross-source-dedup +plan: 06 +subsystem: api +tags: [nestjs, prisma, react, tender-radar, dedup] + +# Dependency graph +requires: + - phase: 13-01 + provides: TenderSource model (sourcePortal, sourceUrl, sourceNoticeId) + backfill of existing DÖE tenders + - phase: 13-03 + provides: dedup-resolver fan-out wiring that attaches additional TenderSource rows to an existing Tender +provides: + - GET /modules/tender-radar/:id now includes the sources[] relation (TenderSource rows) + - Tender API client type carries optional sources[] (sourcePortal, sourceUrl, sourceNoticeId) + - TenderDetail renders one link per source portal with a German portal label, falling back to the single sourceUrl block when sources is missing/empty +affects: [14-i18n-admin-source-ui, tender-radar-detail-view] + +# Tech tracking +tech-stack: + added: [] + patterns: + - "Read-surface include: TenderSource is exposed via Prisma `include` with a minimal `select` (sourcePortal/sourceUrl/sourceNoticeId), same minimal-projection convention as getCoverage's groupBy." + - "Optional relation field on frontend API types (`sources?: [...]`) with an explicit empty/missing fallback to the legacy single-field UI block — safe rollout path for a new relation without breaking older/undeployed callers." + +key-files: + created: [] + modified: + - apps/api/src/tenders/tenders.controller.ts + - apps/api/src/tenders/tenders.controller.spec.ts + - apps/web/src/lib/tender-radar-api.ts + - apps/web/src/app/(portal)/modules/tender-radar/components/TenderDetail.tsx + - apps/web/src/app/(portal)/modules/tender-radar/components/TenderDetail.test.tsx + +key-decisions: + - "portalLabel() map lives inline in TenderDetail.tsx (not a shared constant) — only this component needs it today; unknown sourcePortal slugs fall back to the raw slug so a future portal never silently disappears from the list." + - "Fallback order is sources[]-non-empty -> legacy sourceUrl block -> 'Kein Quell-Link verfügbar' — preserves exact existing UI/tests for tenders that predate the TenderSource backfill or whose sources[] loads empty." + +requirements-completed: [SCHEMA-03] + +coverage: + - id: D1 + description: "GET /modules/tender-radar/:id includes the TenderSource relation (sourcePortal, sourceUrl, sourceNoticeId) via Prisma include, without changing route order or tenant scoping." + requirement: "SCHEMA-03" + verification: + - kind: unit + ref: "apps/api/src/tenders/tenders.controller.spec.ts#GET /:id includes the sources relation so the response carries all TenderSource links (13-06, SCHEMA-03)" + status: pass + human_judgment: false + - id: D2 + description: "TenderDetail renders a list of links (one per source portal, German label) when tender.sources[] has entries, and falls back to the existing single sourceUrl block when sources is missing or empty." + requirement: "SCHEMA-03" + verification: + - kind: unit + ref: "apps/web/.../TenderDetail.test.tsx#renders a link for every source when tender.sources[] has multiple entries (13-06, SCHEMA-03)" + status: pass + - kind: unit + ref: "apps/web/.../TenderDetail.test.tsx#falls back to the single sourceUrl block when tender.sources is missing/empty" + status: pass + human_judgment: false + +# Metrics +duration: 15min +completed: 2026-07-23 +status: complete +--- + +# Phase 13 Plan 06: Multi-Source Read Surface Summary + +**GET /modules/tender-radar/:id now returns the tender's full TenderSource list and the detail drawer renders one clickable link per source portal (DÖE/tender24/DTVP/...), completing the display half of cross-source dedup (Erfolgskriterium 3).** + +## Performance + +- **Duration:** ~15 min +- **Tasks:** 2/2 completed +- **Files modified:** 5 + +## Accomplishments +- `getTender` includes the `sources` relation (minimal `select`: sourcePortal, sourceUrl, sourceNoticeId), route order unchanged (`:id` still declared after all static routes) +- `Tender` API client type gained an optional `sources[]` field mirroring the new backend shape +- `TenderDetail` renders a `