docs(13-06): complete multi-source read surface plan
This commit is contained in:
@@ -439,7 +439,7 @@ Plans:
|
||||
3. A tender that appears via both DÖE and a scraping adapter shows up once in the results list (fuzzy fingerprint dedup on buyer+title+CPV+deadline+value), with links to all of its source portals -- dedup logic only activates once a second source is live
|
||||
4. Attempting to register vergabe24 or aumass as a poll source is refused by the system itself (adapter registry denylist enforced in code), not just documented as forbidden
|
||||
|
||||
**Plans**: 3/6 plans executed
|
||||
**Plans**: 4/6 plans executed
|
||||
|
||||
Plans:
|
||||
|
||||
@@ -448,7 +448,7 @@ Plans:
|
||||
- [x] 13-03-PLAN.md — 3-Stufen-Dedup-Resolver + pollDueSources Fan-out (catch-per-source) + Modul-Wiring (SCHEMA-03)
|
||||
- [ ] 13-04-PLAN.md — NetServer-Adapter (config-getrieben, 3 Portale) + Package-Legitimacy-Checkpoint (INGEST-02)
|
||||
- [ ] 13-05-PLAN.md — cosinex/DTVP-Adapter (separat, best-effort) (INGEST-03)
|
||||
- [ ] 13-06-PLAN.md — Read-Endpoint include sources[] + TenderDetail Multi-Source-Links (SCHEMA-03 Anzeige)
|
||||
- [x] 13-06-PLAN.md — Read-Endpoint include sources[] + TenderDetail Multi-Source-Links (SCHEMA-03 Anzeige)
|
||||
|
||||
### Phase 14: RSS, Email-Alert Ingestion & Module Rollout
|
||||
|
||||
@@ -486,5 +486,5 @@ Phases execute in numeric order: 1 -> 2 -> 3 -> 4 -> 5 -> 6 -> 7 -> 8 -> 9 -> 10
|
||||
| 10. Ausschreibungs-Radar Foundation & DÖE Ingestion | 6/6 | Complete | 2026-07-21 |
|
||||
| 11. Filter Engine, Results UI & Saved Searches | 6/6 | In Progress| |
|
||||
| 12. Tender Notifications | 4/4 | In Progress| |
|
||||
| 13. Scraping Adapters & Cross-Source Deduplication | 3/6 | In Progress| |
|
||||
| 13. Scraping Adapters & Cross-Source Deduplication | 4/6 | In Progress| |
|
||||
| 14. RSS, Email-Alert Ingestion & Module Rollout | 0/TBD | Not started | - |
|
||||
|
||||
+9
-7
@@ -5,15 +5,15 @@ milestone_name: Ausschreibungs-Radar
|
||||
current_phase: 13
|
||||
current_phase_name: scraping-adapters-cross-source-dedup
|
||||
status: executing
|
||||
stopped_at: Completed 13-03-PLAN.md
|
||||
last_updated: "2026-07-23T06:53:28.370Z"
|
||||
stopped_at: Completed 13-06-PLAN.md
|
||||
last_updated: "2026-07-23T06:57:20.973Z"
|
||||
last_activity: 2026-07-23
|
||||
last_activity_desc: Phase 13 execution started
|
||||
progress:
|
||||
total_phases: 13
|
||||
completed_phases: 11
|
||||
total_plans: 62
|
||||
completed_plans: 58
|
||||
completed_plans: 59
|
||||
---
|
||||
|
||||
# Project State
|
||||
@@ -28,11 +28,11 @@ See: .planning/PROJECT.md (updated 2026-07-17)
|
||||
## Current Position
|
||||
|
||||
Phase: 13 (scraping-adapters-cross-source-dedup) — EXECUTING
|
||||
Plan: 4 of 6
|
||||
Plan: 5 of 6
|
||||
Status: Ready to execute
|
||||
Last activity: 2026-07-23 — Phase 13 execution started
|
||||
|
||||
Progress: [█████████░] 94%
|
||||
Progress: [██████████] 95%
|
||||
|
||||
## Performance Metrics
|
||||
|
||||
@@ -93,6 +93,7 @@ Progress: [█████████░] 94%
|
||||
| Phase 13 P01 | 35min | 3 tasks | 6 files |
|
||||
| Phase 13 P02 | 20min | 2 tasks | 4 files |
|
||||
| Phase 13 P03 | 45min | 3 tasks | 5 files |
|
||||
| Phase 13 P06 | 15min | 2 tasks | 5 files |
|
||||
|
||||
## Accumulated Context
|
||||
|
||||
@@ -199,6 +200,7 @@ Recent decisions affecting current work:
|
||||
- [Phase ?]: SourceRegistry.register() throws DeniedPortalError for any portal in DENYLISTED_PORTALS (vergabe24, aumass), enforced at DI-registration time not just documented (INGEST-07)
|
||||
- [Phase ?]: TenderDedupService tier-2 (source:noticeId) match is unconditional (not gated by dedupActive) — idempotent same-source re-poll must work even with a single active source
|
||||
- [Phase ?]: Fingerprint always computed on tender.create regardless of dedupActive, so DÖE-only tenders become fingerprint-matchable the instant a 2nd source activates without further backfill
|
||||
- [Phase ?]: 13-06: portalLabel()-Map bleibt lokal in TenderDetail.tsx (nicht geteilt) — einziger Consumer bisher; Fallback auf sourceUrl-Block bei fehlendem/leerem sources[].
|
||||
|
||||
### Pending Todos
|
||||
|
||||
@@ -236,7 +238,7 @@ Items acknowledged and carried forward from previous milestone close:
|
||||
|
||||
## Session Continuity
|
||||
|
||||
Last session: 2026-07-23T06:53:22.726Z
|
||||
Stopped at: Completed 13-03-PLAN.md
|
||||
Last session: 2026-07-23T06:57:20.960Z
|
||||
Stopped at: Completed 13-06-PLAN.md
|
||||
Resume file: None
|
||||
Last activity: 2026-07-14 - Built LDAP per-user exclude/denylist filter (9d1323f), migration applied on live DB, verified via Playwright: sync deactivated 4 excluded service accounts (administrator/krbtgt/guest/dns-ldap), 2 real LDAP users stay active, 0 wrongly created
|
||||
|
||||
@@ -0,0 +1,125 @@
|
||||
---
|
||||
phase: 13-scraping-adapters-cross-source-dedup
|
||||
plan: 06
|
||||
subsystem: api
|
||||
tags: [nestjs, prisma, react, tender-radar, dedup]
|
||||
|
||||
# Dependency graph
|
||||
requires:
|
||||
- phase: 13-01
|
||||
provides: TenderSource model (sourcePortal, sourceUrl, sourceNoticeId) + backfill of existing DÖE tenders
|
||||
- phase: 13-03
|
||||
provides: dedup-resolver fan-out wiring that attaches additional TenderSource rows to an existing Tender
|
||||
provides:
|
||||
- GET /modules/tender-radar/:id now includes the sources[] relation (TenderSource rows)
|
||||
- Tender API client type carries optional sources[] (sourcePortal, sourceUrl, sourceNoticeId)
|
||||
- TenderDetail renders one link per source portal with a German portal label, falling back to the single sourceUrl block when sources is missing/empty
|
||||
affects: [14-i18n-admin-source-ui, tender-radar-detail-view]
|
||||
|
||||
# Tech tracking
|
||||
tech-stack:
|
||||
added: []
|
||||
patterns:
|
||||
- "Read-surface include: TenderSource is exposed via Prisma `include` with a minimal `select` (sourcePortal/sourceUrl/sourceNoticeId), same minimal-projection convention as getCoverage's groupBy."
|
||||
- "Optional relation field on frontend API types (`sources?: [...]`) with an explicit empty/missing fallback to the legacy single-field UI block — safe rollout path for a new relation without breaking older/undeployed callers."
|
||||
|
||||
key-files:
|
||||
created: []
|
||||
modified:
|
||||
- apps/api/src/tenders/tenders.controller.ts
|
||||
- apps/api/src/tenders/tenders.controller.spec.ts
|
||||
- apps/web/src/lib/tender-radar-api.ts
|
||||
- apps/web/src/app/(portal)/modules/tender-radar/components/TenderDetail.tsx
|
||||
- apps/web/src/app/(portal)/modules/tender-radar/components/TenderDetail.test.tsx
|
||||
|
||||
key-decisions:
|
||||
- "portalLabel() map lives inline in TenderDetail.tsx (not a shared constant) — only this component needs it today; unknown sourcePortal slugs fall back to the raw slug so a future portal never silently disappears from the list."
|
||||
- "Fallback order is sources[]-non-empty -> legacy sourceUrl block -> 'Kein Quell-Link verfügbar' — preserves exact existing UI/tests for tenders that predate the TenderSource backfill or whose sources[] loads empty."
|
||||
|
||||
requirements-completed: [SCHEMA-03]
|
||||
|
||||
coverage:
|
||||
- id: D1
|
||||
description: "GET /modules/tender-radar/:id includes the TenderSource relation (sourcePortal, sourceUrl, sourceNoticeId) via Prisma include, without changing route order or tenant scoping."
|
||||
requirement: "SCHEMA-03"
|
||||
verification:
|
||||
- kind: unit
|
||||
ref: "apps/api/src/tenders/tenders.controller.spec.ts#GET /:id includes the sources relation so the response carries all TenderSource links (13-06, SCHEMA-03)"
|
||||
status: pass
|
||||
human_judgment: false
|
||||
- id: D2
|
||||
description: "TenderDetail renders a list of links (one per source portal, German label) when tender.sources[] has entries, and falls back to the existing single sourceUrl block when sources is missing or empty."
|
||||
requirement: "SCHEMA-03"
|
||||
verification:
|
||||
- kind: unit
|
||||
ref: "apps/web/.../TenderDetail.test.tsx#renders a link for every source when tender.sources[] has multiple entries (13-06, SCHEMA-03)"
|
||||
status: pass
|
||||
- kind: unit
|
||||
ref: "apps/web/.../TenderDetail.test.tsx#falls back to the single sourceUrl block when tender.sources is missing/empty"
|
||||
status: pass
|
||||
human_judgment: false
|
||||
|
||||
# Metrics
|
||||
duration: 15min
|
||||
completed: 2026-07-23
|
||||
status: complete
|
||||
---
|
||||
|
||||
# Phase 13 Plan 06: Multi-Source Read Surface Summary
|
||||
|
||||
**GET /modules/tender-radar/:id now returns the tender's full TenderSource list and the detail drawer renders one clickable link per source portal (DÖE/tender24/DTVP/...), completing the display half of cross-source dedup (Erfolgskriterium 3).**
|
||||
|
||||
## Performance
|
||||
|
||||
- **Duration:** ~15 min
|
||||
- **Tasks:** 2/2 completed
|
||||
- **Files modified:** 5
|
||||
|
||||
## Accomplishments
|
||||
- `getTender` includes the `sources` relation (minimal `select`: sourcePortal, sourceUrl, sourceNoticeId), route order unchanged (`:id` still declared after all static routes)
|
||||
- `Tender` API client type gained an optional `sources[]` field mirroring the new backend shape
|
||||
- `TenderDetail` renders a `<ul>` of source links with hardcoded German portal labels (DÖE, tender24, lhs-vpbw, vergabe.landbw, DTVP), falling back to the pre-existing single `sourceUrl` block when `sources` is missing or empty (older tenders / responses without the include)
|
||||
|
||||
## Task Commits
|
||||
|
||||
Each task was committed atomically:
|
||||
|
||||
1. **Task 1: getTender liefert sources[] via include** - `166194f` (feat)
|
||||
2. **Task 2: API-Client-Typ + TenderDetail Multi-Source-Liste** - `bf61316` (feat)
|
||||
|
||||
**Plan metadata:** pending (this commit)
|
||||
|
||||
## Files Created/Modified
|
||||
- `apps/api/src/tenders/tenders.controller.ts` - `getTender` now does `prisma.tender.findUnique({ where, include: { sources: { select: {...} } } })`
|
||||
- `apps/api/src/tenders/tenders.controller.spec.ts` - fake prisma tender `t1` now carries a two-entry `sources` array; new test asserts the exact `include`/`select` shape and the passthrough response
|
||||
- `apps/web/src/lib/tender-radar-api.ts` - `Tender.sources?` optional field added, documented as detail-only/optional
|
||||
- `apps/web/src/app/(portal)/modules/tender-radar/components/TenderDetail.tsx` - added `portalLabel()` map + multi-source `<ul>` rendering, with fallback to the original single-`sourceUrl` block
|
||||
- `apps/web/src/app/(portal)/modules/tender-radar/components/TenderDetail.test.tsx` - two new tests: multi-link rendering with portal labels, and fallback-to-single-link when `sources` is empty/missing
|
||||
|
||||
## Decisions Made
|
||||
- Kept `portalLabel()` as a local, non-exported function in `TenderDetail.tsx` rather than a shared module — only one consumer today; can be lifted to a shared constants file in Phase 14 (i18n rollout) if a second consumer appears.
|
||||
- Used `select` (not the full `TenderSource` row) in the Prisma `include` to avoid leaking `id`/`tenderId`/`createdAt`/`ocid` to the frontend — matches the existing minimal-projection convention (`getCoverage`'s `groupBy`).
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
None - plan executed exactly as written.
|
||||
|
||||
## Issues Encountered
|
||||
|
||||
None.
|
||||
|
||||
## User Setup Required
|
||||
|
||||
None - no external service configuration required.
|
||||
|
||||
## Next Phase Readiness
|
||||
- Erfolgskriterium 3 (Anzeige aller Quell-Links) is now fully wired end-to-end: backend include (13-06) sits on top of the dedup-resolver fan-out (13-03) and the 13-01 backfill, so any tender that later gains a second `TenderSource` row (once a NetServer/cosinex adapter is active) will automatically surface a second link in the detail drawer with no further frontend work.
|
||||
- Admin-facing "excluded portals" UI (UI-06) and i18n rollout remain explicitly out of scope here, deferred to Phase 14 per 13-CONTEXT.md.
|
||||
|
||||
---
|
||||
*Phase: 13-scraping-adapters-cross-source-dedup*
|
||||
*Completed: 2026-07-23*
|
||||
|
||||
## Self-Check: PASSED
|
||||
|
||||
All modified/created files verified present on disk; both task commits (166194f, bf61316) verified present in git history.
|
||||
Reference in New Issue
Block a user