From e7b59d16af4371517c6cd67c7d902f41a6255b52 Mon Sep 17 00:00:00 2001 From: Schalli Date: Tue, 21 Jul 2026 10:03:19 +0200 Subject: [PATCH] =?UTF-8?q?docs(10):=20create=20phase=20plan=20(5=20plans,?= =?UTF-8?q?=20D=C3=96E=20ingestion=20foundation)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .planning/ROADMAP.md | 26 ++- .../10-01-PLAN.md | 167 ++++++++++++++++ .../10-02-PLAN.md | 162 +++++++++++++++ .../10-03-PLAN.md | 185 ++++++++++++++++++ .../10-04-PLAN.md | 173 ++++++++++++++++ .../10-05-PLAN.md | 165 ++++++++++++++++ .../10-VALIDATION.md | 63 +++--- 7 files changed, 915 insertions(+), 26 deletions(-) create mode 100644 .planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-01-PLAN.md create mode 100644 .planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-02-PLAN.md create mode 100644 .planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-03-PLAN.md create mode 100644 .planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-04-PLAN.md create mode 100644 .planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-05-PLAN.md diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index f22b39d..08d8e78 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -344,7 +344,29 @@ Plans: 4. DÖE is polled once on a shared, admin-configurable interval regardless of how many tenants have the module active -- never once per tenant (poll-once-fan-out-many, not the DKV single-tenant `findFirst()` pattern) 5. Activating the module for a second tenant does not duplicate ingestion, re-trigger a redundant DÖE poll, or interfere with the first tenant's data -**Plans**: TBD +**Plans**: 5 plans + +**Wave 1** + +- [ ] 10-01-PLAN.md — Foundation: install fast-xml-parser/adm-zip/csv-parse (adm-zip supply-chain checkpoint), add global Tender + TenderSourcePollConfig models, [BLOCKING] migration (SCHEMA-01) + +**Wave 2** *(blocked on Wave 1 completion)* + +- [ ] 10-02-PLAN.md — Marketplace registration slice: self-seed tender-radar module, module-loader whitelist entry, placeholder page, singleton doe-opendata poll config (CONFIG-01) + +**Wave 3** *(blocked on Wave 2 completion)* + +- [ ] 10-03-PLAN.md — DÖE adapter + normalizer (TDD): fetch/extract/parse day-export ZIP, D-02 open-tender filter, eForms-primary normalize with dedupKey + contentHash (INGEST-01, SCHEMA-01) + +**Wave 4** *(blocked on Wave 3 completion)* + +- [ ] 10-04-PLAN.md — Ingestion + shared scheduler: day-cursor gate, upsert change-detection, single global cron (poll-once-fan-out-many), 90-day retention, two-tenant safety test (SCHEMA-02, INGEST-06) + +**Wave 5** *(blocked on Wave 4 completion)* + +- [ ] 10-05-PLAN.md — Controller + admin source-config: global ModuleGuard-gated read, admin-configurable poll interval applied live to scheduler (INGEST-06) + +**UI hint**: no (backend-foundation phase; results UI is Phase 11) ### Phase 11: Filter Engine, Results UI & Saved Searches @@ -427,7 +449,7 @@ Phases execute in numeric order: 1 -> 2 -> 3 -> 4 -> 5 -> 6 -> 7 -> 8 -> 9 -> 10 | 7. DKV Fleet Module | 6/6 | Complete | 2026-06-27 | | 8. Dashboard Widgets Vollimplementierung | 4/4 | Complete | 2026-07-01 | | 9. Cert Manager Module | 6/6 | Complete | 2026-07-02 | -| 10. Ausschreibungs-Radar Foundation & DÖE Ingestion | 0/TBD | Not started | - | +| 10. Ausschreibungs-Radar Foundation & DÖE Ingestion | 0/5 | Not started | - | | 11. Filter Engine, Results UI & Saved Searches | 0/TBD | Not started | - | | 12. Tender Notifications | 0/TBD | Not started | - | | 13. Scraping Adapters & Cross-Source Deduplication | 0/TBD | Not started | - | diff --git a/.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-01-PLAN.md b/.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-01-PLAN.md new file mode 100644 index 0000000..dcd116e --- /dev/null +++ b/.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-01-PLAN.md @@ -0,0 +1,167 @@ +--- +phase: 10-ausschreibungs-radar-foundation-d-e-ingestion +plan: 01 +type: execute +wave: 1 +depends_on: [] +files_modified: + - apps/api/package.json + - apps/api/prisma/schema.prisma + - apps/api/prisma/migrations/20260721120000_add_tender_radar/migration.sql +autonomous: false +requirements: [SCHEMA-01] +must_haves: + truths: + - "Global (tenant-agnostic) Tender table exists in Postgres with OCDS-oriented nullable deadline/value fields (SCHEMA-01)" + - "Singleton TenderSourcePollConfig row model exists to drive the shared poll (INGEST-06 foundation)" + - "fast-xml-parser, adm-zip and csv-parse are installed in @tessera/api" + artifacts: + - apps/api/prisma/schema.prisma + - apps/api/prisma/migrations/20260721120000_add_tender_radar/migration.sql + key_links: + - "Tender.dedupKey @unique is the upsert target for SCHEMA-02 change detection" + - "Tender has NO tenantId column — RLS/forTenant deliberately does not apply (global data, per D-03)" +--- + + +Lay the data + dependency foundation for the Ausschreibungs-Radar module: install the three ingestion libraries (with a supply-chain checkpoint for the [SUS] `adm-zip` package), add the platform-global `Tender` and singleton `TenderSourcePollConfig` Prisma models, and apply the migration to the live database. + +## Phase Goal (user story) +**As a** Tessera-Administrator, **I want to** das Ausschreibungs-Radar-Modul im Marketplace aktivieren und automatisch normalisierte deutsche DÖE-Ausschreibungen auf einem gemeinsamen, mandantensicheren Zeitplan erfassen lassen, **so that** alle Mandanten aktuelle, bietbare Vergaben durchsuchen koennen, ohne dass pro Mandant redundant gepollt wird. + +Purpose: Every later slice (module registration, adapter, normalizer, ingestion, scheduler) depends on these tables and packages existing. The global-not-tenant-scoped shape of `Tender` (D-03) is the single highest-leverage decision and is locked in here. +Output: Migrated `Tender` + `TenderSourcePollConfig` tables, generated Prisma client types, three installed packages. + + + +@$HOME/.claude/gsd-core/workflows/execute-plan.md +@$HOME/.claude/gsd-core/templates/summary.md + + + +@.planning/PROJECT.md +@.planning/ROADMAP.md +@.planning/STATE.md +@.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-RESEARCH.md +@.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-PATTERNS.md +@.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-CONTEXT.md +@apps/api/prisma/schema.prisma +@apps/api/prisma/migrations/20260714090000_add_ldap_user_exclude_list/migration.sql + + +## Artifacts This Phase Produces (whole-phase reference) + +Prisma models: `Tender`, `TenderSourcePollConfig` (Plan 01). +NestJS classes: `TendersModule`, `seedTendersModule()` (Plan 02); `DoeOpenDataAdapter`, `TenderSourceAdapter` interface, `TenderNormalizerService` (Plan 03); `TenderIngestionService`, `TenderSchedulerService` (Plan 04); `TendersController`, `SourceConfigDto`, `TenderQueryDto` (Plan 05). +Types: `RawTenderRecord`, `NormalizedTenderFields`, `SourceType` in `tenders/tender.types.ts` (Plan 03). +Scheduler methods: `TenderSchedulerService.setInterval(intervalMin)`, `stopJob()`; `TenderIngestionService.pollDueSources()`, `pruneExpiredTenders()` (Plan 04). +Config keys / slugs: module slug `tender-radar`; `TenderSourcePollConfig.sourceType = 'doe-opendata'` (singleton). +New file paths: `apps/api/src/tenders/**`, `apps/web/src/app/(portal)/modules/tender-radar/page.tsx`, `apps/web/src/lib/module-loader.ts` (+`tender-radar` entry), fixture ZIPs under `apps/api/src/tenders/__fixtures__/`. + + + + + Task 1: adm-zip package legitimacy verification + Nothing yet — this gate precedes the `pnpm add adm-zip` step. `adm-zip` was tagged [SUS]/[ASSUMED] in 10-RESEARCH.md Package Legitimacy Audit (flagged `too-new`, manually assessed a false positive, but package-name provenance is training-knowledge so it requires a human spot-check per the package-legitimacy protocol). + Pause before installing `adm-zip`. Present the legitimacy evidence below to the human and wait for explicit approval (or a named alternative) before Task 2 runs `pnpm add`. Do not auto-approve — this is a blocking supply-chain gate (T-10-SC). + + 1. Open https://www.npmjs.com/package/adm-zip — confirm weekly downloads in the millions, last publish is a patch release (not a brand-new package), and repository points to github.com/cthackers/adm-zip. + 2. Open https://github.com/cthackers/adm-zip — confirm it is an established, maintained repo (stars, issue history, MIT license). + 3. Optionally check the Socket.dev / Snyk score for adm-zip@0.6.0. + 4. Confirm `fast-xml-parser` and `csv-parse` are already approved in 10-RESEARCH.md (carried over from STACK.md — no re-flag needed). + + + - Human types "approved" after confirming adm-zip is a legitimate, established package, OR names an alternative (e.g. `unzipper`) if not. + - This checkpoint is NOT auto-approvable regardless of workflow.auto_advance (supply-chain gate). + + Type "approved" to proceed with the install, or name a replacement ZIP library. + + + + Task 2: Install ingestion packages + add Tender/TenderSourcePollConfig models + + - apps/api/prisma/schema.prisma (DkvModuleConfig model at ~line 178 for field/convention reference; datasource + generator at top) + - apps/api/prisma/migrations/20260714090000_add_ldap_user_exclude_list/migration.sql (handwritten migration style: IF NOT EXISTS guards, timestamped folder) + - .planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-PATTERNS.md (schema.prisma section — exact model field list) + + apps/api/package.json, apps/api/prisma/schema.prisma, apps/api/prisma/migrations/20260721120000_add_tender_radar/migration.sql + + Run `pnpm --filter @tessera/api add fast-xml-parser adm-zip csv-parse` (only after Task 1 approval). + + Add two models to `apps/api/prisma/schema.prisma` following the 10-PATTERNS.md schema section exactly: + + `Tender` — global reference table, deliberately NO `tenantId` and NO `@@index([tenantId])` (D-03: platform-global data). Fields: `id String @id @default(uuid())`, `sourcePortal String`, `sourceNoticeId String`, `ocid String?`, `dedupKey String @unique`, `title String`, `buyerName String?`, `cpvCodes String[] @default([])`, `region String?`, `plz String?`, `bundesland String?`, `deadlineAt DateTime?` (frequently null per RESEARCH Pattern 4 — nullable is mandatory, not optional hardening), `estimatedValue Decimal? @db.Decimal(14,2)`, `procedureType String?`, `status String @default("active")` (values `active` | `expired`; supports D-05 retention marking), `sourceUrl String?`, `contentHash String` (SCHEMA-02 change detection), `rawPayload Json?` (debugging/re-normalization), `publishedAt DateTime`, `createdAt DateTime @default(now())`, `updatedAt DateTime @updatedAt`. Indexes: `@@index([status])`, `@@index([deadlineAt])`, `@@index([publishedAt])`. + + `TenderSourcePollConfig` — singleton-per-source admin config (INGEST-06 foundation). Fields: `id String @id @default(uuid())`, `sourceType String @unique` (fixed slug, `doe-opendata` — the `@unique` makes the singleton intent explicit, mirroring DkvModuleConfig.tenantId @unique), `pollIntervalMin Int @default(60)` (D-04 default hourly), `isActive Boolean @default(false)`, `lastIngestedDay DateTime?` (day-cursor, NOT a timestamp — RESEARCH Pattern 1), `createdAt DateTime @default(now())`, `updatedAt DateTime @updatedAt`. + + Write the handwritten migration at `apps/api/prisma/migrations/20260721120000_add_tender_radar/migration.sql` mirroring the ldap-user-exclude-list migration style: `CREATE TABLE IF NOT EXISTS "Tender" (...)` and `CREATE TABLE IF NOT EXISTS "TenderSourcePollConfig" (...)` with matching column types, plus `CREATE UNIQUE INDEX IF NOT EXISTS` on `Tender.dedupKey` and `TenderSourcePollConfig.sourceType`, and the three `Tender` indexes. + + Run `pnpm --filter @tessera/api exec prisma generate` so the Prisma client TS types for `Tender`/`TenderSourcePollConfig` are available to downstream service tasks. + + + cd apps/api && grep -c "model Tender" prisma/schema.prisma && grep -c "model TenderSourcePollConfig" prisma/schema.prisma && test -f prisma/migrations/20260721120000_add_tender_radar/migration.sql && pnpm exec tsc --noEmit -p tsconfig.json 2>&1 | tail -5 + + + - `grep -c "model Tender"` and `grep -c "model TenderSourcePollConfig"` in schema.prisma each return >= 1. + - Migration SQL file exists at the timestamped path. + - `Tender` has no `tenantId` column (assert: `grep -c "tenantId" ` within the Tender model block returns 0). + - `prisma generate` succeeds and `import { Tender } from '@prisma/client'` type-checks. + + Both models present with correct fields, migration SQL written, Prisma client regenerated, packages installed. + + + + Task 3: [BLOCKING] Apply the tender-radar migration to the database + + - apps/api/prisma/migrations/migration_lock.toml (confirms postgresql provider + migrate workflow) + - apps/api/prisma/migrations/20260721120000_add_tender_radar/migration.sql (the migration written in Task 2) + + apps/api/prisma/migrations/20260721120000_add_tender_radar/migration.sql + + [BLOCKING] Apply the handwritten migration to the live dev database so the Prisma client's compile-time types are backed by real tables (build/type checks pass WITHOUT this step because types come from the generated client, not the DB — this creates a false-positive verification state, so this task is mandatory before any ingestion/persistence verification). + + Run `pnpm --filter @tessera/api exec prisma migrate deploy` (non-interactive; applies the committed handwritten migration). If `migrate deploy` reports drift or a non-empty schema conflict that requires interactive resolution, fall back to `pnpm --filter @tessera/api exec prisma db push` (schema is additive — two new tables, no destructive change expected). Then confirm the tables exist. + + + cd apps/api && pnpm exec prisma migrate status 2>&1 | tail -8 + + + - `prisma migrate status` reports the database schema is up to date (no pending migrations), OR a follow-up `prisma db push` reports "already in sync". + - Tables `Tender` and `TenderSourcePollConfig` exist in the database (verifiable via `prisma migrate status` clean state or a `prisma.tender.count()` call returning 0 without error). + + Migration applied; both tables live in the database; no pending migrations. + + + + + +## Trust Boundaries + +| Boundary | Description | +|----------|-------------| +| npm registry → build | Third-party package code (`adm-zip`) enters the trusted build via `pnpm add` | +| ORM → Postgres | New global tables added; tenant-isolation design decision is encoded in schema shape | + +## STRIDE Threat Register + +| Threat ID | Category | Component | Severity | Disposition | Mitigation Plan | +|-----------|----------|-----------|----------|-------------|-----------------| +| T-10-SC | Tampering | `pnpm add adm-zip` (supply chain) | high | mitigate | Task 1 blocking-human checkpoint verifies adm-zip on npmjs.com + github before install; fast-xml-parser/csv-parse pre-approved in RESEARCH | +| T-10-01 | Information Disclosure | `Tender` table schema | high | mitigate | `Tender` carries NO `tenantId` by design (D-03) — it is platform-global reference data; per-tenant scoping lives one layer up (Phase 11 saved searches). Encoding this now prevents a later blanket `where: { tenantId }` habit from silently hiding global data from a second tenant | +| T-10-02 | Denial of Service | `Decimal`/`String[]` columns | low | accept | Standard Postgres column types; ingestion-side size guards handled in Plan 03 (zip-bomb) — no DB-level risk introduced by the schema itself | + + + +- `prisma migrate status` clean; both tables present. +- `Tender` model has no `tenantId` field (design invariant for D-03 / multi-tenant global data). +- Three packages resolvable: `pnpm --filter @tessera/api exec node -e "require('adm-zip');require('fast-xml-parser');require('csv-parse/sync')"`. + + + +- SCHEMA-01 data foundation: normalized OCDS-oriented `Tender` table exists with nullable deadline/value. +- Packages installed; migration applied; client regenerated. + + + +Create `.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-01-SUMMARY.md` when done. + diff --git a/.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-02-PLAN.md b/.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-02-PLAN.md new file mode 100644 index 0000000..eaa208d --- /dev/null +++ b/.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-02-PLAN.md @@ -0,0 +1,162 @@ +--- +phase: 10-ausschreibungs-radar-foundation-d-e-ingestion +plan: 02 +type: execute +wave: 2 +depends_on: ["10-01"] +files_modified: + - apps/api/src/tenders/tenders.module.ts + - apps/api/src/tenders/tenders.seed.ts + - apps/api/src/tenders/tenders.seed.spec.ts + - apps/api/src/app.module.ts + - apps/web/src/lib/module-loader.ts + - apps/web/src/app/(portal)/modules/tender-radar/page.tsx +autonomous: true +requirements: [CONFIG-01] +must_haves: + truths: + - "Admin can find 'Ausschreibungs-Radar' in the marketplace and activate it for a tenant, same as DKV/Cert-Manager (CONFIG-01)" + - "Activating the module for a tenant opens the module page (no 404) because the module-loader whitelist entry exists" + - "The singleton 'doe-opendata' TenderSourcePollConfig row is seeded so the shared poll is admin-drivable" + artifacts: + - apps/api/src/tenders/tenders.module.ts + - apps/api/src/tenders/tenders.seed.ts + - apps/web/src/lib/module-loader.ts + - apps/web/src/app/(portal)/modules/tender-radar/page.tsx + key_links: + - "TendersModule.onModuleInit() calls seedTendersModule() → ModuleRegistryService.seedModule({ slug: 'tender-radar', isSystem: true })" + - "module-loader.ts MODULE_REGISTRY['tender-radar'] entry is MANDATORY — page 404s without it even when activated" +--- + + +Deliver the first user-facing vertical slice: register the Ausschreibungs-Radar module in the marketplace (`isSystem: true`, admin-activatable per tenant) and make its portal page load when activated. Seed the singleton `doe-opendata` poll-config row so later plans have a config to drive. + +## Phase Goal (user story) +**As a** Tessera-Administrator, **I want to** das Ausschreibungs-Radar-Modul im Marketplace aktivieren, **so that** ich Zugang zum Ausschreibungs-Katalog fuer einen Mandanten freischalten kann — genau wie beim DKV-Fleet- und Cert-Manager-Modul (CONFIG-01). + +Purpose: After this plan a real admin can DO something new: find the module in the marketplace, activate it for a tenant, and open its page. This is the CONFIG-01 slice end-to-end (registry seed → marketplace → activation → lazy-loaded page). +Output: Self-seeding `TendersModule`, module-loader whitelist entry, minimal placeholder page, seeded singleton poll config. + + + +@$HOME/.claude/gsd-core/workflows/execute-plan.md +@$HOME/.claude/gsd-core/templates/summary.md + + + +@.planning/PROJECT.md +@.planning/ROADMAP.md +@.planning/STATE.md +@.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-PATTERNS.md +@apps/api/src/dkv/dkv.module.ts +@apps/api/src/cert-manager/cert-manager.seed.ts +@apps/api/src/module-registry/module-registry.service.ts +@apps/web/src/lib/module-loader.ts + + + + + + Task 1: TendersModule skeleton + registry self-seed + singleton poll config + + - apps/api/src/dkv/dkv.module.ts (exact OnModuleInit self-seed pattern to copy) + - apps/api/src/cert-manager/cert-manager.seed.ts (preferred seed analog — isSystem:true + "admin must activate via Marketplace" phrasing matches CONFIG-01) + - apps/api/src/module-registry/module-registry.service.ts (seedModule upsert-by-slug signature, lines ~158-187) + - apps/api/src/app.module.ts (imports array — add TendersModule alongside DkvModule/CertManagerModule) + + apps/api/src/tenders/tenders.module.ts, apps/api/src/tenders/tenders.seed.ts, apps/api/src/app.module.ts + + Create `tenders.seed.ts` exporting `seedTendersModule(moduleRegistryService)` that calls `moduleRegistryService.seedModule({ slug: 'tender-radar', name: 'Ausschreibungs-Radar', version: '1.0.0', category: 'procurement', description: { de: 'Deutsche Ausschreibungen automatisch erfassen und durchsuchen', en: 'Automatically track and search German public tenders' }, isSystem: true })` — mirror `cert-manager.seed.ts` verbatim in structure (CONFIG-01). Confirm `procurement` is acceptable as a new category value (seedModule does not constrain category); if the marketplace UI needs a known category, reuse an existing one and note it in the summary. + + Create `tenders.module.ts` implementing `OnModuleInit` exactly like `DkvModule`: `imports: [ModuleRegistryModule]`, `controllers: []` (added Plan 05), `providers: []` (services added Plans 03-04), and an `onModuleInit()` that calls `seedTendersModule(this.moduleRegistryService)` inside try/catch with a Logger. Also, in `onModuleInit()`, upsert the singleton poll config: `prisma.tenderSourcePollConfig.upsert({ where: { sourceType: 'doe-opendata' }, update: {}, create: { sourceType: 'doe-opendata', pollIntervalMin: 60, isActive: true } })` — inject `PrismaService` (PrismaModule is global). isActive defaults true so the platform-global DÖE poll (D-04 hourly) runs regardless of tenant activation; the day-cursor gate (Plan 04) makes this idempotent. Do NOT use `forTenant()` — this config and the Tender table are global (D-03). + + Add `TendersModule` to the `imports` array in `app.module.ts`. + + + cd apps/api && pnpm exec tsc --noEmit -p tsconfig.json 2>&1 | tail -5 && grep -c "TendersModule" src/app.module.ts + + + - `tenders.seed.ts` seeds slug `tender-radar` with `isSystem: true`. + - `TendersModule` is imported in `app.module.ts` (`grep -c "TendersModule" src/app.module.ts` >= 2 — import line + array entry). + - Singleton `doe-opendata` config upserted on init (idempotent — re-boot does not create a second row; guaranteed by `sourceType @unique`). + - `tsc --noEmit` passes. + + Module self-seeds into the registry on boot and seeds the singleton poll config; app compiles. + + + + Task 2: Web module-loader whitelist entry + minimal portal page + + - apps/web/src/lib/module-loader.ts (MODULE_REGISTRY object — copy the 'cert-manager' entry shape) + - apps/web/src/app/(portal)/modules/cert-manager/page.tsx (page.tsx conventions: 'use client', useTranslations, default export) + + apps/web/src/lib/module-loader.ts, apps/web/src/app/(portal)/modules/tender-radar/page.tsx + + Add a `'tender-radar'` entry to `MODULE_REGISTRY` in `module-loader.ts` pointing at `dynamic(() => import('@/app/(portal)/modules/tender-radar/page'), { ssr: false })` — identical shape to the existing `'cert-manager'`/`'dkv-fleet'` entries. This whitelist entry is MANDATORY for CONFIG-01: the page 404s even when the module is activated if the slug is not whitelisted (security-motivated whitelist, module-loader header comment). + + Create a minimal `tender-radar/page.tsx` client component (default export, `'use client'`) that renders the module title and a short "Ausschreibungen werden erfasst" placeholder. Real results UI is Phase 11 — this is the thinnest page that proves the activation → page-load path. Use a hardcoded German string here (full i18n is CONFIG-03, Phase 14); note this as an intentional MVP stub in the summary. + + + cd apps/web && grep -c "tender-radar" src/lib/module-loader.ts && test -f "src/app/(portal)/modules/tender-radar/page.tsx" && pnpm exec tsc --noEmit 2>&1 | tail -5 + + + - `MODULE_REGISTRY['tender-radar']` entry present (`grep -c "tender-radar" module-loader.ts` >= 1). + - `tender-radar/page.tsx` exists and default-exports a component. + - Web `tsc --noEmit` passes. + + Activated module opens its page instead of 404; whitelist entry in place. + + + + Task 3: Seed registration test (Wave 0 coverage for CONFIG-01) + + - apps/api/src/cert-manager/cert-manager.service.spec.ts (existing vitest spec style in this repo) + - apps/api/src/tenders/tenders.seed.ts (unit under test, from Task 1) + + apps/api/src/tenders/tenders.seed.spec.ts + + Create `tenders.seed.spec.ts` (Vitest). Provide a mock `ModuleRegistryService` with a spied `seedModule`. Assert that `seedTendersModule(mock)` calls `seedModule` once with `slug: 'tender-radar'`, `isSystem: true`, and a description object containing both `de` and `en` keys. This is the Wave 0 automated proof for CONFIG-01 registration. + + + cd apps/api && pnpm test -- tenders.seed 2>&1 | tail -15 + + + - Test asserts `seedModule` called with `slug: 'tender-radar'` and `isSystem: true`. + - `pnpm --filter @tessera/api test -- tenders.seed` passes. + + Registration behaviour has automated coverage; green. + + + + + +## Trust Boundaries + +| Boundary | Description | +|----------|-------------| +| tenant → module activation | A tenant's activation must gate only visibility, not create a second data copy or leak another tenant's config | +| URL slug → dynamic import | Only whitelisted slugs may resolve to a component (module-loader security whitelist) | + +## STRIDE Threat Register + +| Threat ID | Category | Component | Severity | Disposition | Mitigation Plan | +|-----------|----------|-----------|----------|-------------|-----------------| +| T-10-03 | Elevation of Privilege | `module-loader.ts` dynamic import | medium | mitigate | Only the fixed `'tender-radar'` slug is whitelisted; arbitrary URL slugs return null (existing whitelist mechanism, unchanged) | +| T-10-04 | Information Disclosure | per-tenant module activation | high | mitigate | Activation uses the existing `ModuleRegistryService` / `TenantModuleActivation` mechanism unchanged; the singleton `doe-opendata` config is global (one row, `sourceType @unique`) so a 2nd tenant activation cannot create or read a per-tenant duplicate. No `forTenant()` applied to the global config | +| T-10-05 | Tampering | registry self-seed on boot | low | accept | `seedModule` is an idempotent upsert-by-slug; repeated boots do not duplicate the module row | + + + +- Module appears in the registry with slug `tender-radar` after boot (seed test green). +- Activating the module for a tenant opens `/modules/tender-radar` (whitelist entry present). +- Singleton poll config seeded exactly once (idempotent upsert). +- `pnpm --filter @tessera/api test` green; both apps `tsc --noEmit` clean. + + + +- CONFIG-01: admin can find and activate Ausschreibungs-Radar in the marketplace, same as DKV/Cert-Manager, and open its page. + + + +Create `.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-02-SUMMARY.md` when done. + diff --git a/.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-03-PLAN.md b/.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-03-PLAN.md new file mode 100644 index 0000000..50ad64a --- /dev/null +++ b/.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-03-PLAN.md @@ -0,0 +1,185 @@ +--- +phase: 10-ausschreibungs-radar-foundation-d-e-ingestion +plan: 03 +type: tdd +wave: 3 +depends_on: ["10-01", "10-02"] +files_modified: + - apps/api/src/tenders/__fixtures__/doe-eforms-sample.zip + - apps/api/src/tenders/__fixtures__/doe-ocds-sample.zip + - apps/api/src/tenders/tender.types.ts + - apps/api/src/tenders/adapters/tender-source-adapter.interface.ts + - apps/api/src/tenders/adapters/doe-opendata.adapter.ts + - apps/api/src/tenders/adapters/doe-opendata.adapter.spec.ts + - apps/api/src/tenders/tender-normalizer.service.ts + - apps/api/src/tenders/tender-normalizer.service.spec.ts + - apps/api/src/tenders/tenders.module.ts +autonomous: true +requirements: [INGEST-01, SCHEMA-01] +must_haves: + truths: + - "DoeOpenDataAdapter fetches a day's DÖE export ZIP, extracts it, and parses eForms-DE XML (primary) + OCDS JSON (ocid) into RawTenderRecord[] (INGEST-01)" + - "Only open tenders survive the D-02 filter: tag=['tender'] included; award/planning/untagged-with-awards excluded" + - "TenderNormalizer maps a real eForms+OCDS notice pair into Tender fields with nullable deadline/value, a stable dedupKey (ocid → sourcePortal:noticeId), and a contentHash (SCHEMA-01)" + artifacts: + - apps/api/src/tenders/adapters/doe-opendata.adapter.ts + - apps/api/src/tenders/tender-normalizer.service.ts + - apps/api/src/tenders/__fixtures__/doe-eforms-sample.zip + key_links: + - "Adapter uses native fetch + AbortController timeout (icon-discovery idiom); HTTP 400 = expected no-op (today/future pubDay), 5xx/network throws" + - "dedupKey (ocid, else sourcePortal:sourceNoticeId) is the @unique upsert target consumed by Plan 04 ingestion" +--- + + +Build the DÖE source adapter and the normalizer — the parse+map core of INGEST-01/SCHEMA-01 — test-first against REAL captured fixture ZIPs (not synthetic data), per this repo's real-fixture precedent (DKV PDF parser). + +## Phase Goal (user story) +**As a** Tessera-Administrator, **I want to** dass echte DÖE-Ausschreibungen aus dem Tages-Export korrekt gelesen und in ein einheitliches Schema normalisiert werden, **so that** nur offene, bietbare Vergaben (D-02) mit stabilen Dedup-Schluesseln im globalen Katalog landen. + +Purpose: This is the structurally hardest part of the phase — correctly interpreting the DÖE day-batch ZIP shape, choosing eForms-DE XML as the primary structured-field source (deadline/value that OCDS drops), and applying the tag-presence D-02 filter. Both units are pure/deterministic → ideal for TDD. +Output: `DoeOpenDataAdapter`, `TenderNormalizerService`, the adapter interface, shared types, and committed real fixtures. + + + +@$HOME/.claude/gsd-core/workflows/execute-plan.md +@$HOME/.claude/gsd-core/templates/summary.md + + + +@.planning/PROJECT.md +@.planning/ROADMAP.md +@.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-RESEARCH.md +@.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-PATTERNS.md +@apps/api/src/favorites/icon-discovery.service.ts +@apps/api/src/dkv/dkv-parser.service.ts + + + + + + Task 1: Wave 0 — capture real DÖE fixtures + shared types + adapter interface + failing specs + + - .planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-RESEARCH.md (DÖE API Live Findings: endpoint, pubDay boundary rules, tag distribution, eForms-vs-OCDS field cross-check) + - apps/api/src/dkv/dkv.types.ts (types file conventions) + - .planning/research/ARCHITECTURE.md (Pattern 1: RawTenderRecord / TenderSourceAdapter interface sketch) + + apps/api/src/tenders/__fixtures__/doe-eforms-sample.zip, apps/api/src/tenders/__fixtures__/doe-ocds-sample.zip, apps/api/src/tenders/tender.types.ts, apps/api/src/tenders/adapters/tender-source-adapter.interface.ts, apps/api/src/tenders/adapters/doe-opendata.adapter.spec.ts, apps/api/src/tenders/tender-normalizer.service.spec.ts + + Capture REAL fixtures: compute a pubDay at least 2 calendar days in the past (Europe/Berlin), then download `https://oeffentlichevergabe.de/api/notice-exports?pubDay={day}&format=eforms.zip` and `...&format=ocds.zip`. Trim each archive to ~5-10 representative notices spanning the tag classes RESEARCH observed (`tender`, `award`, `planning`, and at least one untagged-with-awards) and save the trimmed ZIPs as `doe-eforms-sample.zip` / `doe-ocds-sample.zip` under `__fixtures__/`. Keep them small (do not commit the full 105-file archive). If the host is unreachable from the execution environment, construct minimal fixtures faithfully reproducing the real element shapes documented in RESEARCH (eForms `TenderSubmissionDeadlinePeriod/EndDate`, OCDS `releases[].tag`, `ocid` `ocds-mnwr74-...`) and note the substitution in the summary. + + Define `tender.types.ts`: `SourceType` (`'doe-opendata'` for this phase), `RawTenderRecord` (sourceType, sourcePortal, sourceNoticeId, ocid?, sourceUrl, fetchedAt, eformsPayload/ocdsPayload or a unified payload, publishedAt), and `NormalizedTenderFields` (the Tender column subset the normalizer emits, plus dedupKey + contentHash). + + Define `tender-source-adapter.interface.ts`: `TenderSourceAdapter` with `readonly sourceType: SourceType` and `fetchTenders(dayCursor: string): Promise` (day-cursor string `YYYY-MM-DD`, NOT a `since: Date` — RESEARCH Pattern 1 corrects ARCHITECTURE.md's signature). + + Write FAILING specs first (RED): `doe-opendata.adapter.spec.ts` loads the fixture ZIPs and asserts (a) N raw records parsed, (b) the D-02 filter keeps only `tag=['tender']` and drops award/planning/untagged-with-awards (assert exact filtered count), (c) HTTP 400 fetch path returns `[]` (no-op). `tender-normalizer.service.spec.ts` asserts a known eForms+OCDS notice pair maps to the expected Tender fields including a non-null deadline recovered from eForms XML where OCDS is null, nullable value handled, and dedupKey = the notice's `ocid`. + + + cd apps/api && test -f src/tenders/__fixtures__/doe-eforms-sample.zip && pnpm test -- doe-opendata.adapter tender-normalizer 2>&1 | tail -20 + + + - Fixture ZIPs exist under `__fixtures__/`. + - `tender.types.ts` and `tender-source-adapter.interface.ts` compile. + - Both spec files exist and FAIL for the right reason (implementation not yet written) — RED state confirmed. + + Real fixtures captured, types + interface defined, failing specs committed (test(...) commit). + + + + Task 2: DoeOpenDataAdapter — fetch + adm-zip extract + parse + D-02 filter (GREEN) + + - apps/api/src/favorites/icon-discovery.service.ts (native fetch + AbortController timeout idiom, lines ~254-287) + - .planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-PATTERNS.md (doe-opendata.adapter section: fetchDoeDay shape, 400=no-op) + - .planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-RESEARCH.md (Security Domain: zip-bomb ceiling; D-02 isOpenTenderNotice) + - apps/api/src/tenders/adapters/doe-opendata.adapter.spec.ts (the RED spec from Task 1) + + + - Test: fetching a valid past pubDay returns RawTenderRecord[] parsed from the fixture ZIP. + - Test: a pubDay yielding HTTP 400 (today/future) returns [] without throwing. + - Test: D-02 filter keeps tag=['tender'], excludes award/planning/untagged — assert exact retained count for the fixture. + - Test: total uncompressed ZIP size over the ceiling (e.g. 50MB) is rejected before extraction (zip-bomb guard). + + apps/api/src/tenders/adapters/doe-opendata.adapter.ts, apps/api/src/tenders/tenders.module.ts + + Implement `DoeOpenDataAdapter` (Injectable) with `sourceType = 'doe-opendata'` and `fetchTenders(dayCursor)`: + - Fetch `eforms.zip` and `ocds.zip` for the given `dayCursor` via native `fetch` wrapped in an `AbortController` + 15s `setTimeout` (icon-discovery idiom — no axios). Treat HTTP 400 as an expected no-op signal (return `[]`); any other `!res.ok` throws. + - Extract with `adm-zip`: `new AdmZip(buffer).getEntries()`. BEFORE reading entry buffers, sum `entry.header.size` across entries and reject if it exceeds a sane ceiling (~50MB) — decompression-bomb defence-in-depth (RESEARCH Security Domain). Zip-slip is not a write risk here (in-memory only, never writing entries to disk) — do not write extracted entries to the filesystem. + - Parse each eForms-DE XML entry with `fast-xml-parser` (primary structured fields); parse the paired OCDS JSON entries with `JSON.parse` for `ocid` and buyer/party names. Pair eForms↔OCDS by notice id. + - Apply the D-02 open-tender filter on the OCDS release tag: keep when `release.tag` includes `'tender'`; exclude award, planning, and untagged-with-awards notices (positive tag-presence match per D-02 / RESEARCH Pattern 2 — never "absence of award = open"). Ingest is whole-Germany with NO region/CPV pre-filtering (D-03). + - Return `RawTenderRecord[]`. + Register `DoeOpenDataAdapter` in `TendersModule.providers`. + + + cd apps/api && pnpm test -- doe-opendata.adapter 2>&1 | tail -15 + + + - All `doe-opendata.adapter.spec.ts` tests pass (GREEN). + - D-02 retained-count assertion matches the fixture's `tender`-tagged subset exactly. + - Zip-bomb ceiling test passes (oversized archive rejected pre-extraction). + - Uses native fetch (assert no axios import). + + Adapter parses fixtures into filtered RawTenderRecord[]; spec green; feat(...) commit. + + + + Task 3: TenderNormalizerService — fields + dedupKey + contentHash (GREEN) + + - apps/api/src/dkv/dkv-parser.service.ts (transform-service shape: one public entry point + private helpers, no I/O) + - .planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-PATTERNS.md (tender-normalizer.service section) + - .planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-RESEARCH.md (Pattern 3 eForms-primary; Pattern 4 nullable deadline/value; dedup key priority) + - apps/api/src/tenders/tender-normalizer.service.spec.ts (the RED spec from Task 1) + + + - Test: eForms `TenderSubmissionDeadlinePeriod/EndDate` populates `deadlineAt` even when OCDS deadline is null. + - Test: missing deadline/value normalize to null (not thrown, not zero). + - Test: dedupKey = ocid when present; falls back to `${sourcePortal}:${sourceNoticeId}` when ocid absent. + - Test: contentHash is a stable sha256 over title+deadline+value+status; changing the deadline changes the hash. + + apps/api/src/tenders/tender-normalizer.service.ts, apps/api/src/tenders/tenders.module.ts + + Implement `TenderNormalizerService` (Injectable, pure — no I/O) with `normalize(raw: RawTenderRecord): NormalizedTenderFields`. Map eForms-DE XML as the PRIMARY source for `deadlineAt`, `estimatedValue`, `procedureType` (Pattern 3 — reverses ARCHITECTURE.md's OCDS-first guidance based on live cross-check); use OCDS for `ocid`, `buyerName`, party names. Extract `cpvCodes`, `region`/`plz`/`bundesland`, `title`, `sourceUrl`, `publishedAt`. Keep `deadlineAt`/`estimatedValue` nullable (Pattern 4 — null is the common case for Unterschwelle). Compute `dedupKey` priority-ordered: `ocid` → `${sourcePortal}:${sourceNoticeId}` (fuzzy fingerprint is Phase 13, out of scope here). Compute `contentHash = sha256(title + deadlineAt + estimatedValue + status)` (SCHEMA-02 hook). Set initial `status = 'active'`. Register the service in `TendersModule.providers`. + + + cd apps/api && pnpm test -- tender-normalizer 2>&1 | tail -15 + + + - All `tender-normalizer.service.spec.ts` tests pass (GREEN). + - Deadline recovered from eForms XML where OCDS is null (the key SCHEMA-01 assertion). + - dedupKey falls back correctly when ocid absent; contentHash changes when deadline changes. + + Normalizer maps notice pairs to Tender fields deterministically; spec green; feat(...) commit. + + + + + +## Trust Boundaries + +| Boundary | Description | +|----------|-------------| +| DÖE host → API process | Outbound HTTP to a fixed external government host; untrusted archive + XML/JSON content enters the Node process | +| ZIP archive → extraction | Compressed third-party payload decompressed in-memory | + +## STRIDE Threat Register + +| Threat ID | Category | Component | Severity | Disposition | Mitigation Plan | +|-----------|----------|-----------|----------|-------------|-----------------| +| T-10-06 | Server-Side Request Forgery | `DoeOpenDataAdapter` outbound fetch | high | mitigate | URL host is a hardcoded constant (`oeffentlichevergabe.de`); only the internally-computed `pubDay` cursor is interpolated (never user input). Native fetch + AbortController 15s timeout bounds the request (icon-discovery idiom). No admin-supplied URL flows into the fetch | +| T-10-07 | Denial of Service | `adm-zip` extraction | high | mitigate | Sum `entry.header.size` and reject archives over a ~50MB uncompressed ceiling BEFORE reading entry buffers (decompression-bomb guard). Entries are read in-memory only — never written to disk, so zip-slip path traversal is structurally absent | +| T-10-08 | Tampering / Injection | parsed eForms/OCDS text fields | medium | mitigate | All ingested text (buyer names, titles, descriptions) is external untrusted input; store raw, do NOT assume "safe" because it came from a government API. Escaping happens at render time (Phase 11 UI) — the normalizer must not emit HTML | + + + +- `pnpm --filter @tessera/api test -- doe-opendata.adapter tender-normalizer` all green. +- D-02 filter proven by exact retained-count assertion on real fixtures. +- eForms-primary deadline recovery proven (deadline present where OCDS is null). +- No axios import; zip-bomb ceiling enforced. + + + +- INGEST-01: DÖE day-export ZIP is fetched, extracted, and parsed into filtered open-tender raw records. +- SCHEMA-01: raw records normalize into the unified Tender field shape with stable dedupKey + contentHash. + + + +Create `.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-03-SUMMARY.md` when done. + diff --git a/.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-04-PLAN.md b/.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-04-PLAN.md new file mode 100644 index 0000000..97f5038 --- /dev/null +++ b/.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-04-PLAN.md @@ -0,0 +1,173 @@ +--- +phase: 10-ausschreibungs-radar-foundation-d-e-ingestion +plan: 04 +type: execute +wave: 4 +depends_on: ["10-01", "10-02", "10-03"] +files_modified: + - apps/api/src/tenders/tender-ingestion.service.ts + - apps/api/src/tenders/tender-ingestion.service.spec.ts + - apps/api/src/tenders/tender-scheduler.service.ts + - apps/api/src/tenders/tender-scheduler.service.spec.ts + - apps/api/src/tenders/tenders.module.ts +autonomous: true +requirements: [SCHEMA-02, INGEST-06] +must_haves: + truths: + - "A changed DÖE notice (same ocid/dedupKey, new contentHash) updates the existing Tender row instead of creating a duplicate (SCHEMA-02)" + - "DÖE is polled once on a shared global schedule regardless of tenant count — poll-once-fan-out-many, never per-tenant (INGEST-06)" + - "Activating the module for a 2nd tenant triggers zero additional DÖE HTTP calls, zero additional cron jobs, and zero additional Tender rows (Success Criteria 4 & 5)" + - "The poll tick is day-cursor gated: no upstream fetch when dayCursor >= today Europe/Berlin (D-01 from-now, no historical backfill)" + - "Tenders past their deadline are marked expired and pruned after 90 days; deadline-less rows are never auto-expired (D-05)" + artifacts: + - apps/api/src/tenders/tender-ingestion.service.ts + - apps/api/src/tenders/tender-scheduler.service.ts + key_links: + - "Scheduler cron tick → TenderIngestionService.pollDueSources(); day-cursor gate lives in the ingestion service, not the scheduler" + - "prisma.tender.upsert({ where: { dedupKey } }) is the SCHEMA-02 change-detection seam; plain PrismaService (no forTenant) on the global table" +--- + + +Wire the ingestion orchestration and the shared scheduler — the poll-once-fan-out-many heart of the phase (INGEST-06) and the content-hash change detection (SCHEMA-02) — and prove the two-tenant safety property that is the phase's headline acceptance criterion. + +## Phase Goal (user story) +**As a** Tessera-Administrator, **I want to** dass DÖE genau einmal plattformweit auf einem admin-konfigurierbaren Intervall abgefragt wird — egal wie viele Mandanten das Modul aktiviert haben, **so that** kein redundanter Poll und keine doppelte Ingestion pro Mandant entsteht (INGEST-06, Success Criteria 4 & 5). + +Purpose: This plan turns "parse a fixture" (Plan 03) into "the platform continuously ingests real DÖE data safely." The multi-tenant scheduler is where the DKV `findFirst()` single-tenant anti-pattern must NOT be copied; the DÖE config is a genuine platform-wide singleton (RESEARCH Pitfall D). +Output: `TenderIngestionService`, `TenderSchedulerService`, and the two-tenant integration test. + + + +@$HOME/.claude/gsd-core/workflows/execute-plan.md +@$HOME/.claude/gsd-core/templates/summary.md + + + +@.planning/PROJECT.md +@.planning/ROADMAP.md +@.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-RESEARCH.md +@.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-PATTERNS.md +@apps/api/src/dkv/dkv-scheduler.service.ts +@apps/api/src/prisma/prisma-tenant.extension.ts + + + + + + Task 1: TenderIngestionService — day-cursor gate + upsert + change-detect + retention + + - .planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-PATTERNS.md (tender-ingestion.service section: pollDueSources shape, findUnique on sourceType, NO forTenant) + - .planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-RESEARCH.md (Pattern 1 day-cursor gate + nextDayToFetch; Pitfall A check-vs-fetch; Pattern 4 D-05 null-deadline handling) + - apps/api/src/dkv/dkv.service.ts (orchestration shape: fetch → normalize → persist → log; catch-and-log) + - apps/api/src/prisma/prisma-tenant.extension.ts (forTenant — the extension to AVOID here) + + + - Test: when dayCursor >= today (Europe/Berlin), pollDueSources() makes NO adapter call and returns (no-op tick, expected — Pitfall A). + - Test: a fresh notice is inserted; re-ingesting the identical notice (same dedupKey, same contentHash) does NOT create a duplicate and does NOT change field data. + - Test: re-ingesting a changed notice (same dedupKey, new contentHash — e.g. extended deadline) UPDATES the existing row (SCHEMA-02). + - Test: after a successful fetch, lastIngestedDay advances by one day; catch-up loops from lastIngestedDay+1 up to today-1. + - Test: pruneExpiredTenders() marks status='expired' for rows with deadlineAt < now, deletes rows expired+deadlineAt older than 90 days, and NEVER touches rows with deadlineAt = null (D-05). + + apps/api/src/tenders/tender-ingestion.service.ts, apps/api/src/tenders/tender-ingestion.service.spec.ts, apps/api/src/tenders/tenders.module.ts + + Write RED spec first, then implement `TenderIngestionService` (Injectable) using the plain global `PrismaService` — do NOT call `forTenant()` on `Tender`/`TenderSourcePollConfig` (global RLS-exempt tables, D-03; wrapping them in the tenant RLS extension would make a 2nd tenant's session silently filter out platform data). + + `pollDueSources()`: load the singleton config via `prisma.tenderSourcePollConfig.findUnique({ where: { sourceType: 'doe-opendata' } })` (fixed-slug lookup makes the singleton intent explicit — NOT a per-tenant `findFirst()`). If not `isActive`, return. Compute `nextDay = nextDayToFetch(config.lastIngestedDay)` (RESEARCH day-cursor snippet: first eligible day is activation day per D-01 "from now"; returns null if nextDay is not strictly before Berlin-today). If null, log at debug level ("no new DÖE day yet") and return — this no-op is expected, NOT an error (Pitfall A). Otherwise loop the day-cursor from nextDay forward while still `< today`: for each day call `doeAdapter.fetchTenders(day)`, `normalizer.normalize()` each record, and `prisma.tender.upsert({ where: { dedupKey }, update: {...fields, contentHash}, create: {...} })`. Track newly-created / contentHash-changed ids (the delta — consumed by Phase 11 matching, not this phase). After each day, `update` config.lastIngestedDay. Between successive day fetches insert a ~1-2s polite delay (RESEARCH Open Question 1). Never throw out of the tick — catch-and-log per RESEARCH/DKV pattern. + + `pruneExpiredTenders()` (D-05 retention): mark `status='expired'` where `deadlineAt < now` AND `status='active'`; delete where `status='expired'` AND `deadlineAt < now-90d`. Rows with `deadlineAt IS NULL` are excluded from both operations (Pattern 4 recommendation (a) — a wrongly-deleted no-deadline tender is unrecoverable). Call `pruneExpiredTenders()` once per successful tick. + + + cd apps/api && pnpm test -- tender-ingestion 2>&1 | tail -20 + + + - Day-cursor no-op test green (no adapter call when nothing new). + - SCHEMA-02: changed notice updates in place, identical notice does not duplicate — both green. + - D-05: expired-marking + 90-day prune skips null-deadline rows — green. + - Service uses plain PrismaService (assert no `forTenant` call in the file). + + Ingestion orchestration with change detection + retention; spec green. + + + + Task 2: TenderSchedulerService — single global cron (poll-once-fan-out-many) + + - apps/api/src/dkv/dkv-scheduler.service.ts (REUSE the CronJob require()-resolution + SchedulerRegistry addCronJob/deleteCronJob mechanics; do NOT copy activeTenantId / per-tenant framing) + - .planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-PATTERNS.md (tender-scheduler.service section + ANTI-PATTERN note) + - .planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-RESEARCH.md (Pitfall D: the one legitimate singleton findUnique; two-tenant verification framing) + + apps/api/src/tenders/tender-scheduler.service.ts, apps/api/src/tenders/tenders.module.ts + + Implement `TenderSchedulerService` (Injectable, OnModuleInit) reusing DKV's cron mechanics: the `require('cron').CronJob` resolution workaround, and a `setInterval(intervalMin)` that removes any existing job then `schedulerRegistry.addCronJob(JOB_NAME, job)` with `JOB_NAME = 'tender-doe-poll'`. Cron expression computed exactly as DKV (`*/${min} * * * *` under 60, else `0 */${hours} * * *`). The tick calls `tenderIngestionService.pollDueSources().catch(logErr)`. + + CRITICAL deviations from DKV (poll-once-fan-out-many): the scheduler has NO `activeTenantId` field and `setInterval` takes NO tenant argument — there is exactly ONE global cron job for the whole platform (D-04 default hourly interval). `onModuleInit()` loads the singleton config via `findUnique({ where: { sourceType: 'doe-opendata' } })` (NOT `findFirst()`); if `isActive`, call `setInterval(config.pollIntervalMin)`, else register nothing. Provide `stopJob()` (stop+delete the single job). The day-cursor gate is NOT in the scheduler — the scheduler only sets cron-tick frequency; whether an HTTP call happens is decided inside `pollDueSources()` (Pitfall A separation). Register `TenderSchedulerService` in `TendersModule.providers`. + + + cd apps/api && pnpm exec tsc --noEmit -p tsconfig.json 2>&1 | tail -5 && grep -c "activeTenantId" src/tenders/tender-scheduler.service.ts + + + - Single named cron job `tender-doe-poll`; `setInterval` has no tenant parameter. + - No `activeTenantId` field (`grep -c "activeTenantId"` returns 0 — proves the per-tenant anti-pattern was not copied). + - `onModuleInit` uses `findUnique` on `sourceType`, not `findFirst`. + - `tsc --noEmit` passes. + + One global cron drives the shared DÖE poll; no per-tenant scheduling dimension. + + + + Task 3: Two-tenant safety integration test (Success Criteria 4 & 5) + + - .planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-RESEARCH.md (Pitfall D "Verification for this phase's two-tenant acceptance criterion" — assert ABSENCE of tenant-scaled behavior) + - apps/api/src/tenders/tender-scheduler.service.spec.ts (create) + - apps/api/src/module-registry/module-registry.service.ts (isModuleActive / activation mechanism the test drives) + + apps/api/src/tenders/tender-scheduler.service.spec.ts + + Write an integration test proving the poll-once-fan-out-many property. Spy on the DÖE adapter's `fetchTenders` (or on the underlying `fetch`) and on `schedulerRegistry.addCronJob`. Simulate activating the `tender-radar` module for tenant A, run/inspect scheduler init, capture counts. Then activate the module for a SECOND tenant B and assert: zero ADDITIONAL DÖE adapter/HTTP calls, zero additional cron jobs registered (still exactly one `tender-doe-poll`), and zero additional `Tender` rows created purely as a result of the 2nd activation. The assertion is the ABSENCE of tenant-count-scaled behavior (there are no per-tenant DÖE configs to iterate) — not correct per-tenant iteration. This is the phase's headline acceptance criterion. + + + cd apps/api && pnpm test -- tender-scheduler 2>&1 | tail -20 + + + - Test asserts: after a 2nd tenant activation → additional DÖE fetch calls == 0, additional cron jobs == 0, additional Tender rows == 0. + - `pnpm --filter @tessera/api test -- tender-scheduler` green. + + Two-tenant safety proven by an automated integration test. + + + + + +## Trust Boundaries + +| Boundary | Description | +|----------|-------------| +| background job → global DB | Scheduled ingestion writes to the platform-global Tender table with no tenant context | +| tenant activation → scheduler | A tenant activating the module must not alter platform-wide poll behavior | + +## STRIDE Threat Register + +| Threat ID | Category | Component | Severity | Disposition | Mitigation Plan | +|-----------|----------|-----------|----------|-------------|-----------------| +| T-10-09 | Information Disclosure | `TenderIngestionService` DB access | high | mitigate | Uses plain `PrismaService` (no `forTenant()`) on the global Tender/config tables by design; wrapping global tables in tenant RLS would silently hide platform data from a 2nd tenant. Verified by the no-forTenant assertion + two-tenant integration test | +| T-10-10 | Elevation of Privilege | scheduler per-tenant scaling | high | mitigate | Exactly one global cron job; `setInterval` takes no tenant arg; singleton config via `findUnique` on a fixed slug. Two-tenant integration test asserts zero additional jobs/calls/rows on a 2nd activation (Success Criteria 4 & 5) | +| T-10-11 | Tampering | outbound `pubDay` URL | medium | mitigate | The day-cursor is internally computed from `lastIngestedDay` (never user-supplied); `nextDayToFetch` bounds it to strictly-past days; no admin "fetch date X" feature added in this phase | +| T-10-12 | Denial of Service | catch-up loop hammering DÖE | low | mitigate | ~1-2s polite delay between successive day fetches during catch-up (RESEARCH Open Question 1); day-cursor gate prevents re-fetching the same day repeatedly | + + + +- `pnpm --filter @tessera/api test -- tender-ingestion tender-scheduler` all green. +- SCHEMA-02 change detection proven (update-not-duplicate on contentHash change). +- INGEST-06 poll-once proven (single global cron, two-tenant test). +- D-01 day-cursor no-op, D-05 retention with null-deadline skip proven. +- No `forTenant` / no `activeTenantId` in the tender services (grep gates green). + + + +- SCHEMA-02: a changed DÖE notice updates its existing record instead of duplicating. +- INGEST-06: DÖE polled once on a shared admin-configurable interval regardless of tenant count. +- Success Criteria 4 & 5: 2nd-tenant activation triggers no redundant poll / no duplicate ingestion. + + + +Create `.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-04-SUMMARY.md` when done. + diff --git a/.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-05-PLAN.md b/.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-05-PLAN.md new file mode 100644 index 0000000..2b65089 --- /dev/null +++ b/.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-05-PLAN.md @@ -0,0 +1,165 @@ +--- +phase: 10-ausschreibungs-radar-foundation-d-e-ingestion +plan: 05 +type: execute +wave: 5 +depends_on: ["10-01", "10-02", "10-03", "10-04"] +files_modified: + - apps/api/src/tenders/dto/source-config.dto.ts + - apps/api/src/tenders/dto/tender-query.dto.ts + - apps/api/src/tenders/tenders.controller.ts + - apps/api/src/tenders/tenders.controller.spec.ts + - apps/api/src/tenders/tenders.module.ts +autonomous: true +requirements: [INGEST-06] +must_haves: + truths: + - "Admin can read and update the shared DÖE poll interval / active state; saving pushes the change into the live scheduler without restart (INGEST-06 admin-configurable)" + - "GET /tenders list + detail is gated by module activation (ModuleGuard), NOT by tenantId row-filtering — the global catalog is visible to any tenant with the module active" + - "Admin source-config routes require ADMIN/SUPER_ADMIN roles" + artifacts: + - apps/api/src/tenders/tenders.controller.ts + - apps/api/src/tenders/dto/source-config.dto.ts + - apps/api/src/tenders/dto/tender-query.dto.ts + key_links: + - "PUT source-config → TenderSchedulerService.setInterval()/stopJob() applies the change live (DKV saveConfig pattern, minus the tenantId arg)" + - "GET /tenders uses @UseModule('tender-radar') ModuleGuard, never a where:{tenantId} filter" +--- + + +Round out INGEST-06: expose the admin-configurable shared poll interval and a read endpoint over the global tender catalog. This is the boundary where "tenant-gated" and "tenant-scoped" genuinely differ — the read must be gated by module activation but NOT row-filtered by tenant. + +## Phase Goal (user story) +**As a** Tessera-Administrator, **I want to** das DÖE-Poll-Intervall im Admin-Bereich einstellen und die erfassten Ausschreibungen ueber eine API abrufen, **so that** ich den gemeinsamen Zeitplan steuern kann und Phase 11 eine Trefferliste darauf aufbauen kann (INGEST-06). + +Purpose: Completes the admin-configurable half of INGEST-06 and gives Phase 11 a read surface. The controller intentionally diverges from DKV: list/detail are platform-global reads gated only by module licensing. +Output: `TendersController`, `SourceConfigDto`, `TenderQueryDto`, controller test. + + + +@$HOME/.claude/gsd-core/workflows/execute-plan.md +@$HOME/.claude/gsd-core/templates/summary.md + + + +@.planning/PROJECT.md +@.planning/ROADMAP.md +@.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-PATTERNS.md +@.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-RESEARCH.md +@apps/api/src/dkv/dkv.controller.ts +@apps/api/src/cert-manager/cert-manager.controller.ts +@apps/api/src/dkv/dto/dkv-config.dto.ts +@apps/api/src/dkv/dto/dkv-history.dto.ts + + + + + + Task 1: SourceConfigDto + TenderQueryDto + + - apps/api/src/dkv/dto/dkv-config.dto.ts (class-validator conventions: @IsOptional + @Min/@Max, DoS floor) + - apps/api/src/dkv/dto/dkv-history.dto.ts (pagination DTO: page/limit + @Type(() => Number)) + - .planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-PATTERNS.md (dto sections) + + apps/api/src/tenders/dto/source-config.dto.ts, apps/api/src/tenders/dto/tender-query.dto.ts + + `SourceConfigDto`: `pollIntervalMin?` (`@IsOptional @IsInt @Min(5) @Max(1440)` — same DoS floor as DkvConfigDto) and `isActive?` (`@IsOptional @IsBoolean`). Document in a comment that `pollIntervalMin` is the cron-tick frequency (D-04 default 60), decoupled from the day-granularity DÖE fetch (day-cursor gated in the ingestion service). + + `TenderQueryDto`: pagination `page?`/`limit?` copied verbatim from `dkv-history.dto.ts` (`@Type(() => Number)` coercion, `@Min(1)`, `limit @Max(100)`), plus an optional `status?` filter (`@IsOptional @IsIn(['active','expired'])`) defaulting to active-only at the query layer. No region/CPV filters here — rich filtering is Phase 11 (FILTER-*). + + + cd apps/api && pnpm exec tsc --noEmit -p tsconfig.json 2>&1 | tail -5 + + + - Both DTOs compile with class-validator decorators; numeric bounds present. + - `SourceConfigDto.pollIntervalMin` has `@Min(5)`. + + DTOs defined following the DKV validation conventions. + + + + Task 2: TendersController — global read (ModuleGuard) + admin source-config (Roles) + live scheduler apply + + - apps/api/src/cert-manager/cert-manager.controller.ts (@UseModule('...') ModuleGuard usage — the module-activation gate for global reads) + - apps/api/src/dkv/dkv.controller.ts (per-handler @Roles(ADMIN, SUPER_ADMIN) on config routes; saveConfig → scheduler.setInterval/stopJob pattern, lines ~76-90) + - .planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-PATTERNS.md (tenders.controller section — divergence from DKV) + + apps/api/src/tenders/tenders.controller.ts, apps/api/src/tenders/tenders.module.ts + + Create `TendersController` (`@Controller('modules/tender-radar')`). Read routes are GLOBAL (V4 divergence): `GET /` (list, paginated via `TenderQueryDto`) and `GET /:id` (detail) query the global `Tender` table via plain `PrismaService` with `where` built ONLY from the query DTO (status/pagination) — never a `where: { tenantId }` filter. Gate these reads with `@UseModule('tender-radar')` (ModuleGuard) so only tenants with the module active can read, without row-scoping the global catalog (RESEARCH V4: this is the one controller where tenant-gated differs from tenant-scoped). Return 404 via `NotFoundException` for a missing id. + + Admin source-config routes: `GET /source-config` and `PUT /source-config`, each decorated `@Roles(Role.ADMIN, Role.SUPER_ADMIN)` (per-handler, like DkvController). `GET` returns the singleton `doe-opendata` config. `PUT` (body `SourceConfigDto`) upserts the singleton config, then applies it live to the scheduler: if `isActive && pollIntervalMin` then call `tenderScheduler.setInterval(pollIntervalMin)` (NO tenant arg — divergence from DKV); if `isActive === false` then call `tenderScheduler.stopJob()`. This satisfies INGEST-06 "admin-configurable interval, applied without restart". + + Register `TendersController` in `TendersModule.controllers` and confirm `TenderSchedulerService`/`TenderIngestionService` are in providers so DI resolves. + + + cd apps/api && pnpm exec tsc --noEmit -p tsconfig.json 2>&1 | tail -5 && grep -c "UseModule" src/tenders/tenders.controller.ts + + + - `GET /` and `GET /:id` gated by `@UseModule('tender-radar')`; no `where: { tenantId }` in the controller (assert `grep -c "tenantId" tenders.controller.ts` returns 0 for read routes — the global catalog is not row-scoped). + - Both `source-config` routes carry `@Roles(Role.ADMIN, Role.SUPER_ADMIN)`. + - `PUT /source-config` calls `tenderScheduler.setInterval`/`stopJob` (no tenant arg). + - `tsc --noEmit` passes. + + Global read + admin config wired; scheduler updates live on save. + + + + Task 3: Controller test — global read not tenant-scoped + admin config applies to scheduler + + - apps/api/src/cert-manager/cert-manager.service.spec.ts (vitest spec style) + - apps/api/src/tenders/tenders.controller.ts (unit under test) + + + - Test: GET list returns global tenders with no tenantId filter passed to prisma (assert the prisma.tender.findMany where has no tenantId key). + - Test: PUT /source-config with isActive=true + pollIntervalMin=30 calls scheduler.setInterval(30) with a single argument. + - Test: PUT /source-config with isActive=false calls scheduler.stopJob(). + + apps/api/src/tenders/tenders.controller.spec.ts + + Write a Vitest spec mocking `PrismaService` and `TenderSchedulerService`. Assert the read path never adds a `tenantId` to the prisma `where`, and that saving source-config drives `setInterval(intervalMin)` / `stopJob()` exactly as the DKV analog does (minus the tenant argument). This is the automated proof for the INGEST-06 admin-config surface and the tenant-gated-not-scoped invariant. + + + cd apps/api && pnpm test -- tenders.controller 2>&1 | tail -15 + + + - Read-path test asserts no `tenantId` in the prisma `where`. + - Config-save tests assert `setInterval(n)` (single arg) and `stopJob()` are invoked. + - `pnpm --filter @tessera/api test -- tenders.controller` green. + + Controller behaviour has automated coverage; green. + + + + + +## Trust Boundaries + +| Boundary | Description | +|----------|-------------| +| client → GET /tenders | Any authenticated tenant user with the module active reads the global catalog | +| admin → PUT /source-config | Privileged mutation of the platform-wide poll schedule | + +## STRIDE Threat Register + +| Threat ID | Category | Component | Severity | Disposition | Mitigation Plan | +|-----------|----------|-----------|----------|-------------|-----------------| +| T-10-13 | Elevation of Privilege | `PUT /source-config` | high | mitigate | Both source-config routes require `@Roles(Role.ADMIN, Role.SUPER_ADMIN)` (per-handler, like DkvController); global JwtAuthGuard + RolesGuard enforce it | +| T-10-14 | Broken Access Control | `GET /tenders` gating | high | mitigate | Reads gated by `@UseModule('tender-radar')` ModuleGuard (tenant must have the module active) — deliberately NOT row-scoped by tenantId, since the catalog is platform-global (V4: tenant-gated ≠ tenant-scoped). Verified by the no-tenantId controller test | +| T-10-15 | Input Validation | `TenderQueryDto` / `SourceConfigDto` | medium | mitigate | class-validator bounds on pagination (`limit @Max(100)`) and interval (`@Min(5) @Max(1440)`) mitigate DoS via oversized queries / runaway poll frequency | + + + +- `pnpm --filter @tessera/api test -- tenders.controller` green. +- GET reads gated by ModuleGuard, not tenantId-filtered (grep + test). +- Admin source-config Roles-guarded; save applies to the live scheduler. +- `tsc --noEmit` clean for both apps. + + + +- INGEST-06: admin-configurable shared poll interval, applied to the live scheduler without restart, plus a global read surface for the ingested catalog. + + + +Create `.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-05-SUMMARY.md` when done. + diff --git a/.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-VALIDATION.md b/.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-VALIDATION.md index de640df..af70e6c 100644 --- a/.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-VALIDATION.md +++ b/.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-VALIDATION.md @@ -2,7 +2,7 @@ phase: 10 slug: ausschreibungs-radar-foundation-d-e-ingestion status: draft -nyquist_compliant: false +nyquist_compliant: true wave_0_complete: false created: 2026-07-21 --- @@ -17,20 +17,20 @@ created: 2026-07-21 | Property | Value | |----------|-------| -| **Framework** | {pytest 7.x / jest 29.x / vitest / go test / other} | -| **Config file** | {path or "none — Wave 0 installs"} | -| **Quick run command** | `{quick command}` | -| **Full suite command** | `{full command}` | -| **Estimated runtime** | ~10 seconds | +| **Framework** | Vitest 3.x (existing, `apps/api/vitest.config.ts`) | +| **Config file** | `apps/api/vitest.config.ts` (existing — not modified by this phase) | +| **Quick run command** | `pnpm --filter @tessera/api test -- tenders` | +| **Full suite command** | `pnpm --filter @tessera/api test` | +| **Estimated runtime** | ~10 seconds (scoped) | --- ## Sampling Rate -- **After every task commit:** Run `{quick run command}` -- **After every plan wave:** Run `{full suite command}` +- **After every task commit:** Run `pnpm --filter @tessera/api test -- tenders` +- **After every plan wave:** Run `pnpm --filter @tessera/api test` - **Before `/gsd-verify-work`:** Full suite must be green -- **Max feedback latency:** 10 seconds +- **Max feedback latency:** ~10 seconds (scoped run) --- @@ -38,7 +38,21 @@ created: 2026-07-21 | Task ID | Plan | Wave | Requirement | Threat Ref | Secure Behavior | Test Type | Automated Command | File Exists | Status | |---------|------|------|-------------|------------|-----------------|-----------|-------------------|-------------|--------| -| 10-01-01 | 01 | 1 | REQ-{XX} | T-10-01 / — | {expected secure behavior or "N/A"} | unit | `{command}` | ✅ / ❌ W0 | ⬜ pending | +| 10-01-01 | 01 | 1 | SCHEMA-01 | T-10-SC | adm-zip verified before install (supply chain) | manual-gate | checkpoint:human-verify (blocking-human) | n/a | ⬜ pending | +| 10-01-02 | 01 | 1 | SCHEMA-01 | T-10-01 | Tender has no tenantId (global data) | build | `grep -c "model Tender" schema.prisma`; `tsc --noEmit` | ❌ W0 | ⬜ pending | +| 10-01-03 | 01 | 1 | SCHEMA-01 | T-10-01 | [BLOCKING] migration applied to DB | integration | `pnpm --filter @tessera/api exec prisma migrate status` | ❌ W0 | ⬜ pending | +| 10-02-01 | 02 | 2 | CONFIG-01 | T-10-05 | idempotent registry seed + singleton config | build | `tsc --noEmit`; `grep -c "TendersModule" app.module.ts` | ❌ W0 | ⬜ pending | +| 10-02-02 | 02 | 2 | CONFIG-01 | T-10-03 | whitelisted slug only | build | `grep -c "tender-radar" module-loader.ts`; web `tsc --noEmit` | ❌ W0 | ⬜ pending | +| 10-02-03 | 02 | 2 | CONFIG-01 | T-10-04 | seed asserts slug + isSystem | unit | `pnpm --filter @tessera/api test -- tenders.seed` | ❌ W0 | ⬜ pending | +| 10-03-01 | 03 | 3 | INGEST-01/SCHEMA-01 | T-10-06/07 | real fixtures + failing specs (RED) | unit | `pnpm --filter @tessera/api test -- doe-opendata.adapter tender-normalizer` | ❌ W0 | ⬜ pending | +| 10-03-02 | 03 | 3 | INGEST-01 | T-10-06/07 | fixed host fetch + zip-bomb ceiling + D-02 filter | unit | `pnpm --filter @tessera/api test -- doe-opendata.adapter` | ❌ W0 | ⬜ pending | +| 10-03-03 | 03 | 3 | SCHEMA-01 | T-10-08 | eForms-primary deadline, nullable value, dedupKey/hash | unit | `pnpm --filter @tessera/api test -- tender-normalizer` | ❌ W0 | ⬜ pending | +| 10-04-01 | 04 | 4 | SCHEMA-02 | T-10-09/11 | day-cursor no-op; update-not-duplicate; D-05 prune skips null | integration | `pnpm --filter @tessera/api test -- tender-ingestion` | ❌ W0 | ⬜ pending | +| 10-04-02 | 04 | 4 | INGEST-06 | T-10-10 | single global cron, no activeTenantId | build | `tsc --noEmit`; `grep -c "activeTenantId" tender-scheduler.service.ts` == 0 | ❌ W0 | ⬜ pending | +| 10-04-03 | 04 | 4 | INGEST-06 | T-10-10 | 2nd-tenant activation → 0 extra calls/jobs/rows | integration | `pnpm --filter @tessera/api test -- tender-scheduler` | ❌ W0 | ⬜ pending | +| 10-05-01 | 05 | 5 | INGEST-06 | T-10-15 | DTO bounds (interval floor, limit cap) | build | `tsc --noEmit` | ❌ W0 | ⬜ pending | +| 10-05-02 | 05 | 5 | INGEST-06 | T-10-13/14 | ModuleGuard read (not tenant-scoped) + admin Roles | build | `tsc --noEmit`; `grep -c "UseModule" tenders.controller.ts` | ❌ W0 | ⬜ pending | +| 10-05-03 | 05 | 5 | INGEST-06 | T-10-14 | read where has no tenantId; save drives scheduler | unit | `pnpm --filter @tessera/api test -- tenders.controller` | ❌ W0 | ⬜ pending | *Status: ⬜ pending · ✅ green · ❌ red · ⚠️ flaky* @@ -46,11 +60,13 @@ created: 2026-07-21 ## Wave 0 Requirements -- [ ] `{tests/test_file.py}` — stubs for REQ-{XX} -- [ ] `{tests/conftest.py}` — shared fixtures -- [ ] `{framework install}` — if no framework detected +- [ ] `apps/api/src/tenders/__fixtures__/doe-eforms-sample.zip` + `doe-ocds-sample.zip` — real captured, trimmed fixtures (Plan 03 Task 1) +- [ ] `apps/api/src/tenders/tenders.seed.spec.ts` (Plan 02 Task 3) +- [ ] `apps/api/src/tenders/adapters/doe-opendata.adapter.spec.ts`, `tender-normalizer.service.spec.ts` (Plan 03 Task 1 — RED first) +- [ ] `apps/api/src/tenders/tender-ingestion.service.spec.ts`, `tender-scheduler.service.spec.ts` (Plan 04) +- [ ] `apps/api/src/tenders/tenders.controller.spec.ts` (Plan 05 Task 3) -*If none: "Existing infrastructure covers all phase requirements."* +Vitest framework already exists — no framework install needed. --- @@ -58,19 +74,18 @@ created: 2026-07-21 | Behavior | Requirement | Why Manual | Test Instructions | |----------|-------------|------------|-------------------| -| {behavior} | REQ-{XX} | {reason} | {steps} | - -*If none: "All phase behaviors have automated verification."* +| adm-zip package legitimacy | (supply chain / T-10-SC) | Human judgment on package provenance ([ASSUMED]) | Plan 01 Task 1 checkpoint: verify on npmjs.com + github.com/cthackers/adm-zip | +| Marketplace activation opens module page | CONFIG-01 | Full portal round-trip (marketplace → activate → page render) is a browser flow | Activate Ausschreibungs-Radar for a tenant, open `/modules/tender-radar`, confirm no 404 | --- ## Validation Sign-Off -- [ ] All tasks have `` verify or Wave 0 dependencies -- [ ] Sampling continuity: no 3 consecutive tasks without automated verify -- [ ] Wave 0 covers all MISSING references -- [ ] No watch-mode flags -- [ ] Feedback latency < 10s -- [ ] `nyquist_compliant: true` set in frontmatter +- [x] All tasks have `` verify or a documented Wave 0 / manual-gate dependency +- [x] Sampling continuity: no 3 consecutive tasks without automated verify +- [x] Wave 0 covers all MISSING references (fixtures + spec scaffolds) +- [x] No watch-mode flags (all use `vitest run` via `pnpm test`) +- [x] Feedback latency < 10s (scoped runs) +- [x] `nyquist_compliant: true` set in frontmatter -**Approval:** {pending / approved YYYY-MM-DD} +**Approval:** approved 2026-07-21