168 lines
13 KiB
Markdown
168 lines
13 KiB
Markdown
---
|
|
phase: 10-ausschreibungs-radar-foundation-d-e-ingestion
|
|
plan: 01
|
|
type: execute
|
|
wave: 1
|
|
depends_on: []
|
|
files_modified:
|
|
- apps/api/package.json
|
|
- apps/api/prisma/schema.prisma
|
|
- apps/api/prisma/migrations/20260721120000_add_tender_radar/migration.sql
|
|
autonomous: false
|
|
requirements: [SCHEMA-01]
|
|
must_haves:
|
|
truths:
|
|
- "Global (tenant-agnostic) Tender table exists in Postgres with OCDS-oriented nullable deadline/value fields (SCHEMA-01)"
|
|
- "Singleton TenderSourcePollConfig row model exists to drive the shared poll (INGEST-06 foundation)"
|
|
- "fast-xml-parser, adm-zip and csv-parse are installed in @tessera/api"
|
|
artifacts:
|
|
- apps/api/prisma/schema.prisma
|
|
- apps/api/prisma/migrations/20260721120000_add_tender_radar/migration.sql
|
|
key_links:
|
|
- "Tender.dedupKey @unique is the upsert target for SCHEMA-02 change detection"
|
|
- "Tender has NO tenantId column — RLS/forTenant deliberately does not apply (global data, per D-03)"
|
|
---
|
|
|
|
<objective>
|
|
Lay the data + dependency foundation for the Ausschreibungs-Radar module: install the three ingestion libraries (with a supply-chain checkpoint for the [SUS] `adm-zip` package), add the platform-global `Tender` and singleton `TenderSourcePollConfig` Prisma models, and apply the migration to the live database.
|
|
|
|
## Phase Goal (user story)
|
|
**As a** Tessera-Administrator, **I want to** das Ausschreibungs-Radar-Modul im Marketplace aktivieren und automatisch normalisierte deutsche DÖE-Ausschreibungen auf einem gemeinsamen, mandantensicheren Zeitplan erfassen lassen, **so that** alle Mandanten aktuelle, bietbare Vergaben durchsuchen koennen, ohne dass pro Mandant redundant gepollt wird.
|
|
|
|
Purpose: Every later slice (module registration, adapter, normalizer, ingestion, scheduler) depends on these tables and packages existing. The global-not-tenant-scoped shape of `Tender` (D-03) is the single highest-leverage decision and is locked in here.
|
|
Output: Migrated `Tender` + `TenderSourcePollConfig` tables, generated Prisma client types, three installed packages.
|
|
</objective>
|
|
|
|
<execution_context>
|
|
@$HOME/.claude/gsd-core/workflows/execute-plan.md
|
|
@$HOME/.claude/gsd-core/templates/summary.md
|
|
</execution_context>
|
|
|
|
<context>
|
|
@.planning/PROJECT.md
|
|
@.planning/ROADMAP.md
|
|
@.planning/STATE.md
|
|
@.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-RESEARCH.md
|
|
@.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-PATTERNS.md
|
|
@.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-CONTEXT.md
|
|
@apps/api/prisma/schema.prisma
|
|
@apps/api/prisma/migrations/20260714090000_add_ldap_user_exclude_list/migration.sql
|
|
</context>
|
|
|
|
## Artifacts This Phase Produces (whole-phase reference)
|
|
|
|
Prisma models: `Tender`, `TenderSourcePollConfig` (Plan 01).
|
|
NestJS classes: `TendersModule`, `seedTendersModule()` (Plan 02); `DoeOpenDataAdapter`, `TenderSourceAdapter` interface, `TenderNormalizerService` (Plan 03); `TenderIngestionService`, `TenderSchedulerService` (Plan 04); `TendersController`, `SourceConfigDto`, `TenderQueryDto` (Plan 05).
|
|
Types: `RawTenderRecord`, `NormalizedTenderFields`, `SourceType` in `tenders/tender.types.ts` (Plan 03).
|
|
Scheduler methods: `TenderSchedulerService.setInterval(intervalMin)`, `stopJob()`; `TenderIngestionService.pollDueSources()`, `pruneExpiredTenders()` (Plan 04).
|
|
Config keys / slugs: module slug `tender-radar`; `TenderSourcePollConfig.sourceType = 'doe-opendata'` (singleton).
|
|
New file paths: `apps/api/src/tenders/**`, `apps/web/src/app/(portal)/modules/tender-radar/page.tsx`, `apps/web/src/lib/module-loader.ts` (+`tender-radar` entry), fixture ZIPs under `apps/api/src/tenders/__fixtures__/`.
|
|
|
|
<tasks>
|
|
|
|
<task type="checkpoint:human-verify" gate="blocking-human">
|
|
<name>Task 1: adm-zip package legitimacy verification</name>
|
|
<what-built>Nothing yet — this gate precedes the `pnpm add adm-zip` step. `adm-zip` was tagged [SUS]/[ASSUMED] in 10-RESEARCH.md Package Legitimacy Audit (flagged `too-new`, manually assessed a false positive, but package-name provenance is training-knowledge so it requires a human spot-check per the package-legitimacy protocol).</what-built>
|
|
<action>Pause before installing `adm-zip`. Present the legitimacy evidence below to the human and wait for explicit approval (or a named alternative) before Task 2 runs `pnpm add`. Do not auto-approve — this is a blocking supply-chain gate (T-10-SC).</action>
|
|
<how-to-verify>
|
|
1. Open https://www.npmjs.com/package/adm-zip — confirm weekly downloads in the millions, last publish is a patch release (not a brand-new package), and repository points to github.com/cthackers/adm-zip.
|
|
2. Open https://github.com/cthackers/adm-zip — confirm it is an established, maintained repo (stars, issue history, MIT license).
|
|
3. Optionally check the Socket.dev / Snyk score for adm-zip@0.6.0.
|
|
4. Confirm `fast-xml-parser` and `csv-parse` are already approved in 10-RESEARCH.md (carried over from STACK.md — no re-flag needed).
|
|
</how-to-verify>
|
|
<acceptance_criteria>
|
|
- Human types "approved" after confirming adm-zip is a legitimate, established package, OR names an alternative (e.g. `unzipper`) if not.
|
|
- This checkpoint is NOT auto-approvable regardless of workflow.auto_advance (supply-chain gate).
|
|
</acceptance_criteria>
|
|
<resume-signal>Type "approved" to proceed with the install, or name a replacement ZIP library.</resume-signal>
|
|
</task>
|
|
|
|
<task type="auto">
|
|
<name>Task 2: Install ingestion packages + add Tender/TenderSourcePollConfig models</name>
|
|
<read_first>
|
|
- apps/api/prisma/schema.prisma (DkvModuleConfig model at ~line 178 for field/convention reference; datasource + generator at top)
|
|
- apps/api/prisma/migrations/20260714090000_add_ldap_user_exclude_list/migration.sql (handwritten migration style: IF NOT EXISTS guards, timestamped folder)
|
|
- .planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-PATTERNS.md (schema.prisma section — exact model field list)
|
|
</read_first>
|
|
<files>apps/api/package.json, apps/api/prisma/schema.prisma, apps/api/prisma/migrations/20260721120000_add_tender_radar/migration.sql</files>
|
|
<action>
|
|
Run `pnpm --filter @tessera/api add fast-xml-parser adm-zip csv-parse` (only after Task 1 approval).
|
|
|
|
Add two models to `apps/api/prisma/schema.prisma` following the 10-PATTERNS.md schema section exactly:
|
|
|
|
`Tender` — global reference table, deliberately NO `tenantId` and NO `@@index([tenantId])` (D-03: platform-global data). Fields: `id String @id @default(uuid())`, `sourcePortal String`, `sourceNoticeId String`, `ocid String?`, `dedupKey String @unique`, `title String`, `buyerName String?`, `cpvCodes String[] @default([])`, `region String?`, `plz String?`, `bundesland String?`, `deadlineAt DateTime?` (frequently null per RESEARCH Pattern 4 — nullable is mandatory, not optional hardening), `estimatedValue Decimal? @db.Decimal(14,2)`, `procedureType String?`, `status String @default("active")` (values `active` | `expired`; supports D-05 retention marking), `sourceUrl String?`, `contentHash String` (SCHEMA-02 change detection), `rawPayload Json?` (debugging/re-normalization), `publishedAt DateTime`, `createdAt DateTime @default(now())`, `updatedAt DateTime @updatedAt`. Indexes: `@@index([status])`, `@@index([deadlineAt])`, `@@index([publishedAt])`.
|
|
|
|
`TenderSourcePollConfig` — singleton-per-source admin config (INGEST-06 foundation). Fields: `id String @id @default(uuid())`, `sourceType String @unique` (fixed slug, `doe-opendata` — the `@unique` makes the singleton intent explicit, mirroring DkvModuleConfig.tenantId @unique), `pollIntervalMin Int @default(60)` (D-04 default hourly), `isActive Boolean @default(false)`, `lastIngestedDay DateTime?` (day-cursor, NOT a timestamp — RESEARCH Pattern 1), `createdAt DateTime @default(now())`, `updatedAt DateTime @updatedAt`.
|
|
|
|
Write the handwritten migration at `apps/api/prisma/migrations/20260721120000_add_tender_radar/migration.sql` mirroring the ldap-user-exclude-list migration style: `CREATE TABLE IF NOT EXISTS "Tender" (...)` and `CREATE TABLE IF NOT EXISTS "TenderSourcePollConfig" (...)` with matching column types, plus `CREATE UNIQUE INDEX IF NOT EXISTS` on `Tender.dedupKey` and `TenderSourcePollConfig.sourceType`, and the three `Tender` indexes.
|
|
|
|
Run `pnpm --filter @tessera/api exec prisma generate` so the Prisma client TS types for `Tender`/`TenderSourcePollConfig` are available to downstream service tasks.
|
|
</action>
|
|
<verify>
|
|
<automated>cd apps/api && grep -c "model Tender" prisma/schema.prisma && grep -c "model TenderSourcePollConfig" prisma/schema.prisma && test -f prisma/migrations/20260721120000_add_tender_radar/migration.sql && pnpm exec tsc --noEmit -p tsconfig.json 2>&1 | tail -5</automated>
|
|
</verify>
|
|
<acceptance_criteria>
|
|
- `grep -c "model Tender"` and `grep -c "model TenderSourcePollConfig"` in schema.prisma each return >= 1.
|
|
- Migration SQL file exists at the timestamped path.
|
|
- `Tender` has no `tenantId` column (assert: `grep -c "tenantId" ` within the Tender model block returns 0).
|
|
- `prisma generate` succeeds and `import { Tender } from '@prisma/client'` type-checks.
|
|
</acceptance_criteria>
|
|
<done>Both models present with correct fields, migration SQL written, Prisma client regenerated, packages installed.</done>
|
|
</task>
|
|
|
|
<task type="auto">
|
|
<name>Task 3: [BLOCKING] Apply the tender-radar migration to the database</name>
|
|
<read_first>
|
|
- apps/api/prisma/migrations/migration_lock.toml (confirms postgresql provider + migrate workflow)
|
|
- apps/api/prisma/migrations/20260721120000_add_tender_radar/migration.sql (the migration written in Task 2)
|
|
</read_first>
|
|
<files>apps/api/prisma/migrations/20260721120000_add_tender_radar/migration.sql</files>
|
|
<action>
|
|
[BLOCKING] Apply the handwritten migration to the live dev database so the Prisma client's compile-time types are backed by real tables (build/type checks pass WITHOUT this step because types come from the generated client, not the DB — this creates a false-positive verification state, so this task is mandatory before any ingestion/persistence verification).
|
|
|
|
Run `pnpm --filter @tessera/api exec prisma migrate deploy` (non-interactive; applies the committed handwritten migration). If `migrate deploy` reports drift or a non-empty schema conflict that requires interactive resolution, fall back to `pnpm --filter @tessera/api exec prisma db push` (schema is additive — two new tables, no destructive change expected). Then confirm the tables exist.
|
|
</action>
|
|
<verify>
|
|
<automated>cd apps/api && pnpm exec prisma migrate status 2>&1 | tail -8</automated>
|
|
</verify>
|
|
<acceptance_criteria>
|
|
- `prisma migrate status` reports the database schema is up to date (no pending migrations), OR a follow-up `prisma db push` reports "already in sync".
|
|
- Tables `Tender` and `TenderSourcePollConfig` exist in the database (verifiable via `prisma migrate status` clean state or a `prisma.tender.count()` call returning 0 without error).
|
|
</acceptance_criteria>
|
|
<done>Migration applied; both tables live in the database; no pending migrations.</done>
|
|
</task>
|
|
|
|
</tasks>
|
|
|
|
<threat_model>
|
|
## Trust Boundaries
|
|
|
|
| Boundary | Description |
|
|
|----------|-------------|
|
|
| npm registry → build | Third-party package code (`adm-zip`) enters the trusted build via `pnpm add` |
|
|
| ORM → Postgres | New global tables added; tenant-isolation design decision is encoded in schema shape |
|
|
|
|
## STRIDE Threat Register
|
|
|
|
| Threat ID | Category | Component | Severity | Disposition | Mitigation Plan |
|
|
|-----------|----------|-----------|----------|-------------|-----------------|
|
|
| T-10-SC | Tampering | `pnpm add adm-zip` (supply chain) | high | mitigate | Task 1 blocking-human checkpoint verifies adm-zip on npmjs.com + github before install; fast-xml-parser/csv-parse pre-approved in RESEARCH |
|
|
| T-10-01 | Information Disclosure | `Tender` table schema | high | mitigate | `Tender` carries NO `tenantId` by design (D-03) — it is platform-global reference data; per-tenant scoping lives one layer up (Phase 11 saved searches). Encoding this now prevents a later blanket `where: { tenantId }` habit from silently hiding global data from a second tenant |
|
|
| T-10-02 | Denial of Service | `Decimal`/`String[]` columns | low | accept | Standard Postgres column types; ingestion-side size guards handled in Plan 03 (zip-bomb) — no DB-level risk introduced by the schema itself |
|
|
</threat_model>
|
|
|
|
<verification>
|
|
- `prisma migrate status` clean; both tables present.
|
|
- `Tender` model has no `tenantId` field (design invariant for D-03 / multi-tenant global data).
|
|
- Three packages resolvable: `pnpm --filter @tessera/api exec node -e "require('adm-zip');require('fast-xml-parser');require('csv-parse/sync')"`.
|
|
</verification>
|
|
|
|
<success_criteria>
|
|
- SCHEMA-01 data foundation: normalized OCDS-oriented `Tender` table exists with nullable deadline/value.
|
|
- Packages installed; migration applied; client regenerated.
|
|
</success_criteria>
|
|
|
|
<output>
|
|
Create `.planning/phases/10-ausschreibungs-radar-foundation-d-e-ingestion/10-01-SUMMARY.md` when done.
|
|
</output>
|