docs(10): mark phase planned (6 plans) + add pattern map

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-21 10:10:41 +02:00
parent b52a0a796b
commit 66efd08dc0
2 changed files with 409 additions and 3 deletions
+3 -3
View File
@@ -4,9 +4,9 @@ milestone: v1.1
milestone_name: Ausschreibungs-Radar
current_phase: 10
current_phase_name: Ausschreibungs-Radar Foundation & DÖE Ingestion
status: planning
status: executing
stopped_at: Phase 10 context gathered
last_updated: "2026-07-17T11:49:35.931Z"
last_updated: "2026-07-21T08:10:24.552Z"
last_activity: 2026-07-17
last_activity_desc: ROADMAP.md created for v1.1 (Phases 10-14), 29 requirements mapped, 100% coverage
progress:
@@ -30,7 +30,7 @@ See: .planning/PROJECT.md (updated 2026-07-17)
Phase: 10 of 14 (Ausschreibungs-Radar Foundation & DÖE Ingestion)
Plan: — (not yet planned)
Status: Ready to plan
Status: Ready to execute
Last activity: 2026-07-17 — ROADMAP.md created for v1.1 (Phases 10-14), 29 requirements mapped, 100% coverage
Progress: [░░░░░░░░░░] 0%
@@ -0,0 +1,406 @@
# Phase 10: Ausschreibungs-Radar Foundation & DÖE Ingestion - Pattern Map
**Mapped:** 2026-07-21
**Files analyzed:** 12 (new)
**Analogs found:** 12 / 12
## File Classification
| New/Modified File | Role | Data Flow | Closest Analog | Match Quality |
|--------------------|------|-----------|-----------------|---------------|
| `apps/api/src/tenders/tenders.module.ts` | module | request-response | `apps/api/src/dkv/dkv.module.ts` | exact |
| `apps/api/src/tenders/tenders.seed.ts` | config/registry-seed | event-driven (OnModuleInit) | `apps/api/src/cert-manager/cert-manager.seed.ts` | exact |
| `apps/api/src/tenders/tenders.controller.ts` | controller | request-response | `apps/api/src/dkv/dkv.controller.ts` | exact |
| `apps/api/src/tenders/tender-scheduler.service.ts` | service (cron) | event-driven | `apps/api/src/dkv/dkv-scheduler.service.ts` | role-match (anti-pattern noted below) |
| `apps/api/src/tenders/tender-ingestion.service.ts` | service | CRUD + batch | `apps/api/src/dkv/dkv.service.ts` (`processInbox`/orchestration) | role-match |
| `apps/api/src/tenders/tender-normalizer.service.ts` | service (transform) | transform | `apps/api/src/dkv/dkv-parser.service.ts` | role-match |
| `apps/api/src/tenders/adapters/tender-source-adapter.interface.ts` | interface | — | `apps/api/src/dkv/providers/imap.provider.ts` + `exchange-inbox.provider.ts` (provider interface pattern) | role-match |
| `apps/api/src/tenders/adapters/doe-opendata.adapter.ts` | service (HTTP client) | file-I/O (ZIP fetch/parse) | `apps/api/src/favorites/icon-discovery.service.ts` (native `fetch` + guard patterns) | role-match |
| `apps/api/src/tenders/tender.types.ts` | types | — | `apps/api/src/dkv/dkv.types.ts` | exact |
| `apps/api/src/tenders/dto/source-config.dto.ts` | dto/validation | request-response | `apps/api/src/dkv/dto/dkv-config.dto.ts` | exact |
| `apps/api/src/tenders/dto/tender-query.dto.ts` | dto/validation | request-response | `apps/api/src/dkv/dto/dkv-history.dto.ts` | exact |
| `apps/api/prisma/schema.prisma` (add `Tender`, `TenderSourcePollConfig` models) | model | CRUD | `Module`/`DkvModuleConfig` models in same file | exact |
| `apps/api/prisma/migrations/<timestamp>_add_tender_radar/migration.sql` | migration | — | `apps/api/prisma/migrations/20260714090000_add_ldap_user_exclude_list/migration.sql` | exact (style: hand-written SQL, `IF NOT EXISTS`) |
| `apps/web/src/lib/module-loader.ts` (add `tender-radar` entry) | config (frontend registry) | request-response | existing `dkv-fleet`/`cert-manager` entries in same file | exact |
## Pattern Assignments
### `apps/api/src/tenders/tenders.module.ts` (module, request-response)
**Analog:** `apps/api/src/dkv/dkv.module.ts` (full file read, 73 lines)
**Core pattern** — self-seeding module via `OnModuleInit`, imports `ModuleRegistryModule`:
```typescript
@Module({
imports: [ModuleRegistryModule /* + SettingsModule if admin poll-interval config reuses SettingsService */],
controllers: [TendersController],
providers: [
TenderIngestionService,
TenderSchedulerService,
TenderNormalizerService,
DoeOpenDataAdapter,
],
exports: [TenderIngestionService],
})
export class TendersModule implements OnModuleInit {
private readonly logger = new Logger(TendersModule.name);
constructor(private readonly moduleRegistryService: ModuleRegistryService) {}
async onModuleInit(): Promise<void> {
try {
await seedTendersModule(this.moduleRegistryService);
this.logger.log('Ausschreibungs-Radar module seeded in registry');
} catch (error) {
this.logger.error('Failed to seed Ausschreibungs-Radar module', error);
}
}
}
```
Note: `PrismaModule` is global — no explicit import needed (same as DKV). `ScheduleModule.forRoot()` is already registered in `AppModule` (do not re-register).
---
### `apps/api/src/tenders/tenders.seed.ts` (config/registry-seed)
**Analog:** `apps/api/src/cert-manager/cert-manager.seed.ts` (full file, 27 lines) — preferred over `dkv.seed.ts` because `isSystem: true` + "admin must activate via Marketplace" phrasing matches this phase's CONFIG-01 exactly (DKV's comment says "available to all tenants" which is not the desired behavior here).
**Core pattern:**
```typescript
import { ModuleRegistryService } from '../module-registry/module-registry.service';
export async function seedTendersModule(
moduleRegistryService: ModuleRegistryService,
): Promise<void> {
await moduleRegistryService.seedModule({
slug: 'tender-radar',
name: 'Ausschreibungs-Radar',
version: '1.0.0',
category: 'procurement', // or an existing category value — confirm during planning
description: {
de: 'Deutsche Ausschreibungen automatisch erfassen und durchsuchen',
en: 'Automatically track and search German public tenders',
},
isSystem: true,
});
}
```
`seedModule()` is an **upsert by slug** (`ModuleRegistryService.seedModule`, `apps/api/src/module-registry/module-registry.service.ts:158-187`) — reuse unmodified, no new registry mechanism needed.
---
### `apps/api/src/tenders/tenders.controller.ts` (controller, request-response)
**Analog:** `apps/api/src/dkv/dkv.controller.ts` (full file, 243 lines)
**Imports pattern** (lines 1-24):
```typescript
import { Body, Controller, Get, NotFoundException, Param, Post, Put, Query, Req } from '@nestjs/common';
import { Role } from '@prisma/client';
import { Roles } from '../auth/decorators/roles.decorator';
```
**Auth/Guard pattern** (per-handler, lines 59-60, 76-77, etc.) — every admin route decorated individually, no controller-level `@Roles`:
```typescript
@Get('source-config')
@Roles(Role.ADMIN, Role.SUPER_ADMIN)
async getSourceConfig(@Req() req: any) { ... }
```
V4 note from RESEARCH.md: admin source-config routes must be `@Roles(Role.ADMIN, Role.SUPER_ADMIN)`-guarded like `DkvController`, but the **list/detail `GET /tenders` routes must NOT be tenant-row-filtered** (global table) — only gated by `TenantModuleActivation` (module licensing), which is the one place this controller structurally diverges from DKV. Use `ModuleGuard` (`apps/api/src/module-registry/module.guard.ts`) for that gate rather than `req.tenantId` row-scoping.
**Tenant-extraction helper pattern** (lines 234-241) — reuse verbatim for the admin config routes only (not for the global tender list):
```typescript
private _requireTenant(req: any): string {
const tenantId = req.tenantId as string | undefined;
if (!tenantId) {
throw new BadRequestException('No tenant context');
}
return tenantId;
}
```
**Scheduler-interaction pattern on config save** (lines 76-90) — after admin updates `pollIntervalMin`/`isActive`, push the change into the live scheduler without restart:
```typescript
@Put('source-config')
@Roles(Role.ADMIN, Role.SUPER_ADMIN)
async saveSourceConfig(@Body() dto: SourceConfigDto) {
const result = await this.tenderIngestionService.saveSourceConfig(dto);
if (dto.isActive && dto.pollIntervalMin) {
this.tenderScheduler.setInterval(dto.pollIntervalMin);
} else if (dto.isActive === false) {
this.tenderScheduler.stopJob();
}
return result;
}
```
Difference from DKV: no `tenantId` argument to `setInterval()` — the DÖE source config is a platform-wide singleton (per RESEARCH.md Pitfall D), so the scheduler signature drops the tenant parameter entirely.
---
### `apps/api/src/tenders/tender-scheduler.service.ts` (service/cron, event-driven)
**Analog:** `apps/api/src/dkv/dkv-scheduler.service.ts` (full file, 146 lines) — reuse the **mechanics**, do not reuse the **per-tenant framing**.
**Reuse verbatim — CronJob resolution + SchedulerRegistry dynamic job pattern** (lines 1-14, 96-130):
```typescript
// eslint-disable-next-line @typescript-eslint/no-require-imports
const CronJobClass: new (cronTime: string, onTick: () => void) => { start(): void } =
// eslint-disable-next-line @typescript-eslint/no-unsafe-member-access
require('cron').CronJob as new (cronTime: string, onTick: () => void) => { start(): void };
setInterval(intervalMin: number): void {
try {
this.schedulerRegistry.getCronJob(this.JOB_NAME).stop();
this.schedulerRegistry.deleteCronJob(this.JOB_NAME);
} catch { /* not yet registered — expected on first call */ }
const cronExpr = intervalMin < 60
? `*/${intervalMin} * * * *`
: `0 */${Math.floor(intervalMin / 60)} * * *`;
const job = new CronJobClass(cronExpr, () => {
this.tenderIngestionService.pollDueSources().catch((err) =>
this.logger.error(`Tender poll tick failed: ${(err as Error).message}`),
);
});
// eslint-disable-next-line @typescript-eslint/no-explicit-any
this.schedulerRegistry.addCronJob(this.JOB_NAME, job as any);
job.start();
}
```
**ANTI-PATTERN — DO NOT COPY (lines 43, 59-69):** DKV's `onModuleInit()` loads config via `this.dkvService.loadConfig()` which internally calls `findFirst()` scoped to "first active config, single-tenant assumption" and stores `this.activeTenantId` on the service instance. RESEARCH.md's Pitfall D explains why this specific instance is actually *safe* for Phase 10 (DÖE has exactly one platform-wide config row, so `findUnique({ where: { sourceType: 'doe-opendata' } })` on a **fixed known slug** is correct — not the same anti-pattern as a hypothetical per-tenant `findFirst()`). Concretely:
- Do NOT store `activeTenantId` on the scheduler instance — there is no tenant dimension here.
- DO use `findUnique({ where: { sourceType: 'doe-opendata' } })` (or equivalent fixed-slug lookup), not `findFirst()` on an unfiltered/ordered query, to make the "singleton config" intent explicit in code (self-documenting against future copy-paste into a per-tenant source).
- The two-tenant acceptance test (ROADMAP.md Success Criteria 4/5) must assert **zero additional HTTP calls / cron jobs / `Tender` rows** when a 2nd tenant activates the module — i.e. prove absence of tenant-scaled behavior, not correct per-tenant iteration.
- Day-cursor gating (RESEARCH.md Pattern 1) must live inside `TenderIngestionService.pollDueSources()`, not the scheduler — the scheduler only controls cron-tick frequency (can stay hourly per D-04); the ingestion service decides whether `dayCursor < today(Europe/Berlin)` before making any HTTP call (Pitfall A).
---
### `apps/api/src/tenders/tender-ingestion.service.ts` (service, CRUD + batch)
**Analog:** `apps/api/src/dkv/dkv.service.ts` (orchestration methods `processInbox`/`checkNow` — read for the overall shape: fetch → parse → normalize → persist → log).
**Core pattern (orchestration + day-cursor gate + upsert-by-dedupKey):**
```typescript
async pollDueSources(): Promise<void> {
const config = await this.prisma.tenderSourcePollConfig.findUnique({
where: { sourceType: 'doe-opendata' },
});
if (!config?.isActive) return;
const nextDay = nextDayToFetch(config.lastIngestedDay); // RESEARCH.md "Day-cursor gate" snippet
if (!nextDay) return; // no-op tick — expected, not a bug (Pitfall A)
const rawRecords = await this.doeAdapter.fetchTenders(nextDay);
const normalized = rawRecords.map((r) => this.normalizer.normalize(r));
for (const tender of normalized) {
await this.prisma.tender.upsert({
where: { dedupKey: tender.dedupKey },
update: { ...tender, contentHash: tender.contentHash }, // SCHEMA-02 change detection
create: tender,
});
}
await this.prisma.tenderSourcePollConfig.update({
where: { sourceType: 'doe-opendata' },
data: { lastIngestedDay: new Date(nextDay) },
});
}
```
**Note (multi-tenancy):** This service must NOT call `forTenant()` (`apps/api/src/prisma/prisma-tenant.extension.ts`) for `Tender`/`TenderSourcePollConfig` writes — these are global, RLS-exempt tables by design (RESEARCH.md Architectural Responsibility Map + CONTEXT.md Integration Points). Use the plain (non-tenant-extended) `PrismaService` client for this service's queries.
**Error handling pattern** — catch-and-log per tick, never throw out of the cron callback (mirrors `dkv-scheduler.service.ts` lines 113-118):
```typescript
this.tenderIngestionService.pollDueSources().catch((err) =>
this.logger.error(`Tender poll tick failed: ${(err as Error).message}`),
);
```
---
### `apps/api/src/tenders/tender-normalizer.service.ts` (service/transform, transform)
**Analog:** `apps/api/src/dkv/dkv-parser.service.ts` (transform-style service — parse raw input into a typed internal shape; read for method shape, not copied verbatim since parsing target differs entirely — XML/JSON vs. PDF text).
**Core pattern (structure only):** one public `normalize(raw: RawTenderRecord): NormalizedTenderFields` entry point, private helper methods per field group, pure functions wherever possible (no I/O inside the normalizer — I/O stays in the adapter). Apply RESEARCH.md's filter/hash logic directly:
```typescript
function isOpenTenderNotice(release: { tag?: string[] }): boolean {
if (release.tag?.includes('tender')) return true;
return false; // conservative: missing/other tags excluded (Pitfall C)
}
```
---
### `apps/api/src/tenders/adapters/doe-opendata.adapter.ts` (service/HTTP client, file-I/O)
**Analog:** `apps/api/src/favorites/icon-discovery.service.ts` (full file, 380 lines) — the **native `fetch` + AbortController timeout** convention is the established project pattern for outbound HTTP (no axios anywhere in the codebase).
**Core fetch pattern to copy** (lines 254-265, adapted — DÖE needs no SSRF guard since it is a single fixed trusted government host, but keep the timeout/AbortController convention):
```typescript
async function fetchDoeDay(pubDay: string, format: 'eforms.zip' | 'ocds.zip'): Promise<Buffer | null> {
const controller = new AbortController();
const timeout = setTimeout(() => controller.abort(), 15_000);
try {
const res = await fetch(
`https://oeffentlichevergabe.de/api/notice-exports?pubDay=${pubDay}&format=${format}`,
{ signal: controller.signal },
);
if (res.status === 400) return null; // pubDay is today/future — nothing to fetch yet
if (!res.ok) throw new Error(`DÖE fetch failed: ${res.status}`);
return Buffer.from(await res.arrayBuffer());
} finally {
clearTimeout(timeout);
}
}
```
**Error handling pattern:** treat HTTP 400 as an expected no-op signal (not an error to throw), everything else `!res.ok` throws — matches `icon-discovery.service.ts`'s `if (!response.ok) return null;` early-return-on-failure convention, adapted to distinguish "expected no-op" (400) from "real failure" (5xx/network).
---
### `apps/api/src/tenders/dto/source-config.dto.ts` (dto/validation, request-response)
**Analog:** `apps/api/src/dkv/dto/dkv-config.dto.ts` (full file, 111 lines)
**Validation pattern to copy directly:**
```typescript
import { IsBoolean, IsInt, IsOptional, Max, Min } from 'class-validator';
export class SourceConfigDto {
/** Poll interval in minutes. Cron-tick frequency (D-04 default 60), NOT DÖE fetch frequency (day-cursor gated separately). */
@IsOptional()
@IsInt()
@Min(5) // same DoS-mitigation floor as DkvConfigDto.pollIntervalMin
@Max(1440)
pollIntervalMin?: number;
@IsOptional()
@IsBoolean()
isActive?: boolean;
}
```
---
### `apps/api/src/tenders/dto/tender-query.dto.ts` (dto/validation, request-response)
**Analog:** `apps/api/src/dkv/dto/dkv-history.dto.ts` (full file, 37 lines) — copy the pagination pattern verbatim (page/limit + `@Type(() => Number)` coercion, same bounds):
```typescript
import { Type } from 'class-transformer';
import { IsInt, IsOptional, Max, Min } from 'class-validator';
export class TenderQueryDto {
@IsOptional() @IsInt() @Min(1) @Type(() => Number) page?: number;
@IsOptional() @IsInt() @Min(1) @Max(100) @Type(() => Number) limit?: number;
// + filter fields specific to tenders (e.g. status=open|expired) — new, no analog needed, plain @IsOptional @IsIn
}
```
---
### `apps/api/prisma/schema.prisma` (model, CRUD)
**Analog:** `Module`/`TenantModuleActivation`/`DkvModuleConfig` models (same file, lines 92-197) — field-naming and structural conventions to follow:
```prisma
model Tender {
id String @id @default(uuid())
sourcePortal String // e.g. 'doe-opendata'
sourceNoticeId String
ocid String? // stable dedup key across notice versions, when present
dedupKey String @unique // ocid, or fallback sourcePortal:sourceNoticeId
title String
buyerName String?
cpvCodes String[] @default([])
region String?
deadlineAt DateTime? // frequently null per RESEARCH.md Pattern 4 — do not assume present
estimatedValue Decimal? @db.Decimal(14, 2)
procedureType String?
status String @default("active") // 'active' | 'expired'
sourceUrl String?
contentHash String // SCHEMA-02 change detection
publishedAt DateTime
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
@@index([status])
@@index([dedupKey])
// Deliberately NO tenantId field and NO @@index([tenantId]) — global table, CONTEXT.md Integration Points
}
model TenderSourcePollConfig {
id String @id @default(uuid())
sourceType String @unique // fixed slug, e.g. 'doe-opendata' — singleton row (RESEARCH.md Pitfall D)
pollIntervalMin Int @default(60)
isActive Boolean @default(false)
lastIngestedDay DateTime? // day-cursor, not a timestamp (RESEARCH.md Pattern 1)
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
}
```
Conventions confirmed from existing models: `id String @id @default(uuid())`, `createdAt`/`updatedAt` pair with `@default(now())`/`@updatedAt`, `@@index` on frequently-filtered columns, `@unique` for natural singleton/lookup keys (mirrors `DkvModuleConfig.tenantId @unique` — here `sourceType @unique` plays the equivalent singleton role).
---
### `apps/api/prisma/migrations/<timestamp>_add_tender_radar/migration.sql` (migration)
**Analog:** `apps/api/prisma/migrations/20260714090000_add_ldap_user_exclude_list/migration.sql` — hand-written, timestamped folder under `apps/api/prisma/migrations/`, `IF NOT EXISTS` guards:
```sql
-- CreateTable
CREATE TABLE IF NOT EXISTS "Tender" ( ... );
CREATE TABLE IF NOT EXISTS "TenderSourcePollConfig" ( ... );
```
Confirms project convention: migrations are **handwritten** (not auto-generated via `prisma migrate dev` in this repo's workflow), timestamp-prefixed folder naming (`YYYYMMDDHHMMSS_description`).
---
### `apps/web/src/lib/module-loader.ts` (config, frontend registry)
**Analog:** existing `dkv-fleet`/`cert-manager` entries in the same `MODULE_REGISTRY` object (lines 31-49):
```typescript
'tender-radar': {
component: dynamic(
() => import('@/app/(portal)/modules/tender-radar/page'),
{ ssr: false },
),
},
```
Critical: RESEARCH.md flags this explicitly — a module can be seeded in the DB and activated per-tenant, but **the page 404s unless this MODULE_REGISTRY entry also exists** (whitelist-only dynamic import, security-motivated per the file's own header comment lines 5-8). This entry is mandatory for CONFIG-01, not optional/cosmetic.
## Shared Patterns
### Module self-registration (CONFIG-01)
**Source:** `apps/api/src/module-registry/module-registry.service.ts` (`seedModule`, lines 158-187) + `apps/api/src/cert-manager/cert-manager.seed.ts`
**Apply to:** `tenders.module.ts`, `tenders.seed.ts`
Unmodified reuse — no new registry mechanism. `isSystem: true` + admin-driven Marketplace activation (not auto-activated per tenant), matching cert-manager's model over DKV's ("available to all tenants" is wrong for this phase).
### Native `fetch` as sole HTTP client convention
**Source:** `apps/api/src/favorites/icon-discovery.service.ts` (lines 254-287), `apps/api/src/calendar/providers/ics.provider.ts`
**Apply to:** `doe-opendata.adapter.ts`
No axios anywhere in the codebase — AbortController + `setTimeout`-based timeout is the established idiom for any outbound call.
### Dynamic `SchedulerRegistry` cron job (not static `@Cron()`)
**Source:** `apps/api/src/dkv/dkv-scheduler.service.ts` (lines 96-130)
**Apply to:** `tender-scheduler.service.ts`
Reuse mechanics (CronJob require()-resolution workaround, addCronJob/deleteCronJob replace-on-change pattern) but drop the per-tenant framing entirely (see anti-pattern note above).
### Tenant-scoped Prisma extension — explicitly NOT applied to Tender tables
**Source:** `apps/api/src/prisma/prisma-tenant.extension.ts` (`forTenant`, full file, 25 lines)
**Apply to:** N/A for `Tender`/`TenderSourcePollConfig` (explicitly excluded); DO apply if any tenant-scoped admin-audit table is added in this phase (none currently planned — Suchprofile/Matches are Phase 11).
This is the one pattern to actively **avoid misapplying**: RLS via `set_config('app.current_tenant', ...)` must not wrap queries against the global `Tender` table, or a second tenant's session would silently filter out platform-wide data it should see.
### class-validator DTO conventions
**Source:** `apps/api/src/dkv/dto/dkv-config.dto.ts`, `dkv-history.dto.ts`
**Apply to:** `source-config.dto.ts`, `tender-query.dto.ts`
`@IsOptional()` + explicit `@Min`/`@Max` bounds on every numeric field (DoS mitigation precedent, e.g. `pollIntervalMin` floor of 5); `@Type(() => Number)` required on any query-string-sourced numeric field before validation runs.
## No Analog Found
None — every planned file has at least a role-match analog in the existing DKV/cert-manager/module-registry/favorites code. The `adapters/tender-source-adapter.interface.ts` is the weakest match (DKV's provider files are concrete IMAP/Exchange implementations without a formally separated interface file) — planner should treat ARCHITECTURE.md's `TenderSourceAdapter` interface sketch as primary and the DKV providers only as a loose structural reference (constructor-injected, single primary method).
## Metadata
**Analog search scope:** `apps/api/src/dkv/`, `apps/api/src/cert-manager/`, `apps/api/src/module-registry/`, `apps/api/src/favorites/`, `apps/api/src/calendar/providers/`, `apps/api/src/prisma/`, `apps/api/prisma/schema.prisma`, `apps/api/prisma/migrations/`, `apps/web/src/lib/module-loader.ts`
**Files scanned:** 13 read in full (dkv-scheduler.service.ts, dkv.seed.ts, dkv.module.ts, dkv.controller.ts, prisma-tenant.extension.ts, module-registry.service.ts, icon-discovery.service.ts, cert-manager.seed.ts, dkv-config.dto.ts, dkv-vehicle.dto.ts, dkv-history.dto.ts, schema.prisma excerpt, one migration.sql) + module-loader.ts grep excerpt
**Pattern extraction date:** 2026-07-21