tessera-ctl/.planning/research/ARCHITECTURE.md

# Architecture Research — Ausschreibungs-Radar Module Integration

**Domain:** Multi-source tender/procurement-notice ingestion module for an existing NestJS 11 modular-monolith platform (Tessera)
**Researched:** 2026-07-17
**Confidence:** HIGH (module registration, tenant scoping, inbox/mail reuse — verified against existing `dkv/` and `module-registry/` source) / MEDIUM (OCDS field mapping, DÖE API pagination — verified via OCDS spec + DÖE Swagger listing, not a live API call)

> **Note:** This supersedes the v1.0 platform-level `ARCHITECTURE.md` (2026-06-18, described a generic Fastify/Traefik shell that predates the stack decisions now recorded in `CLAUDE.md`). This file is scoped to the v1.1 Ausschreibungs-Radar milestone and describes how the new module integrates into the actual, current NestJS 11 + Next.js codebase — not a from-scratch platform design.

## Summary

The Ausschreibungs-Radar module ("tenders" module) is a **new, self-contained NestJS module** built strictly on top of existing Tessera infrastructure — module-registry self-seeding, `@nestjs/schedule` dynamic cron jobs, the `ImapProvider`/`ExchangeInboxProvider` inbox abstraction, tenant-scoped SMTP via `SmtpConfig` + fresh-transport-per-send, and the module-loader-driven Next.js portal route. The one architectural decision that **breaks from the DKV template** is deliberate and important: **tender data is platform-global, not tenant-owned.** DKV invoices belong to one tenant; a DÖE/AI-NetServer/cosinex tender notice is a public fact relevant to *every* tenant. Ingestion, normalization, and dedup therefore run once for the whole platform; only **saved searches, matches, and notification preferences** are tenant-scoped. Getting this split right is the single highest-leverage decision in this design — getting it wrong means N-times redundant scraping/storage and N-times the anti-bot exposure per portal.

## Standard Architecture

### System Overview

```
+---------------------------------------------------------------------------+
|                         SOURCE ADAPTERS (new)                             |
|  +-----------+ +--------------+ +-----------+ +--------+ +--------------+|
|  |DoeOpenData| |AiNetServer   | |Cosinex    | |Rss     | |EmailAlert    ||
|  |Adapter    | |Adapter       | |Adapter    | |Adapter | |Adapter       ||
|  |(API,auth- | |(HTML scrape, | |(HTML      | |(feed   | |(reuses Imap/ ||
|  | free)     | | public       | | scrape,   | | parse) | | Exchange     ||
|  |           | | search)      | | public)   | |        | | InboxProvider||
|  +-----+-----+ +------+-------+ +-----+-----+ +---+----+ +------+-------+|
|        |  all implement TenderSourceAdapter -> RawTenderRecord[] |       |
+--------+--------------+---------------+-----------+-----------------+---+
         `--------------`-------+-------`-----------`-------------'
                                v
+---------------------------------------------------------------------------+
|                 TenderIngestionService (new, global -- no tenantId)       |
|  normalize(raw, sourceType) -> Tender (OCDS-oriented)                     |
|  computeDedupKey() -> upsert by dedupKey -> diff contentHash -> mark changed|
+-------------------------------+-------------------------------------------+
                                 v (only NEW / CHANGED tenders this poll)
+---------------------------------------------------------------------------+
|           TenderMatchingService (new) -- DB-query filter evaluation       |
|  for each active TenderSavedSearch: Prisma `where` over the delta batch   |
|  (structured filters) + Postgres full-text search (keywords) -> TenderMatch|
+-------------------------------+-------------------------------------------+
                                 v
+---------------------------------------------------------------------------+
|        TenderMailService (new) -- reuses SmtpConfig + fresh-transport     |
|  instant: send on TenderMatch create   digest: cron batch per SavedSearch |
+---------------------------------------------------------------------------+
                         |
+------------------------+---------------------------------------------------+
|  TendersController (new) -- GET /tenders (search+filter+paginate),         |
|  /tenders/:id, /tenders/saved-searches (CRUD, tenant-scoped),              |
|  /tenders/source-config (admin, global), /tenders/check-now                |
+------------------------+---------------------------------------------------+
                          v
      apps/web/.../modules/tender-radar/  (new Next.js module UI)
```

### Component Responsibilities

| Component | Responsibility | Scope | New/Modified |
|-----------|----------------|-------|---------------|
| `TenderSourceAdapter` implementations | Fetch raw records from one source, no normalization | Global | New |
| `TenderNormalizerService` | Map each source's raw shape → unified `Tender` fields, compute `dedupKey`/`contentHash` | Global | New |
| `TenderIngestionService` | Orchestrate poll → normalize → upsert → change-detect per source | Global | New |
| `TenderSchedulerService` | Dynamic cron per global source + per-tenant cron for email-alert ingestion | Mixed | New |
| `TenderMatchingService` | Evaluate active saved searches against the new/changed delta | Per-tenant read, global data | New |
| `TenderMailService` | Send instant/digest notification emails via tenant SMTP | Per-tenant | New |
| `TendersController` | REST endpoints for list/detail/saved-search CRUD/admin source-config | Mixed | New |
| Shared `InboxModule` (relocated) | `InboxProvider` interface + `ImapProvider`/`ExchangeInboxProvider` | Platform-shared | New (extracted from `dkv/`) |
| `ModuleRegistryService` | Self-seed `tender-radar` module row | Platform | Reused unmodified |
| `SettingsService` / `SmtpConfig` | Decrypted per-tenant SMTP for notification sends | Per-tenant | Reused unmodified |
| `CalendarCryptoService` | AES-256-GCM encryption for any stored credentials | Platform | Reused unmodified |
| `module-loader.ts` `MODULE_REGISTRY` | Whitelist entry mapping slug → lazy component | Platform | Modified (one entry) |
| `AppModule` | Import `TendersModule` | Platform | Modified (one import) |

## Recommended Project Structure

```
apps/api/src/
├── inbox/                          # NEW -- extracted shared module (was dkv/providers/)
│   ├── inbox.module.ts             # exports ImapProvider, ExchangeInboxProvider
│   ├── inbox-provider.interface.ts # InboxProvider contract (moved from dkv.types.ts)
│   ├── imap.provider.ts            # moved verbatim from dkv/providers/
│   ├── exchange-inbox.provider.ts  # moved verbatim from dkv/providers/
│   └── inbox.types.ts              # InboxConfig / InboxEmail / InboxAttachment
│
├── dkv/                             # MODIFIED -- imports InboxModule instead of local providers/
│   └── ...                          # (providers/ folder removed, dkv.types.ts trimmed)
│
├── tenders/                         # NEW -- Ausschreibungs-Radar module
│   ├── tenders.module.ts
│   ├── tenders.controller.ts        # public list/detail + tenant saved-search CRUD + admin source-config
│   ├── tenders.seed.ts              # seeds 'tender-radar' into ModuleRegistry
│   ├── tender-ingestion.service.ts  # poll → normalize → upsert → change-detect (per source)
│   ├── tender-matching.service.ts   # SavedSearch → Prisma where-clause → TenderMatch
│   ├── tender-mail.service.ts       # instant + digest notification sends
│   ├── tender-scheduler.service.ts  # SchedulerRegistry cron: 1 per global source + 1 per tenant (email-alert)
│   ├── tender.types.ts              # RawTenderRecord, NormalizedTenderFields, SourceType
│   ├── dto/
│   │   ├── saved-search.dto.ts
│   │   ├── source-config.dto.ts
│   │   └── tender-query.dto.ts      # pagination + filter query params for GET /tenders
│   └── adapters/
│       ├── tender-source-adapter.interface.ts   # fetchTenders(config, since) → RawTenderRecord[]
│       ├── doe-opendata.adapter.ts              # Build order Phase A
│       ├── ai-netserver.adapter.ts              # Build order Phase B
│       ├── cosinex.adapter.ts                   # Build order Phase B
│       ├── rss.adapter.ts                       # Build order Phase C
│       └── email-alert.adapter.ts               # Build order Phase C (uses InboxModule)
│
apps/web/src/app/(portal)/modules/
├── tender-radar/                    # follow the existing static per-module convention (dkv-fleet, cert-manager)
│   ├── page.tsx                     # searchable trefferliste + filter sidebar
│   ├── [id]/page.tsx                # tender detail view
│   ├── saved-searches/page.tsx      # saved search CRUD UI
│   └── settings/page.tsx            # admin: source poll config, email-alert inbox config
```

### Structure Rationale

- **`inbox/` extraction is a prerequisite, not optional.** `ImapProvider`/`ExchangeInboxProvider` are already generic over `InboxConfig`/`InboxEmail` — nothing in them is DKV-specific. Today they live in `dkv/providers/` and their types live in `dkv.types.ts`, so `tenders/` would otherwise have to import from inside another feature module's internals (`../dkv/providers/imap.provider`), which couples two unrelated features and breaks if DKV is ever restructured. Moving them to a shared `inbox/` module once, and updating `dkv.module.ts` to import `InboxModule` instead, costs one small refactor now and pays for every future module that needs inbox polling (already two: DKV, Tenders).
- **`tenders/adapters/` mirrors `dkv/providers/`** — same rationale as DKV: consumers (`TenderIngestionService`) depend only on the `TenderSourceAdapter` interface, never on a concrete adapter. This is what makes "ship DÖE first, add AI-NetServer/cosinex/RSS/email later" possible without touching the ingestion/matching/notification pipeline.
- **Normalizer is separate from adapters**, unlike DKV where `DkvParserService` is a single PDF parser. Here there are 5 structurally incompatible raw shapes (eForms/OCDS JSON, two flavors of scraped HTML, RSS/Atom XML, free-text alert emails). Each adapter can either normalize inline or delegate to a per-source mapping function inside `TenderNormalizerService` — either way, the *interface boundary* is `RawTenderRecord[] → Tender[]`, so the ingestion orchestrator never branches on source type.
- **Web module UI follows the existing static `modules/<slug>/` folder pattern** seen in `dkv-fleet/` and `cert-manager/` (not the dynamic `[category]/[moduleSlug]/` route also present in the codebase for vehicle/settings sub-pages) — match whichever of the two conventions the team is actively converging on at execution time; both are already present, so this is a phase-planning decision, not an open architectural question.

## Architectural Patterns

### Pattern 1: Source-Adapter Abstraction (`TenderSourceAdapter`)

**What:** One interface, N implementations — directly analogous to `InboxProvider` (`fetchPdfAttachments` → `fetchTenders`).

```typescript
// tenders/adapters/tender-source-adapter.interface.ts
export interface RawTenderRecord {
  sourceType: SourceType;          // 'doe-opendata' | 'ai-netserver' | 'cosinex' | 'rss' | 'email-alert'
  sourcePortal: string;            // 'doe' | 'lhs-vpbw' | 'tender24' | 'vergabe.landbw' | 'dtvp' | 'subreport-elvis' | 'service.bund.de'
  sourceRawId: string;             // portal-native id/notice number, pre-normalization
  sourceUrl: string;
  fetchedAt: Date;
  payload: unknown;                // raw JSON/HTML-extract/RSS-item/email-body — kept for rawPayload + reprocessing
}

export interface TenderSourceAdapter {
  readonly sourceType: SourceType;
  /** since: only fetch records new/changed after this timestamp (cursor from TenderSourcePollConfig.lastPolledAt) */
  fetchTenders(config: TenderSourceConfig, since?: Date): Promise<RawTenderRecord[]>;
  testConnection?(config: TenderSourceConfig): Promise<{ success: boolean; message?: string }>;
}
```

**When to use:** Any time a pipeline must ingest structurally different sources into one output shape without the orchestrator knowing about each source. Same pattern Tessera already uses for `ImapProvider`/`ExchangeInboxProvider`.

**Trade-offs:** Each adapter owns its own retry/rate-limit/anti-bot logic (AI-NetServer and cosinex adapters need polite scraping delays; DÖE/RSS don't). The interface intentionally does *not* prescribe HTTP client or scraping library — `TenderSourceConfig` is adapter-specific (a discriminated union or `Json` blob per `sourceType`), same as `InboxConfig` covers both IMAP and Exchange with one shape only because both providers happen to share fields; here they mostly won't, so `TenderSourceConfig` should be `Json` on the Prisma side with adapter-specific Zod/DTO validation, not a single flat interface.

### Pattern 2: Normalized OCDS-Oriented Schema + Cross-Source Dedup Key

**What:** One `Tender` Prisma model absorbing eForms/OCDS (DÖE), scraped HTML (AI-NetServer, cosinex), RSS, and email-alert text — with a **stable dedup key** so the same real-world procurement notice appearing on multiple sources (e.g. an AI-NetServer notice that later also appears on DÖE once it crosses the EU threshold, or the same DÖE OCID reappearing on a poll re-run) collapses to one row.

```prisma
model Tender {
  id              String   @id @default(uuid())

  // OCDS-oriented core (see standard.open-contracting.org/latest/en/schema/reference/)
  ocid            String?  // Open Contracting ID, e.g. "ocds-mnwr74-XXXXXXXX" — present when sourced via DÖE/TED
  noticeId        String?  // portal-native notice/procedure number (AI-NetServer, cosinex, RSS items)
  title           String
  description     String?  @db.Text
  buyerName       String?
  buyerId         String?  // e.g. Vergabestelle-ID if the source exposes it
  procedureType   String?  // OCDS tender.procurementMethod / procurementMethodDetails
  status          String   @default("active") // 'active' | 'awarded' | 'cancelled' | 'expired'
  cpvCodes        String[] @default([])
  region          String?
  plz             String?
  bundesland      String?
  estimatedValue  Decimal? @db.Decimal(14, 2)
  currency        String?  @default("EUR")
  publishedAt     DateTime?
  deadlineAt      DateTime?

  // Source + dedup
  sourceType      String   // 'doe-opendata' | 'ai-netserver' | 'cosinex' | 'rss' | 'email-alert'
  sourcePortal    String   // 'doe' | 'lhs-vpbw' | 'dtvp' | 'subreport-elvis' | ...
  sourceUrl       String?
  dedupKey        String   @unique   // see dedup strategy below
  contentHash     String             // hash of normalized fields — detects "changed" vs "identical re-poll"
  rawPayload      Json               // original adapter payload — debugging + future re-normalization

  firstSeenAt     DateTime @default(now())
  lastSeenAt      DateTime @updatedAt
  createdAt       DateTime @default(now())

  matches         TenderMatch[]

  @@index([sourceType])
  @@index([deadlineAt])
  @@index([publishedAt])
  @@index([bundesland])
}
```

**Dedup key strategy (priority order, computed by `TenderNormalizerService`):**
1. `ocid` — when the source provides an OCDS Open Contracting ID (always true for DÖE/TED; the platform's registered OCDS prefix is `ocds-mnwr74`). OCID is designed exactly for this — joining the same contracting process across publishers.
2. `${sourcePortal}:${noticeId}` — when the portal exposes a stable native notice/procedure number (AI-NetServer Bietercockpit ID, cosinex Vergabenummer, RSS item guid, email-alert reference number). This is the *primary* key for scraped/RSS/email sources, since they never carry an OCID.
3. `sha256(sourcePortal + normalizedTitle + buyerName + deadlineAt)` — last-resort fallback only when a source gives neither an OCID nor a stable ID (should be rare; flag these rows for manual review via a `dedupConfidence: 'low'` marker if this path is hit).

`dedupKey` is the `@unique` upsert target: `prisma.tender.upsert({ where: { dedupKey }, ... })`. `contentHash` (hash of title+deadline+value+status) is separate from `dedupKey` — it answers "did anything about this same notice change since we last saw it" (deadline extension, cancellation), which is what should trigger re-matching and potentially a "notice updated" notification, whereas an unchanged re-poll should just bump `lastSeenAt` and stop.

**Trade-offs:** A single wide table is simpler to query/filter/index than per-source tables + a union view, and matches how the UI wants to browse ("one trefferliste across all sources"). The cost is that source-specific fields that don't map cleanly (e.g. cosinex-specific metadata) live only in `rawPayload` (Json, unindexed) — acceptable, since the feasibility research shows the cross-source overlap (title, buyer, deadline, value, CPV, region) covers what filtering/notification actually need.

### Pattern 3: Global Data, Per-Tenant Filtering (the key deviation from the DKV template)

**What:** `Tender` rows carry **no `tenantId`** — they are platform-wide. Per-tenant scoping happens one layer up, in `TenderSavedSearch` and `TenderMatch`.

```prisma
model TenderSavedSearch {
  id                String   @id @default(uuid())
  tenantId          String
  userId            String?  // null = tenant-wide search, set = personal search
  name              String
  keywords          String[] @default([])   // full-text match against title+description
  bundeslaender     String[] @default([])
  plzPrefixes       String[] @default([])
  cpvCodes          String[] @default([])
  minValue          Decimal? @db.Decimal(14, 2)
  maxValue          Decimal? @db.Decimal(14, 2)
  deadlineWithinDays Int?
  notifyMode        String   @default("digest") // 'none' | 'digest' | 'instant'
  digestHour        Int?     @default(7)        // for digest mode: hour-of-day to send
  isActive          Boolean  @default(true)
  createdAt         DateTime @default(now())
  updatedAt         DateTime @updatedAt
  matches           TenderMatch[]

  @@index([tenantId])
}

model TenderMatch {
  id             String            @id @default(uuid())
  tenderId       String
  tender         Tender            @relation(fields: [tenderId], references: [id], onDelete: Cascade)
  savedSearchId  String
  savedSearch    TenderSavedSearch @relation(fields: [savedSearchId], references: [id], onDelete: Cascade)
  tenantId       String            // denormalized for fast tenant-scoped queries/RLS
  matchedAt      DateTime          @default(now())
  notifiedAt     DateTime?         // null = not yet sent (instant) or not yet in a digest

  @@unique([tenderId, savedSearchId])
  @@index([tenantId])
  @@index([notifiedAt])
}
```

**Why this beats a `tenantId` on `Tender`:** DÖE alone publishes thousands of Oberschwelle notices; duplicating that table N times (once per tenant) multiplies storage for zero benefit — every tenant sees the same underlying notice, just filtered differently. It also means the DÖE/AI-NetServer/cosinex/RSS pollers run **once for the whole platform**, not once per active tenant — critical for the scraping sources, where running the same scrape N times per tenant multiplies anti-bot/ToS exposure on portals that already sit at "Niedrig-Mittel" risk per the feasibility research. `TenantModuleActivation` still gates whether a tenant sees the module at all (standard Tessera marketplace pattern) — but activation controls *visibility*, not a second data copy.

**When this pattern does NOT apply:** email-alert ingestion. A tenant's alert emails arrive in *that tenant's own mailbox* (their own registered "gespeicherte Suche" on a portal) — so `TenderInboxConfig` (credentials, mirroring `DkvModuleConfig.encryptedInboxCreds`) is legitimately per-tenant, even though the `Tender` rows it produces still land in the same global table (deduped against whatever DÖE/AI-NetServer/cosinex already ingested for the same notice).

### Pattern 4: Filter Evaluation — DB Query Against the Delta, Not In-Memory Full-Table Scan

**What:** Filtering happens as a **Postgres query scoped to the just-ingested batch**, not (a) a full in-memory scan of all tenders per saved search, nor (b) a full re-scan of the entire `Tender` table on every poll.

```typescript
// tender-matching.service.ts (sketch)
async matchDelta(newOrChangedTenderIds: string[]): Promise<void> {
  const savedSearches = await this.prisma.tenderSavedSearch.findMany({ where: { isActive: true } });
  for (const search of savedSearches) {
    const where = this._buildWhereClause(search, newOrChangedTenderIds); // structured filters
    const matches = await this.prisma.tender.findMany({ where });        // DB does the heavy lifting
    // full-text keyword refinement, if keywords present, folded into the same query via
    // a raw `to_tsvector('german', title || ' ' || description) @@ plainto_tsquery(...)`
    for (const tender of matches) {
      await this.prisma.tenderMatch.upsert({
        where: { tenderId_savedSearchId: { tenderId: tender.id, savedSearchId: search.id } },
        create: { tenderId: tender.id, savedSearchId: search.id, tenantId: search.tenantId },
        update: {}, // matchedAt stays as first-seen; re-match is idempotent
      });
    }
  }
}
```

**Why DB, not in-memory:** structured filters (CPV array overlap, numeric value range, date range, region) are exactly what Postgres indexes/arrays/range queries are built for, and the corpus (all German public tenders) will reach tens of thousands of rows within the first year — loading that into Node to filter per saved search per tenant does not scale and duplicates work Postgres already does better. Keyword matching specifically should use a **Postgres full-text `tsvector` GIN index** on `title || description` (German text-search config) rather than `ILIKE '%term%'` scans — `ILIKE` on a growing table degrades linearly, GIN full-text does not.

**Why "scoped to the delta" and not the whole table every poll:** re-evaluating every saved search against the *entire* `Tender` table on every poll cycle is O(searches × table-size) repeated hourly — wasteful and re-creates matches that already exist (idempotent upsert hides the waste but not the cost). Instead, `TenderIngestionService` passes the list of tender IDs that were newly created or had a `contentHash` change in *this* poll run to `TenderMatchingService.matchDelta()`, so the query is `WHERE id IN (delta) AND <search filters>` — bounded by poll batch size (dozens to low hundreds), not table size.

**Trade-off:** if a saved search is created or edited *after* a tender was ingested, that tender won't retroactively appear in the search's matches until the next time it's re-touched (re-poll sees no change → no delta → not re-evaluated). Mitigate with an explicit "backfill" action: when a `TenderSavedSearch` is created/edited, run `matchDelta()` once against *all* tenders from the last N days (bounded, on-demand, not a recurring cost) rather than the full historical table.

### Pattern 5: Scheduled Polling — Global Cron Per Source + Per-Tenant Cron Only for Email-Alerts

**What:** Extends `DkvSchedulerService`'s `SchedulerRegistry.addCronJob()` pattern, but with two distinct job populations:

- **Global source jobs** (DÖE, RSS feeds, AI-NetServer, cosinex): one cron job per row in `TenderSourcePollConfig` (admin-managed, not tenant-scoped), named `tender-poll-${sourceConfigId}`. Interval is source-appropriate — DÖE can poll frequently (auth-free API, cheap), scraping sources should poll less often and with jitter to stay polite.
- **Per-tenant email-alert jobs**: one cron job per tenant with an active `TenderInboxConfig`, named `tender-inbox-poll-${tenantId}` — this is the one place Tessera's existing per-tenant scheduling gap (DKV's scheduler is documented as "v1 single-tenant, `findFirst()`") must actually be solved properly, since two tenants could each have their own mailbox subscribed to different portal alerts. Loop `TenderInboxConfig.findMany({ where: { isActive: true } })` on `onModuleInit` and register one job per row, mirroring `setInterval(intervalMin, tenantId)` but keyed by tenant instead of a single global slot.
- **Digest cron**: one platform-wide cron (e.g. hourly) that queries `TenderMatch` rows where `notifiedAt IS NULL` and the owning `TenderSavedSearch.notifyMode = 'digest'` and `digestHour` matches the current hour, groups by tenant+savedSearch, and hands off to `TenderMailService`.

**Change detection:** `contentHash` (see Pattern 2) is the mechanism — `TenderIngestionService.upsert()` compares the freshly computed hash against the stored one; identical → touch `lastSeenAt` only, skip matching; different → update fields, recompute `contentHash`, add to this poll's "changed" delta so it flows through matching/notification again (a deadline extension should re-surface in a saved search, a brand-new notice obviously should).

### Pattern 6: Notification Dispatch Reusing `SmtpConfig` + Fresh-Transport, Not the Global `MailerService`

**What:** `TenderMailService` is built exactly like `DkvMailService` — `nodemailer.createTransport()` freshly per send, using `SettingsService.getDecryptedSmtpConfig(tenantId)` — **not** the platform's global `MailerService`/`@nestjs-modules/mailer` (which is reserved for system emails: password reset, welcome mail, configured once at bootstrap with a static transport). This distinction already exists in the codebase (`DkvMailService` vs `MailService`) and should hold here too: tender notifications are tenant-directed business content, and the tenant may have configured their own outbound SMTP relay that differs from the platform's.

- **Instant:** triggered synchronously (or via a lightweight in-process queue, matching DKV's direct-call style — no message broker in this stack) right after `TenderMatchingService` creates a `TenderMatch` with `savedSearch.notifyMode === 'instant'`.
- **Digest:** triggered by the digest cron (Pattern 5), batching all unnotified matches for a tenant+savedSearch into one summary email, then setting `notifiedAt` on each included `TenderMatch` — same "mark as sent" idempotency DKV uses for `DkvInvoiceHistory`.

**Trade-off:** reusing per-send transport creation means every notification email opens/closes its own SMTP connection (as DKV already accepts) — fine at Tessera's realistic tender-volume/tenant-count, and it guarantees an admin's SMTP config change takes effect on the very next send without a service restart (same Pitfall-3 mitigation DKV already documents).

## Data Flow

### Ingestion → Notification Flow

```
[Cron tick: TenderSchedulerService]
    v
[Adapter.fetchTenders(config, since)] -> RawTenderRecord[]
    v
[TenderNormalizerService.normalize()] -> { fields, dedupKey, contentHash }
    v
[TenderIngestionService.upsert()] -> prisma.tender.upsert({ where: { dedupKey } })
    v (only rows that were newly created OR whose contentHash changed)
[TenderMatchingService.matchDelta(deltaIds)]
    v (per active TenderSavedSearch, DB-scoped query)
[prisma.tenderMatch.upsert()]
    v
  +--------------------------+---------------------------+
  | notifyMode='instant'     | notifyMode='digest'        |
  v                          v
[TenderMailService.sendInstant()]   [Digest cron batches unnotified matches -> sendDigest()]
    v                          v
[nodemailer via tenant SmtpConfig, fresh transport per send]
```

### Read Flow (Portal UI)

```
[User opens Ausschreibungs-Radar]
    v
GET /tenders?keywords=&bundesland=&cpv=&minValue=&deadlineBefore=  (TendersController)
    v
prisma.tender.findMany({ where: <same structured-filter builder as TenderMatchingService> })
    v
[Trefferliste UI] -> click -> GET /tenders/:id -> [Detail view, incl. rawPayload debug panel for admins]

[User manages saved searches]
    v
POST/PUT/DELETE /tenders/saved-searches  (tenant-scoped, req.tenantId)
    v
prisma.tenderSavedSearch.upsert({ ..., tenantId })
    v (on create/edit) -> one-off matchDelta() backfill against recent tenders (Pattern 4 mitigation)
```

## Scaling Considerations

| Scale | Architecture Adjustments |
|-------|---------------------------|
| Single tenant (current, internal test phase) | Exactly as designed above — global `Tender` table, one poller per source, no extra work needed even though only one tenant exists yet, because the schema is already tenant-agnostic at the data layer. |
| Multiple tenants, few saved searches each | `matchDelta()` cost scales with (poll batch size × active saved searches), both small — no changes needed. |
| Many tenants, many saved searches, high tender volume | Add a GIN full-text index on `title`/`description` (Pattern 4) before this becomes necessary, not after. If `matchDelta()` ever becomes a bottleneck, batch saved-search evaluation into a single SQL query per poll (`Tender × SavedSearch` cross-join filtered in one statement) instead of one query per search — straightforward migration since the where-clause builder is already centralized. |

### Scaling Priorities

1. **First bottleneck:** keyword filtering via `ILIKE` if the full-text index is skipped in the first slice — fix before it matters, it's a single migration (`CREATE INDEX ... USING GIN (to_tsvector('german', title || ' ' || coalesce(description,'')))`).
2. **Second bottleneck:** AI-NetServer/cosinex scraping adapters getting rate-limited or blocked as tender volume/poll frequency grows — mitigate with per-adapter jittered intervals and respecting any `Retry-After`, not a platform-wide fix.

## Anti-Patterns

### Anti-Pattern 1: Tenant-Scoping the `Tender` Table Like DKV Data

**What people do:** Copy the DKV template literally — add `tenantId` to `Tender`, run every adapter poll once per active tenant (matching `DkvSchedulerService`'s per-tenant cron intent).
**Why it's wrong:** Multiplies scraping requests against AI-NetServer/cosinex by tenant count (worse ToS exposure on portals already flagged "Niedrig-Mittel" risk), multiplies storage for identical public data, and makes cross-tenant dedup impossible (the same DÖE notice would need deduping *and* tenant-duplicating, which is incoherent).
**Do this instead:** Global `Tender` table (Pattern 3); tenant scoping lives one layer up in `TenderSavedSearch`/`TenderMatch`. Only the email-alert path (genuinely per-tenant mailbox) needs per-tenant scheduling.

### Anti-Pattern 2: One Adapter Per Portal Instead of Per Platform

**What people do:** Build a `LhsVpbwAdapter`, `Tender24Adapter`, `VergabeLandbwAdapter` as three separate classes because they're three separate portal URLs.
**Why it's wrong:** The feasibility research already established these three (plus many unlisted others) share the same AI AG NetServer fingerprint (`/NetServer/…ControllerServlet`) — one HTML/DOM shape. Three adapter classes triple the maintenance burden for zero behavioral difference; only the base URL and possibly a search-form parameter differ, which belongs in `TenderSourceConfig`, not in three code paths.
**Do this instead:** One `AiNetServerAdapter`, config-driven per portal instance (base URL + optional search params), same for a future `CosinexAdapter` covering DTVP and other cosinex Vergabemarktplatz instances.

### Anti-Pattern 3: In-Memory Filtering Across the Whole Table

**What people do:** `prisma.tender.findMany()` with no `where`, then `.filter()` in TypeScript per saved search.
**Why it's wrong:** Works fine in a demo with 50 rows, degrades badly once DÖE's Oberschwelle backbone plus scraped Unterschwelle notices accumulate over months, and re-does full-table work on every poll instead of scoping to the delta.
**Do this instead:** Pattern 4 — structured Prisma `where` + Postgres full-text index, scoped to the newly-changed batch.

## Integration Points

### External Services

| Service | Integration Pattern | Notes |
|---------|----------------------|-------|
| DÖE OpenData API (`oeffentlichevergabe.de`) | Auth-free HTTP GET against the OpenData/Swagger-documented endpoints; paginate; filter by `publishedAt`/last-poll cursor | eForms-DE / **OCDS `ocds-mnwr74`** / CSV formats available — prefer the OCDS export for direct field alignment with the `Tender` schema; ~75% of market value by € per feasibility doc, zero scraping/ToS risk |
| AI AG NetServer portals (lhs-vpbw, tender24, vergabe.landbw, + others) | HTML scrape of the public search result pages (no login required for search) | One adapter, config-driven per instance; feasibility doc rates ToS risk "Niedrig-Mittel" — implement politely (rate limit, honest UA string, cache ETags if offered) |
| cosinex Vergabemarktplatz (DTVP + other Länder instances) | HTML scrape of public search, structurally incompatible with AI-NetServer — separate adapter | Reusable across NRW/BB/NI/RLP cosinex instances per feasibility doc |
| RSS feeds (subreport-elvis, service.bund.de) | Standard RSS/Atom parse (e.g. `rss-parser` or `fast-xml-parser`), config-driven feed URL list | Lowest ToS risk of the scraped sources; service.bund.de rated "Niedrig" |
| Portal email alerts (Unterschwelle long tail, 8 of 10 portals) | Reuses `InboxProvider` (extracted `ImapProvider`/`ExchangeInboxProvider`) — per-tenant mailbox subscribed to each portal's native "gespeicherte Suche" alert | Requires manual one-time setup per portal (register a saved search on the portal itself); the module only ingests+parses the resulting alert emails, does not create the portal-side saved search |
| TED API v3 | Optional, deprioritized — keyless, EU-wide, largely redundant to DÖE for DE-only coverage | Build order: only if EU-wide coverage becomes a requirement later |

### Internal Boundaries

| Boundary | Communication | Notes |
|----------|----------------|-------|
| `TendersModule` ↔ `ModuleRegistryModule` | Direct DI import, `OnModuleInit` self-seed (`tenders.seed.ts`) — identical to `dkv.seed.ts` | New module import, no registry changes needed beyond the self-seed call |
| `TendersModule` ↔ `InboxModule` (new, extracted) | Direct DI import; `EmailAlertAdapter` depends on `ImapProvider`/`ExchangeInboxProvider` | Requires the one-time extraction of `inbox/` out of `dkv/` (see Structure Rationale) |
| `TendersModule` ↔ `SettingsModule` | Direct DI import, reused unmodified — `SettingsService.getDecryptedSmtpConfig(tenantId)` | Same pattern as `DkvModule` |
| `TendersModule` ↔ `TenantMiddleware`/`TenantGuard` | `req.tenantId` extraction for saved-search/notification endpoints only — **not** for `GET /tenders` list/detail, which is platform-global read access gated only by `TenantModuleActivation` (module licensing), not by tenant-owned data | This is the one controller where "tenant-scoped" and "tenant-gated" genuinely differ — worth flagging explicitly in the phase plan so it isn't implemented as a blanket `where: { tenantId }` by habit |
| `TendersController` ↔ `module-loader.ts` (web) | New `MODULE_REGISTRY['tender-radar']` entry, same as `cert-manager`/`dkv-fleet` | One-line addition, whitelist pattern (`T-03-09`) — must not be skipped or the module page 404s even if activated |
| `AppModule` ↔ `TendersModule` | New import in `app.module.ts` | One line |

## Build Order — Ships DÖE-First as a Usable Slice

Ordered by dependency; each step after step 8 is additive and doesn't touch the pipeline built before it (the point of the adapter abstraction).

**Phase A — DÖE-only usable slice (schema + one source + filter + UI + notification, end-to-end):**
1. Prisma migration: `Tender`, `TenderSourcePollConfig`, `TenderSavedSearch`, `TenderMatch` (+ GIN full-text index on `title`/`description`).
2. `TendersModule` skeleton + `tenders.seed.ts` (module-registry self-seed, category e.g. `procurement`).
3. `TenderSourceAdapter` interface + `DoeOpenDataAdapter` (auth-free, OCDS-formatted fetch).
4. `TenderNormalizerService` (OCDS release → `Tender` fields, `ocid`-based dedup key, `contentHash`).
5. `TenderIngestionService` (poll → normalize → upsert → change-detect) + `TenderSchedulerService` (single global cron for DÖE).
6. `TenderMatchingService` (DB-query filter evaluation, Pattern 4) + `TendersController` (`GET /tenders`, `GET /tenders/:id`, saved-search CRUD).
7. `apps/web/.../modules/tender-radar/` — trefferliste + filter UI + saved-search management + `MODULE_REGISTRY` entry. **This alone is already a usable, demoable slice** — DÖE covers ~75% of market value by € per feasibility doc.
8. `TenderMailService` (instant + digest, reusing `SmtpConfig`) + digest cron. Closes the loop on the milestone's "optional E-Mail-Versand" requirement using only the DÖE source.

**Phase B — Scraping adapters for the Unterschwelle long tail (pipeline unchanged, adapters only):**
9. `AiNetServerAdapter` (covers lhs-vpbw, tender24, vergabe.landbw + any future AI AG portal via config).
10. `CosinexAdapter` (covers DTVP; reusable for other cosinex Länder marketplaces later).

**Phase C — RSS + email-alert (lowest ROI per feasibility doc, do last):**
11. `RssAdapter` (subreport-elvis, service.bund.de).
12. Extract `inbox/` shared module out of `dkv/` (prerequisite refactor).
13. `TenderInboxConfig` (per-tenant, mirrors `DkvModuleConfig` credential pattern) + `EmailAlertAdapter` + per-tenant scheduler jobs.

**Explicitly out of this build order:** TED API v3 (redundant to DÖE for DE-only), vergabe24/aumass (AGB-prohibited scraping — feasibility doc flags these as avoid).

## New vs Modified — Explicit Inventory

**New:**
- `apps/api/src/tenders/` — entire module (controller, services, scheduler, seed, dto/, adapters/, types)
- `apps/api/src/inbox/` — extracted shared inbox module
- Prisma models: `Tender`, `TenderSourcePollConfig`, `TenderSavedSearch`, `TenderMatch`, `TenderInboxConfig`
- `apps/web/src/app/(portal)/modules/tender-radar/` — list, detail, saved-search, settings pages

**Modified:**
- `apps/api/prisma/schema.prisma` — add the 5 new models + migration
- `apps/api/src/dkv/` — `providers/` folder removed, imports `InboxModule` instead; `dkv.types.ts` trimmed of `InboxConfig`/`InboxEmail`/`InboxAttachment` (moved to `inbox/inbox.types.ts`)
- `apps/api/src/app.module.ts` — import `TendersModule` (and `InboxModule` if not auto-imported via `TendersModule`'s own imports)
- `apps/web/src/lib/module-loader.ts` — add `'tender-radar'` entry to `MODULE_REGISTRY`

**Explicitly NOT modified:** `module-registry.service.ts`, `prisma-tenant.extension.ts` (`forTenant`), `mail.module.ts`/`mail.service.ts` (global system mailer stays untouched — tender notifications use the DKV-style per-tenant transport pattern instead), `settings.service.ts`.

## Sources

- Existing codebase (verified by direct read): `apps/api/src/dkv/*`, `apps/api/src/module-registry/*`, `apps/api/src/mail/*`, `apps/api/src/prisma/prisma-tenant.extension.ts`, `apps/api/src/tenant/tenant.middleware.ts`, `apps/api/prisma/schema.prisma`, `apps/web/src/lib/module-loader.ts`, `apps/web/src/app/(portal)/modules/dkv-fleet/*` — HIGH confidence, ground truth.
- `.planning/research/ausschreibungs-portale-feasibility.md` (2026-07-16) — portal platform fingerprints, ToS risk ratings, DÖE/TED coverage estimates — HIGH confidence (project's own prior research).
- [OCDS Release Reference — Open Contracting Data Standard 1.1.5](https://standard.open-contracting.org/latest/en/schema/reference/) — core schema fields (ocid, release, tender, parties, buyer) — MEDIUM confidence (public spec, not project-specific).
- [OCDS Building Blocks](https://standard.open-contracting.org/latest/en/getting_started/building_blocks/) — OCID composition (registered prefix + publisher-chosen process id) — MEDIUM confidence.
- [oeffentlichevergabe.de OpenData Swagger UI](https://oeffentlichevergabe.de/documentation/swagger-ui/opendata/index.html) — confirms `ocds-mnwr74` as the registered German federal OCDS prefix (registered 2023-02-06) and CC-Zero licensing — MEDIUM confidence (page requires JS to render full endpoint/pagination detail; exact pagination parameters remain an open verification point, already flagged in the feasibility doc).

---
*Architecture research for: Tessera Ausschreibungs-Radar module (v1.1 milestone)*
*Researched: 2026-07-17*