Files
schalli d2ac1997d6 docs: complete project research
Ausschreibungs-Radar (v1.1) STACK/FEATURES/ARCHITECTURE/PITFALLS research plus SUMMARY.md synthesis.
2026-07-17 10:12:59 +02:00

40 KiB
Raw Permalink Blame History

Architecture Research — Ausschreibungs-Radar Module Integration

Domain: Multi-source tender/procurement-notice ingestion module for an existing NestJS 11 modular-monolith platform (Tessera) Researched: 2026-07-17 Confidence: HIGH (module registration, tenant scoping, inbox/mail reuse — verified against existing dkv/ and module-registry/ source) / MEDIUM (OCDS field mapping, DÖE API pagination — verified via OCDS spec + DÖE Swagger listing, not a live API call)

Note: This supersedes the v1.0 platform-level ARCHITECTURE.md (2026-06-18, described a generic Fastify/Traefik shell that predates the stack decisions now recorded in CLAUDE.md). This file is scoped to the v1.1 Ausschreibungs-Radar milestone and describes how the new module integrates into the actual, current NestJS 11 + Next.js codebase — not a from-scratch platform design.

Summary

The Ausschreibungs-Radar module ("tenders" module) is a new, self-contained NestJS module built strictly on top of existing Tessera infrastructure — module-registry self-seeding, @nestjs/schedule dynamic cron jobs, the ImapProvider/ExchangeInboxProvider inbox abstraction, tenant-scoped SMTP via SmtpConfig + fresh-transport-per-send, and the module-loader-driven Next.js portal route. The one architectural decision that breaks from the DKV template is deliberate and important: tender data is platform-global, not tenant-owned. DKV invoices belong to one tenant; a DÖE/AI-NetServer/cosinex tender notice is a public fact relevant to every tenant. Ingestion, normalization, and dedup therefore run once for the whole platform; only saved searches, matches, and notification preferences are tenant-scoped. Getting this split right is the single highest-leverage decision in this design — getting it wrong means N-times redundant scraping/storage and N-times the anti-bot exposure per portal.

Standard Architecture

System Overview

+---------------------------------------------------------------------------+
|                         SOURCE ADAPTERS (new)                             |
|  +-----------+ +--------------+ +-----------+ +--------+ +--------------+|
|  |DoeOpenData| |AiNetServer   | |Cosinex    | |Rss     | |EmailAlert    ||
|  |Adapter    | |Adapter       | |Adapter    | |Adapter | |Adapter       ||
|  |(API,auth- | |(HTML scrape, | |(HTML      | |(feed   | |(reuses Imap/ ||
|  | free)     | | public       | | scrape,   | | parse) | | Exchange     ||
|  |           | | search)      | | public)   | |        | | InboxProvider||
|  +-----+-----+ +------+-------+ +-----+-----+ +---+----+ +------+-------+|
|        |  all implement TenderSourceAdapter -> RawTenderRecord[] |       |
+--------+--------------+---------------+-----------+-----------------+---+
         `--------------`-------+-------`-----------`-------------'
                                v
+---------------------------------------------------------------------------+
|                 TenderIngestionService (new, global -- no tenantId)       |
|  normalize(raw, sourceType) -> Tender (OCDS-oriented)                     |
|  computeDedupKey() -> upsert by dedupKey -> diff contentHash -> mark changed|
+-------------------------------+-------------------------------------------+
                                 v (only NEW / CHANGED tenders this poll)
+---------------------------------------------------------------------------+
|           TenderMatchingService (new) -- DB-query filter evaluation       |
|  for each active TenderSavedSearch: Prisma `where` over the delta batch   |
|  (structured filters) + Postgres full-text search (keywords) -> TenderMatch|
+-------------------------------+-------------------------------------------+
                                 v
+---------------------------------------------------------------------------+
|        TenderMailService (new) -- reuses SmtpConfig + fresh-transport     |
|  instant: send on TenderMatch create   digest: cron batch per SavedSearch |
+---------------------------------------------------------------------------+
                         |
+------------------------+---------------------------------------------------+
|  TendersController (new) -- GET /tenders (search+filter+paginate),         |
|  /tenders/:id, /tenders/saved-searches (CRUD, tenant-scoped),              |
|  /tenders/source-config (admin, global), /tenders/check-now                |
+------------------------+---------------------------------------------------+
                          v
      apps/web/.../modules/tender-radar/  (new Next.js module UI)

Component Responsibilities

Component Responsibility Scope New/Modified
TenderSourceAdapter implementations Fetch raw records from one source, no normalization Global New
TenderNormalizerService Map each source's raw shape → unified Tender fields, compute dedupKey/contentHash Global New
TenderIngestionService Orchestrate poll → normalize → upsert → change-detect per source Global New
TenderSchedulerService Dynamic cron per global source + per-tenant cron for email-alert ingestion Mixed New
TenderMatchingService Evaluate active saved searches against the new/changed delta Per-tenant read, global data New
TenderMailService Send instant/digest notification emails via tenant SMTP Per-tenant New
TendersController REST endpoints for list/detail/saved-search CRUD/admin source-config Mixed New
Shared InboxModule (relocated) InboxProvider interface + ImapProvider/ExchangeInboxProvider Platform-shared New (extracted from dkv/)
ModuleRegistryService Self-seed tender-radar module row Platform Reused unmodified
SettingsService / SmtpConfig Decrypted per-tenant SMTP for notification sends Per-tenant Reused unmodified
CalendarCryptoService AES-256-GCM encryption for any stored credentials Platform Reused unmodified
module-loader.ts MODULE_REGISTRY Whitelist entry mapping slug → lazy component Platform Modified (one entry)
AppModule Import TendersModule Platform Modified (one import)
apps/api/src/
├── inbox/                          # NEW -- extracted shared module (was dkv/providers/)
│   ├── inbox.module.ts             # exports ImapProvider, ExchangeInboxProvider
│   ├── inbox-provider.interface.ts # InboxProvider contract (moved from dkv.types.ts)
│   ├── imap.provider.ts            # moved verbatim from dkv/providers/
│   ├── exchange-inbox.provider.ts  # moved verbatim from dkv/providers/
│   └── inbox.types.ts              # InboxConfig / InboxEmail / InboxAttachment
│
├── dkv/                             # MODIFIED -- imports InboxModule instead of local providers/
│   └── ...                          # (providers/ folder removed, dkv.types.ts trimmed)
│
├── tenders/                         # NEW -- Ausschreibungs-Radar module
│   ├── tenders.module.ts
│   ├── tenders.controller.ts        # public list/detail + tenant saved-search CRUD + admin source-config
│   ├── tenders.seed.ts              # seeds 'tender-radar' into ModuleRegistry
│   ├── tender-ingestion.service.ts  # poll → normalize → upsert → change-detect (per source)
│   ├── tender-matching.service.ts   # SavedSearch → Prisma where-clause → TenderMatch
│   ├── tender-mail.service.ts       # instant + digest notification sends
│   ├── tender-scheduler.service.ts  # SchedulerRegistry cron: 1 per global source + 1 per tenant (email-alert)
│   ├── tender.types.ts              # RawTenderRecord, NormalizedTenderFields, SourceType
│   ├── dto/
│   │   ├── saved-search.dto.ts
│   │   ├── source-config.dto.ts
│   │   └── tender-query.dto.ts      # pagination + filter query params for GET /tenders
│   └── adapters/
│       ├── tender-source-adapter.interface.ts   # fetchTenders(config, since) → RawTenderRecord[]
│       ├── doe-opendata.adapter.ts              # Build order Phase A
│       ├── ai-netserver.adapter.ts              # Build order Phase B
│       ├── cosinex.adapter.ts                   # Build order Phase B
│       ├── rss.adapter.ts                       # Build order Phase C
│       └── email-alert.adapter.ts               # Build order Phase C (uses InboxModule)
│
apps/web/src/app/(portal)/modules/
├── tender-radar/                    # follow the existing static per-module convention (dkv-fleet, cert-manager)
│   ├── page.tsx                     # searchable trefferliste + filter sidebar
│   ├── [id]/page.tsx                # tender detail view
│   ├── saved-searches/page.tsx      # saved search CRUD UI
│   └── settings/page.tsx            # admin: source poll config, email-alert inbox config

Structure Rationale

  • inbox/ extraction is a prerequisite, not optional. ImapProvider/ExchangeInboxProvider are already generic over InboxConfig/InboxEmail — nothing in them is DKV-specific. Today they live in dkv/providers/ and their types live in dkv.types.ts, so tenders/ would otherwise have to import from inside another feature module's internals (../dkv/providers/imap.provider), which couples two unrelated features and breaks if DKV is ever restructured. Moving them to a shared inbox/ module once, and updating dkv.module.ts to import InboxModule instead, costs one small refactor now and pays for every future module that needs inbox polling (already two: DKV, Tenders).
  • tenders/adapters/ mirrors dkv/providers/ — same rationale as DKV: consumers (TenderIngestionService) depend only on the TenderSourceAdapter interface, never on a concrete adapter. This is what makes "ship DÖE first, add AI-NetServer/cosinex/RSS/email later" possible without touching the ingestion/matching/notification pipeline.
  • Normalizer is separate from adapters, unlike DKV where DkvParserService is a single PDF parser. Here there are 5 structurally incompatible raw shapes (eForms/OCDS JSON, two flavors of scraped HTML, RSS/Atom XML, free-text alert emails). Each adapter can either normalize inline or delegate to a per-source mapping function inside TenderNormalizerService — either way, the interface boundary is RawTenderRecord[] → Tender[], so the ingestion orchestrator never branches on source type.
  • Web module UI follows the existing static modules/<slug>/ folder pattern seen in dkv-fleet/ and cert-manager/ (not the dynamic [category]/[moduleSlug]/ route also present in the codebase for vehicle/settings sub-pages) — match whichever of the two conventions the team is actively converging on at execution time; both are already present, so this is a phase-planning decision, not an open architectural question.

Architectural Patterns

Pattern 1: Source-Adapter Abstraction (TenderSourceAdapter)

What: One interface, N implementations — directly analogous to InboxProvider (fetchPdfAttachments → fetchTenders).

// tenders/adapters/tender-source-adapter.interface.ts
export interface RawTenderRecord {
  sourceType: SourceType;          // 'doe-opendata' | 'ai-netserver' | 'cosinex' | 'rss' | 'email-alert'
  sourcePortal: string;            // 'doe' | 'lhs-vpbw' | 'tender24' | 'vergabe.landbw' | 'dtvp' | 'subreport-elvis' | 'service.bund.de'
  sourceRawId: string;             // portal-native id/notice number, pre-normalization
  sourceUrl: string;
  fetchedAt: Date;
  payload: unknown;                // raw JSON/HTML-extract/RSS-item/email-body — kept for rawPayload + reprocessing
}

export interface TenderSourceAdapter {
  readonly sourceType: SourceType;
  /** since: only fetch records new/changed after this timestamp (cursor from TenderSourcePollConfig.lastPolledAt) */
  fetchTenders(config: TenderSourceConfig, since?: Date): Promise<RawTenderRecord[]>;
  testConnection?(config: TenderSourceConfig): Promise<{ success: boolean; message?: string }>;
}

When to use: Any time a pipeline must ingest structurally different sources into one output shape without the orchestrator knowing about each source. Same pattern Tessera already uses for ImapProvider/ExchangeInboxProvider.

Trade-offs: Each adapter owns its own retry/rate-limit/anti-bot logic (AI-NetServer and cosinex adapters need polite scraping delays; DÖE/RSS don't). The interface intentionally does not prescribe HTTP client or scraping library — TenderSourceConfig is adapter-specific (a discriminated union or Json blob per sourceType), same as InboxConfig covers both IMAP and Exchange with one shape only because both providers happen to share fields; here they mostly won't, so TenderSourceConfig should be Json on the Prisma side with adapter-specific Zod/DTO validation, not a single flat interface.

Pattern 2: Normalized OCDS-Oriented Schema + Cross-Source Dedup Key

What: One Tender Prisma model absorbing eForms/OCDS (DÖE), scraped HTML (AI-NetServer, cosinex), RSS, and email-alert text — with a stable dedup key so the same real-world procurement notice appearing on multiple sources (e.g. an AI-NetServer notice that later also appears on DÖE once it crosses the EU threshold, or the same DÖE OCID reappearing on a poll re-run) collapses to one row.

model Tender {
  id              String   @id @default(uuid())

  // OCDS-oriented core (see standard.open-contracting.org/latest/en/schema/reference/)
  ocid            String?  // Open Contracting ID, e.g. "ocds-mnwr74-XXXXXXXX" — present when sourced via DÖE/TED
  noticeId        String?  // portal-native notice/procedure number (AI-NetServer, cosinex, RSS items)
  title           String
  description     String?  @db.Text
  buyerName       String?
  buyerId         String?  // e.g. Vergabestelle-ID if the source exposes it
  procedureType   String?  // OCDS tender.procurementMethod / procurementMethodDetails
  status          String   @default("active") // 'active' | 'awarded' | 'cancelled' | 'expired'
  cpvCodes        String[] @default([])
  region          String?
  plz             String?
  bundesland      String?
  estimatedValue  Decimal? @db.Decimal(14, 2)
  currency        String?  @default("EUR")
  publishedAt     DateTime?
  deadlineAt      DateTime?

  // Source + dedup
  sourceType      String   // 'doe-opendata' | 'ai-netserver' | 'cosinex' | 'rss' | 'email-alert'
  sourcePortal    String   // 'doe' | 'lhs-vpbw' | 'dtvp' | 'subreport-elvis' | ...
  sourceUrl       String?
  dedupKey        String   @unique   // see dedup strategy below
  contentHash     String             // hash of normalized fields — detects "changed" vs "identical re-poll"
  rawPayload      Json               // original adapter payload — debugging + future re-normalization

  firstSeenAt     DateTime @default(now())
  lastSeenAt      DateTime @updatedAt
  createdAt       DateTime @default(now())

  matches         TenderMatch[]

  @@index([sourceType])
  @@index([deadlineAt])
  @@index([publishedAt])
  @@index([bundesland])
}

Dedup key strategy (priority order, computed by TenderNormalizerService):

  1. ocid — when the source provides an OCDS Open Contracting ID (always true for DÖE/TED; the platform's registered OCDS prefix is ocds-mnwr74). OCID is designed exactly for this — joining the same contracting process across publishers.
  2. ${sourcePortal}:${noticeId} — when the portal exposes a stable native notice/procedure number (AI-NetServer Bietercockpit ID, cosinex Vergabenummer, RSS item guid, email-alert reference number). This is the primary key for scraped/RSS/email sources, since they never carry an OCID.
  3. sha256(sourcePortal + normalizedTitle + buyerName + deadlineAt) — last-resort fallback only when a source gives neither an OCID nor a stable ID (should be rare; flag these rows for manual review via a dedupConfidence: 'low' marker if this path is hit).

dedupKey is the @unique upsert target: prisma.tender.upsert({ where: { dedupKey }, ... }). contentHash (hash of title+deadline+value+status) is separate from dedupKey — it answers "did anything about this same notice change since we last saw it" (deadline extension, cancellation), which is what should trigger re-matching and potentially a "notice updated" notification, whereas an unchanged re-poll should just bump lastSeenAt and stop.

Trade-offs: A single wide table is simpler to query/filter/index than per-source tables + a union view, and matches how the UI wants to browse ("one trefferliste across all sources"). The cost is that source-specific fields that don't map cleanly (e.g. cosinex-specific metadata) live only in rawPayload (Json, unindexed) — acceptable, since the feasibility research shows the cross-source overlap (title, buyer, deadline, value, CPV, region) covers what filtering/notification actually need.

Pattern 3: Global Data, Per-Tenant Filtering (the key deviation from the DKV template)

What: Tender rows carry no tenantId — they are platform-wide. Per-tenant scoping happens one layer up, in TenderSavedSearch and TenderMatch.

model TenderSavedSearch {
  id                String   @id @default(uuid())
  tenantId          String
  userId            String?  // null = tenant-wide search, set = personal search
  name              String
  keywords          String[] @default([])   // full-text match against title+description
  bundeslaender     String[] @default([])
  plzPrefixes       String[] @default([])
  cpvCodes          String[] @default([])
  minValue          Decimal? @db.Decimal(14, 2)
  maxValue          Decimal? @db.Decimal(14, 2)
  deadlineWithinDays Int?
  notifyMode        String   @default("digest") // 'none' | 'digest' | 'instant'
  digestHour        Int?     @default(7)        // for digest mode: hour-of-day to send
  isActive          Boolean  @default(true)
  createdAt         DateTime @default(now())
  updatedAt         DateTime @updatedAt
  matches           TenderMatch[]

  @@index([tenantId])
}

model TenderMatch {
  id             String            @id @default(uuid())
  tenderId       String
  tender         Tender            @relation(fields: [tenderId], references: [id], onDelete: Cascade)
  savedSearchId  String
  savedSearch    TenderSavedSearch @relation(fields: [savedSearchId], references: [id], onDelete: Cascade)
  tenantId       String            // denormalized for fast tenant-scoped queries/RLS
  matchedAt      DateTime          @default(now())
  notifiedAt     DateTime?         // null = not yet sent (instant) or not yet in a digest

  @@unique([tenderId, savedSearchId])
  @@index([tenantId])
  @@index([notifiedAt])
}

Why this beats a tenantId on Tender: DÖE alone publishes thousands of Oberschwelle notices; duplicating that table N times (once per tenant) multiplies storage for zero benefit — every tenant sees the same underlying notice, just filtered differently. It also means the DÖE/AI-NetServer/cosinex/RSS pollers run once for the whole platform, not once per active tenant — critical for the scraping sources, where running the same scrape N times per tenant multiplies anti-bot/ToS exposure on portals that already sit at "Niedrig-Mittel" risk per the feasibility research. TenantModuleActivation still gates whether a tenant sees the module at all (standard Tessera marketplace pattern) — but activation controls visibility, not a second data copy.

When this pattern does NOT apply: email-alert ingestion. A tenant's alert emails arrive in that tenant's own mailbox (their own registered "gespeicherte Suche" on a portal) — so TenderInboxConfig (credentials, mirroring DkvModuleConfig.encryptedInboxCreds) is legitimately per-tenant, even though the Tender rows it produces still land in the same global table (deduped against whatever DÖE/AI-NetServer/cosinex already ingested for the same notice).

Pattern 4: Filter Evaluation — DB Query Against the Delta, Not In-Memory Full-Table Scan

What: Filtering happens as a Postgres query scoped to the just-ingested batch, not (a) a full in-memory scan of all tenders per saved search, nor (b) a full re-scan of the entire Tender table on every poll.

// tender-matching.service.ts (sketch)
async matchDelta(newOrChangedTenderIds: string[]): Promise<void> {
  const savedSearches = await this.prisma.tenderSavedSearch.findMany({ where: { isActive: true } });
  for (const search of savedSearches) {
    const where = this._buildWhereClause(search, newOrChangedTenderIds); // structured filters
    const matches = await this.prisma.tender.findMany({ where });        // DB does the heavy lifting
    // full-text keyword refinement, if keywords present, folded into the same query via
    // a raw `to_tsvector('german', title || ' ' || description) @@ plainto_tsquery(...)`
    for (const tender of matches) {
      await this.prisma.tenderMatch.upsert({
        where: { tenderId_savedSearchId: { tenderId: tender.id, savedSearchId: search.id } },
        create: { tenderId: tender.id, savedSearchId: search.id, tenantId: search.tenantId },
        update: {}, // matchedAt stays as first-seen; re-match is idempotent
      });
    }
  }
}

Why DB, not in-memory: structured filters (CPV array overlap, numeric value range, date range, region) are exactly what Postgres indexes/arrays/range queries are built for, and the corpus (all German public tenders) will reach tens of thousands of rows within the first year — loading that into Node to filter per saved search per tenant does not scale and duplicates work Postgres already does better. Keyword matching specifically should use a Postgres full-text tsvector GIN index on title || description (German text-search config) rather than ILIKE '%term%' scans — ILIKE on a growing table degrades linearly, GIN full-text does not.

Why "scoped to the delta" and not the whole table every poll: re-evaluating every saved search against the entire Tender table on every poll cycle is O(searches × table-size) repeated hourly — wasteful and re-creates matches that already exist (idempotent upsert hides the waste but not the cost). Instead, TenderIngestionService passes the list of tender IDs that were newly created or had a contentHash change in this poll run to TenderMatchingService.matchDelta(), so the query is WHERE id IN (delta) AND <search filters> — bounded by poll batch size (dozens to low hundreds), not table size.

Trade-off: if a saved search is created or edited after a tender was ingested, that tender won't retroactively appear in the search's matches until the next time it's re-touched (re-poll sees no change → no delta → not re-evaluated). Mitigate with an explicit "backfill" action: when a TenderSavedSearch is created/edited, run matchDelta() once against all tenders from the last N days (bounded, on-demand, not a recurring cost) rather than the full historical table.

Pattern 5: Scheduled Polling — Global Cron Per Source + Per-Tenant Cron Only for Email-Alerts

What: Extends DkvSchedulerService's SchedulerRegistry.addCronJob() pattern, but with two distinct job populations:

  • Global source jobs (DÖE, RSS feeds, AI-NetServer, cosinex): one cron job per row in TenderSourcePollConfig (admin-managed, not tenant-scoped), named tender-poll-${sourceConfigId}. Interval is source-appropriate — DÖE can poll frequently (auth-free API, cheap), scraping sources should poll less often and with jitter to stay polite.
  • Per-tenant email-alert jobs: one cron job per tenant with an active TenderInboxConfig, named tender-inbox-poll-${tenantId} — this is the one place Tessera's existing per-tenant scheduling gap (DKV's scheduler is documented as "v1 single-tenant, findFirst()") must actually be solved properly, since two tenants could each have their own mailbox subscribed to different portal alerts. Loop TenderInboxConfig.findMany({ where: { isActive: true } }) on onModuleInit and register one job per row, mirroring setInterval(intervalMin, tenantId) but keyed by tenant instead of a single global slot.
  • Digest cron: one platform-wide cron (e.g. hourly) that queries TenderMatch rows where notifiedAt IS NULL and the owning TenderSavedSearch.notifyMode = 'digest' and digestHour matches the current hour, groups by tenant+savedSearch, and hands off to TenderMailService.

Change detection: contentHash (see Pattern 2) is the mechanism — TenderIngestionService.upsert() compares the freshly computed hash against the stored one; identical → touch lastSeenAt only, skip matching; different → update fields, recompute contentHash, add to this poll's "changed" delta so it flows through matching/notification again (a deadline extension should re-surface in a saved search, a brand-new notice obviously should).

Pattern 6: Notification Dispatch Reusing SmtpConfig + Fresh-Transport, Not the Global MailerService

What: TenderMailService is built exactly like DkvMailService — nodemailer.createTransport() freshly per send, using SettingsService.getDecryptedSmtpConfig(tenantId) — not the platform's global MailerService/@nestjs-modules/mailer (which is reserved for system emails: password reset, welcome mail, configured once at bootstrap with a static transport). This distinction already exists in the codebase (DkvMailService vs MailService) and should hold here too: tender notifications are tenant-directed business content, and the tenant may have configured their own outbound SMTP relay that differs from the platform's.

  • Instant: triggered synchronously (or via a lightweight in-process queue, matching DKV's direct-call style — no message broker in this stack) right after TenderMatchingService creates a TenderMatch with savedSearch.notifyMode === 'instant'.
  • Digest: triggered by the digest cron (Pattern 5), batching all unnotified matches for a tenant+savedSearch into one summary email, then setting notifiedAt on each included TenderMatch — same "mark as sent" idempotency DKV uses for DkvInvoiceHistory.

Trade-off: reusing per-send transport creation means every notification email opens/closes its own SMTP connection (as DKV already accepts) — fine at Tessera's realistic tender-volume/tenant-count, and it guarantees an admin's SMTP config change takes effect on the very next send without a service restart (same Pitfall-3 mitigation DKV already documents).

Data Flow

Ingestion → Notification Flow

[Cron tick: TenderSchedulerService]
    v
[Adapter.fetchTenders(config, since)] -> RawTenderRecord[]
    v
[TenderNormalizerService.normalize()] -> { fields, dedupKey, contentHash }
    v
[TenderIngestionService.upsert()] -> prisma.tender.upsert({ where: { dedupKey } })
    v (only rows that were newly created OR whose contentHash changed)
[TenderMatchingService.matchDelta(deltaIds)]
    v (per active TenderSavedSearch, DB-scoped query)
[prisma.tenderMatch.upsert()]
    v
  +--------------------------+---------------------------+
  | notifyMode='instant'     | notifyMode='digest'        |
  v                          v
[TenderMailService.sendInstant()]   [Digest cron batches unnotified matches -> sendDigest()]
    v                          v
[nodemailer via tenant SmtpConfig, fresh transport per send]

Read Flow (Portal UI)

[User opens Ausschreibungs-Radar]
    v
GET /tenders?keywords=&bundesland=&cpv=&minValue=&deadlineBefore=  (TendersController)
    v
prisma.tender.findMany({ where: <same structured-filter builder as TenderMatchingService> })
    v
[Trefferliste UI] -> click -> GET /tenders/:id -> [Detail view, incl. rawPayload debug panel for admins]

[User manages saved searches]
    v
POST/PUT/DELETE /tenders/saved-searches  (tenant-scoped, req.tenantId)
    v
prisma.tenderSavedSearch.upsert({ ..., tenantId })
    v (on create/edit) -> one-off matchDelta() backfill against recent tenders (Pattern 4 mitigation)

Scaling Considerations

Scale Architecture Adjustments
Single tenant (current, internal test phase) Exactly as designed above — global Tender table, one poller per source, no extra work needed even though only one tenant exists yet, because the schema is already tenant-agnostic at the data layer.
Multiple tenants, few saved searches each matchDelta() cost scales with (poll batch size × active saved searches), both small — no changes needed.
Many tenants, many saved searches, high tender volume Add a GIN full-text index on title/description (Pattern 4) before this becomes necessary, not after. If matchDelta() ever becomes a bottleneck, batch saved-search evaluation into a single SQL query per poll (Tender × SavedSearch cross-join filtered in one statement) instead of one query per search — straightforward migration since the where-clause builder is already centralized.

Scaling Priorities

  1. First bottleneck: keyword filtering via ILIKE if the full-text index is skipped in the first slice — fix before it matters, it's a single migration (CREATE INDEX ... USING GIN (to_tsvector('german', title || ' ' || coalesce(description,'')))).
  2. Second bottleneck: AI-NetServer/cosinex scraping adapters getting rate-limited or blocked as tender volume/poll frequency grows — mitigate with per-adapter jittered intervals and respecting any Retry-After, not a platform-wide fix.

Anti-Patterns

Anti-Pattern 1: Tenant-Scoping the Tender Table Like DKV Data

What people do: Copy the DKV template literally — add tenantId to Tender, run every adapter poll once per active tenant (matching DkvSchedulerService's per-tenant cron intent). Why it's wrong: Multiplies scraping requests against AI-NetServer/cosinex by tenant count (worse ToS exposure on portals already flagged "Niedrig-Mittel" risk), multiplies storage for identical public data, and makes cross-tenant dedup impossible (the same DÖE notice would need deduping and tenant-duplicating, which is incoherent). Do this instead: Global Tender table (Pattern 3); tenant scoping lives one layer up in TenderSavedSearch/TenderMatch. Only the email-alert path (genuinely per-tenant mailbox) needs per-tenant scheduling.

Anti-Pattern 2: One Adapter Per Portal Instead of Per Platform

What people do: Build a LhsVpbwAdapter, Tender24Adapter, VergabeLandbwAdapter as three separate classes because they're three separate portal URLs. Why it's wrong: The feasibility research already established these three (plus many unlisted others) share the same AI AG NetServer fingerprint (/NetServer/…ControllerServlet) — one HTML/DOM shape. Three adapter classes triple the maintenance burden for zero behavioral difference; only the base URL and possibly a search-form parameter differ, which belongs in TenderSourceConfig, not in three code paths. Do this instead: One AiNetServerAdapter, config-driven per portal instance (base URL + optional search params), same for a future CosinexAdapter covering DTVP and other cosinex Vergabemarktplatz instances.

Anti-Pattern 3: In-Memory Filtering Across the Whole Table

What people do: prisma.tender.findMany() with no where, then .filter() in TypeScript per saved search. Why it's wrong: Works fine in a demo with 50 rows, degrades badly once DÖE's Oberschwelle backbone plus scraped Unterschwelle notices accumulate over months, and re-does full-table work on every poll instead of scoping to the delta. Do this instead: Pattern 4 — structured Prisma where + Postgres full-text index, scoped to the newly-changed batch.

Integration Points

External Services

Service Integration Pattern Notes
DÖE OpenData API (oeffentlichevergabe.de) Auth-free HTTP GET against the OpenData/Swagger-documented endpoints; paginate; filter by publishedAt/last-poll cursor eForms-DE / OCDS ocds-mnwr74 / CSV formats available — prefer the OCDS export for direct field alignment with the Tender schema; ~75% of market value by € per feasibility doc, zero scraping/ToS risk
AI AG NetServer portals (lhs-vpbw, tender24, vergabe.landbw, + others) HTML scrape of the public search result pages (no login required for search) One adapter, config-driven per instance; feasibility doc rates ToS risk "Niedrig-Mittel" — implement politely (rate limit, honest UA string, cache ETags if offered)
cosinex Vergabemarktplatz (DTVP + other Länder instances) HTML scrape of public search, structurally incompatible with AI-NetServer — separate adapter Reusable across NRW/BB/NI/RLP cosinex instances per feasibility doc
RSS feeds (subreport-elvis, service.bund.de) Standard RSS/Atom parse (e.g. rss-parser or fast-xml-parser), config-driven feed URL list Lowest ToS risk of the scraped sources; service.bund.de rated "Niedrig"
Portal email alerts (Unterschwelle long tail, 8 of 10 portals) Reuses InboxProvider (extracted ImapProvider/ExchangeInboxProvider) — per-tenant mailbox subscribed to each portal's native "gespeicherte Suche" alert Requires manual one-time setup per portal (register a saved search on the portal itself); the module only ingests+parses the resulting alert emails, does not create the portal-side saved search
TED API v3 Optional, deprioritized — keyless, EU-wide, largely redundant to DÖE for DE-only coverage Build order: only if EU-wide coverage becomes a requirement later

Internal Boundaries

Boundary Communication Notes
TendersModule ↔ ModuleRegistryModule Direct DI import, OnModuleInit self-seed (tenders.seed.ts) — identical to dkv.seed.ts New module import, no registry changes needed beyond the self-seed call
TendersModule ↔ InboxModule (new, extracted) Direct DI import; EmailAlertAdapter depends on ImapProvider/ExchangeInboxProvider Requires the one-time extraction of inbox/ out of dkv/ (see Structure Rationale)
TendersModule ↔ SettingsModule Direct DI import, reused unmodified — SettingsService.getDecryptedSmtpConfig(tenantId) Same pattern as DkvModule
TendersModule ↔ TenantMiddleware/TenantGuard req.tenantId extraction for saved-search/notification endpoints only — not for GET /tenders list/detail, which is platform-global read access gated only by TenantModuleActivation (module licensing), not by tenant-owned data This is the one controller where "tenant-scoped" and "tenant-gated" genuinely differ — worth flagging explicitly in the phase plan so it isn't implemented as a blanket where: { tenantId } by habit
TendersController ↔ module-loader.ts (web) New MODULE_REGISTRY['tender-radar'] entry, same as cert-manager/dkv-fleet One-line addition, whitelist pattern (T-03-09) — must not be skipped or the module page 404s even if activated
AppModule ↔ TendersModule New import in app.module.ts One line

Build Order — Ships DÖE-First as a Usable Slice

Ordered by dependency; each step after step 8 is additive and doesn't touch the pipeline built before it (the point of the adapter abstraction).

Phase A — DÖE-only usable slice (schema + one source + filter + UI + notification, end-to-end):

  1. Prisma migration: Tender, TenderSourcePollConfig, TenderSavedSearch, TenderMatch (+ GIN full-text index on title/description).
  2. TendersModule skeleton + tenders.seed.ts (module-registry self-seed, category e.g. procurement).
  3. TenderSourceAdapter interface + DoeOpenDataAdapter (auth-free, OCDS-formatted fetch).
  4. TenderNormalizerService (OCDS release → Tender fields, ocid-based dedup key, contentHash).
  5. TenderIngestionService (poll → normalize → upsert → change-detect) + TenderSchedulerService (single global cron for DÖE).
  6. TenderMatchingService (DB-query filter evaluation, Pattern 4) + TendersController (GET /tenders, GET /tenders/:id, saved-search CRUD).
  7. apps/web/.../modules/tender-radar/ — trefferliste + filter UI + saved-search management + MODULE_REGISTRY entry. This alone is already a usable, demoable slice — DÖE covers ~75% of market value by € per feasibility doc.
  8. TenderMailService (instant + digest, reusing SmtpConfig) + digest cron. Closes the loop on the milestone's "optional E-Mail-Versand" requirement using only the DÖE source.

Phase B — Scraping adapters for the Unterschwelle long tail (pipeline unchanged, adapters only): 9. AiNetServerAdapter (covers lhs-vpbw, tender24, vergabe.landbw + any future AI AG portal via config). 10. CosinexAdapter (covers DTVP; reusable for other cosinex Länder marketplaces later).

Phase C — RSS + email-alert (lowest ROI per feasibility doc, do last): 11. RssAdapter (subreport-elvis, service.bund.de). 12. Extract inbox/ shared module out of dkv/ (prerequisite refactor). 13. TenderInboxConfig (per-tenant, mirrors DkvModuleConfig credential pattern) + EmailAlertAdapter + per-tenant scheduler jobs.

Explicitly out of this build order: TED API v3 (redundant to DÖE for DE-only), vergabe24/aumass (AGB-prohibited scraping — feasibility doc flags these as avoid).

New vs Modified — Explicit Inventory

New:

  • apps/api/src/tenders/ — entire module (controller, services, scheduler, seed, dto/, adapters/, types)
  • apps/api/src/inbox/ — extracted shared inbox module
  • Prisma models: Tender, TenderSourcePollConfig, TenderSavedSearch, TenderMatch, TenderInboxConfig
  • apps/web/src/app/(portal)/modules/tender-radar/ — list, detail, saved-search, settings pages

Modified:

  • apps/api/prisma/schema.prisma — add the 5 new models + migration
  • apps/api/src/dkv/ — providers/ folder removed, imports InboxModule instead; dkv.types.ts trimmed of InboxConfig/InboxEmail/InboxAttachment (moved to inbox/inbox.types.ts)
  • apps/api/src/app.module.ts — import TendersModule (and InboxModule if not auto-imported via TendersModule's own imports)
  • apps/web/src/lib/module-loader.ts — add 'tender-radar' entry to MODULE_REGISTRY

Explicitly NOT modified: module-registry.service.ts, prisma-tenant.extension.ts (forTenant), mail.module.ts/mail.service.ts (global system mailer stays untouched — tender notifications use the DKV-style per-tenant transport pattern instead), settings.service.ts.

Sources

  • Existing codebase (verified by direct read): apps/api/src/dkv/*, apps/api/src/module-registry/*, apps/api/src/mail/*, apps/api/src/prisma/prisma-tenant.extension.ts, apps/api/src/tenant/tenant.middleware.ts, apps/api/prisma/schema.prisma, apps/web/src/lib/module-loader.ts, apps/web/src/app/(portal)/modules/dkv-fleet/* — HIGH confidence, ground truth.
  • .planning/research/ausschreibungs-portale-feasibility.md (2026-07-16) — portal platform fingerprints, ToS risk ratings, DÖE/TED coverage estimates — HIGH confidence (project's own prior research).
  • OCDS Release Reference — Open Contracting Data Standard 1.1.5 — core schema fields (ocid, release, tender, parties, buyer) — MEDIUM confidence (public spec, not project-specific).
  • OCDS Building Blocks — OCID composition (registered prefix + publisher-chosen process id) — MEDIUM confidence.
  • oeffentlichevergabe.de OpenData Swagger UI — confirms ocds-mnwr74 as the registered German federal OCDS prefix (registered 2023-02-06) and CC-Zero licensing — MEDIUM confidence (page requires JS to render full endpoint/pagination detail; exact pagination parameters remain an open verification point, already flagged in the feasibility doc).

Architecture research for: Tessera Ausschreibungs-Radar module (v1.1 milestone) Researched: 2026-07-17