Commit Graph

9 Commits

Author SHA1 Message Date
schalli 5a03b75f1e refactor(quick-260921-jt4): ueberfluessige case-Marke in tender-normalizer.service.ts entfernt (Restposten 2)
Die Fallmarke 'doe-opendata' stand unmittelbar ueber default und fiel
in denselben Zweig — 260921-bi2 liess sie stehen, weil der Kommentar
darunter Absicht dokumentiert. Die Absicht laesst sich ohne die
Fallmarke ausdruecken und wird dabei deutlicher: der erweiterte
Kommentar traegt jetzt beide Aussagen (DÖE-Quelle landet hier UND
kuenftige additive SourceType-Mitglieder sollen ebenfalls hier landen
statt zu scheitern). Kein Verhaltenswechsel — derselbe Zweig wie
vorher. Belegt durch die vorhandenen 18 tender-normalizer-Tests.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TPPB4ApQxzSU1rwV2Ffj9J
2026-09-21 14:57:40 +02:00
schalli 636fe0df8f refactor(quick-260921-bi2): maschinelle Lint-Fixe und toten Code abbauen
- Aufgabe 2: vier sichere Biome-Regeln (useImportType pfadgebunden auf
  apps/web+packages, noUselessEscapeInRegex, useConst,
  useExponentiationOperator) sowie fuenf ungesicherte Regeln
  (useNodejsImportProtocol, useLiteralKeys, useOptionalChain, useTemplate,
  useParseIntRadix) angewendet und den gesamten Diff von Hand gelesen
  (ldap.service.ts zeichenweise gegen Gross-/Kleinschreibung der
  AD-Merkmale, auth.service.ts/jwt.strategy.ts gegen Durchwinken bei
  fehlender Sitzung geprueft)
- noUselessSwitchCase bleibt bewusst stehen (tender-normalizer.service.ts:60,
  die Fallmarke dokumentiert Absicht)
- Toter Code (D-03): fuenf folgenlose Auffangvariablen entfernt, eine
  nicht benutzte Funktion (forSystemQuery, Pruefskript) entfernt, ein
  positionsgebundener Dekoratorparameter umbenannt (current-user.decorator.ts),
  fuenf Symptomfunde entfernt und als Folgeaufgaben zu melden (siehe unten)
- Sechs weitere, im Plan nicht namentlich gelistete aber
  gleich-kategorische Dead-Code-Fundstellen in Testdateien zusaetzlich
  bereinigt (groups.service.spec.ts, cert-manager.test.tsx,
  ldap.service.spec.ts, prisma-tenant.extension.spec.ts x3) — noetig, um
  die vom Plan selbst verlangten Nullstaende bei noUnusedVariables/
  noUnusedImports/noUnusedFunctionParameters zu erreichen

Dekoratordaten aus apps/api unveraendert (593 Zeilen, sha256 6e1583f1...).
Endstand 620 Befunde (541 echt, 79 Test) statt der im Plan geschaetzten
621/542 — eine Differenz von 1, weil das Streichen des Namens aus
`catch (e: any)` in calendar.service.ts (Symptom-Fix) den dort ebenfalls
gemeldeten noExplicitAny-Befund miteliminiert; das ist eine erwuenschte
Nebenwirkung, keine Regression. Fehlerstufe 0, beide Testlaeufe
punktgleich gruen (69/1124, 66/459), pnpm type-check 4/4, pnpm lint
--force 5/5.

Folgeaufgaben aus D-03 (nicht in diesem Vorgang behoben):
- force-password-change.interceptor.ts: Freigabeliste prueft nur den Pfad,
  nicht die HTTP-Methode
- change-password/page.tsx: nach erzwungenem Wechsel bleibt die Person auf
  der Seite stehen (keine Weiterleitung, keine Aktualisierung der
  Benutzerablage)
- VehicleTable.tsx: Loeschschaltflaeche hat keinen Besetztzustand, laesst
  sich doppelt ausloesen
- SplitTab.tsx: downloadAllAsZip erhielt eine ungenutzte
  Uebersetzungsfunktion, Hinweis auf fest verdrahtete Texte im Zip-Pfad

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TPPB4ApQxzSU1rwV2Ffj9J
2026-09-21 08:58:32 +02:00
schalli 1be6b15249 feat(14-03): add per-tenant encrypted TenderEmailConfig + ownerTenantId write-side (D-13)
Prisma: new TenderEmailConfig model (per-tenant, tenantId @unique, mirrors
DkvModuleConfig) + Tender.ownerTenantId nullable column + index (D-13:
null = global/platform-wide, unchanged for all existing rows and every
public source; set = visible only to that tenant). Migration
20260723113917_tender_email_config_owner_tenant_id applied locally.

TenderEmailConfigService: safe-select admin CRUD (GET never returns the
password, only hasPassword — T-07-12) with DkvService's encrypt-preserve-
empty semantics, via CalendarCryptoService (AES-256-GCM).

RawTenderRecord/NormalizedTenderFields gain optional ownerTenantId,
threaded through TenderNormalizerService.assemble() unchanged.
TenderDedupService's CREATE branch writes ownerTenantId (defaulting to
null); the UPDATE branch deliberately never references it, so a tender
later also seen on a public source is never retroactively hidden.

EmailAlertAdapter.fetchTenders() now does the real per-tenant fan-out:
findMany({isActive:true}) across ALL tenants (deliberate, documented
cross-tenant platform-scheduler read, never forTenant()/RLS), decrypts
each tenant's credentials, picks imap/exchange provider, and tags every
extracted candidate with ownerTenantId — catch-per-tenant so one broken
mailbox never blocks the others.

tenders.module.ts: imports CalendarModule/InboxModule, registers
EmailAlertAdapter + TenderEmailConfigService, seeds an 'email-alert'
TenderSourcePollConfig row (pollGranularity='tick', isActive=false —
no default mailbox to activate yet, D-02 framework-ready stance).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 13:45:11 +02:00
schalli 8983231196 feat(14-03): add EmailAlertAdapter generic extraction + 'email-alert' normalizer dispatch
GREEN phase (TDD) for Task 1: extractCandidateLinks (cheerio a[href] +
footer-noise filter + MAX_LINKS_PER_EMAIL cap, plaintext regex fallback),
titleFromEmail (subject -> first body line -> fallback), and
sourceNoticeIdFor (sha256 link hash) implement D-04's generic, no-portal-
specific-parser evaluation of alert emails.

SourceType gains 'email-alert'; TenderNormalizerService routes it through
the existing normalizeBag() path (same as ai-netserver/cosinex-dtvp/rss).
EmailAlertAdapter.fetchTenders() is a Task-1 placeholder — Task 2 wires the
real per-tenant fan-out.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 13:38:36 +02:00
schalli 3a96cbbbe6 feat(14-02): add RssAdapter.parseFeed + 'rss' normalizer dispatch
Fixture-first RSS parsing (INGEST-04): parses live-captured
service.bund.de (pubDate present, numeric-HTML-entity titles) and
subreport-elvis (pubDate absent, CDATA titles) feed shapes into
RawTenderRecord[] via fast-xml-parser, mirroring the DoeOpenDataAdapter
config. SourceType extended with 'rss'; normalize() dispatches 'rss'
through the existing normalizeBag() path unchanged (D-04/D-05).

Rule 1 fix: fast-xml-parser only decodes the 5 predefined XML entities,
not numeric character references — added an explicit decode step so
service.bund.de titles ("&#220;bermittlung...") render correctly
instead of leaking raw entity syntax.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 13:17:02 +02:00
schalli 82c9a48305 feat(quick-260723-e7i): per-sourceType dispatch in TenderNormalizerService
- Extract shared assemble() tail (status/dedupKey/contentHash/publishedAt)
  so it is computed identically across all sources, not duplicated
- Move existing DOE eForms/OCDS extraction into normalizeDoe() (byte-identical
  behavior, regression guard)
- Add normalizeBag() for the flat ocdsPayload bag shared by the NetServer and
  cosinex/DTVP scraper adapters ({title, buyerName, procedureType,
  legalFramework, deadlineAt}); legalFramework deliberately not mapped
- normalize() dispatches on raw.sourceType, defaulting to the DOE path so the
  additive SourceType union never throws

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 10:18:16 +02:00
schalli f9591dc491 feat(11-03): cpvDivisions column — normalizer + backfill migration
Adds Tender.cpvDivisions String[] (@@index Gin) derived at ingestion time
via cpv-catalog.ts's divisionOf() — replaces exact-match cpvCodes
comparison with a typesafe, GIN-indexable hasSome target (FILTER-03,
Pitfall 2). Backfill migration 20260721150000_tender_cpv_divisions_backfill
applied locally: 1612/1671 rows populated across all observed divisions
(741 rows carry division '45' — Bauarbeiten); idempotent (second run:
UPDATE 0, ADD COLUMN IF NOT EXISTS / CREATE INDEX IF NOT EXISTS both skip
cleanly). Applied via docker exec psql + `prisma migrate resolve
--applied` + `prisma generate` against the local dev DB only — no
Docker deploy on the test server.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 16:11:40 +02:00
schalli e4db602597 feat(11-02): normalizer derives bundesland + backfill migration for existing rows
Replace the Phase-10-deferred `bundesland = null` assignment with
bundeslandFromRegion(region) so new ingests are Bundesland-filterable
immediately. Add @@index([bundesland]) for filter performance. New
handwritten migration 20260721140000_tender_bundesland_backfill backfills
the ~1671 pre-existing rows (idempotent UPDATE, only where bundesland IS
NULL AND region IS NOT NULL) — applied locally via
`docker exec tessera-ctl-db-1 psql`, resolved as applied in
_prisma_migrations, and prisma generate re-run.

Verified on the local dev DB: 933/1671 rows now have bundesland set
across all 16 Länder (738 remain NULL where region itself is NULL).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 16:01:32 +02:00
schalli 31607df48c feat(10-03): implement TenderNormalizerService — fields, dedupKey, hash
- normalize(raw): eForms-DE XML primary for deadlineAt/estimatedValue/
  procedureType (RESEARCH Pattern 3); OCDS primary for ocid/buyerName/
  title/cpvCodes/region/plz
- deadlineAt/estimatedValue nullable by mandate (RESEARCH Pattern 4) —
  missing data normalizes to null, never thrown or zero
- dedupKey priority: ocid -> sourcePortal:sourceNoticeId fallback
- contentHash = sha256(title+deadlineAt+estimatedValue+status), stable
  across repeat calls, changes when the deadline changes (SCHEMA-02 hook)
- Register TenderNormalizerService in TendersModule.providers
- All tender-normalizer.service.spec.ts tests green (6/6); full API
  suite green (59/59); tsc --noEmit clean
2026-07-21 10:58:33 +02:00