Commit Graph

7 Commits

Author SHA1 Message Date
schalli 1be6b15249 feat(14-03): add per-tenant encrypted TenderEmailConfig + ownerTenantId write-side (D-13)
Prisma: new TenderEmailConfig model (per-tenant, tenantId @unique, mirrors
DkvModuleConfig) + Tender.ownerTenantId nullable column + index (D-13:
null = global/platform-wide, unchanged for all existing rows and every
public source; set = visible only to that tenant). Migration
20260723113917_tender_email_config_owner_tenant_id applied locally.

TenderEmailConfigService: safe-select admin CRUD (GET never returns the
password, only hasPassword — T-07-12) with DkvService's encrypt-preserve-
empty semantics, via CalendarCryptoService (AES-256-GCM).

RawTenderRecord/NormalizedTenderFields gain optional ownerTenantId,
threaded through TenderNormalizerService.assemble() unchanged.
TenderDedupService's CREATE branch writes ownerTenantId (defaulting to
null); the UPDATE branch deliberately never references it, so a tender
later also seen on a public source is never retroactively hidden.

EmailAlertAdapter.fetchTenders() now does the real per-tenant fan-out:
findMany({isActive:true}) across ALL tenants (deliberate, documented
cross-tenant platform-scheduler read, never forTenant()/RLS), decrypts
each tenant's credentials, picks imap/exchange provider, and tags every
extracted candidate with ownerTenantId — catch-per-tenant so one broken
mailbox never blocks the others.

tenders.module.ts: imports CalendarModule/InboxModule, registers
EmailAlertAdapter + TenderEmailConfigService, seeds an 'email-alert'
TenderSourcePollConfig row (pollGranularity='tick', isActive=false —
no default mailbox to activate yet, D-02 framework-ready stance).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 13:45:11 +02:00
schalli 8983231196 feat(14-03): add EmailAlertAdapter generic extraction + 'email-alert' normalizer dispatch
GREEN phase (TDD) for Task 1: extractCandidateLinks (cheerio a[href] +
footer-noise filter + MAX_LINKS_PER_EMAIL cap, plaintext regex fallback),
titleFromEmail (subject -> first body line -> fallback), and
sourceNoticeIdFor (sha256 link hash) implement D-04's generic, no-portal-
specific-parser evaluation of alert emails.

SourceType gains 'email-alert'; TenderNormalizerService routes it through
the existing normalizeBag() path (same as ai-netserver/cosinex-dtvp/rss).
EmailAlertAdapter.fetchTenders() is a Task-1 placeholder — Task 2 wires the
real per-tenant fan-out.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 13:38:36 +02:00
schalli 3a96cbbbe6 feat(14-02): add RssAdapter.parseFeed + 'rss' normalizer dispatch
Fixture-first RSS parsing (INGEST-04): parses live-captured
service.bund.de (pubDate present, numeric-HTML-entity titles) and
subreport-elvis (pubDate absent, CDATA titles) feed shapes into
RawTenderRecord[] via fast-xml-parser, mirroring the DoeOpenDataAdapter
config. SourceType extended with 'rss'; normalize() dispatches 'rss'
through the existing normalizeBag() path unchanged (D-04/D-05).

Rule 1 fix: fast-xml-parser only decodes the 5 predefined XML entities,
not numeric character references — added an explicit decode step so
service.bund.de titles ("&#220;bermittlung...") render correctly
instead of leaking raw entity syntax.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 13:17:02 +02:00
schalli 82c9a48305 feat(quick-260723-e7i): per-sourceType dispatch in TenderNormalizerService
- Extract shared assemble() tail (status/dedupKey/contentHash/publishedAt)
  so it is computed identically across all sources, not duplicated
- Move existing DOE eForms/OCDS extraction into normalizeDoe() (byte-identical
  behavior, regression guard)
- Add normalizeBag() for the flat ocdsPayload bag shared by the NetServer and
  cosinex/DTVP scraper adapters ({title, buyerName, procedureType,
  legalFramework, deadlineAt}); legalFramework deliberately not mapped
- normalize() dispatches on raw.sourceType, defaulting to the DOE path so the
  additive SourceType union never throws

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 10:18:16 +02:00
schalli f9591dc491 feat(11-03): cpvDivisions column — normalizer + backfill migration
Adds Tender.cpvDivisions String[] (@@index Gin) derived at ingestion time
via cpv-catalog.ts's divisionOf() — replaces exact-match cpvCodes
comparison with a typesafe, GIN-indexable hasSome target (FILTER-03,
Pitfall 2). Backfill migration 20260721150000_tender_cpv_divisions_backfill
applied locally: 1612/1671 rows populated across all observed divisions
(741 rows carry division '45' — Bauarbeiten); idempotent (second run:
UPDATE 0, ADD COLUMN IF NOT EXISTS / CREATE INDEX IF NOT EXISTS both skip
cleanly). Applied via docker exec psql + `prisma migrate resolve
--applied` + `prisma generate` against the local dev DB only — no
Docker deploy on the test server.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 16:11:40 +02:00
schalli e4db602597 feat(11-02): normalizer derives bundesland + backfill migration for existing rows
Replace the Phase-10-deferred `bundesland = null` assignment with
bundeslandFromRegion(region) so new ingests are Bundesland-filterable
immediately. Add @@index([bundesland]) for filter performance. New
handwritten migration 20260721140000_tender_bundesland_backfill backfills
the ~1671 pre-existing rows (idempotent UPDATE, only where bundesland IS
NULL AND region IS NOT NULL) — applied locally via
`docker exec tessera-ctl-db-1 psql`, resolved as applied in
_prisma_migrations, and prisma generate re-run.

Verified on the local dev DB: 933/1671 rows now have bundesland set
across all 16 Länder (738 remain NULL where region itself is NULL).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 16:01:32 +02:00
schalli 31607df48c feat(10-03): implement TenderNormalizerService — fields, dedupKey, hash
- normalize(raw): eForms-DE XML primary for deadlineAt/estimatedValue/
  procedureType (RESEARCH Pattern 3); OCDS primary for ocid/buyerName/
  title/cpvCodes/region/plz
- deadlineAt/estimatedValue nullable by mandate (RESEARCH Pattern 4) —
  missing data normalizes to null, never thrown or zero
- dedupKey priority: ocid -> sourcePortal:sourceNoticeId fallback
- contentHash = sha256(title+deadlineAt+estimatedValue+status), stable
  across repeat calls, changes when the deadline changes (SCHEMA-02 hook)
- Register TenderNormalizerService in TendersModule.providers
- All tender-normalizer.service.spec.ts tests green (6/6); full API
  suite green (59/59); tsc --noEmit clean
2026-07-21 10:58:33 +02:00