Commit Graph

187 Commits

Author SHA1 Message Date
schalli 954cd6e171 feat(15-05): static widget-to-module registration table
- WIDGET_MODULE_MAP + getModuleSlugForWidgetType (apps/api/src/dashboard/widget-module-map.ts)
- table intentionally empty at end of phase: all 8 existing widget types are module-free platform widgets (D-22)
- code constant chosen over a WidgetInstance schema column — no migration for a field empty on every row
2026-08-04 15:30:51 +02:00
schalli 33838dde39 feat(15-02): automatische Standardgruppen-Mitgliedschaft an genau einem Ort
- UserService.create ruft nach der Anlage GroupsService.addUserToDefaultGroup
  auf (D-11/D-12) — einziger Erzeugungspunkt für Benutzer, erbt LdapService
  ohne eigene Kopie der Regel
- try/catch mit Logger: gescheiterte Gruppenzuordnung bricht weder die
  Benutzeranlage noch einen LDAP-Sync-Lauf ab (T-15-14)
- UserModule importiert GroupsModule, keine Zirkularität
- 4 Tests in user.service.spec.ts; ldap.service.ts unverändert
2026-08-04 15:23:01 +02:00
schalli 69494d7549 feat(15-02): GroupsModule — CRUD für Gruppen, Mitgliedschaften und Löschauswirkung
- GroupsService: listForTenant/create/update/remove/getImpact/listMembers/addMembers/removeMember/addUserToDefaultGroup, jede Query tenantId-gescoped (T-15-02/T-15-12)
- isDefault:true läuft in einer Transaktion (updateMany+update), D-13
- getImpact liefert { memberCount, grantCount } für den Löschdialog (D-17)
- removeMember beschränkt sich auf source:MANUAL (D-19)
- GroupsController: 8 rollengeschützte Routen unter /groups
- 19 Tests in groups.service.spec.ts, hand-rolled In-Memory-Fake
2026-08-04 15:21:41 +02:00
schalli 92e8eaffa5 feat(15-01): RLS policies for Group/GroupMembership/ModuleGrant (T-15-11)
- Second, deliberately separate migration (pure hand-SQL, no Prisma-
  generated DDL): ENABLE/FORCE ROW LEVEL SECURITY plus a
  tenant_isolation_policy for each of the three new tables, following
  the pattern of 20260618112133_rls_policies (Auth-Kerntabellen)
  rather than the RLS-exempt Tender* app-layer tables
- Group/ModuleGrant compare tenantId directly against
  current_tenant_id(); GroupMembership has no own tenantId and follows
  the PasswordResetToken join pattern (groupId IN (SELECT id FROM
  Group WHERE tenantId = ...))
- migration-sql.spec.ts extended with a second describe block covering
  both migration files (6x ROW LEVEL SECURITY, 3x CREATE POLICY, the
  join vs. direct-comparison shape)
- Re-ran the Task-2 end-to-end proof after applying this migration:
  identical result (USER without grant 403 + empty list, USER with
  direct grant 200 + slug present, ADMIN 200) — the app's DB role
  (tessera) is a Postgres superuser with rolbypassrls=true, so it
  bypasses RLS as documented as an acceptable outcome by the plan;
  RLS remains the defense-in-depth net for any future non-superuser
  connection
2026-08-04 15:11:33 +02:00
schalli 9a4ba8a33c feat(15-01): ModuleAccessService as single source of truth for module access (D-01)
- ModuleAccessService.getAccessibleModuleIds(tenantId, userId, role):
  ADMIN/SUPER_ADMIN bypass (D-03) via one query, otherwise a single
  Promise.all of direct + group ModuleGrant lookups intersected against
  active TenantModuleActivation (D-02) — no N+1 over the user's groups
- findAccessibleModules() adds the name-asc sort for stable sidebar order
- ModuleGuard now resolves userId/role from request.user (JWT-sourced,
  never body/params) and calls getAccessibleModuleIds instead of the
  tenant-only isModuleActive check; caches the result on
  request.moduleAccessIds for same-request reuse (D-09, no cross-request
  caching)
- ModuleRegistryController.findActive delegates to
  ModuleAccessService.findAccessibleModules instead of
  findActiveForTenant, which stays untouched for Plan 15-03's
  tenant-wide marketplace catalog
- ModuleRegistryModule exports ModuleAccessService for Plan 15-03/15-05
- module-access.service.spec.ts / module.guard.spec.ts cover every case
  in the plan's <behavior> list with a hand-rolled Prisma mock
- End-to-end verified against the running local API: a USER without a
  grant gets 403 on a @UseModule-protected endpoint and an empty
  /modules/active list; the same USER with a direct grant gets 200 plus
  the slug in the list; an ADMIN without any grant also gets 200 (D-03)
2026-08-04 15:09:10 +02:00
schalli c5c704bae9 feat(15-01): Group/GroupMembership/ModuleGrant schema + D-06 backfill migration
- Group/GroupMembership/ModuleGrant models plus MembershipSource enum
  (D-05), placed under TenantModuleActivation with German block comment
- Hand-SQL appended to the generated migration: partial unique index for
  one default group per tenant (D-13), CHECK num_nonnulls xor-constraint
  plus two partial unique indexes for ModuleGrant (D-04), and the D-06
  backfill (Group -> GroupMembership -> ModuleGrant, each INSERT guarded
  by WHERE NOT EXISTS for idempotent re-runs on `prisma migrate deploy`)
- apps/api/src/groups/migration-sql.spec.ts verifies the hand-SQL by
  reading migration.sql directly, no DB required
- Verified against the local DB: default-group count matches tenant
  count, membership/grant counts match existing users/active
  activations, and the XOR constraint rejects a group+user-less insert
2026-08-04 15:03:48 +02:00
schalli 5cbd530a87 feat(260729-d3k): multi-base LDAP sync scope + re-keyed no-op guard
- parseBaseDns() splits the newline-separated baseDn field into a list
- syncUsersForTenant no-op guard re-keyed on empty parsed base-DN list
  (was empty groupFilterDns) — the sole condition that skips search +
  the deactivation loop, preventing mass-deactivation on an
  unconfigured config
- collectSearchEntries/listGroups/searchUsers loop every base DN and
  merge/dedupe results by entry dn
- empty groupFilterDns is no longer a no-op: it now performs a normal
  multi-base search with no memberOf restriction
- groupFilterDns ou= entries stay additional search bases; group DNs
  become an optional memberOf constraint applied to every base search
- spec: replaced empty-groupFilterDns no-op test with empty-base-DN
  no-op test, added multi-base merge/dedup test

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-29 09:36:55 +02:00
schalli 57bc7f96b3 feat(260728-lih): make LDAP sync strictly selective (empty selection = no-op)
- collectSearchEntries() returns [] on empty/undefined groupFilterDns
  instead of scanning the whole baseDn subtree
- syncUsersForTenant() early-returns an empty successful result before
  any LDAP search or the deactivation loop when groupFilterDns is empty,
  so an empty selection can never mass-deactivate existing LDAP users
- Updated exclude-list tests to use a non-empty groupFilterDns; added a
  dedicated no-op test proving empty selection performs zero search/
  create/update/deactivate operations

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-28 15:41:33 +02:00
schalli c54e424c05 feat(260728-lih): default LDAP syncIntervalMin to 0 (auto-sync off)
- LdapConfig.syncIntervalMin default changed 60 -> 0
- New migration sets column DEFAULT only, no data rewrite
- isActive @default(true) left unchanged (gates LDAP login only)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-28 15:40:28 +02:00
schalli 7502f97e85 feat(260723-lvg): add admin-gated POST /poll-now endpoint for tender radar
Manual "Jetzt abrufen" trigger delegates to
TenderIngestionService.pollDueSources() — the same fan-out tick the
scheduler cron runs. Gated to ADMIN/SUPER_ADMIN (T-lvg-01, DoS) and
declared before @Get(':id') per the established route-order convention.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 15:52:44 +02:00
schalli e1358d70fd fix(tenders): register poll cron in onApplicationBootstrap (fresh-DB bootstrap)
Tessera CI/CD / Lint & Type Check (push) Successful in 48s
Tessera CI/CD / Tests (push) Successful in 48s
Tessera CI/CD / Build & Publish Images (push) Successful in 26s
On a fresh database the DÖE poll cron was never registered: TenderScheduler
read the doe-opendata poll config in its onModuleInit, which raced ahead of
TendersModule.onModuleInit seeding that config. The scheduler saw the config
absent → skipped registering the single global cron that drives pollDueSources
(DÖE + RSS + email-alert) → the platform ingested NOTHING until a second restart.
Observed live on a fresh prod DB (0 tenders, 'doe-opendata config inactive —
cron job not registered', lastIngestedDay null despite isActive=true).

Move the scheduler to onApplicationBootstrap, which runs after every module's
onModuleInit, so the seed is guaranteed complete before the config is read.
Adds a regression test asserting the lifecycle choice.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 15:30:04 +02:00
schalli 28c6c7fee1 feat(14-04): denylisted-portals read endpoint sourced from DENYLISTED_PORTALS
- Add PORTAL_URLS map (vergabe24, aumass) in source-registry.ts, keyed off
  the existing DENYLISTED_PORTALS constant so the portal set is never
  re-declared
- Add GET /modules/tender-radar/denylisted-portals, declared before
  @Get(':id') (route-order pitfall), mapping over DENYLISTED_PORTALS
- Extend tenders.controller.spec.ts: response shape + route-order guard

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 13:58:25 +02:00
schalli 48e12523f3 feat(14-03): add email-config admin routes + D-13 read-side visibility filter + EmailAlertConfigForm
buildTenderWhere gains an optional ownerTenantId param: a resolved
requesting tenant sees global tenders (null) plus its own private ones
(OR[global, mine]); an unresolved requester fails CLOSED to global-only —
never an accidental cross-tenant leak.

TendersController: listTenders/getTender resolve the requesting tenant
leniently from the auth context (resolveRequestingTenantId, never throws)
and apply the D-13 filter; getTender 404s (not a distinct "forbidden") when
a tender's non-null ownerTenantId doesn't match the requester, so no
cross-tenant detail leak. New GET/PUT /modules/tender-radar/email-config
routes (Roles ADMIN/SUPER_ADMIN, tenantId from auth context, never the
body) delegate to TenderEmailConfigService — declared before @Get(':id')
per the project's NestJS route-order convention.

Web: EmailAlertConfig type + fetchEmailConfig/saveEmailConfig client
functions; EmailAlertConfigForm mirrors the DKV InboxConfigForm (password
blank on load, only sent when typed — T-07-12), added as a new
"E-Mail-Alerts" section on the existing tender-radar settings page.
Hardcoded German strings — i18n is Plan 14-05.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 13:53:17 +02:00
schalli 1be6b15249 feat(14-03): add per-tenant encrypted TenderEmailConfig + ownerTenantId write-side (D-13)
Prisma: new TenderEmailConfig model (per-tenant, tenantId @unique, mirrors
DkvModuleConfig) + Tender.ownerTenantId nullable column + index (D-13:
null = global/platform-wide, unchanged for all existing rows and every
public source; set = visible only to that tenant). Migration
20260723113917_tender_email_config_owner_tenant_id applied locally.

TenderEmailConfigService: safe-select admin CRUD (GET never returns the
password, only hasPassword — T-07-12) with DkvService's encrypt-preserve-
empty semantics, via CalendarCryptoService (AES-256-GCM).

RawTenderRecord/NormalizedTenderFields gain optional ownerTenantId,
threaded through TenderNormalizerService.assemble() unchanged.
TenderDedupService's CREATE branch writes ownerTenantId (defaulting to
null); the UPDATE branch deliberately never references it, so a tender
later also seen on a public source is never retroactively hidden.

EmailAlertAdapter.fetchTenders() now does the real per-tenant fan-out:
findMany({isActive:true}) across ALL tenants (deliberate, documented
cross-tenant platform-scheduler read, never forTenant()/RLS), decrypts
each tenant's credentials, picks imap/exchange provider, and tags every
extracted candidate with ownerTenantId — catch-per-tenant so one broken
mailbox never blocks the others.

tenders.module.ts: imports CalendarModule/InboxModule, registers
EmailAlertAdapter + TenderEmailConfigService, seeds an 'email-alert'
TenderSourcePollConfig row (pollGranularity='tick', isActive=false —
no default mailbox to activate yet, D-02 framework-ready stance).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 13:45:11 +02:00
schalli 8983231196 feat(14-03): add EmailAlertAdapter generic extraction + 'email-alert' normalizer dispatch
GREEN phase (TDD) for Task 1: extractCandidateLinks (cheerio a[href] +
footer-noise filter + MAX_LINKS_PER_EMAIL cap, plaintext regex fallback),
titleFromEmail (subject -> first body line -> fallback), and
sourceNoticeIdFor (sha256 link hash) implement D-04's generic, no-portal-
specific-parser evaluation of alert emails.

SourceType gains 'email-alert'; TenderNormalizerService routes it through
the existing normalizeBag() path (same as ai-netserver/cosinex-dtvp/rss).
EmailAlertAdapter.fetchTenders() is a Task-1 placeholder — Task 2 wires the
real per-tenant fan-out.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 13:38:36 +02:00
schalli 4d6fbb136e test(14-03): add failing spec for email-alert generic link/subject extraction
RED phase (TDD) for Task 1: pure-function tests for extractCandidateLinks,
titleFromEmail, sourceNoticeIdFor — covers HTML + plaintext bodies,
footer-noise removal, link cap, and D-04 no-portal-specific-parser restraint.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 13:37:37 +02:00
schalli 48a2dc1026 feat(14-02): add RSS feed admin routes, API client, and settings UI
Adds GET/POST/DELETE /modules/tender-radar/rss-feeds (Roles-guarded
ADMIN/SUPER_ADMIN), declared before the existing @Get(':id') handler to
avoid NestJS route-order shadowing. Delegates to
TenderRssFeedSourceService; the denylist/SSRF rejection (D-14) surfaces
as a 400 unchanged.

Web: tender-radar-api.ts gains listRssFeeds/createRssFeed/deleteRssFeed
(relaying the backend's specific rejection message via
extractErrorMessage), and a new RssFeedListForm client component renders
an "RSS-Feeds" section on the tender-radar settings page (D-09) — list,
add (with inline denylist error), and remove global feed URLs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 13:29:07 +02:00
schalli e812738c3a feat(14-02): add TenderRssFeedSource CRUD, tick poll gate, and wire RssAdapter
Global admin-managed RSS feed list (TenderRssFeedSource, D-08/D-14) with
a save-time hostname/SSRF guard (TenderRssFeedSourceService) — RSS feed
URLs are runtime admin input, so the code-level SourceRegistry denylist
gate does not cover them; a separate check rejects DENYLISTED_PORTALS
hostnames, non-http(s) schemes, and private/loopback hosts.

Adds TenderSourcePollConfig.pollGranularity ('day' | 'tick', D-15):
pollDueSources() branches per source — 'day' sources keep the existing
lastIngestedDay gate byte-unchanged, 'tick' sources (rss) fetch on every
active scheduler tick regardless of lastIngestedDay, since the day-cursor
gate was built for a genuine daily batch-export API and would otherwise
silently cap RSS to one fetch per calendar day.

Wires RssAdapter.fetchTenders() to fan out over active feed rows (native
fetch + AbortController 15s + response-size ceiling, catch-per-feed),
registers it in tenders.module.ts, and seeds the 'rss' poll config
active with pollGranularity='tick' plus a default-active service.bund.de
feed row (subreport-elvis has no single canonical URL — zero rows seeded,
admin adds relevant municipality feeds).

Migration applied locally per project convention (host -> container IP).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 13:24:36 +02:00
schalli 3a96cbbbe6 feat(14-02): add RssAdapter.parseFeed + 'rss' normalizer dispatch
Fixture-first RSS parsing (INGEST-04): parses live-captured
service.bund.de (pubDate present, numeric-HTML-entity titles) and
subreport-elvis (pubDate absent, CDATA titles) feed shapes into
RawTenderRecord[] via fast-xml-parser, mirroring the DoeOpenDataAdapter
config. SourceType extended with 'rss'; normalize() dispatches 'rss'
through the existing normalizeBag() path unchanged (D-04/D-05).

Rule 1 fix: fast-xml-parser only decodes the 5 predefined XML entities,
not numeric character references — added an explicit decode step so
service.bund.de titles ("&#220;bermittlung...") render correctly
instead of leaking raw entity syntax.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 13:17:02 +02:00
schalli c404954bee feat(14-01): add fetchMessages() to InboxProvider + both implementations
- Add InboxMessage type (subject + html/text body) and fetchMessages() to
  the InboxProvider interface, ImapProvider, and ExchangeInboxProvider
- IMAP: findBodyParts() walks the MIME tree for first text/html + text/plain
  parts, reusing the connect/lock/search/fetchAll skeleton; marks \Seen
- EWS: new getItemBodySoap() requests item:Body, extracts BodyType via the
  existing extractAttr/extractAll helpers, marks IsRead via markReadSoap
- Net-new spec coverage (imap.provider.spec.ts, exchange-inbox.provider.spec.ts)
  mocking ImapFlow and httpntlm.post (via require.cache stub, since httpntlm
  is loaded with a raw require() that vi.mock cannot intercept)
- fetchPdfAttachments untouched in both providers (D-02); full API suite
  (301 tests) + tsc --noEmit stay green

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 13:09:45 +02:00
schalli eb668fd5c8 refactor(14-01): extract DKV inbox providers into shared inbox/ module
- Move ImapProvider, ExchangeInboxProvider, InboxProvider into apps/api/src/inbox/
- Move InboxConfig/InboxAttachment/InboxEmail into new inbox.types.ts
- dkv.types.ts re-exports the moved types so existing DKV imports keep compiling
- DKV switches import paths to ../inbox/... and imports InboxModule
- Pure move + import-path swap: fetchPdfAttachments and all DKV logic unchanged (D-01/D-02)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 13:02:15 +02:00
schalli 988100576f test(quick-260723-e7i): cover ai-netserver/cosinex bag normalization + DOE regression
- bagRecord() helper builds inline RawTenderRecords for the flat ocdsPayload
  bag shape (no fixtures exist for NetServer/cosinex-DTVP)
- ai-netserver and cosinex-dtvp full-bag mapping, null/empty-field fallback,
  and contentHash-format assertions
- Existing DOE fixture-based assertions untouched, confirming the refactor
  didn't change DOE-path behavior

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 10:19:00 +02:00
schalli 82c9a48305 feat(quick-260723-e7i): per-sourceType dispatch in TenderNormalizerService
- Extract shared assemble() tail (status/dedupKey/contentHash/publishedAt)
  so it is computed identically across all sources, not duplicated
- Move existing DOE eForms/OCDS extraction into normalizeDoe() (byte-identical
  behavior, regression guard)
- Add normalizeBag() for the flat ocdsPayload bag shared by the NetServer and
  cosinex/DTVP scraper adapters ({title, buyerName, procedureType,
  legalFramework, deadlineAt}); legalFramework deliberately not mapped
- normalize() dispatches on raw.sourceType, defaulting to the DOE path so the
  additive SourceType union never throws

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 10:18:16 +02:00
schalli 6fe0dd07fe feat(13-05): register CosinexAdapter as tenders module provider
Adds CosinexAdapter to TendersModule's providers and registers it with
SourceRegistry at DI boot, alongside DoeOpenDataAdapter/NetServerAdapter
(cosinex-dtvp is not AGB-denylisted, so registration succeeds). Seeds a
cosinex-dtvp TenderSourcePollConfig row with isActive: false, matching
the ai-netserver "framework ready, activation deferred" stance (D-02).

tsc --noEmit clean; src/tenders slice: 21 files, 221/221 tests pass
(205 pre-existing + 16 new cosinex.adapter.spec.ts).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 09:49:19 +02:00
schalli 12fc5ac02d feat(13-05): add cosinex/DTVP source adapter with fixture tests
Separate HTML adapter for the cosinex Vergabemarktplatz (DTVP) satellite
(sourceType='cosinex-dtvp'), distinct from the NetServer adapter since
cosinex markup differs structurally. Live inspection (2026-07-23) found
the "Aktuelle Bekanntmachungen" results table is fully server-rendered
(not JS-dependent as D-01 anticipated), so selectors are fully populated
rather than falling back to a needs-JS stub — parses publish date,
deadline (or "nv"), title, legal framework/procedure type, buyer name,
and a real per-notice deep link (pid) into RawTenderRecord[].

Rule 1 fix: cosinex serves charset=ISO-8859-1 with raw Latin-1 bytes for
umlauts (not HTML entities); Response.text() always UTF-8-decodes per
the Fetch spec, so the adapter reads arrayBuffer() and decodes explicitly
via TextDecoder('iso-8859-1') to avoid mojibake.

16 spec tests pass against a live-captured fixture (20 rows, transcoded
to UTF-8 on disk): full-fixture parse, deadline/publish date parsing,
nested-<abbr> procedure-type extraction, umlaut decoding, empty/broken
HTML and missing-pid row fallback, fetch-throw/non-2xx fallback, no-axios
and no-input-interpolated-URL guards.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 09:48:21 +02:00
schalli 88572f5694 feat(13-04): NetServerAdapter im Modul registriert (INGEST-02)
- NetServerAdapter als Provider ergaenzt, additiv neben DoeOpenDataAdapter
  in onModuleInit ueber SourceRegistry.register() registriert (Denylist-
  Gate erlaubt tender24/lhs-vpbw/vergabe.landbw, keine auf der Denylist)
- TenderSourcePollConfig-Row fuer 'ai-netserver' geseedet, isActive:false
  (Aktivierung bleibt Admin/Seed-Entscheidung, D-02)
- tsc --noEmit clean, src/tenders-Suite 20 Dateien/205 Tests gruen

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 09:14:24 +02:00
schalli 043ada9438 feat(13-04): NetServer-Adapter fuer tender24/lhs-vpbw/vergabe.landbw (INGEST-02)
- Package-Legitimacy-Checkpoint bestaetigt: cheerio (sauberer Gate-Pass,
  27M Downloads/Woche) statt node-html-parser installiert
- EIN config-getriebener NetServerAdapter bedient alle 3 AI-AG-Portale
  via PublicationSearchControllerServlet-Trefferliste, parst <table> mit
  cheerio zu RawTenderRecord[], sourcePortal je Zeile korrekt
- sourceNoticeId = data-oid (stabil, per Zeile eindeutig); sourceUrl =
  Such-URL als dokumentierter Best-Effort-Fallback (kein Deep-Link ohne
  JS-Ausfuehrung ermittelbar, Open Question 2)
- Fehlertoleranz: try/catch pro Zeile + pro Portal, []-Fallback bei
  Totalausfall, AbortController 15s-Timeout, hardcodierte Portal-URLs
  (SSRF-Guard T-13-04-01)
- Live-gecapturte Fixture (tender24.de, 2026-07-23) mit Umlaut-/&amp;-
  Buyer, gefuellter und leerer Deadline; 14 Spec-Tests gruen

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 09:13:01 +02:00
schalli 166194fa04 feat(13-06): getTender include sources[] (SCHEMA-03 read surface)
GET /modules/tender-radar/:id now includes the TenderSource relation
(sourcePortal, sourceUrl, sourceNoticeId) so a cross-source-deduped
tender's detail response carries links to all its source portals, not
just the single primary sourceUrl column. Route order unchanged (:id
stays after all static routes).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 08:55:02 +02:00
schalli 1d4f9cf1b3 feat(13-03): wire SourceRegistry + TenderDedupService into TendersModule
Register SourceRegistry and TenderDedupService as providers.
onModuleInit registers DoeOpenDataAdapter with the registry before the
scheduler's first tick — the DI-boot-time enforcement point for the
INGEST-07 denylist gate (D-06). This is Wave 2's sole writer of
tenders.module.ts; 13-04 (NetServer) and 13-05 (cosinex) add their
own registry.register(...) calls additively in later waves.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 08:51:47 +02:00
schalli d453dbbd1f feat(13-03): pollDueSources fan-out over all active sources (SCHEMA-03)
Replace the DÖE-only findUnique with findMany({isActive:true}) fan-out
(poll-once-fan-out-many, D-01). Each active TenderSourcePollConfig is
resolved through SourceRegistry.get(sourceType) and processed inside
its own try/catch (catch-per-source, D-01) — one broken/blocking source
no longer aborts the tick for the others. dedupActive =
activePortalCount >= 2 (D-05) is computed once per tick and passed to
TenderDedupService.resolve(), which now replaces the direct
tender.upsert call. Delta-only matchDelta boundary (D-07) preserved:
only genuinely-created tender IDs across all sources are collected.

Extended tender-ingestion.service.spec.ts: multi-config fan-out,
catch-per-source isolation, dedupActive gate assertion, adapter-missing
skip, plus the existing SCHEMA-02/D-07/retention/day-cursor suites
updated to the new registry+dedup constructor shape (all green).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 08:50:57 +02:00
schalli 1cd2fcfa01 feat(13-03): implement TenderDedupService three-tier resolver (D-04/D-05)
resolve(n, {dedupActive}) matches OCID -> source:noticeId -> fingerprint
(fingerprint tier hard-gated by dedupActive, D-05). On any match the
existing Tender gets an additional TenderSource attached (D-03 merge)
instead of a new Tender row; SCHEMA-02 change-detection is preserved
inline (matched Tender's mutable fields refresh when contentHash
differs, exactly as the old direct tender.upsert UPDATE branch did).
No match -> tender.create (with computed fingerprint) + tenderSource.create.
Plain PrismaService, no forTenant()/RLS (T-10-09).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 08:48:12 +02:00
schalli 605ea4cf80 test(13-03): add failing spec for TenderDedupService (D-04/D-05)
RED-first: three-tier dedup resolver spec (OCID -> source:noticeId ->
fingerprint), D-05 inert-proof (dedupActive=false skips fingerprint
tier -> two Tender rows despite equal fingerprints), and SCHEMA-02
change-detection preservation (matched contentHash change still
updates mutable Tender fields).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 08:47:37 +02:00
schalli 0fa9567571 feat(13-02): implement SourceRegistry with hard denylist gate
GREEN — SourceRegistry.register() throws DeniedPortalError when any
of an adapter's declared portals is in DENYLISTED_PORTALS
(vergabe24, aumass), enforced at DI-registration time (INGEST-07/
D-06), not just documented. get()/activeAdapters() support the
Plan 13-03 poll-once-fan-out-many scheduler. 6/6 tests pass, no
Prisma/scraping import.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 08:43:29 +02:00
schalli 78b17ef28b test(13-02): add failing SourceRegistry denylist-gate spec
RED — proves Erfolgskriterium 4 (INGEST-07): registering an adapter
whose portals include vergabe24 or aumass must throw DeniedPortalError,
including a mixed portals array with one denylisted entry. Also covers
legitimate register/get/activeAdapters happy paths. Fake adapter stub,
no real scraping.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 08:42:51 +02:00
schalli 1b11ada112 feat(13-02): generalize adapter contract with portals[] array
TenderSourceAdapter gains a readonly portals: readonly string[] field
so one adapter can serve multiple portals (NetServer: 3, Plan 13-04)
and so SourceRegistry can gate registration per-portal (INGEST-07).
DoeOpenDataAdapter declares portals = ['doe-opendata'] additively,
no behavior change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 08:42:23 +02:00
schalli c2a60212fe feat(13-01): widen SourceType to open union, add NormalizedTenderFields.fingerprint
SourceType now covers 'doe-opendata' | 'ai-netserver' | 'cosinex-dtvp'
(13-RESEARCH Pattern 1) so the Plan 13-04/05 adapters can register
without further type-contract changes. NormalizedTenderFields gains an
optional fingerprint field for the SCHEMA-03 dedup resolver (Plan
13-03) to populate later. tsc --noEmit clean; full API suite green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 08:38:59 +02:00
schalli 447fb74e0c feat(13-01): add TenderSource model + Tender.fingerprint, backfill 2851 rows
Additive schema change (SCHEMA-03/D-03/D-04): new model TenderSource
(1:n Tender, @@unique[sourcePortal, sourceNoticeId], onDelete Cascade)
and a nullable Tender.fingerprint column + index. dedupKey stays
unchanged as the SCHEMA-02 upsert target.

Migration 20260723120000_add_tender_source applies in strict order
(Pitfall 5): table+column create, then one TenderSource row per
pre-existing Tender via SQL INSERT/SELECT, then the unique constraint.
Applied locally against the tessera dev DB (container IP, no host
port) — verified via psql: TenderSource count == Tender count == 2851.

backfill-tender-source.ts is a one-time script that computes
Tender.fingerprint via the Task-1 tenderFingerprint() function
(Decimal->number conversion for estimatedValue, T-13-01-03) — run via
the compiled dist/ output (source uses standard extensionless TS
imports for tsc compatibility). Confirmed: 2851/2851 rows backfilled,
idempotent re-run verified.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 08:38:26 +02:00
schalli c6cac696ff feat(13-01): implement tenderFingerprint pure NULL-tolerant dedup key
GREEN: title+buyer dominant, CPV division (order-independent, dedup'd),
value bucketed by order-of-magnitude, deadline truncated to day-grain.
sha256 hex, deterministic, no I/O — foundation for the Task-2 backfill
and the Plan 13-03 dedup resolver's fingerprint tier.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 08:33:40 +02:00
schalli 063ba5b180 test(13-01): add failing test for tenderFingerprint (SCHEMA-03)
RED: NULL-tolerant fingerprint (title+buyer+cpv dominant, value-bucket,
deadline-day), collision guard, umlaut normalization, deterministic
sha256. Implementation follows in the next commit.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 08:33:17 +02:00
schalli a1cf05404c feat(auth): LDAP login — authenticate imported users against the directory
Tessera CI/CD / Lint & Type Check (push) Successful in 47s
Tessera CI/CD / Tests (push) Successful in 47s
Tessera CI/CD / Build & Publish Images (push) Successful in 2m21s
LDAP-imported users have no local passwordHash, and validateUser only checked
the local password, so they could never log in. Now a passwordless user with
an ldapDn is authenticated by binding as their OWN DN with the entered
password against the tenant's active LDAP config (reusing the ldaps TLS-skip
option). Empty passwords are rejected before binding to avoid AD's
unauthenticated-bind bypass. Local-password users are unchanged.

LdapService.verifyUserCredentials added; LdapModule now exports
LdapConfigService; AuthModule imports LdapModule (no circular dep). 8 new
specs (bind success/fail, empty-password guard, login via bind, wrong pw, no
config, no ldapDn, inactive). API 226 green, tsc clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 14:51:23 +02:00
schalli af9e968c6f feat(ldap): opt-in skip TLS verification for ldaps (internal CA)
Tessera CI/CD / Lint & Type Check (push) Successful in 45s
Tessera CI/CD / Tests (push) Successful in 49s
Tessera CI/CD / Build & Publish Images (push) Successful in 1m45s
Add a per-tenant "Skip TLS certificate verification" toggle to the LDAP
admin page so admins can connect to an AD whose ldaps:// certificate is
signed by an internal/self-signed CA (Node error: "unable to verify the
first certificate"). When enabled, ldapts is given
tlsOptions.rejectUnauthorized=false; the flag is ignored for plain ldap://
(no TLS). Defaults to full verification.

New Boolean column LdapConfig.tlsRejectUnauthorized (@default(true)) +
migration; wired through DTOs, config service, all Client creations
(test/groups/user-search/import/sync) and the test-connection endpoint. UI
checkbox with an insecure-network warning (de/en). 3 new service specs;
API 218 green, web 131 green, both apps tsc clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 14:18:55 +02:00
schalli 38face43b4 feat(ldap): individual user search + selective import with dedup
Tessera CI/CD / Lint & Type Check (push) Successful in 49s
Tessera CI/CD / Tests (push) Successful in 47s
Tessera CI/CD / Build & Publish Images (push) Successful in 1m45s
Add an AD single-user search (by cn/sAMAccountName/displayName/mail) and a
selective import to the LDAP admin page, alongside the existing group/OU
filter. Imported users are deduped against existing ones by (ldapDn, then
username): a manually-imported user carries its ldapDn, so a later
department/group sync matches and updates it in place instead of creating a
duplicate. Search results flag alreadyImported; import skips existing users
and links a missing ldapDn. Extracted shared mapEntry/upsertMappedUser
helpers so sync and manual import resolve identity identically.

Backend: GET /ldap/users/search, POST /ldap/users/import (RFC-4515 escaped
query, ADMIN-guarded). 6 new service specs (search flags, create, skip,
ldapDn-link, denylist). Full API suite 215 green, both apps tsc clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 13:52:50 +02:00
schalli 9e7ba5353d feat(12-04): tender-radar notification-pref service + routes, instantAlert passthrough
- TenderNotificationPrefService: per-user digestInterval CRUD (default
  'daily', upsert on @@unique userId, D-01/D-03)
- UpdateNotificationPrefDto: @IsIn(['daily','weekly','off']) validation (V5)
- GET/PUT /modules/tender-radar/notification-pref, declared before
  @Get(':id') (route-order pitfall)
- instantAlert passthrough in Create/UpdateSavedSearchDto and
  TenderSavedSearchService.create/update (NOTIFY-02, D-04)
- All pref/profile routes scoped strictly via extractTriageContext(req),
  never from body/query (T-12-14, IDOR)
- Updated tenders.controller.spec.ts fakes for the new constructor param

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 09:28:02 +02:00
schalli 5395ce6b6d test(12-03): prove instant+digest single-mail guarantee across both channels
Integration spec runs TenderMatchingService.matchDelta and
TenderDigestScheduler.runDigest against one shared mocked-Prisma store (no
live DB, mocked TenderMailService, no live SMTP) to prove the NOTIFY-03
core invariant end-to-end:

- instantAlert=true: matchDelta sends exactly one instant mail and stamps
  notifiedAt='instant'; the subsequent digest run then sees zero eligible
  matches for that user and sends zero digest mails
  (sendInstant=1, sendDigest=0).
- instantAlert=false: matchDelta never dispatches instant; the digest run
  is the only channel and sends exactly one mail
  (sendInstant=0, sendDigest=1).

Both scenarios assert total mail count across channels === 1.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 09:22:04 +02:00
schalli ec032ad89d feat(12-03): instant-alert dispatch at end of matchDelta
GREEN: after all match upserts of a poll tick are written, profiles with
instantAlert=true are checked for fresh (notifiedAt=NULL, tenderId IN
newTenderIds) matches. If any exist they are bundled into one
TenderMailService.sendInstant call per profile per tick (D-05). notifiedAt
is stamped 'instant' only on a successful send (D-06) -- the same
eligibility gate the digest reads, so a tender x profile pair can never be
notified twice across instant and digest. Instant dispatch runs
synchronously in the tick, before any later digest run.

Each profile's dispatch is wrapped in its own try/catch so a send
failure/thrown error never aborts the tick or the remaining profiles'
dispatch; notifiedAt stays NULL on failure and is retried next tick/digest.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 09:20:49 +02:00
schalli 86c184f8d5 test(12-03): add failing tests for instant-alert dispatch in matchDelta
RED: covers D-04 (instantAlert=true only), D-05 (bundling per profile/tick),
D-06 (stamp notifiedAt/channel=instant only after success), retry-safety on
send failure (per-profile catch, other profiles unaffected), and no-op when
a profile has no fresh notifiedAt=NULL matches this tick.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 09:20:43 +02:00
schalli 32441678c1 feat(12-02): register TenderMailService/TenderDigestScheduler, import SettingsModule
TendersModule imports SettingsModule so TenderMailService can inject
SettingsService (getDecryptedSmtpConfig), and registers
TenderMailService + TenderDigestScheduler as providers alongside the
existing TenderMatchingService (12-01). ScheduleModule.forRoot() is
already global in AppModule — not re-imported. DI graph verified
resolvable via npx tsc --noEmit; full apps/api suite green (192/192).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 09:14:40 +02:00
schalli c0a8906d6a feat(12-02): TenderDigestScheduler — ein globaler Cron, findMany über fällige Nutzer
A single platform-wide @nestjs/schedule cron (daily 07:00), registered
via SchedulerRegistry exactly like TenderSchedulerService — NOT the
DkvSchedulerService single-tenant pattern (Pitfall 1). Selects
candidate users as distinct userId with an open TenderMatch
(notifiedAt IS NULL) via findMany across all tenants, resolves each
user's TenderNotificationPref.digestInterval (missing row -> daily
default, D-01: daily always due, weekly only on Monday Europe/Berlin,
off never), groups their un-notified matches by saved-search profile
name into one TenderMailService.sendDigest call per user (D-02), and
stamps notifiedAt+channel='digest' ONLY after a successful send — the
shared notifiedAt-IS-NULL eligibility gate that guarantees no
double-send with instant alerts (D-06).

Each candidate user is processed in its own try/catch: a missing SMTP
config, a send failure, or an unexpected thrown error for one
user/tenant leaves that user's matches notifiedAt=NULL (retried next
run) and never aborts the run for the rest (Pitfall 6).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 09:13:56 +02:00
schalli 2f305840a2 test(12-02): add failing TenderDigestScheduler spec (RED)
Covers NOTIFY-01/03: due-date selection (daily always, weekly only on
Monday Europe/Berlin, off never, missing pref row defaults to daily —
D-01), multi-tenant safety via findMany over ALL due users across ALL
tenants (never findFirst — the documented DkvSchedulerService v1-gap,
Pitfall 1), one sectioned mail per user grouping matches by saved
search (D-02), the no-double-send notifiedAt eligibility gate (only
notifiedAt=NULL selected, stamped notifiedAt+channel=digest only after
a successful send — D-06), and per-user robustness so one failing/
skipped/throwing user never aborts the run for the rest (Pitfall 6).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 09:12:54 +02:00
schalli 53e72dfa05 feat(12-02): TenderMailService — mandanten-SMTP-Versand (DkvMailService-Klon)
Structural clone of DkvMailService (RESEARCH.md Pattern F / D-08): a
fresh nodemailer transport is built from
settingsService.getDecryptedSmtpConfig(tenantId) on every send, never
a cached/global mailer, and transport.close() always runs in finally
(WR-01 socket-leak guard).

Unlike DkvMailService, sendDigest/sendInstant never throw — a missing
SmtpConfig or a send failure both resolve to false so the digest
scheduler (Task 2) can decide whether to stamp TenderMatch.notifiedAt
without a per-caller try/catch, and a cron run never crashes because
one tenant lacks SMTP config (Pitfall 6).

sendDigest builds ONE mail sectioned by saved-search profile name
(D-02); sendInstant builds ONE collective mail per profile (D-05).
estimatedValue is formatted via String() only, never Number()-coerced
(mostly-null Decimal field). Tender titles/profile names are
HTML-escaped before interpolation (T-12-08).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 09:11:57 +02:00