Normalizer-Gap (quick 260723-e7i) closed criteria 1/2; re-verification
lifts Phase 13 from 1/4 to 3/4. Truth 3 (cross-source fingerprint dedup)
stays PARTIAL: newly-found cpvDivisions asymmetry (scrapers always [],
~79% of DÖE tenders populated) makes exact-match fingerprint miss most
real overlaps. User decision: accept as dormant deferral — dedup is inert
(both scrapers isActive=false), decide with live data once a 2nd source
is activated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- bagRecord() helper builds inline RawTenderRecords for the flat ocdsPayload
bag shape (no fixtures exist for NetServer/cosinex-DTVP)
- ai-netserver and cosinex-dtvp full-bag mapping, null/empty-field fallback,
and contentHash-format assertions
- Existing DOE fixture-based assertions untouched, confirming the refactor
didn't change DOE-path behavior
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Extract shared assemble() tail (status/dedupKey/contentHash/publishedAt)
so it is computed identically across all sources, not duplicated
- Move existing DOE eForms/OCDS extraction into normalizeDoe() (byte-identical
behavior, regression guard)
- Add normalizeBag() for the flat ocdsPayload bag shared by the NetServer and
cosinex/DTVP scraper adapters ({title, buyerName, procedureType,
legalFramework, deadlineAt}); legalFramework deliberately not mapped
- normalize() dispatches on raw.sourceType, defaulting to the DOE path so the
additive SourceType union never throws
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds CosinexAdapter to TendersModule's providers and registers it with
SourceRegistry at DI boot, alongside DoeOpenDataAdapter/NetServerAdapter
(cosinex-dtvp is not AGB-denylisted, so registration succeeds). Seeds a
cosinex-dtvp TenderSourcePollConfig row with isActive: false, matching
the ai-netserver "framework ready, activation deferred" stance (D-02).
tsc --noEmit clean; src/tenders slice: 21 files, 221/221 tests pass
(205 pre-existing + 16 new cosinex.adapter.spec.ts).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Separate HTML adapter for the cosinex Vergabemarktplatz (DTVP) satellite
(sourceType='cosinex-dtvp'), distinct from the NetServer adapter since
cosinex markup differs structurally. Live inspection (2026-07-23) found
the "Aktuelle Bekanntmachungen" results table is fully server-rendered
(not JS-dependent as D-01 anticipated), so selectors are fully populated
rather than falling back to a needs-JS stub — parses publish date,
deadline (or "nv"), title, legal framework/procedure type, buyer name,
and a real per-notice deep link (pid) into RawTenderRecord[].
Rule 1 fix: cosinex serves charset=ISO-8859-1 with raw Latin-1 bytes for
umlauts (not HTML entities); Response.text() always UTF-8-decodes per
the Fetch spec, so the adapter reads arrayBuffer() and decodes explicitly
via TextDecoder('iso-8859-1') to avoid mojibake.
16 spec tests pass against a live-captured fixture (20 rows, transcoded
to UTF-8 on disk): full-fixture parse, deadline/publish date parsing,
nested-<abbr> procedure-type extraction, umlaut decoding, empty/broken
HTML and missing-pid row fallback, fetch-throw/non-2xx fallback, no-axios
and no-input-interpolated-URL guards.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Tender.sources[] added to the API client type (sourcePortal, sourceUrl,
sourceNoticeId). TenderDetail now renders one link per TenderSource
with a German portal label (DÖE/tender24/DTVP/...), falling back to
the existing single sourceUrl block when sources is missing or empty
(older responses, single-source tenders).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
GET /modules/tender-radar/:id now includes the TenderSource relation
(sourcePortal, sourceUrl, sourceNoticeId) so a cross-source-deduped
tender's detail response carries links to all its source portals, not
just the single primary sourceUrl column. Route order unchanged (:id
stays after all static routes).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Register SourceRegistry and TenderDedupService as providers.
onModuleInit registers DoeOpenDataAdapter with the registry before the
scheduler's first tick — the DI-boot-time enforcement point for the
INGEST-07 denylist gate (D-06). This is Wave 2's sole writer of
tenders.module.ts; 13-04 (NetServer) and 13-05 (cosinex) add their
own registry.register(...) calls additively in later waves.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the DÖE-only findUnique with findMany({isActive:true}) fan-out
(poll-once-fan-out-many, D-01). Each active TenderSourcePollConfig is
resolved through SourceRegistry.get(sourceType) and processed inside
its own try/catch (catch-per-source, D-01) — one broken/blocking source
no longer aborts the tick for the others. dedupActive =
activePortalCount >= 2 (D-05) is computed once per tick and passed to
TenderDedupService.resolve(), which now replaces the direct
tender.upsert call. Delta-only matchDelta boundary (D-07) preserved:
only genuinely-created tender IDs across all sources are collected.
Extended tender-ingestion.service.spec.ts: multi-config fan-out,
catch-per-source isolation, dedupActive gate assertion, adapter-missing
skip, plus the existing SCHEMA-02/D-07/retention/day-cursor suites
updated to the new registry+dedup constructor shape (all green).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
resolve(n, {dedupActive}) matches OCID -> source:noticeId -> fingerprint
(fingerprint tier hard-gated by dedupActive, D-05). On any match the
existing Tender gets an additional TenderSource attached (D-03 merge)
instead of a new Tender row; SCHEMA-02 change-detection is preserved
inline (matched Tender's mutable fields refresh when contentHash
differs, exactly as the old direct tender.upsert UPDATE branch did).
No match -> tender.create (with computed fingerprint) + tenderSource.create.
Plain PrismaService, no forTenant()/RLS (T-10-09).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
GREEN — SourceRegistry.register() throws DeniedPortalError when any
of an adapter's declared portals is in DENYLISTED_PORTALS
(vergabe24, aumass), enforced at DI-registration time (INGEST-07/
D-06), not just documented. get()/activeAdapters() support the
Plan 13-03 poll-once-fan-out-many scheduler. 6/6 tests pass, no
Prisma/scraping import.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
RED — proves Erfolgskriterium 4 (INGEST-07): registering an adapter
whose portals include vergabe24 or aumass must throw DeniedPortalError,
including a mixed portals array with one denylisted entry. Also covers
legitimate register/get/activeAdapters happy paths. Fake adapter stub,
no real scraping.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
TenderSourceAdapter gains a readonly portals: readonly string[] field
so one adapter can serve multiple portals (NetServer: 3, Plan 13-04)
and so SourceRegistry can gate registration per-portal (INGEST-07).
DoeOpenDataAdapter declares portals = ['doe-opendata'] additively,
no behavior change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
SourceType now covers 'doe-opendata' | 'ai-netserver' | 'cosinex-dtvp'
(13-RESEARCH Pattern 1) so the Plan 13-04/05 adapters can register
without further type-contract changes. NormalizedTenderFields gains an
optional fingerprint field for the SCHEMA-03 dedup resolver (Plan
13-03) to populate later. tsc --noEmit clean; full API suite green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Additive schema change (SCHEMA-03/D-03/D-04): new model TenderSource
(1:n Tender, @@unique[sourcePortal, sourceNoticeId], onDelete Cascade)
and a nullable Tender.fingerprint column + index. dedupKey stays
unchanged as the SCHEMA-02 upsert target.
Migration 20260723120000_add_tender_source applies in strict order
(Pitfall 5): table+column create, then one TenderSource row per
pre-existing Tender via SQL INSERT/SELECT, then the unique constraint.
Applied locally against the tessera dev DB (container IP, no host
port) — verified via psql: TenderSource count == Tender count == 2851.
backfill-tender-source.ts is a one-time script that computes
Tender.fingerprint via the Task-1 tenderFingerprint() function
(Decimal->number conversion for estimatedValue, T-13-01-03) — run via
the compiled dist/ output (source uses standard extensionless TS
imports for tsc compatibility). Confirmed: 2851/2851 rows backfilled,
idempotent re-run verified.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
GREEN: title+buyer dominant, CPV division (order-independent, dedup'd),
value bucketed by order-of-magnitude, deadline truncated to day-grain.
sha256 hex, deterministic, no I/O — foundation for the Task-2 backfill
and the Plan 13-03 dedup resolver's fingerprint tier.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
6 plans (INGEST-02/03/07, SCHEMA-03) + Nyquist validation.
Core (registry, fingerprint, dedup, TenderSource, backfill) lands
first and is green independent of live scraping (D-01).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
LDAP-imported users have no local passwordHash, and validateUser only checked
the local password, so they could never log in. Now a passwordless user with
an ldapDn is authenticated by binding as their OWN DN with the entered
password against the tenant's active LDAP config (reusing the ldaps TLS-skip
option). Empty passwords are rejected before binding to avoid AD's
unauthenticated-bind bypass. Local-password users are unchanged.
LdapService.verifyUserCredentials added; LdapModule now exports
LdapConfigService; AuthModule imports LdapModule (no circular dep). 8 new
specs (bind success/fail, empty-password guard, login via bind, wrong pw, no
config, no ldapDn, inactive). API 226 green, tsc clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a per-tenant "Skip TLS certificate verification" toggle to the LDAP
admin page so admins can connect to an AD whose ldaps:// certificate is
signed by an internal/self-signed CA (Node error: "unable to verify the
first certificate"). When enabled, ldapts is given
tlsOptions.rejectUnauthorized=false; the flag is ignored for plain ldap://
(no TLS). Defaults to full verification.
New Boolean column LdapConfig.tlsRejectUnauthorized (@default(true)) +
migration; wired through DTOs, config service, all Client creations
(test/groups/user-search/import/sync) and the test-connection endpoint. UI
checkbox with an insecure-network warning (de/en). 3 new service specs;
API 218 green, web 131 green, both apps tsc clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add an AD single-user search (by cn/sAMAccountName/displayName/mail) and a
selective import to the LDAP admin page, alongside the existing group/OU
filter. Imported users are deduped against existing ones by (ldapDn, then
username): a manually-imported user carries its ldapDn, so a later
department/group sync matches and updates it in place instead of creating a
duplicate. Search results flag alreadyImported; import skips existing users
and links a missing ldapDn. Extracted shared mapEntry/upsertMappedUser
helpers so sync and manual import resolve identity identically.
Backend: GET /ldap/users/search, POST /ldap/users/import (RFC-4515 escaped
query, ADMIN-guarded). 6 new service specs (search flags, create, skip,
ldapDn-link, denylist). Full API suite 215 green, both apps tsc clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- fetchNotificationPref/saveNotificationPref for GET/PUT
/modules/tender-radar/notification-pref (NOTIFY-01)
- SavedSearch/Create/UpdateSavedSearchPayload now carry instantAlert
(NOTIFY-02, D-04)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- TenderNotificationPrefService: per-user digestInterval CRUD (default
'daily', upsert on @@unique userId, D-01/D-03)
- UpdateNotificationPrefDto: @IsIn(['daily','weekly','off']) validation (V5)
- GET/PUT /modules/tender-radar/notification-pref, declared before
@Get(':id') (route-order pitfall)
- instantAlert passthrough in Create/UpdateSavedSearchDto and
TenderSavedSearchService.create/update (NOTIFY-02, D-04)
- All pref/profile routes scoped strictly via extractTriageContext(req),
never from body/query (T-12-14, IDOR)
- Updated tenders.controller.spec.ts fakes for the new constructor param
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Integration spec runs TenderMatchingService.matchDelta and
TenderDigestScheduler.runDigest against one shared mocked-Prisma store (no
live DB, mocked TenderMailService, no live SMTP) to prove the NOTIFY-03
core invariant end-to-end:
- instantAlert=true: matchDelta sends exactly one instant mail and stamps
notifiedAt='instant'; the subsequent digest run then sees zero eligible
matches for that user and sends zero digest mails
(sendInstant=1, sendDigest=0).
- instantAlert=false: matchDelta never dispatches instant; the digest run
is the only channel and sends exactly one mail
(sendInstant=0, sendDigest=1).
Both scenarios assert total mail count across channels === 1.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
GREEN: after all match upserts of a poll tick are written, profiles with
instantAlert=true are checked for fresh (notifiedAt=NULL, tenderId IN
newTenderIds) matches. If any exist they are bundled into one
TenderMailService.sendInstant call per profile per tick (D-05). notifiedAt
is stamped 'instant' only on a successful send (D-06) -- the same
eligibility gate the digest reads, so a tender x profile pair can never be
notified twice across instant and digest. Instant dispatch runs
synchronously in the tick, before any later digest run.
Each profile's dispatch is wrapped in its own try/catch so a send
failure/thrown error never aborts the tick or the remaining profiles'
dispatch; notifiedAt stays NULL on failure and is retried next tick/digest.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
RED: covers D-04 (instantAlert=true only), D-05 (bundling per profile/tick),
D-06 (stamp notifiedAt/channel=instant only after success), retry-safety on
send failure (per-profile catch, other profiles unaffected), and no-op when
a profile has no fresh notifiedAt=NULL matches this tick.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
TendersModule imports SettingsModule so TenderMailService can inject
SettingsService (getDecryptedSmtpConfig), and registers
TenderMailService + TenderDigestScheduler as providers alongside the
existing TenderMatchingService (12-01). ScheduleModule.forRoot() is
already global in AppModule — not re-imported. DI graph verified
resolvable via npx tsc --noEmit; full apps/api suite green (192/192).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A single platform-wide @nestjs/schedule cron (daily 07:00), registered
via SchedulerRegistry exactly like TenderSchedulerService — NOT the
DkvSchedulerService single-tenant pattern (Pitfall 1). Selects
candidate users as distinct userId with an open TenderMatch
(notifiedAt IS NULL) via findMany across all tenants, resolves each
user's TenderNotificationPref.digestInterval (missing row -> daily
default, D-01: daily always due, weekly only on Monday Europe/Berlin,
off never), groups their un-notified matches by saved-search profile
name into one TenderMailService.sendDigest call per user (D-02), and
stamps notifiedAt+channel='digest' ONLY after a successful send — the
shared notifiedAt-IS-NULL eligibility gate that guarantees no
double-send with instant alerts (D-06).
Each candidate user is processed in its own try/catch: a missing SMTP
config, a send failure, or an unexpected thrown error for one
user/tenant leaves that user's matches notifiedAt=NULL (retried next
run) and never aborts the run for the rest (Pitfall 6).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Covers NOTIFY-01/03: due-date selection (daily always, weekly only on
Monday Europe/Berlin, off never, missing pref row defaults to daily —
D-01), multi-tenant safety via findMany over ALL due users across ALL
tenants (never findFirst — the documented DkvSchedulerService v1-gap,
Pitfall 1), one sectioned mail per user grouping matches by saved
search (D-02), the no-double-send notifiedAt eligibility gate (only
notifiedAt=NULL selected, stamped notifiedAt+channel=digest only after
a successful send — D-06), and per-user robustness so one failing/
skipped/throwing user never aborts the run for the rest (Pitfall 6).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
schalli