Commit Graph

4 Commits

Author SHA1 Message Date
schalli 3a96cbbbe6 feat(14-02): add RssAdapter.parseFeed + 'rss' normalizer dispatch
Fixture-first RSS parsing (INGEST-04): parses live-captured
service.bund.de (pubDate present, numeric-HTML-entity titles) and
subreport-elvis (pubDate absent, CDATA titles) feed shapes into
RawTenderRecord[] via fast-xml-parser, mirroring the DoeOpenDataAdapter
config. SourceType extended with 'rss'; normalize() dispatches 'rss'
through the existing normalizeBag() path unchanged (D-04/D-05).

Rule 1 fix: fast-xml-parser only decodes the 5 predefined XML entities,
not numeric character references — added an explicit decode step so
service.bund.de titles ("Übermittlung...") render correctly
instead of leaking raw entity syntax.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-23 13:17:02 +02:00
schalli 12fc5ac02d feat(13-05): add cosinex/DTVP source adapter with fixture tests
Separate HTML adapter for the cosinex Vergabemarktplatz (DTVP) satellite
(sourceType='cosinex-dtvp'), distinct from the NetServer adapter since
cosinex markup differs structurally. Live inspection (2026-07-23) found
the "Aktuelle Bekanntmachungen" results table is fully server-rendered
(not JS-dependent as D-01 anticipated), so selectors are fully populated
rather than falling back to a needs-JS stub — parses publish date,
deadline (or "nv"), title, legal framework/procedure type, buyer name,
and a real per-notice deep link (pid) into RawTenderRecord[].

Rule 1 fix: cosinex serves charset=ISO-8859-1 with raw Latin-1 bytes for
umlauts (not HTML entities); Response.text() always UTF-8-decodes per
the Fetch spec, so the adapter reads arrayBuffer() and decodes explicitly
via TextDecoder('iso-8859-1') to avoid mojibake.

16 spec tests pass against a live-captured fixture (20 rows, transcoded
to UTF-8 on disk): full-fixture parse, deadline/publish date parsing,
nested-<abbr> procedure-type extraction, umlaut decoding, empty/broken
HTML and missing-pid row fallback, fetch-throw/non-2xx fallback, no-axios
and no-input-interpolated-URL guards.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 09:48:21 +02:00
schalli 043ada9438 feat(13-04): NetServer-Adapter fuer tender24/lhs-vpbw/vergabe.landbw (INGEST-02)
- Package-Legitimacy-Checkpoint bestaetigt: cheerio (sauberer Gate-Pass,
  27M Downloads/Woche) statt node-html-parser installiert
- EIN config-getriebener NetServerAdapter bedient alle 3 AI-AG-Portale
  via PublicationSearchControllerServlet-Trefferliste, parst <table> mit
  cheerio zu RawTenderRecord[], sourcePortal je Zeile korrekt
- sourceNoticeId = data-oid (stabil, per Zeile eindeutig); sourceUrl =
  Such-URL als dokumentierter Best-Effort-Fallback (kein Deep-Link ohne
  JS-Ausfuehrung ermittelbar, Open Question 2)
- Fehlertoleranz: try/catch pro Zeile + pro Portal, []-Fallback bei
  Totalausfall, AbortController 15s-Timeout, hardcodierte Portal-URLs
  (SSRF-Guard T-13-04-01)
- Live-gecapturte Fixture (tender24.de, 2026-07-23) mit Umlaut-/&amp;-
  Buyer, gefuellter und leerer Deadline; 14 Spec-Tests gruen

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 09:13:01 +02:00
schalli f88e2a8141 test(10-03): add failing specs + real DÖE fixtures for adapter/normalizer
- Capture real, trimmed DÖE day-export fixtures (pubDay=2026-07-19, 8
  notices spanning tender/award/planning/untagged-with-awards tag classes)
  under tenders/__fixtures__/, live-downloaded from oeffentlichevergabe.de
- Define RawTenderRecord/NormalizedTenderFields/SourceType (tender.types.ts)
  and TenderSourceAdapter (day-cursor fetchTenders signature per RESEARCH
  Pattern 1)
- RED: doe-opendata.adapter.spec.ts asserts D-02 exact-count filtering,
  HTTP-400 no-op, and a zip-bomb ceiling guard (T-10-07)
- RED: tender-normalizer.service.spec.ts asserts eForms-primary deadline
  recovery where OCDS is null (RESEARCH Pattern 3), nullable deadline/value,
  dedupKey priority (ocid -> sourcePortal:sourceNoticeId), and a stable
  sha256 contentHash
2026-07-21 10:56:13 +02:00