tessera-ctl/.planning/phases/14-rss-email-alert-ingestion-module-rollout/14-02-SUMMARY.md

---

phase: 14-rss-email-alert-ingestion-module-rollout plan: 02 subsystem: api tags: [nestjs, prisma, fast-xml-parser, rss, ssrf, cron, nextjs]

Dependency graph

requires:

• phase: 13-additional-tender-sources-cross-source-dedup provides: TenderSourceAdapter interface, SourceRegistry denylist gate, normalizeBag() generic-bag normalizer path, pollDueSources fan-out provides:
• RssAdapter (fast-xml-parser, admin-feed internal fan-out) parsing RSS 2.0 into RawTenderRecord[]
• 'rss' SourceType + normalizer dispatch through normalizeBag()
• TenderRssFeedSource Prisma model (global admin-managed feed list) + save-time hostname/SSRF guard service
• TenderSourcePollConfig.pollGranularity ('day' | 'tick') gate in pollDueSources — RSS bypasses the day-cursor entirely
• GET/POST/DELETE /modules/tender-radar/rss-feeds admin routes + RssFeedListForm settings UI affects: [14-03-email-alert-ingestion, 14-04-admin-config-encryption, 14-05-i18n]

Tech tracking

tech-stack: added: [] patterns: - "RSS parsing reuses the existing fast-xml-parser XMLParser config (removeNSPrefix/ignoreAttributes/attributeNamePrefix) from doe-opendata.adapter.ts — no new dependency" - "Internal-fan-out adapter: RssAdapter.fetchTenders() reads all active TenderRssFeedSource rows and fetches each independently (catch-per-feed), same shape as NetServerAdapter's multi-portal loop" - "pollGranularity branch in pollDueSources(): 'day' sources keep the byte-unchanged lastIngestedDay gate; 'tick' sources fetch unconditionally every active tick" - "Save-time hostname/SSRF guard as a SEPARATE enforcement point from the code-level SourceRegistry denylist — required whenever a source's target is runtime admin input rather than a hardcoded portal"

key-files: created: - apps/api/src/tenders/adapters/rss.adapter.ts - apps/api/src/tenders/adapters/rss.adapter.spec.ts - apps/api/src/tenders/fixtures/service-bund-feed.xml - apps/api/src/tenders/fixtures/subreport-elvis-feed.xml - apps/api/src/tenders/tender-rss-feed.service.ts - apps/api/src/tenders/tender-rss-feed.service.spec.ts - apps/api/src/tenders/dto/tender-rss-feed.dto.ts - apps/api/prisma/migrations/20260723111946_add_rss_feed_source_and_poll_granularity/migration.sql - apps/web/src/app/(portal)/modules/tender-radar/settings/components/RssFeedListForm.tsx - apps/web/src/app/(portal)/modules/tender-radar/settings/components/RssFeedListForm.test.tsx modified: - apps/api/src/tenders/tender.types.ts - apps/api/src/tenders/tender-normalizer.service.ts - apps/api/src/tenders/tender-normalizer.service.spec.ts - apps/api/src/tenders/tender-ingestion.service.ts - apps/api/src/tenders/tender-ingestion.service.spec.ts - apps/api/src/tenders/tenders.module.ts - apps/api/src/tenders/tenders.controller.ts - apps/api/src/tenders/tenders.controller.spec.ts - apps/api/prisma/schema.prisma - apps/web/src/lib/tender-radar-api.ts - apps/web/src/app/(portal)/modules/tender-radar/settings/page.tsx

key-decisions:

• "fast-xml-parser only decodes the 5 predefined XML entities, not numeric character refs (Ü) — added an explicit decodeNumericEntities() step so service.bund.de titles render correctly instead of leaking raw entity syntax (Rule 1 fix)"
• "Seeded service.bund.de as an active-by-default TenderRssFeedSource row; zero subreport-elvis rows seeded (no single canonical URL — per-municipality instances, admin adds relevant feeds)"
• "'rss' TenderSourcePollConfig seeded isActive:true/pollGranularity:'tick' from day one — unlike ai-netserver/cosinex-dtvp's 'framework ready, activation deferred' stance from Phase 13"
• "Save-time SSRF guard is string/IP-literal-based only, no DNS resolution — matches the existing T-10-06 scope; resolving a hostname to check its IP would itself be an SSRF-adjacent action"

patterns-established:

• "pollGranularity as the mechanism the upcoming email-alert adapter (Plan 14-03) reuses for its own tick-driven polling need"

requirements-completed: [INGEST-04, CONFIG-02]

coverage:

• id: D1 description: "RssAdapter.parseFeed maps live-captured service.bund.de (pubDate present, numeric-entity titles) and subreport-elvis (pubDate absent, CDATA titles) shapes into RawTenderRecord[]; malformed/empty XML and single-item link-loss never throw" requirement: INGEST-04 verification:
  • kind: unit ref: "apps/api/src/tenders/adapters/rss.adapter.spec.ts (18 tests)" status: pass human_judgment: false
• id: D2 description: "'rss' sourceType routes through TenderNormalizerService.normalizeBag() with title-through mapping and empty cpvDivisions" requirement: INGEST-04 verification:
  • kind: unit ref: "apps/api/src/tenders/tender-normalizer.service.spec.ts (rss describe block, 2 tests)" status: pass human_judgment: false
• id: D3 description: "Admin can add/list/remove global RSS feed URLs; vergabe24/aumass and private/loopback hosts are rejected at save time with a domain-specific 400 message" requirement: CONFIG-02 verification:
  • kind: unit ref: "apps/api/src/tenders/tender-rss-feed.service.spec.ts (14 tests)" status: pass
  • kind: unit ref: "apps/api/src/tenders/tenders.controller.spec.ts (RSS-feeds describe block, 4 tests + 1 route-order test)" status: pass human_judgment: false
• id: D4 description: "RSS is polled every active scheduler tick honoring pollIntervalMin (via the single global cron), NOT gated to once per calendar day like doe-opendata/ai-netserver/cosinex-dtvp" requirement: INGEST-04 verification:
  • kind: unit ref: "apps/api/src/tenders/tender-ingestion.service.spec.ts (pollGranularity 'tick' gate describe block, 3 tests)" status: pass human_judgment: false
• id: D5 description: "RssFeedListForm renders an admin-manageable RSS feed list on the tender-radar settings page: load, add (denylist error surfaced inline), remove" requirement: CONFIG-02 verification:
  • kind: unit ref: "apps/web/src/app/(portal)/modules/tender-radar/settings/components/RssFeedListForm.test.tsx (6 tests)" status: pass human_judgment: false

duration: 30min completed: 2026-07-23 status: complete

Phase 14 Plan 02: RSS Ingestion Slice Summary

RssAdapter (fast-xml-parser, admin-CRUD feed fan-out) + global TenderRssFeedSource list with a save-time denylist/SSRF guard + pollGranularity='tick' gate so RSS honors its poll interval instead of DÖE's day-cursor

Performance

• Duration: ~30 min (including live-fixture capture and a pre-existing migration checksum-drift fix)
• Started: 2026-07-23T13:09:00Z (context reads) / first commit 13:17:02
• Completed: 2026-07-23T13:29:07Z
• Tasks: 3/3 completed
• Files modified: 21 (10 created, 11 modified)

Accomplishments

• RssAdapter.parseFeed() maps two live-captured RSS shapes (service.bund.de with pubDate + numeric-HTML-entity titles; subreport-elvis without pubDate, CDATA titles) into RawTenderRecord[], routed through the existing generic normalizeBag() normalizer path (D-04/D-05, zero normalizer-specific RSS code)
• TenderRssFeedSource Prisma model (global, no tenantId, D-08) with full admin CRUD (TenderRssFeedSourceService) gated by a save-time hostname/SSRF guard (D-14) — rejects DENYLISTED_PORTALS hostnames (vergabe24/aumass, same constant as the code-level gate), non-http(s) schemes, and private/loopback/link-local hosts
• TenderSourcePollConfig.pollGranularity ('day' | 'tick') added; pollDueSources() branches per source — 'day' sources (doe-opendata/ai-netserver/cosinex-dtvp) keep the exact pre-existing lastIngestedDay gate, 'tick' sources (rss) fetch unconditionally every active tick (D-15)
• RssAdapter registered in tenders.module.ts; 'rss' poll config seeded isActive: true, pollGranularity: 'tick'; a default-active service.bund.de feed row seeded (zero subreport-elvis rows — no single canonical URL to default to)
• GET/POST/DELETE /modules/tender-radar/rss-feeds admin routes (@Roles(ADMIN, SUPER_ADMIN)), declared before @Get(':id') per the project's NestJS route-order convention
• Web: tender-radar-api.ts RSS feed client functions (relaying the backend's specific denylist/SSRF rejection message) + RssFeedListForm.tsx rendering an "RSS-Feeds" section on the existing tender-radar settings page (D-09)

Task Commits

1. Task 1: RssAdapter.parseFeed + 'rss' normalizer dispatch (fixture-first) - 3a96cbb (feat)
2. Task 2: TenderRssFeedSource model + CRUD service (denylist guard) + tick-gate + wiring - e812738 (feat)
3. Task 3: RSS feed admin routes + client + RssFeedListForm settings section - 48a2dc1 (feat)

No separate plan-metadata commit yet — this SUMMARY/STATE/ROADMAP update is the final commit for this plan.

Files Created/Modified

• apps/api/src/tenders/adapters/rss.adapter.ts - RssAdapter implements TenderSourceAdapter; pure parseFeed() + internal fan-out fetchTenders() over active TenderRssFeedSource rows (native fetch + AbortController 15s + 10MB size ceiling, catch-per-feed)
• apps/api/src/tenders/adapters/rss.adapter.spec.ts - Fixture-driven parseFeed tests (both feed shapes) + fetchTenders fan-out/catch-per-feed/size-ceiling tests (18 tests)
• apps/api/src/tenders/__fixtures__/service-bund-feed.xml / subreport-elvis-feed.xml - Live-captured (2026-07-23) real RSS XML, trimmed to a representative item subset (6 and 5 items respectively)
• apps/api/src/tenders/tender.types.ts - SourceType extended with 'rss'
• apps/api/src/tenders/tender-normalizer.service.ts - normalize() dispatches 'rss' through normalizeBag()
• apps/api/src/tenders/tender-rss-feed.service.ts - Admin CRUD (list/create/remove) + assertUrlAllowed() save-time hostname/SSRF guard
• apps/api/src/tenders/dto/tender-rss-feed.dto.ts - TenderRssFeedDto (coarse @IsUrl check; substantive guard lives in the service)
• apps/api/src/tenders/tender-ingestion.service.ts - pollDueSources() pollGranularity branch (D-15)
• apps/api/src/tenders/tenders.module.ts - RssAdapter/TenderRssFeedSourceService providers, registry registration, poll-config + default-feed seeds
• apps/api/src/tenders/tenders.controller.ts - rss-feeds routes (declared before :id)
• apps/api/prisma/schema.prisma + migration - TenderRssFeedSource model, TenderSourcePollConfig.pollGranularity column
• apps/web/src/lib/tender-radar-api.ts - RssFeedSource/CreateRssFeedPayload types + listRssFeeds/createRssFeed/deleteRssFeed + extractErrorMessage helper
• apps/web/src/app/(portal)/modules/tender-radar/settings/components/RssFeedListForm.tsx + .test.tsx - Admin feed-list UI (load/add/remove, inline denylist error)
• apps/web/src/app/(portal)/modules/tender-radar/settings/page.tsx - "RSS-Feeds" section added below SourceConfigForm

Decisions Made

• Trimmed the live-captured service.bund.de fixture (500 items, 360KB) down to the first 6 items (5.3KB) — a representative real-data sample, not a fabricated one, matching the sizing convention of the existing cosinex fixture (20 rows)
• Chose a string/IP-literal-based private-host check (no DNS resolution) for the SSRF guard — resolving a hostname at save time would itself be an SSRF-adjacent action and is out of scope; matches the existing T-10-06 pattern's scope
• pollIntervalMin: 60 seeded for the 'rss' poll config for consistency with other seeds, even though the single global cron's actual tick rate is still governed by the doe-opendata config's setInterval() call (pre-existing architecture, unchanged this plan)

Deviations from Plan

Auto-fixed Issues

1. [Rule 1 - Bug] fast-xml-parser does not decode numeric HTML character references

• Found during: Task 1 (writing rss.adapter.ts against the live-captured service.bund.de fixture)
• Issue: fast-xml-parser v5.10.1 only decodes the 5 predefined XML entities (& < > " '); service.bund.de item titles use raw numeric character references (Übermittlung...) rather than CDATA — without a fix, RSS-derived tender titles would literally display Übermittlung... to users instead of Übermittlung....
• Fix: Added decodeNumericEntities() (decimal &#NNN; and hex &#xHH;) applied to the extracted title only (never to <description>, which is never read at all — no HTML risk).
• Files modified: apps/api/src/tenders/adapters/rss.adapter.ts
• Verification: rss.adapter.spec.ts asserts the decoded title contains "Ü" and no residual &# sequence.
• Committed in: 3a96cbb (part of Task 1 commit)

2. [Rule 3 - Blocking] Pre-existing Prisma migration checksum drift blocked migrate dev

• Found during: Task 2 (running the RSS schema migration against the local dev DB)
• Issue: prisma migrate dev refused to proceed ("The migration 20260722100000_add_tender_notifications was modified after it was applied... reset the schema"), because the _prisma_migrations.checksum column for that pre-existing migration was empty in the DB — a leftover from an earlier out-of-band resolution (see project memory: "Failed Prisma migration resolved and marked as applied", 2026-07-21). migrate status reported "up to date" (no pending drift), confirming this was purely a stale-checksum bookkeeping issue, not an actual schema mismatch.
• Fix: Verified via migrate status that no real drift existed, then updated the recorded checksum in _prisma_migrations to match the on-disk migration file's sha256 (no migrate reset, no data loss) before re-running migrate dev for this plan's new migration.
• Files modified: none (DB-only fix, no source file changes)
• Verification: prisma migrate dev --name add_rss_feed_source_and_poll_granularity then succeeded cleanly; prisma validate passes; \d "TenderRssFeedSource" / \d "TenderSourcePollConfig" confirmed the new schema live.
• Committed in: N/A (database-only, no commit)

---

Total deviations: 2 auto-fixed (1 Rule 1 bug fix, 1 Rule 3 blocking-issue fix) Impact on plan: Both fixes were necessary for correctness (Rule 1: user-visible garbled titles) or to unblock the required local migration (Rule 3: pre-existing DB bookkeeping issue, unrelated to this plan's schema changes). No scope creep.

Issues Encountered

None beyond the two deviations documented above.

User Setup Required

None - no external service configuration required. The migration was applied locally per the project's documented convention (host → container IP 172.19.0.2, tessera:tessera_dev); no Docker rebuild/restart was performed (per CLAUDE.md instruction).

Next Phase Readiness

• The pollGranularity mechanism (D-15) is now proven end-to-end and ready for Plan 14-03 (email-alert ingestion) to reuse for its own tick-driven polling need, as anticipated by this plan's purpose statement.
• Full API suite: 343/343 tests passing (29 files). Full web suite: 139/139 tests passing (24 files). Both tsc --noEmit clean.
• No blockers for Plan 14-03/14-04/14-05.

---

Phase: 14-rss-email-alert-ingestion-module-rollout Completed: 2026-07-23

Self-Check: PASSED

All created files verified present on disk; all 3 task commit hashes (3a96cbb, e812738, 48a2dc1) verified present in git history.