feat(14-02): add TenderRssFeedSource CRUD, tick poll gate, and wire RssAdapter
Global admin-managed RSS feed list (TenderRssFeedSource, D-08/D-14) with
a save-time hostname/SSRF guard (TenderRssFeedSourceService) — RSS feed
URLs are runtime admin input, so the code-level SourceRegistry denylist
gate does not cover them; a separate check rejects DENYLISTED_PORTALS
hostnames, non-http(s) schemes, and private/loopback hosts.
Adds TenderSourcePollConfig.pollGranularity ('day' | 'tick', D-15):
pollDueSources() branches per source — 'day' sources keep the existing
lastIngestedDay gate byte-unchanged, 'tick' sources (rss) fetch on every
active scheduler tick regardless of lastIngestedDay, since the day-cursor
gate was built for a genuine daily batch-export API and would otherwise
silently cap RSS to one fetch per calendar day.
Wires RssAdapter.fetchTenders() to fan out over active feed rows (native
fetch + AbortController 15s + response-size ceiling, catch-per-feed),
registers it in tenders.module.ts, and seeds the 'rss' poll config
active with pollGranularity='tick' plus a default-active service.bund.de
feed row (subreport-elvis has no single canonical URL — zero rows seeded,
admin adds relevant municipality feeds).
Migration applied locally per project convention (host -> container IP).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -419,6 +419,31 @@ model TenderSourcePollConfig {
|
||||
pollIntervalMin Int @default(60) // D-04 default hourly
|
||||
isActive Boolean @default(false)
|
||||
lastIngestedDay DateTime? // day-cursor, NOT a timestamp (RESEARCH Pattern 1)
|
||||
// Phase 14, Plan 02 (D-15/Pitfall 1): 'day' | 'tick'. 'day' sources
|
||||
// (doe-opendata/ai-netserver/cosinex-dtvp) keep the lastIngestedDay-gated
|
||||
// once-per-calendar-day fetch, unchanged. 'tick' sources (rss) are
|
||||
// fetched on every active scheduler tick, ignoring lastIngestedDay
|
||||
// entirely — the day-cursor gate was built for a genuine daily
|
||||
// batch-export API and would otherwise silently cap RSS to one fetch
|
||||
// per day regardless of pollIntervalMin.
|
||||
pollGranularity String @default("day")
|
||||
createdAt DateTime @default(now())
|
||||
updatedAt DateTime @updatedAt
|
||||
}
|
||||
|
||||
// Phase 14, Plan 02 (INGEST-04, D-14) — admin-managed GLOBAL RSS feed list.
|
||||
// Deliberately NO tenantId (mirrors TenderSourcePollConfig's global/
|
||||
// RLS-exempt stance, D-08: RSS feeds are public and identical for every
|
||||
// tenant). Feed URLs are RUNTIME admin input — unlike the hardcoded
|
||||
// NETSERVER_PORTALS/COSINEX_BASE_URL constants, the code-level
|
||||
// SourceRegistry denylist gate does NOT cover this data (RESEARCH.md
|
||||
// Pitfall 3); TenderRssFeedSourceService enforces a SEPARATE save-time
|
||||
// hostname/SSRF guard (T-14-02-01) on create/update.
|
||||
model TenderRssFeedSource {
|
||||
id String @id @default(uuid())
|
||||
url String @unique
|
||||
label String
|
||||
isActive Boolean @default(true)
|
||||
createdAt DateTime @default(now())
|
||||
updatedAt DateTime @updatedAt
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user