feat(14-02): add TenderRssFeedSource CRUD, tick poll gate, and wire RssAdapter
Global admin-managed RSS feed list (TenderRssFeedSource, D-08/D-14) with
a save-time hostname/SSRF guard (TenderRssFeedSourceService) — RSS feed
URLs are runtime admin input, so the code-level SourceRegistry denylist
gate does not cover them; a separate check rejects DENYLISTED_PORTALS
hostnames, non-http(s) schemes, and private/loopback hosts.
Adds TenderSourcePollConfig.pollGranularity ('day' | 'tick', D-15):
pollDueSources() branches per source — 'day' sources keep the existing
lastIngestedDay gate byte-unchanged, 'tick' sources (rss) fetch on every
active scheduler tick regardless of lastIngestedDay, since the day-cursor
gate was built for a genuine daily batch-export API and would otherwise
silently cap RSS to one fetch per calendar day.
Wires RssAdapter.fetchTenders() to fan out over active feed rows (native
fetch + AbortController 15s + response-size ceiling, catch-per-feed),
registers it in tenders.module.ts, and seeds the 'rss' poll config
active with pollGranularity='tick' plus a default-active service.bund.de
feed row (subreport-elvis has no single canonical URL — zero rows seeded,
admin adds relevant municipality feeds).
Migration applied locally per project convention (host -> container IP).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,41 @@
|
||||
import {
|
||||
IsBoolean,
|
||||
IsOptional,
|
||||
IsString,
|
||||
IsUrl,
|
||||
MaxLength,
|
||||
MinLength,
|
||||
} from 'class-validator';
|
||||
|
||||
/**
|
||||
* DTO for admin-managed RSS feed sources (`TenderRssFeedSource`, D-14/D-08).
|
||||
*
|
||||
* `@IsUrl` here is only a COARSE well-formedness check (http/https,
|
||||
* protocol required). The SUBSTANTIVE SSRF/denylist guard — rejecting
|
||||
* DENYLISTED_PORTALS hostnames and private/loopback hosts — is enforced in
|
||||
* `TenderRssFeedSourceService.assertUrlAllowed()`, NOT here (RESEARCH.md
|
||||
* Pitfall 3: an admin-supplied RSS feed URL is runtime data, added long
|
||||
* after `SourceRegistry.register()`'s DI-boot-time denylist check runs —
|
||||
* this DTO alone provides zero protection against a feed URL pointing at
|
||||
* vergabe24/aumass or an internal host). `require_tld: false` deliberately
|
||||
* lets IP-literal URLs pass THIS validation layer so the service-layer
|
||||
* check can reject them with a clear, domain-specific SSRF error message
|
||||
* instead of a generic "invalid URL" one.
|
||||
*/
|
||||
export class TenderRssFeedDto {
|
||||
@IsUrl({
|
||||
protocols: ['http', 'https'],
|
||||
require_protocol: true,
|
||||
require_tld: false,
|
||||
})
|
||||
url!: string;
|
||||
|
||||
@IsString()
|
||||
@MinLength(1)
|
||||
@MaxLength(200)
|
||||
label!: string;
|
||||
|
||||
@IsOptional()
|
||||
@IsBoolean()
|
||||
isActive?: boolean;
|
||||
}
|
||||
Reference in New Issue
Block a user