feat(10-04): implement TenderIngestionService — day-cursor gate, upsert change-detect, D-05 retention
- pollDueSources(): singleton doe-opendata config via findUnique (fixed slug,
not findFirst); day-cursor gate (nextDayToFetch) no-ops when nothing new
(Pitfall A); catch-up loop from lastIngestedDay+1 to today-1 with a polite
1.5s delay between successive day-fetches
- prisma.tender.upsert({ where: { dedupKey } }) — SCHEMA-02 change-detection
seam: identical notice does not duplicate, changed contentHash updates in
place
- pruneExpiredTenders(): marks active+past-deadline rows 'expired', deletes
expired rows older than 90 days, never touches deadlineAt=null rows (D-05)
- Plain PrismaService throughout — no tenant RLS extension on the global
Tender/TenderSourcePollConfig tables (D-03, T-10-09)
- Registered in TendersModule.providers
This commit is contained in:
@@ -4,16 +4,19 @@ import { ModuleRegistryService } from '../module-registry/module-registry.servic
|
||||
import { PrismaService } from '../prisma/prisma.service';
|
||||
import { DoeOpenDataAdapter } from './adapters/doe-opendata.adapter';
|
||||
import { seedTendersModule } from './tenders.seed';
|
||||
import { TenderIngestionService } from './tender-ingestion.service';
|
||||
import { TenderNormalizerService } from './tender-normalizer.service';
|
||||
|
||||
/**
|
||||
* NestJS module for the Ausschreibungs-Radar feature.
|
||||
*
|
||||
* Wave 2 registered the module in the marketplace and seeded the
|
||||
* singleton DÖE poll config so the shared poll is admin-drivable. This
|
||||
* plan (03) adds the DÖE source adapter + normalizer (parse+map core of
|
||||
* INGEST-01/SCHEMA-01). Ingestion orchestration/scheduler/controller
|
||||
* wiring is added in Plans 04-05.
|
||||
* singleton DÖE poll config so the shared poll is admin-drivable. Plan 03
|
||||
* added the DÖE source adapter + normalizer (parse+map core of
|
||||
* INGEST-01/SCHEMA-01). This plan (04) adds ingestion orchestration
|
||||
* (TenderIngestionService: day-cursor gate, SCHEMA-02 change detection,
|
||||
* D-05 retention) and the shared global scheduler. Controller/DTO wiring
|
||||
* is added in Plan 05.
|
||||
*
|
||||
* PrismaModule is global (no explicit import needed).
|
||||
*
|
||||
@@ -23,7 +26,7 @@ import { TenderNormalizerService } from './tender-normalizer.service';
|
||||
@Module({
|
||||
imports: [ModuleRegistryModule],
|
||||
controllers: [],
|
||||
providers: [DoeOpenDataAdapter, TenderNormalizerService],
|
||||
providers: [DoeOpenDataAdapter, TenderNormalizerService, TenderIngestionService],
|
||||
})
|
||||
export class TendersModule implements OnModuleInit {
|
||||
private readonly logger = new Logger(TendersModule.name);
|
||||
|
||||
Reference in New Issue
Block a user