fix(tenders): link DOE notices to the notice page, not the API
The doe-opendata adapter stored the OCDS document's own `uri` as sourceUrl. That is the API address of the record and serves OCDS JSON by design, so anyone following the link from the results list, the detail view, or an alert mail landed on raw JSON instead of the notice. Build the human-readable page from the notice id the row already carries instead. `/ui/de/search/details?noticeId=...` is the redirect target of `/ui/de/notices/...`, so it needs no redirect. Verified in a browser for both id shapes the feed uses -- numeric (25673764 -> "Feuerwehr-Geraetehaus Miehlen Fliesenarbeiten") and UUID (7085ba12-... -> "Holzfassade"). The page is a single-page app that answers 200 with an identical shell for any id, so this had to be checked on rendered content; a status code proves nothing. The adapter alone only fixes new ingests, so a backfill migration rewrites the rows already stored -- in Tender and in TenderSource, since the detail view lists per-source links separately. It touches only rows still pointing at /api/notices/ and only ids of a shape that was actually verified, which makes it idempotent and keeps an unexpected id from being pasted into a URL. Counted read-only against the live database beforehand: 2846 DOE rows affected, none skipped. Closes the 2026-08-05 backlog item, which was deliberately held until Phase 16 was done. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,43 @@
|
|||||||
|
-- Backfill for the DÖE notice links.
|
||||||
|
--
|
||||||
|
-- Until 2026-08-11 the doe-opendata adapter stored the OCDS document's own
|
||||||
|
-- `uri` as Tender.sourceUrl. That is the API address of the record: it serves
|
||||||
|
-- OCDS JSON by design, so every user who followed the link from the results
|
||||||
|
-- list, the detail view, or an alert mail landed on raw JSON instead of the
|
||||||
|
-- notice. The adapter now builds the human-readable notice page instead, but
|
||||||
|
-- rows already ingested keep the broken URL until this migration rewrites
|
||||||
|
-- them.
|
||||||
|
--
|
||||||
|
-- The id inside the API URL is the same value the row already carries in
|
||||||
|
-- sourceNoticeId (verified against the live feed on 2026-08-11: for a single
|
||||||
|
-- notice, `uri`'s id and `releases[0].id` are identical). The page address is
|
||||||
|
-- therefore derivable from stored data alone -- no refetch, no user input
|
||||||
|
-- interpolated.
|
||||||
|
--
|
||||||
|
-- Two shapes of notice id occur in the feed and both were verified to render
|
||||||
|
-- the correct notice in a browser: numeric ("25673764") and UUID
|
||||||
|
-- ("7085ba12-c7f4-4f8c-8599-2fe9e4e2737c"). The WHERE clause restricts the
|
||||||
|
-- rewrite to ids made of those characters, so a row with an unexpected id
|
||||||
|
-- shape keeps its old value rather than getting a URL built from something
|
||||||
|
-- that was never checked.
|
||||||
|
--
|
||||||
|
-- Idempotent: only rows still pointing at /api/notices/ are touched. Running
|
||||||
|
-- it twice changes nothing the second time.
|
||||||
|
|
||||||
|
UPDATE "Tender"
|
||||||
|
SET "sourceUrl" =
|
||||||
|
'https://oeffentlichevergabe.de/ui/de/search/details?noticeId=' || "sourceNoticeId"
|
||||||
|
WHERE "sourcePortal" = 'doe-opendata'
|
||||||
|
AND "sourceUrl" LIKE 'https://oeffentlichevergabe.de/api/notices/%'
|
||||||
|
AND "sourceNoticeId" ~ '^[A-Za-z0-9-]+$';
|
||||||
|
|
||||||
|
-- Same rewrite for the per-source rows behind a deduplicated tender. The
|
||||||
|
-- detail view lists these separately (tenders.controller.ts), so leaving them
|
||||||
|
-- untouched would keep broken links visible even after the primary column is
|
||||||
|
-- fixed.
|
||||||
|
UPDATE "TenderSource"
|
||||||
|
SET "sourceUrl" =
|
||||||
|
'https://oeffentlichevergabe.de/ui/de/search/details?noticeId=' || "sourceNoticeId"
|
||||||
|
WHERE "sourcePortal" = 'doe-opendata'
|
||||||
|
AND "sourceUrl" LIKE 'https://oeffentlichevergabe.de/api/notices/%'
|
||||||
|
AND "sourceNoticeId" ~ '^[A-Za-z0-9-]+$';
|
||||||
@@ -2,7 +2,7 @@ import AdmZip from 'adm-zip';
|
|||||||
import { readFileSync } from 'fs';
|
import { readFileSync } from 'fs';
|
||||||
import { join } from 'path';
|
import { join } from 'path';
|
||||||
import { afterEach, describe, expect, it, vi } from 'vitest';
|
import { afterEach, describe, expect, it, vi } from 'vitest';
|
||||||
import { DoeOpenDataAdapter } from './doe-opendata.adapter';
|
import { buildDoeNoticeUrl, DoeOpenDataAdapter } from './doe-opendata.adapter';
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Real, trimmed DÖE day-export fixtures (8 notices, captured live from
|
* Real, trimmed DÖE day-export fixtures (8 notices, captured live from
|
||||||
@@ -129,6 +129,42 @@ describe('DoeOpenDataAdapter', () => {
|
|||||||
await expect(adapter.fetchTenders(TEST_PUBDAY)).rejects.toThrow();
|
await expect(adapter.fetchTenders(TEST_PUBDAY)).rejects.toThrow();
|
||||||
});
|
});
|
||||||
|
|
||||||
|
// Backlog item 2026-08-05: sourceUrl used to be the OCDS document's own
|
||||||
|
// `uri`, which is the API address and answers with JSON. Users following a
|
||||||
|
// link from the results list, the detail view or an alert mail landed on raw
|
||||||
|
// JSON instead of the notice.
|
||||||
|
it('points sourceUrl at the human-readable notice page, never at the API', async () => {
|
||||||
|
stubFetchWithFixtures();
|
||||||
|
const adapter = new DoeOpenDataAdapter();
|
||||||
|
|
||||||
|
const records = await adapter.fetchTenders(TEST_PUBDAY);
|
||||||
|
|
||||||
|
expect(records.length).toBe(EXPECTED_TENDER_TAGGED_COUNT);
|
||||||
|
for (const record of records) {
|
||||||
|
expect(record.sourceUrl).toBe(
|
||||||
|
`https://oeffentlichevergabe.de/ui/de/search/details?noticeId=${record.sourceNoticeId}`,
|
||||||
|
);
|
||||||
|
// The exact shape that was broken — an API address serving OCDS JSON.
|
||||||
|
expect(record.sourceUrl).not.toContain('/api/notices/');
|
||||||
|
expect(record.sourceUrl).not.toContain('format=ocds');
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it('builds the notice URL from both id shapes the feed uses, escaping the id', () => {
|
||||||
|
// Numeric and UUID ids both occur in the live feed; both were verified in
|
||||||
|
// a browser on 2026-08-11 to render the correct notice.
|
||||||
|
expect(buildDoeNoticeUrl('25673764')).toBe(
|
||||||
|
'https://oeffentlichevergabe.de/ui/de/search/details?noticeId=25673764',
|
||||||
|
);
|
||||||
|
expect(buildDoeNoticeUrl('7085ba12-c7f4-4f8c-8599-2fe9e4e2737c')).toBe(
|
||||||
|
'https://oeffentlichevergabe.de/ui/de/search/details?noticeId=7085ba12-c7f4-4f8c-8599-2fe9e4e2737c',
|
||||||
|
);
|
||||||
|
// An id is directory data, not something to paste into a URL unchecked.
|
||||||
|
expect(buildDoeNoticeUrl('a b&c=d')).toBe(
|
||||||
|
'https://oeffentlichevergabe.de/ui/de/search/details?noticeId=a%20b%26c%3Dd',
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
it('never imports or uses axios (native fetch is the sole HTTP client convention)', () => {
|
it('never imports or uses axios (native fetch is the sole HTTP client convention)', () => {
|
||||||
const source = readFileSync(
|
const source = readFileSync(
|
||||||
join(__dirname, 'doe-opendata.adapter.ts'),
|
join(__dirname, 'doe-opendata.adapter.ts'),
|
||||||
|
|||||||
@@ -43,6 +43,27 @@ interface OcdsDocument {
|
|||||||
releases?: OcdsRelease[];
|
releases?: OcdsRelease[];
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Human-readable notice page for a DÖE notice id.
|
||||||
|
*
|
||||||
|
* The OCDS document's own `uri` field is the API address of the record — it
|
||||||
|
* serves OCDS JSON by design, so a user who follows it lands on raw JSON
|
||||||
|
* instead of the notice. The id in that URL is the same `releases[0].id` the
|
||||||
|
* adapter already stores as sourceNoticeId, so the page can be addressed
|
||||||
|
* without a second lookup.
|
||||||
|
*
|
||||||
|
* `/ui/de/search/details?noticeId=…` is the redirect target of
|
||||||
|
* `/ui/de/notices/…` and therefore the form that needs no redirect. Verified
|
||||||
|
* in a browser on 2026-08-11 for both id shapes that occur in the feed: the
|
||||||
|
* numeric one (25673764 -> "Feuerwehr-Gerätehaus Miehlen Fliesenarbeiten")
|
||||||
|
* and the UUID one (7085ba12-… -> "Holzfassade"). Note that the page is a
|
||||||
|
* single-page app: it answers HTTP 200 with an identical shell for ANY id,
|
||||||
|
* so a status-code check proves nothing here — only rendered content does.
|
||||||
|
*/
|
||||||
|
export function buildDoeNoticeUrl(noticeId: string): string {
|
||||||
|
return `https://oeffentlichevergabe.de/ui/de/search/details?noticeId=${encodeURIComponent(noticeId)}`;
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* D-02 open-tender filter (RESEARCH.md Pattern 2 / Pitfall C): positive
|
* D-02 open-tender filter (RESEARCH.md Pattern 2 / Pitfall C): positive
|
||||||
* tag-presence match only. Missing/other tags (award, planning, or no tag
|
* tag-presence match only. Missing/other tags (award, planning, or no tag
|
||||||
@@ -124,7 +145,8 @@ export class DoeOpenDataAdapter implements TenderSourceAdapter {
|
|||||||
sourcePortal: 'doe-opendata',
|
sourcePortal: 'doe-opendata',
|
||||||
sourceNoticeId: release.id,
|
sourceNoticeId: release.id,
|
||||||
ocid: release.ocid,
|
ocid: release.ocid,
|
||||||
sourceUrl: ocdsDocument.uri,
|
// NOT ocdsDocument.uri — that is the API address and serves JSON.
|
||||||
|
sourceUrl: buildDoeNoticeUrl(release.id),
|
||||||
fetchedAt,
|
fetchedAt,
|
||||||
publishedAt: ocdsDocument.publishedDate
|
publishedAt: ocdsDocument.publishedDate
|
||||||
? new Date(ocdsDocument.publishedDate)
|
? new Date(ocdsDocument.publishedDate)
|
||||||
|
|||||||
@@ -0,0 +1,68 @@
|
|||||||
|
import { readdirSync, readFileSync } from 'node:fs';
|
||||||
|
import { join } from 'node:path';
|
||||||
|
import { describe, expect, it } from 'vitest';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Prüft das hand-geschriebene Backfill-SQL für die DÖE-Bekanntmachungslinks
|
||||||
|
* ohne Datenbank — reiner Textabgleich, gleiches Muster wie
|
||||||
|
* groups/migration-sql.spec.ts.
|
||||||
|
*
|
||||||
|
* Der Adapter allein repariert nur neue Importe. Ohne den Nachlauf behalten
|
||||||
|
* die bereits importierten Ausschreibungen ihre kaputte API-URL, und genau
|
||||||
|
* die sieht der Nutzer heute in Trefferliste, Detailansicht und Alarm-Mail.
|
||||||
|
*/
|
||||||
|
|
||||||
|
const MIGRATIONS_DIR = join(__dirname, '../../prisma/migrations');
|
||||||
|
|
||||||
|
function readMigrationSql(suffix: string): string {
|
||||||
|
const dirs = readdirSync(MIGRATIONS_DIR, { withFileTypes: true })
|
||||||
|
.filter((entry) => entry.isDirectory() && entry.name.endsWith(suffix))
|
||||||
|
.map((entry) => entry.name);
|
||||||
|
|
||||||
|
if (dirs.length !== 1) {
|
||||||
|
throw new Error(
|
||||||
|
`Expected exactly one migration directory ending in "${suffix}", found ${dirs.length}: ${dirs.join(', ')}`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
return readFileSync(join(MIGRATIONS_DIR, dirs[0], 'migration.sql'), 'utf-8');
|
||||||
|
}
|
||||||
|
|
||||||
|
describe('doe_notice_url_backfill migration.sql', () => {
|
||||||
|
const sql = readMigrationSql('_doe_notice_url_backfill');
|
||||||
|
|
||||||
|
it('schreibt die Bekanntmachungsseite, nicht die API-Adresse', () => {
|
||||||
|
expect(sql).toContain(
|
||||||
|
"'https://oeffentlichevergabe.de/ui/de/search/details?noticeId=' || \"sourceNoticeId\"",
|
||||||
|
);
|
||||||
|
expect(sql).not.toMatch(/SET\s+"sourceUrl"\s*=\s*'[^']*\/api\/notices\//);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('fasst beide Tabellen an — sonst bleiben die Quell-Links der zusammengefuehrten Tender kaputt', () => {
|
||||||
|
expect(sql).toMatch(/UPDATE\s+"Tender"/);
|
||||||
|
expect(sql).toMatch(/UPDATE\s+"TenderSource"/);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('ist idempotent: nur Zeilen, die noch auf die API zeigen', () => {
|
||||||
|
const updates = sql.match(/UPDATE\s+"[A-Za-z]+"[\s\S]*?;/g) ?? [];
|
||||||
|
expect(updates.length).toBe(2);
|
||||||
|
for (const stmt of updates) {
|
||||||
|
expect(stmt).toContain(
|
||||||
|
`"sourceUrl" LIKE 'https://oeffentlichevergabe.de/api/notices/%'`,
|
||||||
|
);
|
||||||
|
expect(stmt).toContain(`"sourcePortal" = 'doe-opendata'`);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it('baut keine URL aus einer Kennung unerwarteter Form', () => {
|
||||||
|
const updates = sql.match(/UPDATE\s+"[A-Za-z]+"[\s\S]*?;/g) ?? [];
|
||||||
|
for (const stmt of updates) {
|
||||||
|
expect(stmt).toContain(`"sourceNoticeId" ~ '^[A-Za-z0-9-]+$'`);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it('fasst ausschliesslich DÖE-Zeilen an — andere Portale behalten ihre Links', () => {
|
||||||
|
expect(sql).not.toMatch(/UPDATE[\s\S]*?service-bund/);
|
||||||
|
expect(sql).not.toMatch(/UPDATE[\s\S]*?cosinex/);
|
||||||
|
});
|
||||||
|
});
|
||||||
Reference in New Issue
Block a user