Files
tessera-ctl/.planning/phases/07-dkv-fleet-module/07-01-SUMMARY.md
T
schalli 6bf032ef94
Tessera CI/CD / Build & Deploy (push) Blocked by required conditions
Tessera CI/CD / Lint & Type Check (push) Successful in 39s
Tessera CI/CD / Tests (push) Waiting to run
docs(07-01): complete DKV backend foundation plan
- 07-01-SUMMARY.md: execution record with PDF parser analysis, regex deviation, DB push method
- STATE.md: plan counter advanced to 2/6, decisions added, session updated
2026-06-26 19:39:18 +02:00

125 lines
6.7 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
phase: 07-dkv-fleet-module
plan: 01
subsystem: dkv-backend-foundation
tags: [dkv, pdf-parse, prisma, nestjs, schedule, imapflow, xlsx]
dependency_graph:
requires: [CalendarModule, PrismaModule]
provides: [DkvParserService, DkvModuleConfig, DkvVehicleMaster, DkvInvoiceHistory, SmtpConfig, ScheduleModule, CalendarCryptoService (exported)]
affects:
- apps/api/package.json
- apps/api/prisma/schema.prisma
- apps/api/src/app.module.ts
- apps/api/src/calendar/calendar.module.ts
tech_stack:
added: [imapflow@1.4.3, pdf-parse@2.4.5, xlsx@0.18.5]
patterns:
- "pdf-parse v2 class API: new PDFParse({data:buffer}).getText().destroy()"
- "DKV PDF dual-format parser: tab-split (single-tx) + columnar transpose (multi-tx)"
- "German number parsing: replace(/\\./ g,'').replace(',','.')"
- "ScheduleModule.forRoot() for dynamic cron job support"
- "CalendarCryptoService re-exported for DKV credential encryption"
key_files:
created:
- apps/api/src/dkv/dkv.types.ts
- apps/api/src/dkv/dkv-parser.service.ts
- apps/api/src/dkv/dkv-parser.validate.ts
modified:
- apps/api/package.json
- apps/api/prisma/schema.prisma
- apps/api/src/app.module.ts
- apps/api/src/calendar/calendar.module.ts
- pnpm-lock.yaml
decisions:
- "DKV PDF uses two extraction formats: single-tx (tab-separated row) vs multi-tx (columnar transpose) — both handled"
- "KM and Produkt are merged into single tab field in single-tx format (no tab between them in PDF layout)"
- "Vehicle block regex: /VEHICLE:\\s+([A-Z0-9 ._\\-]+?)\\s+CARD NO\\.:\\s+(\\S+)/ reliably matches all 27 vehicles"
- "Multi-tx columnar: n = count of consecutive date-lines at block start; groups indexed as 0×n..n-1 per column"
- "DB push via docker exec tessera-ctl-api-1 (DB port not exposed to host)"
- "prisma generate runs successfully on host (postinstall script)"
metrics:
duration: 13min
completed: "2026-06-26T17:36:00Z"
tasks: 3
files_created: 3
files_modified: 5
---
# Phase 07 Plan 01: DKV Backend Foundation — Deps + Schema + PDF Parser Summary
DKV backend foundation established: three npm packages installed (imapflow, pdf-parse, xlsx), four Prisma tables pushed to live PostgreSQL, ScheduleModule registered in AppModule, CalendarCryptoService re-exported, and the highest-risk component (PDF parser) empirically validated against the real April 2026 DKV invoice — 27 vehicle blocks, 66 transactions extracted, DKV-02 parsing risk retired.
## Tasks Completed
| Task | Name | Commit | Key Files |
|------|------|--------|-----------|
| 1 | Package legitimacy gate | (human-verify, no commit) | — |
| 2 | Install deps, add Prisma models, wire ScheduleModule + crypto | c4b39cc | package.json, schema.prisma, app.module.ts, calendar.module.ts, dkv.types.ts |
| 3 | Push schema to live DB | (no code commit — DB op only) | Database: 4 new tables created |
| 4 | DKV PDF parser — validate + implement service | 6235aaf | dkv-parser.validate.ts, dkv-parser.service.ts |
## Verification Results
- `imapflow + pdf-parse + xlsx` loadable: PASS
- `grep -c "^model " apps/api/prisma/schema.prisma` → 15 (was 11, +4 new): PASS
- `ScheduleModule.forRoot()` in AppModule: PASS
- `CalendarCryptoService` in CalendarModule exports: PASS
- `prisma db push` (2nd run): "The database is already in sync" PASS
- `dkv-parser.validate.ts` exits 0: 27 vehicle blocks, 66 transactions PASS
- `pnpm --filter @tessera/api type-check` exits 0: PASS
## Deviations from Plan
### Auto-fixed Issues
**1. [Rule 1 - Bug] Research Pattern 4 regex replaced with dual-format tab parser**
- **Found during:** Task 4
- **Issue:** Research Pattern 4 used a space-based regex (`/(\d{2}\.\d{2}\.\d{4})\s+(.+?)\s{2,}([\d.,]+)\s+km\s+(\S+)\s+([\d.,]+)\s+(\w+)/g`) that assumed `km` suffix after the odometer value and whitespace-based column separation. Actual invoice.pdf uses tab-separated fields; there is no ` km ` text; single-tx and multi-tx formats differ structurally.
- **Fix:** Replaced with a dual-format dispatcher:
- **Single-TX:** Tab-split → `fields[5] = "KM PRODUKT"` (km and product merged in one tab cell), `fields[6] = "CODE UNIT"`, `fields[7] = menge`
- **Multi-TX (columnar):** Count leading date lines → n; extract groups at offsets 0×n, 2×n (orts), 5×n (kms), 6×n (products), 8×n (units), 9×n (quantities), 12×n (brutto), 13×n (netto)
- **Files modified:** apps/api/src/dkv/dkv-parser.validate.ts, apps/api/src/dkv/dkv-parser.service.ts
- **Commits:** 6235aaf
**2. [Rule 3 - Blocking] Temporary debug-pdf.ts removed before type-check**
- **Found during:** Task 4
- **Issue:** Debug script used top-level `await` which TypeScript rejects under the API's CommonJS module target.
- **Fix:** Deleted before committing (it was a scratch file, never staged).
## PDF Parser: Empirical Results vs Research Assumptions
| Research Assumption | Actual Format (invoice.pdf) | Resolution |
|---------------------|-----------------------------|------------|
| `{ data: buffer }` LoadParameters | CONFIRMED: pdf-parse v2 accepts `{ data: Buffer }` | Assumption A1 VERIFIED |
| VEHICLE: / CARD NO.: markers | CONFIRMED: exact strings present, all 27 vehicles matched | Assumption A2 PARTIALLY verified |
| Space/whitespace column separation | WRONG: tabs separate columns in single-tx format | Regex rewritten |
| `NNN.NNN km` suffix pattern | WRONG: no "km" text in extracted PDF; odometer is bare number | Regex rewritten |
| Single regex for all transactions | WRONG: two distinct formats (single-tx tabs vs multi-tx columnar) | Dual-format dispatcher |
**Actual vehicle count from invoice.pdf:** 27 (matches expected from CONTEXT.md)
**Actual transaction count:** 66 across 27 vehicles (varies 1–7 per vehicle)
**DB push method used:** `docker exec tessera-ctl-api-1 sh -c "cd /app/apps/api && npx prisma db push --skip-generate"` — same as Phase 05 pattern. Note: `prisma generate` ran on host (postinstall), but container generate failed due to permission error on `node_modules/.pnpm/prisma*/`. Host-side generation is sufficient for downstream type-check.
## Known Stubs
None — all functionality in this plan is fully implemented and empirically validated.
## Threat Flags
None. All STRIDE threats in the plan's threat register were mitigated:
- T-07-SC: Package legitimacy gate (Task 1 human-verify) cleared by user.
- T-07-01: `destroy()` called after every PDF extraction.
- T-07-02: Generic error messages only on parse failure — no PDF content in logs.
## Self-Check: PASSED
Files verified present:
- apps/api/src/dkv/dkv.types.ts ✓
- apps/api/src/dkv/dkv-parser.service.ts ✓
- apps/api/src/dkv/dkv-parser.validate.ts ✓
Commits verified in git log:
- c4b39cc ✓ (feat(07-01): install DKV deps...)
- 6235aaf ✓ (feat(07-01): DKV PDF parser...)