docs(07-01): complete DKV backend foundation plan
- 07-01-SUMMARY.md: execution record with PDF parser analysis, regex deviation, DB push method - STATE.md: plan counter advanced to 2/6, decisions added, session updated
This commit is contained in:
+17
-14
@@ -2,15 +2,15 @@
|
|||||||
gsd_state_version: 1.0
|
gsd_state_version: 1.0
|
||||||
milestone: v1.0
|
milestone: v1.0
|
||||||
milestone_name: milestone
|
milestone_name: milestone
|
||||||
status: completed
|
status: executing
|
||||||
stopped_at: Phase 7 context gathered
|
stopped_at: Phase 07 Plan 01 complete
|
||||||
last_updated: "2026-06-26T12:22:38.551Z"
|
last_updated: "2026-06-26T17:36:00.000Z"
|
||||||
last_activity: 2026-06-25 -- Phase 06 complete (CI green, AppImage built)
|
last_activity: 2026-06-26 -- Phase 07 Plan 01 completed (DKV backend foundation)
|
||||||
progress:
|
progress:
|
||||||
total_phases: 7
|
total_phases: 7
|
||||||
completed_phases: 5
|
completed_phases: 5
|
||||||
total_plans: 24
|
total_plans: 30
|
||||||
completed_plans: 23
|
completed_plans: 24
|
||||||
percent: 71
|
percent: 71
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -21,14 +21,14 @@ progress:
|
|||||||
See: .planning/PROJECT.md (updated 2026-06-18)
|
See: .planning/PROJECT.md (updated 2026-06-18)
|
||||||
|
|
||||||
**Core value:** Eine zentrale Plattform, in der beliebige Workflow-Tools als Module lizenziert, aktiviert und genutzt werden koennen -- ohne zwischen verschiedenen Anwendungen wechseln zu muessen.
|
**Core value:** Eine zentrale Plattform, in der beliebige Workflow-Tools als Module lizenziert, aktiviert und genutzt werden koennen -- ohne zwischen verschiedenen Anwendungen wechseln zu muessen.
|
||||||
**Current focus:** Phase 06 — desktop-client-ci-cd
|
**Current focus:** Phase 07 — dkv-fleet-module
|
||||||
|
|
||||||
## Current Position
|
## Current Position
|
||||||
|
|
||||||
Phase: 06 (desktop-client-ci-cd) — COMPLETE
|
Phase: 07 (dkv-fleet-module) — EXECUTING
|
||||||
Plan: 3 of 3
|
Plan: 2 of 6
|
||||||
Status: All plans complete
|
Status: Executing Phase 07 (Plan 01 complete)
|
||||||
Last activity: 2026-06-25 -- Phase 06 complete (CI green, AppImage built)
|
Last activity: 2026-06-26 -- Phase 07 Plan 01 completed (DKV backend foundation)
|
||||||
|
|
||||||
Progress: [██████████] 100%
|
Progress: [██████████] 100%
|
||||||
|
|
||||||
@@ -96,6 +96,9 @@ Recent decisions affecting current work:
|
|||||||
- [Phase ?]: Plain docker compose build statt build-push-action (Gitea JWT Pitfall 4)
|
- [Phase ?]: Plain docker compose build statt build-push-action (Gitea JWT Pitfall 4)
|
||||||
- [Phase ?]: Ephemeral runner mode (GITEA_RUNNER_EPHEMERAL=1) fuer Credential-Revokation pro Job
|
- [Phase ?]: Ephemeral runner mode (GITEA_RUNNER_EPHEMERAL=1) fuer Credential-Revokation pro Job
|
||||||
- [Phase ?]: Separate docker-compose.ci.yml fuer opt-in CI-Infrastruktur
|
- [Phase ?]: Separate docker-compose.ci.yml fuer opt-in CI-Infrastruktur
|
||||||
|
- [07-01]: DKV PDF uses two extraction formats: single-tx (tab-separated) vs multi-tx (columnar) — both handled in DkvParserService
|
||||||
|
- [07-01]: Research Pattern 4 regex replaced with empirical dual-format tab/columnar parser after testing against real invoice.pdf
|
||||||
|
- [07-01]: CalendarCryptoService exported from CalendarModule for DKV credential encryption reuse
|
||||||
|
|
||||||
### Pending Todos
|
### Pending Todos
|
||||||
|
|
||||||
@@ -115,6 +118,6 @@ Items acknowledged and carried forward from previous milestone close:
|
|||||||
|
|
||||||
## Session Continuity
|
## Session Continuity
|
||||||
|
|
||||||
Last session: 2026-06-26T12:22:38.543Z
|
Last session: 2026-06-26T17:36:00.000Z
|
||||||
Stopped at: Phase 7 context gathered
|
Stopped at: Phase 07 Plan 01 complete (DKV backend foundation)
|
||||||
Resume file: .planning/phases/07-dkv-fleet-module/07-CONTEXT.md
|
Resume file: .planning/phases/07-dkv-fleet-module/07-02-PLAN.md
|
||||||
|
|||||||
@@ -0,0 +1,124 @@
|
|||||||
|
---
|
||||||
|
phase: 07-dkv-fleet-module
|
||||||
|
plan: 01
|
||||||
|
subsystem: dkv-backend-foundation
|
||||||
|
tags: [dkv, pdf-parse, prisma, nestjs, schedule, imapflow, xlsx]
|
||||||
|
dependency_graph:
|
||||||
|
requires: [CalendarModule, PrismaModule]
|
||||||
|
provides: [DkvParserService, DkvModuleConfig, DkvVehicleMaster, DkvInvoiceHistory, SmtpConfig, ScheduleModule, CalendarCryptoService (exported)]
|
||||||
|
affects:
|
||||||
|
- apps/api/package.json
|
||||||
|
- apps/api/prisma/schema.prisma
|
||||||
|
- apps/api/src/app.module.ts
|
||||||
|
- apps/api/src/calendar/calendar.module.ts
|
||||||
|
tech_stack:
|
||||||
|
added: [imapflow@1.4.3, pdf-parse@2.4.5, xlsx@0.18.5]
|
||||||
|
patterns:
|
||||||
|
- "pdf-parse v2 class API: new PDFParse({data:buffer}).getText().destroy()"
|
||||||
|
- "DKV PDF dual-format parser: tab-split (single-tx) + columnar transpose (multi-tx)"
|
||||||
|
- "German number parsing: replace(/\\./ g,'').replace(',','.')"
|
||||||
|
- "ScheduleModule.forRoot() for dynamic cron job support"
|
||||||
|
- "CalendarCryptoService re-exported for DKV credential encryption"
|
||||||
|
key_files:
|
||||||
|
created:
|
||||||
|
- apps/api/src/dkv/dkv.types.ts
|
||||||
|
- apps/api/src/dkv/dkv-parser.service.ts
|
||||||
|
- apps/api/src/dkv/dkv-parser.validate.ts
|
||||||
|
modified:
|
||||||
|
- apps/api/package.json
|
||||||
|
- apps/api/prisma/schema.prisma
|
||||||
|
- apps/api/src/app.module.ts
|
||||||
|
- apps/api/src/calendar/calendar.module.ts
|
||||||
|
- pnpm-lock.yaml
|
||||||
|
decisions:
|
||||||
|
- "DKV PDF uses two extraction formats: single-tx (tab-separated row) vs multi-tx (columnar transpose) — both handled"
|
||||||
|
- "KM and Produkt are merged into single tab field in single-tx format (no tab between them in PDF layout)"
|
||||||
|
- "Vehicle block regex: /VEHICLE:\\s+([A-Z0-9 ._\\-]+?)\\s+CARD NO\\.:\\s+(\\S+)/ reliably matches all 27 vehicles"
|
||||||
|
- "Multi-tx columnar: n = count of consecutive date-lines at block start; groups indexed as 0×n..n-1 per column"
|
||||||
|
- "DB push via docker exec tessera-ctl-api-1 (DB port not exposed to host)"
|
||||||
|
- "prisma generate runs successfully on host (postinstall script)"
|
||||||
|
metrics:
|
||||||
|
duration: 13min
|
||||||
|
completed: "2026-06-26T17:36:00Z"
|
||||||
|
tasks: 3
|
||||||
|
files_created: 3
|
||||||
|
files_modified: 5
|
||||||
|
---
|
||||||
|
|
||||||
|
# Phase 07 Plan 01: DKV Backend Foundation — Deps + Schema + PDF Parser Summary
|
||||||
|
|
||||||
|
DKV backend foundation established: three npm packages installed (imapflow, pdf-parse, xlsx), four Prisma tables pushed to live PostgreSQL, ScheduleModule registered in AppModule, CalendarCryptoService re-exported, and the highest-risk component (PDF parser) empirically validated against the real April 2026 DKV invoice — 27 vehicle blocks, 66 transactions extracted, DKV-02 parsing risk retired.
|
||||||
|
|
||||||
|
## Tasks Completed
|
||||||
|
|
||||||
|
| Task | Name | Commit | Key Files |
|
||||||
|
|------|------|--------|-----------|
|
||||||
|
| 1 | Package legitimacy gate | (human-verify, no commit) | — |
|
||||||
|
| 2 | Install deps, add Prisma models, wire ScheduleModule + crypto | c4b39cc | package.json, schema.prisma, app.module.ts, calendar.module.ts, dkv.types.ts |
|
||||||
|
| 3 | Push schema to live DB | (no code commit — DB op only) | Database: 4 new tables created |
|
||||||
|
| 4 | DKV PDF parser — validate + implement service | 6235aaf | dkv-parser.validate.ts, dkv-parser.service.ts |
|
||||||
|
|
||||||
|
## Verification Results
|
||||||
|
|
||||||
|
- `imapflow + pdf-parse + xlsx` loadable: PASS
|
||||||
|
- `grep -c "^model " apps/api/prisma/schema.prisma` → 15 (was 11, +4 new): PASS
|
||||||
|
- `ScheduleModule.forRoot()` in AppModule: PASS
|
||||||
|
- `CalendarCryptoService` in CalendarModule exports: PASS
|
||||||
|
- `prisma db push` (2nd run): "The database is already in sync" PASS
|
||||||
|
- `dkv-parser.validate.ts` exits 0: 27 vehicle blocks, 66 transactions PASS
|
||||||
|
- `pnpm --filter @tessera/api type-check` exits 0: PASS
|
||||||
|
|
||||||
|
## Deviations from Plan
|
||||||
|
|
||||||
|
### Auto-fixed Issues
|
||||||
|
|
||||||
|
**1. [Rule 1 - Bug] Research Pattern 4 regex replaced with dual-format tab parser**
|
||||||
|
- **Found during:** Task 4
|
||||||
|
- **Issue:** Research Pattern 4 used a space-based regex (`/(\d{2}\.\d{2}\.\d{4})\s+(.+?)\s{2,}([\d.,]+)\s+km\s+(\S+)\s+([\d.,]+)\s+(\w+)/g`) that assumed `km` suffix after the odometer value and whitespace-based column separation. Actual invoice.pdf uses tab-separated fields; there is no ` km ` text; single-tx and multi-tx formats differ structurally.
|
||||||
|
- **Fix:** Replaced with a dual-format dispatcher:
|
||||||
|
- **Single-TX:** Tab-split → `fields[5] = "KM PRODUKT"` (km and product merged in one tab cell), `fields[6] = "CODE UNIT"`, `fields[7] = menge`
|
||||||
|
- **Multi-TX (columnar):** Count leading date lines → n; extract groups at offsets 0×n, 2×n (orts), 5×n (kms), 6×n (products), 8×n (units), 9×n (quantities), 12×n (brutto), 13×n (netto)
|
||||||
|
- **Files modified:** apps/api/src/dkv/dkv-parser.validate.ts, apps/api/src/dkv/dkv-parser.service.ts
|
||||||
|
- **Commits:** 6235aaf
|
||||||
|
|
||||||
|
**2. [Rule 3 - Blocking] Temporary debug-pdf.ts removed before type-check**
|
||||||
|
- **Found during:** Task 4
|
||||||
|
- **Issue:** Debug script used top-level `await` which TypeScript rejects under the API's CommonJS module target.
|
||||||
|
- **Fix:** Deleted before committing (it was a scratch file, never staged).
|
||||||
|
|
||||||
|
## PDF Parser: Empirical Results vs Research Assumptions
|
||||||
|
|
||||||
|
| Research Assumption | Actual Format (invoice.pdf) | Resolution |
|
||||||
|
|---------------------|-----------------------------|------------|
|
||||||
|
| `{ data: buffer }` LoadParameters | CONFIRMED: pdf-parse v2 accepts `{ data: Buffer }` | Assumption A1 VERIFIED |
|
||||||
|
| VEHICLE: / CARD NO.: markers | CONFIRMED: exact strings present, all 27 vehicles matched | Assumption A2 PARTIALLY verified |
|
||||||
|
| Space/whitespace column separation | WRONG: tabs separate columns in single-tx format | Regex rewritten |
|
||||||
|
| `NNN.NNN km` suffix pattern | WRONG: no "km" text in extracted PDF; odometer is bare number | Regex rewritten |
|
||||||
|
| Single regex for all transactions | WRONG: two distinct formats (single-tx tabs vs multi-tx columnar) | Dual-format dispatcher |
|
||||||
|
|
||||||
|
**Actual vehicle count from invoice.pdf:** 27 (matches expected from CONTEXT.md)
|
||||||
|
**Actual transaction count:** 66 across 27 vehicles (varies 1–7 per vehicle)
|
||||||
|
|
||||||
|
**DB push method used:** `docker exec tessera-ctl-api-1 sh -c "cd /app/apps/api && npx prisma db push --skip-generate"` — same as Phase 05 pattern. Note: `prisma generate` ran on host (postinstall), but container generate failed due to permission error on `node_modules/.pnpm/prisma*/`. Host-side generation is sufficient for downstream type-check.
|
||||||
|
|
||||||
|
## Known Stubs
|
||||||
|
|
||||||
|
None — all functionality in this plan is fully implemented and empirically validated.
|
||||||
|
|
||||||
|
## Threat Flags
|
||||||
|
|
||||||
|
None. All STRIDE threats in the plan's threat register were mitigated:
|
||||||
|
- T-07-SC: Package legitimacy gate (Task 1 human-verify) cleared by user.
|
||||||
|
- T-07-01: `destroy()` called after every PDF extraction.
|
||||||
|
- T-07-02: Generic error messages only on parse failure — no PDF content in logs.
|
||||||
|
|
||||||
|
## Self-Check: PASSED
|
||||||
|
|
||||||
|
Files verified present:
|
||||||
|
- apps/api/src/dkv/dkv.types.ts ✓
|
||||||
|
- apps/api/src/dkv/dkv-parser.service.ts ✓
|
||||||
|
- apps/api/src/dkv/dkv-parser.validate.ts ✓
|
||||||
|
|
||||||
|
Commits verified in git log:
|
||||||
|
- c4b39cc ✓ (feat(07-01): install DKV deps...)
|
||||||
|
- 6235aaf ✓ (feat(07-01): DKV PDF parser...)
|
||||||
Reference in New Issue
Block a user