Engagement Plan (Week-by-Week)
Week 0 is discovery: access, data audit, and risk inventory.
Week 1 builds instrumentation and a baseline eval set.
Week 2 ships a prototype with retrieval + structured output.
Week 3 runs a pilot with guardrails and audit logging.
Week 4 hardens the system and delivers runbooks.
Week 0 — Discovery + Access
- Stakeholder map and workflow walkthroughs.
- Data inventory: notes, job events, equipment, manuals, attachments.
- Threat model draft and trust boundary diagram.
Mitigation
Run a sampling audit of 50 recent jobs to quantify evidence coverage and identify gaps.
Week 1 — Instrumentation + Baseline
- RAG pipeline skeleton with structured context builder.
- Initial eval set (golden questions) with scoring rubric.
- Logging schema for citations, abstentions, and latency.
Tradeoff
Prioritize low-latency structured context for first responses, then enrich with retrieval.
Week 2 — Prototype + Eval Harness
- Working copilot endpoint with citations and JSON output.
- Automatic regression tests for retrieval and response schema.
- First reliability dashboard.
Mitigation
Introduce hybrid retrieval (vector + keyword) and limit to job-scoped filters.
Week 3 — Pilot + Guardrails
- Limited pilot with a small technician cohort.
- Refusal policies for high-risk requests.
- Audit trail per answer (context + evidence + output).
Risk
Any answer without citations must be an abstain, not a best guess.
Week 4 — Hardening + Runbooks
- Incident response playbooks.
- Metrics reviews (latency, safety, adoption).
- Security review and tenancy boundary verification.
Mitigation
Ship runbooks alongside monitoring dashboards and alert routing.