AI/ML Practices¶

🎯 We treat AI components like any other production system: evaluated, monitored, and revisited — not deployed and forgotten.

Model & Approach Selection¶

Question	Why it matters
Does this need an LLM, or does a simpler rule/heuristic solve it?	Avoid unnecessary complexity and cost
Fine-tune vs. RAG vs. prompting?	Match approach to data volume, update frequency, and latency needs
What's the fallback if the model is wrong or unavailable?	Production systems need a degraded-but-safe path

Define success metrics before building (accuracy, latency, cost per call, human-review rate).
Test against a representative sample of real (or realistic) data, not just happy-path examples.
Document known failure modes and how the system handles them.

Client data usage and retention terms are clear and agreed
No client data used to train or fine-tune models for other clients
PII handling reviewed (masking, minimal retention, access control)
Model outputs have a human-in-the-loop review step where decisions are high-stakes
Monitoring in place for drift, cost spikes, and failure rate after launch