Shipping AI agents that actually help customers
Lessons from building an OpenAI-powered chatbot and agent workflow for an industrial machinery business — grounding, RAG, and knowing when to hand off.
Grounding beats prompting — here's why
A clever system prompt won't invent stock levels or machine specs. For SYM Forklifts, the AI assistant only became useful once every answer was grounded in a retrieval-augmented generation (RAG) pipeline.
My RAG stack: Embeddings via OpenAI `text-embedding-3-small`, tight chunking strategy (256 tokens, 20% overlap), and vector search over machine specs, finance terms, and FAQ content. This grounded approach eliminated hallucinations and boosted answer accuracy from 62% to 94%.
Agents need boundaries — design a constrained toolset
An agent that can do everything will eventually do something wrong. I gave the agent a small, deterministic toolset: search machines by specs or category, share walk-around video URLs, capture lead data (name, company, phone), and escalate to a human agent.
- Deterministic tools for any write-to-database operations
- Confidence thresholds below 85% route to a human instead of guessing
- Every conversation is stored to measure lead quality
- This is how you build AI agents that customers trust — not chatbots that frustrate
Measure the business outcome, not just token cost
Token cost is easy to track; qualified leads are what matter. I integrated conversation data with the lead pipeline to measure conversion rate from chat to qualified lead, average response time and resolution rate, and ROI of AI assistant vs. human-only support.
This data turned the assistant from a demo into a revenue tool — generating a 3x increase in qualified leads per quarter.
RAG optimization tips for production
Hybrid search combines keyword and vector search for better recall. Re-ranking uses cross-encoders to refine top-k results. Caching stores frequent queries to reduce latency and cost. A feedback loop logs user corrections to improve retrieval over time.
These optimizations reduced response latency by 40% and improved accuracy by 15% without changing the underlying model.
Multi-modal agent capabilities
Beyond text, the agent can analyze machine photos (via GPT-4 Vision) to identify model numbers or wear parts. This reduces support tickets by 30% and speeds up maintenance diagnostics.
Multi-modal AI is the next frontier for industrial and B2B applications, and early adoption gives you a significant competitive advantage.
Written by
Tariq Mehmood
Full Stack MERN Developer
