How to Build a Production AI Sales Agent System (Step-by-Step)

1. Start from the business outcome

Define "success" in numbers before writing prompts. Typical targets: time-to-first-reply for inbound visitors, % of sessions that become qualified leads, cost per session, and human handoff rate. If you skip this, you will optimize for clever dialogue instead of pipeline—which is how teams burn API budgets without moving revenue metrics.

Write down the minimum data you need on a lead: company size, use case, budget band, and consent for follow-up. If the agent cannot collect those fields reliably, the downstream CRM or sales motion stays broken.

2. Use multiple specialized agents (or nodes)

Split by responsibility: greeting, discovery, expertise, objection handling, portfolio proof, push to calendar, and safety / moderation. In graph-based frameworks (e.g. LangGraph), these map cleanly to nodes with explicit edges, so you can test each path with fixtures instead of re-running a giant prompt every time.

3. Add RAG for grounded answers

Retrieval augments the model with your FAQs, case studies, pricing rules, and product boundaries. Design chunking and metadata (source URL, product line, "internal only" flags) as carefully as the embedding model—bad chunks produce confident nonsense.

In a production build, expect hundreds of chunks once you add portfolio projects, help articles, and structured snippets. The Ramy case study used on the order of 196 vector chunks in Qdrant; your scale may differ, but the operational pattern is the same: refresh, diff, and measure hallucination rate on a fixed test set of questions.

4. Route models by difficulty (cost control)

A practical pipeline is: rules / regex for no-LLM paths → small / cheap model for "easy" turns → flagship model for negotiation, long context, or high-revenue risk. Log which tier handled every turn. That is how you get large savings without trashing user-visible quality.

Summarize or trim thread history before expensive calls. If you pass 20k tokens to a flagship model on every turn, you will not need competitors to beat you on unit economics—your own bill will.

5. Classify intent and score leads

Map user utterances to intents and a conversation phase (e.g. discovery → evaluation → conversion). Combine lightweight classifiers and LLM for edge cases. Downstream, emit a structured object that your CRM, email, or Slack expects—not a free-form chat transcript.

6. Ship operator tools

Production means prompts change, knowledge updates, and cost monitoring. At minimum, plan for versioned prompts, basic analytics, and conversation replay. Without replay, you cannot answer "why did it say that?" in an audit, sales dispute, or bug report.

7. Deploy like any critical service

Health checks, structured logging, rate limits, secrets management, and staged rollouts. If your agent shares an API with the rest of the product, it needs the same SLO thinking—not a one-off serverless function with a huge timeout.

8. Code sketch — LangGraph-style node (Python)

You will not copy-paste this into production, but the shape is what reviewers look for: explicit state in, state out, a single clear side-effect surface (e.g. tool calls, not hidden globals).

def discovery_node(state: GraphState) -> GraphState:
    user_msg = state["messages"][-1].content
    if looks_like_pricing_intent(user_msg):
        return {**state, "phase": "evaluation", "route": "pricing_agent"}
    retrieved = retriever.query(user_msg, k=4, filter={"audience": "prospect"})
    reply = small_model.generate(SYSTEM + format_chunks(retrieved) + user_msg)
    return {**state, "messages": state["messages"] + [("assistant", reply)]}

9. Chatbot vs agent system (quick comparison)

10. Mistakes to avoid

• One system prompt to rule them all (un-testable, expensive, brittle).
• No negative examples in safety—so the model over-promises SLAs, pricing, or features.
• Skipping human-readable logs for what was retrieved vs what the model was told.
• Launching without a frozen test set of 50+ real visitor questions and expected behaviors.

11. Pre-launch checklist (condensed)

• Test suite of prompts with expected tools / phases.
• PII and prompt-injection playbooks, including off-topic jailbreaks.
• Budget per session at expected traffic, with a kill switch on spend.
• Replay and export for a compliance or sales review.
• On-call: who is paged if the agent errors above X% in 10 minutes.