An owner deployed an AI voice agent for his HVAC business with the default template. The first week, customers kept hearing "I am not sure about that — let me have someone call you back" for questions like "do you service ductless mini-splits" and "is the diagnostic fee waived if I proceed." The AI was working — it just did not know enough about his specific business. Two hours of focused knowledge-base work later, the "I don't know" rate dropped from 18% to under 4%, and the booking rate rose 40%.
This is the gap most AI voice agent deployments fail to close: the AI is technically configured and live, but it has not been trained on the specific knowledge of your specific business. This guide walks through how to train your AI phone agent on your FAQ and knowledge base properly — from document structure to common mistakes to the iterative refinement loop.
What an AI Phone Agent's "Knowledge Base" Actually Is
Modern AI voice agents use Retrieval-Augmented Generation (RAG). At a high level: when a caller asks a question, the AI searches your uploaded documents for relevant content, retrieves the most relevant chunks, and uses them to generate a response. The quality of the response is bounded by:
- Whether the answer exists in your documents at all.
- Whether it is structured in a way the retrieval system can find.
- Whether the response prompt knows when to use the knowledge vs. when to escalate.
The Three Layers of Knowledge to Train
Layer 1: Structured FAQ (highest priority)
The 30–80 questions your business answers over and over. Each entry is a clean Q&A pair:
- Question: Phrased the way customers actually ask it. Use multiple variations if customers ask the same thing differently.
- Answer: 1–3 sentences, specific, in your brand voice.
Example for a plumbing shop:
Q: Do you service tankless water heaters? / Q: Can you work on my Rinnai? / Q: Do you do tankless?
A: Yes — we service and install tankless water heaters from all major brands including Rinnai, Navien, and Noritz. A diagnostic visit is $89 (waived if you proceed); a typical service call runs $180–$485 depending on the issue.
Layer 2: Service / product / pricing reference
The "spec sheet" of your business. Services you offer, pricing ranges, what is included, what is not. This is denser than FAQ and used when callers ask specific questions about offerings.
Format as structured tables or clear bullet lists, not free-flowing prose. The retrieval system finds structured data more reliably.
Layer 3: Long-form policy / process documents
Less commonly retrieved but useful for edge cases. Includes things like warranty terms, refund policies, regulatory disclosures, complex service-area maps, multi-step processes.
Keep these separated from the FAQ. The AI uses them when relevant and ignores them otherwise.
The Right Format for Each Layer
For FAQ
Use a simple Q-A markdown or text file:
Q: What are your hours?
A: We are open Monday-Friday 7 AM to 6 PM, Saturday 8 AM to 4 PM. We have an after-hours emergency line for true emergencies (no heat, no water, gas smell).
Q: Do you take credit cards?
A: Yes, all major credit cards plus financing through Synchrony for jobs over $1,000.Most platforms accept either uploaded files (CSV, JSON, MD) or direct dashboard entry. Either works.
For services / pricing
A simple table or structured list:
| Service | Price Range | Notes |
|---------|------------|-------|
| Diagnostic | $89 | Waived if you proceed |
| Drain cleaning | $180-$385 | Depends on length and access |
| Water heater install (40-gal gas) | $1,400-$2,200 | Includes haul-away |For policy / process
Section headers + bulletized content. Avoid 1,000-word prose paragraphs — chunk into 3–5 sentence blocks per topic.
The Iterative Refinement Loop
Training is not one-and-done. The right pattern is:
Day 1: Initial knowledge base
30–60 FAQ entries, services list, top policies. Get to "good enough to launch."
Days 2–7: Listen and refine
Spend 15 minutes/day reviewing transcripts. Look for:
- Questions where the AI said "I don't know" — add to FAQ.
- Answers that were wrong or imprecise — fix the underlying knowledge.
- Patterns of similar questions phrased differently — add variant phrasings.
- Calls that escalated when they should not have — refine the escalation criteria, not just the knowledge.
Week 2 onwards: Steady-state
Add new entries as new business patterns emerge. Pricing change? Update the table. New service? Add an FAQ entry. Most businesses add 3–10 entries/month after the initial sprint.
Common Training Mistakes
1. Uploading a 50-page PDF and expecting good answers
Long unstructured documents are hard to retrieve from. Convert to structured FAQ format first.
2. Single-phrasing questions
Customers ask the same question 5 different ways. "Do you take Cigna?" / "Is Cigna in-network?" / "Do you accept my insurance?" — the AI needs to recognize all of these.
3. Vague answers
"We have flexible pricing" loses to "diagnostic is $89, drain cleaning $180–$385." Specific answers convert; vague ones do not.
4. Forgetting to include "what we don't do"
"Do you do roofing?" — for a plumber, the right answer is "No, we don't do roofing — we can refer you to [partner]." Configure these explicitly.
5. No update cadence
Pricing changes, services evolve, policies shift. If your knowledge base goes stale, customer experience degrades. Set a quarterly review cadence at minimum.
6. Mixing escalation logic into knowledge content
"If the customer is angry, escalate" is not knowledge — it is an escalation rule. Keep these separate. The knowledge tells the AI what; the rules tell it when.
For most SMBs:
- 40–60 FAQ entries — covers 85% of inbound questions.
- Service/pricing table — 10–30 services with ranges.
- 5–10 policy documents — warranty, refund, process, regulatory disclosures.
This is achievable in 2–4 hours of focused work. The next 10% of accuracy requires iteration over the first month; the final 5% requires ongoing maintenance.
Measuring Knowledge-Base Quality
Track three metrics weekly:
- "I don't know" rate — share of calls where the AI could not answer. Target under 5%.
- Escalation rate — share of calls escalated to a human. Should land 8–15% for service businesses (lower means too aggressive AI; higher means knowledge gaps).
- Booking conversion rate — share of intent-qualified calls that ended in a booking or commitment. Should be 60–80%+ for service businesses.
If any of these is off, the knowledge base is the most likely root cause.
Vertical-Specific Tips
Home services
Include service-area ZIPs, after-hours pricing uplift, common-service ranges by year/make/model where relevant. Configure permit-required jobs with proper qualification questions.
Medical / dental / veterinary
Include accepted insurance, new-patient process, what-to-bring lists, urgent-vs-routine triage criteria. HIPAA-BAA tier required.
Restaurants
Build the dish-by-dish allergen / dietary database. This is the highest-impact knowledge investment for restaurants.
Real estate / leasing
Property-by-property facts (rent, deposits, pet policy, parking, amenities). Connect to your property management software for live availability.
Salons / spas
Stylist-level pricing tiers, service durations, color-correction triage flow.
The Bottom Line
The AI voice agent is only as good as what you train it on. 2–4 hours of focused FAQ and knowledge-base work is the difference between a 70% accuracy agent that frustrates customers and a 95%+ accuracy agent that books calls smoothly. The work is finite; the payoff compounds with every call.
If you want to start, begin a JagCall trial. For deeper context, see our 15-minute setup guide, our AI voice agent explainer, our platform comparison, or our small-business automation playbook.
Frequently Asked Questions
How many FAQ entries do I need?
40–60 covers 85% of inbound questions for most SMBs. Add the remaining 10–20 over the first month based on transcript review.
Can I just upload my website?
Some platforms support web crawl. It works for basic FAQs but typically yields lower accuracy than purpose-written Q&A pairs because website prose is not optimized for retrieval.
What about pricing — should I include exact prices?
Use ranges for service businesses (diagnostic + service ranges by category). Exact prices for fixed-price items (gift cards, fees, copays). Avoid quoting firm prices on uninspected work.
How often should I update the knowledge base?
Quarterly review minimum. Update immediately when pricing changes, services evolve, or policies shift.
Will the AI use the knowledge base for all calls?
The AI uses it when caller questions match. For routine intake / booking flows, the AI follows the configured script. The knowledge base is for FAQ-style questions.
What if the AI gives a wrong answer?
Fix the underlying knowledge entry. Most "wrong answers" trace back to either missing knowledge or ambiguous knowledge. Iterate.
Can I split knowledge by use case (sales vs. support vs. after-hours)?
Yes — most platforms support multiple knowledge bases or context tagging. Useful when the same business has distinct call types.
Should I include competitor information?
Optional but useful for "are you cheaper than X" questions. Be factual and brand-appropriate.
Does HIPAA-BAA tiering affect knowledge-base structure?
The knowledge base itself is the same. PHI handling rules apply to call recording / transcription / retention — not the knowledge documents you upload.
How fast can I get from 70% to 95% accuracy?
Most businesses hit 90%+ within 2 weeks of daily refinement. The final 5% takes ongoing iteration as new question patterns emerge.