- AI Engineering
- Posts
- [Hands-On] Build a Customer Support Voice Agent
[Hands-On] Build a Customer Support Voice Agent
... PLUS: Vector Search Is Not Agent Memory
In today’s newsletter:
The Current State of Vector Databases (Q2 2026)
[Hands-On] Build a Customer Support Voice Agent
Reading time: 5 minutes.
Last year, many predicted that dedicated vector databases would disappear. PostgreSQL, MongoDB, SQL Server, and other databases were all adding native vector search, making specialized vector databases seem unnecessary.
That is not what is happening.
Instead, the market is splitting based on scale, workload, and deployment requirements. The right database in 2026 depends less on whether you need vector search and more on how you are using it.
Actian's State of Vector Databases Q2 2026 report takes a look at how vector databases have changed over the past year. One of its biggest findings challenges a prediction many people made in 2025.
What Changed Since 2025
Small workloads are staying inside existing databases. If you are storing millions of vectors alongside application data, PostgreSQL with pgvector is often simpler than managing a separate vector database.
At larger scales, purpose-built vector databases still outperform general-purpose databases. They handle billion-scale indexes, distributed search, and predictable latency that traditional databases struggle to maintain.
The Three Forces Reshaping the Market
Scale: PostgreSQL fits many small and medium workloads, while dedicated engines dominate at larger scales.
Workload patterns: AI agents generate frequent writes, making traditional read-optimized RAG architectures less effective.
Deployment requirements: Regulated industries increasingly require on-premises and edge deployments instead of cloud-only services.
What the Report Found
The strongest insight from the report is that vector search has become a feature, not a product category.
General-purpose databases are getting good enough for many production workloads. Dedicated vector databases are still the best choice for massive-scale retrieval. AI-native applications are creating a third category that needs high write throughput, fast retrieval, and flexible deployment options.
That is a very different market from the one we had a year ago.
Building a voice agent usually means integrating speech-to-text, a language model, text-to-speech, telephony, and more into a single pipeline.
This project skips almost all of that complexity. Built on Telnyx's AI Assistant Builder, the entire voice pipeline is configured through a no-code portal instead of written by hand.
What's Actually Yours to Build
Speech-to-text, the LLM, text-to-speech, and phone routing are all handled by the platform this project is built on. Configuring an agent means filling in a system prompt, picking a model, and assigning a phone number, all inside a web portal.
The only backend code in the repo is one webhook. It runs once, right when a call starts, and its job is to hand the agent a few pieces of live information before the conversation begins.
A Prompt That Updates Itself
A static system prompt goes stale immediately. If a support queue is backed up right now, or there's an active incident affecting logins, a prompt written last week doesn't know that. Hardcoding that context means editing the prompt by hand every time something changes.
The system prompt here uses placeholders instead, filled in right before each call connects:
The platform sends a POST request to the webhook the moment a call starts, and whatever dictionary comes back gets substituted into those placeholders. The keys have to match exactly.

In this demo those values are hardcoded stubs. In production, each one becomes a real lookup: a status page API, a support queue count, PagerDuty for incidents.
Verifying the Call Is Real
Anyone who finds the webhook URL could send it a fake request and try to feed the agent false context. The webhook verifies the request's signature before it does anything else.
Every real request carries a signature, checked against a public key, and only a request that passes gets a real answer back. A forged request gets a 401 instead of a chance to poison the prompt.
Skip the Vector Database
The bank's entire support knowledge base lives directly inside the system prompt: account tiers, card policies, wire transfer cutoffs, fraud procedures. Not in a vector database. Not behind a retrieval step. Just plain text, read in full on every call.
That works because the knowledge base is small, under roughly 8,000 tokens. A model reading the whole thing every time is simpler than building a retrieval pipeline, and for a support playbook that size, simpler wins. Retrieval earns its complexity once a knowledge base grows past what a prompt can hold, not before.
Run It
git clone https://github.com/Sumanth077/Hands-On-AI-Engineering.git
cd Hands-On-AI-Engineering/audio/customer_support_voice_agent
uv venv && source .venv/bin/activate
uv pip install -e .
cp .env.example .env Add your Telnyx API key and public key to .env, then start the webhook:
python main.py
ngrok http 8000Less Code Doesn't Mean No Code
The interesting part of this project isn't the phone call. It's how little of the system you actually have to build once the pipeline itself isn't your job. What's left is one small, well-scoped question: what does the agent need to know right now, and how does it get that before the conversation starts.
That’s all for today. Thank you for reading today’s edition. See you in the next issue with more AI Engineering insights.
PS: We curate this AI Engineering content for free, and your support means everything. If you find value in what you read, consider sharing it with a friend or two.
Your feedback is valuable: If there’s a topic you’re stuck on or curious about, reply to this email. We’re building this for you, and your feedback helps shape what we send.
WORK WITH US
Looking to promote your company, product, or service to 200K+ AI developers? Get in touch today by replying to this email.



