Join me as I chat with Tony Ge about building Voice AI Agents. Tony breakdown his process of creating a voice AI where called hundreds of luxury watch dealers to negotiate prices on a Rolex Daytona. Tony demonstrates how he built the system using VAPI, explains the technical setup including prompt engineering and tool integration, and discusses how the data collected could be used for arbitrage opportunities. The conversation highlights the potential for voice AI to disrupt industries reliant on phone communication.
Timestamps:
00:00 – Intro
01:28 – Overview of Negotiation Voice AI Agent
02:46 – VAPI Overview
03:11 – Prompt engineering techniques
06:01 – Choosing the best Model and Temperature
08:56 – Choosing the best Voice for your AI Agent
09:57 – Explanation of tool integration with databases
12:36 – Combining Lindy and Voice AI Agents
14:32 – Integrating with Twilio for phone number registration
15:44 – Voice AI Startup Ideas
Key Points:
• Tony Ge created a voice AI called "Lowballer9000" that called luxury watch dealers to negotiate prices on a $35,000 Rolex Daytona
• The AI successfully gathered data from over 800 calls, identifying which dealers offered the best prices
• VAPI (Voice API) was used to build the AI agent with Gemini 2.0 Flash as the model and Cartesia voice
• Tony believes voice AI will automate phone-based work within 1-3 years
1) Tony created "Lowballer9000" (aka Alex) using VAPI to call 800+ luxury watch dealers and negotiate prices on a Rolex Daytona.
The results? He found MASSIVE arbitrage opportunities – buying at $19K and potentially selling at $22K for a $3K profit PER WATCH!
2) The TECH STACK behind this voice AI:
• VAPI (Voice API) platform
• Gemini 2.0 Flash model
• "New York Man" voice (definitely NOT a Sopranos reference)
• Twilio for calls
• Airtable to track results
• Custom prompts
All connected with tool calls to create a seamless system.
3) PROMPT ENGINEERING SECRETS:
• Keep responses concise (1-2 sentences max)
• Use 6th grade English level (dramatically increased call duration!)
• Temperature setting of 1 for more creative responses
• Structured conversation flow to sound more human
These tweaks made the AI significantly more convincing.
4) The BIGGEST CHALLENGE wasn’t the tech – it was getting calls to actually connect!
Tony had to navigate "Shaken and Stir" FCC regulations and KYC requirements for phone numbers.
About 300 calls went straight to voicemail before he figured this out.
5) What makes this POWERFUL is the data collection.
Tony’s system automatically:
• Records all conversations
• Identifies which calls resulted in offers
• Tracks original price, lowball amount, and final price
• Organizes everything in a searchable database
Creating an UNFAIR ADVANTAGE.
6) Beyond watches, this approach works for ANY business where data = arbitrage opportunity.
Some use cases:
• Real estate prospecting at scale
• Collecting customer feedback (Tony built this for @StartupEmpire)
• Market research
• Lead qualification
7) BUSINESS IDEA ALERT:
Tony and Greg realized this feedback collection system could be packaged as a service:
"For $300/month, we’ll set up a voice AI feedback system that posts summaries to your Slack and sends weekly reports."
Companies would PAY for this!
8) The KEY INSIGHT: Voice AI isn’t just about replacing humans – it’s about creating transparency in markets where information asymmetry exists.
When you can make 800+ calls to find the best deal, you unlock opportunities others can’t access.
Notable Quotes:
"I think people are drastically underestimating what voice AIs are capable of. Basically, it’s my belief that any work that you do that’s over the phone could and may get automated in the next one to three years." – Tony Ge
"Most conversations happen at like a sixth grade English level… when I changed the prompt to say like only use like sixth grade English, I got way longer call durations." – Tony Ge
LCA helps Fortune 500s and fast-growing startups build their future – from Warner Music to Fortnite to Dropbox. We turn ‘what if’ into reality with AI, apps, and next-gen products https://latecheckout.agency/
BoringMarketing — Vibe Marketing for Sale: http://boringmarketing.com/
Startup Empire – a membership for builders who want to build cash-flowing businesses https://www.startupempire.co
FIND ME ON SOCIAL
X/Twitter: https://twitter.com/gregisenberg
Instagram: https://instagram.com/gregisenberg/
LinkedIn: https://www.linkedin.com/in/gisenberg/
FIND TONY ON SOCIAL
Youtube: https://www.youtube.com/@tonyxge
Patreon: https://www.patreon.com/tonyge
