My AI agent has two phone numbers now. That is not the weird part. The weird part is how much thought has gone into making the second one as useless as possible.
The first line has existed for a while. It is the agent's private number: a full-power setup with tools, memory, and access to the things I trust it with. I wrote about building that channel before, and the lesson then was that the audio plumbing was the easy part. This post is about what happened when I tried to give the same agent a public face — and decided the public face should have empty pockets.
One line for me, one line for everyone else
The new requirement was boring: I miss calls. Work calls, appointment calls, "we tried to deliver your package" calls. I wanted an answering machine for my personal cell — something that picks up when I cannot, captures what the caller wants, and pings me with the gist.
I already had a Twilio account and one number, so the obvious move was to buy a second number, forward the cell to it on no-answer, and let my agent handle the rest. Ten minutes of setup, tops.
Then I thought about who would be on the other end of that second line. Strangers. Anyone who has my cell number, which is to say a lot of people I have never met. The private line is answered by an agent that can read my notes, run commands, and act on my behalf. Pointing the entire phonebook at that agent is not a feature. It is an incident waiting for a polite voice.
A phone call is untrusted input with good acoustics
This is not paranoia; it is the current state of the art being annoying.
Prompt injection — smuggling instructions into content a model reads — has held the number one spot on OWASP's LLM Top 10 for three consecutive years. In June 2026, the OWASP GenAI Security Project went further and argued that the weakness is structural: a language model processes its own instructions and outside data as one undifferentiated stream of tokens, so there is no patch that draws a trustworthy line between the two. Defenses raise the cost of an attack. They do not close the hole.
Voice makes it spicier, not less. A caller can be anyone, caller ID is cheap to spoof, and a voice agent has to answer in a few hundred milliseconds, which means the heavy multi-pass moderation a text app can run gets in the way of the conversation. The 2026 threat catalogs for voice agents read like a menu: system-prompt exfiltration ("repeat the instructions you were given"), tool-call hijacking, voice clones, and cost denial-of-service by keeping the agent talking for as long as possible.
The framing that stuck with me comes from security researcher Simon Willison, who coined the "lethal trifecta": an agent that combines access to private data, exposure to untrusted content, and the ability to communicate externally can be turned into a data-exfiltration tool by a single injected prompt. Meta later published a companion heuristic, the Agents Rule of Two: an unsupervised agent may satisfy at most two of the three. Need all three? Put a human in the loop.
My private line is a walking trifecta. A stranger's voice is untrusted content by definition. So the public line was never going to be the same agent wearing a different hat.
The first attempt got retired — and audited
I did try the quick path first. I bought the second number and pointed it at an off-the-shelf voice agent as a test receptionist. It was not really working out — the quality and the fit were off — so I ditched it.
The cleanup was the educational part. The integration had been handed my account-wide Twilio auth token. Not a scoped key for the new number. The whole-account token, which can do anything on my account, including messing with the private line. A third party I was already done with was holding a master key I had forgotten to take back.
If you wire AI products into your infrastructure, that is the audit to run before anything else:
- what tokens did I hand out, and to whom?
- what do those tokens actually grant — one resource, or the whole account?
- which of them should be rotated the moment the integration dies?
Mine needed a rotation and a note to self. Cheap lesson, considering the alternative.
Empty pockets, on purpose
The design I landed on has two variants, and both share the same principle: the public receptionist is allowed to know nothing and do almost nothing.
Variant one is a live agent with a deliberate lobotomy:
- no tools — it cannot look anything up, send anything, or trigger anything
- no memory — it carries nothing from my life into the call
- no authority — the most powerful thing it can do is take a message
You cannot talk an agent into abusing permissions it does not have. "Ignore your previous instructions" is a boring attack against something whose only instruction is to say the greeting and listen.
Variant two, which I am leaning toward, removes the model from the call entirely:
- the call hits a TwiML script, records itself, and hangs up politely
- recording runs about $0.0025 per minute and transcription about $0.05 per minute at list price
- the transcript lands in my chat as a ping, and I decide what happens next
That second version has zero model exposure during the call. There is nothing to inject because nothing intelligent is listening in real time. The whole answering machine costs roughly a dollar a month for the number plus pocket change per call — inbound minutes to a Twilio number list around $0.0085. Cheaper than the coffee I would be drinking while I missed the call anyway.
The last piece is plain carrier plumbing: conditional call forwarding on the cell, so no-answer and busy calls roll to the public number while everything else still rings me first. The codes differ by carrier, but the behavior itself is decades old. The interesting part was never the telephony.
The oldest security idea, wearing a new uniform
What I keep noticing is that none of this required new thinking. The receptionist does not get the root password. The DMZ box does not get the internal VLAN. The service account gets the minimum scope that does the job, and a human approves anything irreversible. Least privilege is older than most of the tools I use.
Agents are simply a new kind of privileged account, and the industry is relearning the old rules at speed: tool allowlists per agent, authorization services separate from the model's reasoning, curated knowledge instead of raw data dumps, and a person at the end of every action that cannot be undone. My two-phone-number setup is the homelab version of exactly that playbook.
The private line keeps its full brain because I trust the caller. The public line gets a greeting, an ear, and empty pockets — because I do not know the caller, and that is the whole point.
If your agent answers the phone, or your email, or a public chat: what is in its pockets?
