Should I let AI handle customer service calls for my business in 2026?

What Kinney Drugs' AI Rollback Teaches Founders

Kinney Drugs pulled its AI phone assistant Burt on August 7, 2026 after patients reported wrong dosages and missed refill alerts — proof that Daugherty and Wilson's Radically Human argument that trust is a business imperative, not an ethics footnote, is now a P&L line item; before deploying customer-facing AI, run a wrong-answer-cost audit, not just a capability demo.

I spend a chunk of most weeks telling founders to automate something. So when a story like Kinney Drugs lands in my feed, I read it twice — once as the guy who builds these systems, once as the guy who has to explain to a client why "the AI sounded confident" isn't the same as "the AI was right."

Here's what happened. Kinney Drugs, a Vermont and New York pharmacy chain, introduced an AI phone assistant named Burt back in May 2026, built to handle prescription and refill calls. By early August, hundreds of customers had reported incoherent calls, wrong dosage information, and missed prescription notifications. On August 7, president John Marraffa announced the company was pulling Burt back and returning to the old touch-tone phone system for incoming calls. His quote is the part I keep rereading: "Getting privacy and security right does not mean we got the experience right. We did not, and we own that." Burt survives only for outbound texts — refill reminders — and only for patients who opt in.

What strikes me is that this wasn't a security failure or a jailbreak. Burt was HIPAA compliant, closed-source, not manipulating data. It failed on the much more boring axis: accuracy under real-world pressure, in a domain where being wrong has an immediate physical cost. That's the gap most "should we deploy AI here" conversations skip. Founders demo the tool, it sounds articulate, and articulate gets mistaken for reliable.

Paul Daugherty and H. James Wilson named this exact blind spot in Radically Human, years before this incident: "Trust has been thrust to the forefront by biased algorithms, data breaches, and surveillance concerns... Trust is not just a moral imperative — it's a business imperative." Their point wasn't abstract ethics — it was that trust failures show up on the income statement, in churned customers and reversed rollouts, faster than most companies budget for. Kinney didn't lose money because Burt lacked capability. They lost trust because nobody had war-gamed what a wrong answer costs when the topic is a medication dose.

I run a version of this audit with every client before we let an AI touch anything customer-facing, and Kinney's collapse is a clean enough case study that I've tightened it into four questions. I call it the trust-before-capability audit, and it takes about twenty minutes with a whiteboard.

The trust-before-capability audit (run before any customer-facing AI launch)

Map the wrong-answer cost. Not "can it fail" — what does the single worst plausible output look like, and who absorbs it? A wrong dosage is not the same failure class as a wrong shipping estimate.
Draw the humane fallback line. What still routes to a human or a dumb, boring system no matter what? Kinney's fallback was literally the old touch-tone menu — unglamorous, but it never hallucinates a prescription.
Instrument for silent failure. Did anyone at Kinney see the complaint volume rising in real time, or did it take months of accumulation before leadership acted? If your only failure signal is a churn report, you'll find out too late.
Default to opt-in for anything irreversible. Kinney kept Burt for outbound texts, but made them opt-in only. Irreversible or high-stakes actions earn consent gates; low-stakes, reversible ones don't need them.

The honest limit here: this audit doesn't make AI safe for every customer-facing job. It tells you which jobs are still too early. Medical dosing, legal advice, anything with an "I trusted the AI and it cost me money or health" failure mode — that's not a prompting problem you engineer away this quarter. It's a category where the fallback needs to stay boring and human for a while longer. I've told two clients this year to shelve a customer-facing agent idea after running this audit, not because the demo was bad, but because the wrong-answer cost was a lawsuit, not an apology email.

What I do deploy without hesitation is AI on the internal side of that same workflow — drafting the refill reminder copy, summarizing call transcripts, flagging accounts likely to have a bad interaction — anywhere a human still reviews the output before a patient or customer sees it. Radically Human's whole thesis is that the winning move isn't choosing between automation and humans; it's being deliberate about which side of the trust line each task sits on. Kinney's mistake wasn't building Burt. It was putting him in the highest-stakes seat in the building before the trust infrastructure existed to catch him.

Sources: WCAX, "Kinney Drugs pulls back AI phone assistant after hundreds of customer complaints" (Aug 7, 2026); VTDigger, original reporting; KWQC follow-up coverage; Paul Daugherty and H. James Wilson, Radically Human (Harvard Business Review Press, 2022).