A chatbot on a crisis support page opens with: "I’m so sorry you’re experiencing this. I’m here to help." Research quoted in a recent MIT Technology Review Insights report found something worth sitting with: some people genuinely appreciate that line. Others find it grating, precisely because it is performed. The split runs roughly 42% who respond well to it against 20% who find it frustrating. Not a landslide either way. A real, uncomfortable divide.
Harvard Business Review has a name for what produces the 20%: engineered insincerity. Empathy manufactured at scale, deployed by a system that cannot feel anything, aimed at a person who can tell.
The next two years will bring an enormous amount of pressure on community organisations to adopt AI, chatbots included. Some of that pressure will be well-founded. Some of it will not. The evidence supports a considered, discriminating position rather than a wholesale embrace or a wholesale rejection, and for organisations whose visitors sometimes arrive in crisis, getting that line right is not a branding decision. It is a duty of care question.
Where automation earns its place
Start with the legitimate case, because it is real and it is larger than the scepticism above might suggest.
Most of what people want from automation is simply their time back. Confirming an appointment. Checking opening hours. Reordering something they have ordered before. Finding a form. Being told which office to visit. None of this needs a human, and pretending otherwise wastes a visitor’s time as surely as a broken link does.
For a community organisation, this covers more ground than it might first appear: intake logistics, service area lookups, appointment scheduling, frequently asked questions about eligibility, document checklists before a first meeting. Automating these tasks does not cheapen the organisation’s care. It protects the time of the people who provide it, so that care can go where it is actually needed.
Where it should never go
Then there is the other kind of moment. Disclosure of family violence. A first call about a criminal charge. A person describing a mental health crisis. A parent explaining a custody dispute they do not yet have words for.
Here the research is specific rather than vague: 64% of customers say the ability to reach a human within an AI-assisted process would improve their experience. For a general business, that is a service quality metric. For an organisation supporting someone in crisis, it is closer to a minimum standard. A visitor who is disclosing something difficult and is met with a scripted, sympathetic-sounding reply, however well designed, has been handed exactly the kind of insincerity the research warns about, at exactly the worst possible moment to receive it.
This is not an argument against AI as a category. The same body of research found 81% of people open to AI recommendations, provided the trust conditions are met: transparency about what is automated, an easy route to a human, and evidence that the organisation understands the difference between the two. People are not rejecting automation. They are rejecting automation that pretends to be something it is not, offered where something it is not was needed.
A practical framework: automate the findable, humanise the fragile
A useful test for any proposed automation is simple to state, if not always simple to apply. Ask whether the interaction is findable or fragile.
Findable interactions have a correct answer that does not depend on the visitor’s circumstances: hours, locations, eligibility criteria, document requirements, appointment slots. These are well suited to automation, and automating them frees staff time for the interactions that need it.
Fragile interactions depend entirely on the visitor’s circumstances, and the correct response cannot be scripted because it has to be heard first. Anything involving disclosure, distress, or a first approach to your organisation about something serious sits here. These need a human, reachable quickly, without the visitor having to fight through a script to get there.
A board or leadership team considering any automation proposal can ask three questions before approving it.
Is this interaction findable or fragile? If the answer is genuinely uncertain, treat it as fragile until proven otherwise.
Is the exit to a human immediate and obvious? Not buried three menu layers down. Visible from the first message.
Does the automated language make a claim it cannot back? "I understand" and "I’m sorry you’re going through this" are claims of feeling, made by something that does not feel. Automated systems can be efficient, clear, and genuinely useful without pretending to empathy they do not have. The insincerity is not in the automation. It is in the performance of care layered on top of it.
Governance, not gadgetry
The organisations that get this right treat automation decisions as governance decisions, not as feature additions bolted on by whoever is comfortable with the software. A chatbot deployed on a service page belongs in the same review process as a change to intake procedure, because functionally, that is what it is.
This means documenting what is automated and why, reviewing it against the findable/fragile test before launch, and revisiting the decision on the same schedule as other policy reviews rather than leaving it running indefinitely because nobody flagged a problem. It also means being honest with your own board about where the line sits, rather than letting a vendor’s sales pitch set it for you.
For organisations working with vulnerable people, this sits naturally alongside the other risk areas a board already tracks: security, accessibility, funding compliance. Automation is not a separate, exciting new category. It is one more thing that needs the same calm scrutiny as everything else your digital presence does on your behalf.
The line is not about the technology
None of this is an argument that chatbots are bad, or that automation is beneath a caring organisation. Plenty of good, careful organisations use both well. The line that matters is not automation versus no automation. It is performed care versus genuine access.
A visitor checking your opening hours does not need to feel understood by a machine. A visitor disclosing something they have never told anyone does. Getting that distinction right, deliberately and on paper rather than by accident, is what separates a digital presence that serves people from one that merely sounds like it does.
If you are weighing an automation decision, or want an honest read on where your current setup sits, that conversation fits naturally within a Digital Capacity Diagnosis. For organisations in the regulated professions, there is a dedicated starting point for regulated professions.
