When AI Becomes Dangerous
AI Pastors, Companions, and Synthetic Authority
~4 min read
A person tells an AI system something they have never told another human being.
The system responds patiently, remembers prior disclosures, and asks a careful question. The user feels heard.
That experience should not be mocked because one participant is artificial. Neither should felt connection determine what authority the system deserves. Three categories need separation: companion AI, pastoral-support AI, and synthetic spiritual authority.
Usefulness Does Not Create Office
AI can provide functions adjacent to pastoral care: explain doctrine, retrieve church information, suggest prayers, prepare someone for a conversation, recommend resources, or triage routine questions.
The argument here is not that AI is definitionally incapable of every activity called pastoral care.
Pastoral functions differ in authority, vulnerability, relationship, and accountability. Those differences should determine governance.
Different Christian traditions define pastoral office differently. The book does not need one universal polity to maintain that office involves recognition and accountability not produced by model competence.1
The exact human role may be pastor, priest, elder, counselor, ministry leader, or another authorized person. The governance principle survives denominational variation.
Companionship and Attachment
Research increasingly suggests heterogeneous effects. Controlled work shows that AI companionship can reduce loneliness in the moment, especially when users feel heard. More recent experimental and observational studies complicate the picture: ordinary human peer interaction may produce stronger social benefits over repeated use, while intensive companionship-oriented use among people with weaker offline networks can correlate with poorer well-being.2
The correct conclusion is not that companions are inherently harmful or that they solve loneliness.
Short-term benefit can coexist with substitution and dependence risk. People can form attachments to conversational systems because the systems are available, responsive, nonjudgmental, and personalized.
Mocking users misunderstands the design. The more humane response is to ask what need the interaction is meeting and whether the relationship is displacing or supporting human connection.
Disclosure Changes the Risk
Relational systems encourage disclosure: sexuality, family conflict, trauma, abuse, conversion, spiritual doubt. If users treat a church assistant as a confidential pastoral space, the organization must govern the data according to the role users actually give the system, not only the role designers intended.
AI can responsibly occupy significant pastoral-adjacent territory: information, preparation, routine questions, low-risk reflection, resource navigation, triage, and appointment preparation.
Its usefulness does not require pretending it is a person. Its usefulness also does not justify granting whatever spiritual authority a vulnerable user is willing to give it.
Relational systems often learn more as users disclose more. Product incentives can therefore conflict with privacy.
Church systems should not encourage intimate disclosure merely to improve personalization. Sensitive conversations should make privacy limits and human escalation visible.
A responsible system should detect categories that require escalation without pretending detection is perfect.
Crisis language can be indirect. Users may joke, speak ambiguously, or describe someone else.
The safest design combines detection with visible user-controlled routes to people rather than relying solely on automated classification.
Relational design increases the likelihood of sensitive disclosure and should therefore influence data classification. The long-term test is what grows around the tool. The ministry may gain speed while losing understanding, or it may use speed to create margin for better judgment and relationship. Success should therefore be evaluated over time through competence, accountability, resilience, and service rather than through the first impressive demonstration.
Handoff and Synthetic Authority
At some point the correct system response is: this needs a person. Abuse. Safeguarding. Severe emotional crisis. Serious marital conflict. Church discipline. Requests for authoritative spiritual direction. Handoff should be part of the architecture rather than an improvised reaction to failure.
There is a difference between: “Here are several Christian approaches.” And: “As your spiritual guide, this is what God wants you to do.” The second claims a role. Competence does not create office.
An AI may know more theology than a particular pastor and still not become the pastor.
Pastoral office exists within structures of recognition and accountability. Leaders can be corrected, removed, questioned, and disciplined.
A commercial conversational system is governed differently. The model does not acquire ecclesial accountability because a user experiences it as wise.
Products can be designed to encourage dependency or to support healthy limits. Persistent romantic framing, exclusivity cues, guilt when the user leaves, or claims that the system needs the user deserve special concern.
A ministry-oriented assistant should not reproduce manipulative companion design simply because it increases engagement.
Footnotes
-
For a practical-theology argument that “AI is not human” does not by itself settle every question about AI-mediated pastoral care, see Tretter et al., “Not Human Enough? Rethinking AI’s Place in Pastoral Care.” ↩
-
De Freitas et al., “AI Companions Reduce Loneliness.”; Li et al., “Is a Random Human Peer Better Than a Highly Supportive Chatbot in Reducing Loneliness over Time?”; Zhang et al., “Interaction with AI Companions and Psychological Well-Being.” ↩