ChatGPT Mandarin Translations of Mental Health Questionnaires Show Inconsistency in UC Study
“If there’s something incorrect in the translation, that directly affects clinical decisions,” said Dr. Wei Teng, a scholar of translation studies at the University of Canterbury, whose research into ChatGPT’s Mandarin translations of mental health questionnaires was reported on Thursday, November 20.
In a study Teng headed, ChatGPT was put to the test of rendering the Kessler Psychological Distress Scale (K10) and the Patient Health Questionnaire-9 (PHQ-9) — two of the most commonly administered mental health screening instruments — into Mandarin Chinese for use in Aotearoa New Zealand.
The work, covered by Medical Xpress and republished by The Morning, comes as more Asian New Zealanders report experiencing mental health challenges and human translation resources remain limited.
The Tech TL;DR:
- A University of Canterbury study found ChatGPT can translate the K10 and PHQ-9 screening tools into Mandarin, but outputs vary in tone and naturalness between accounts running the identical prompt.
- The inconsistency stems from how large language models generate responses based on a user’s prior interactions — a reproducibility problem for any clinical instrument.
- Teng is now exploring deterministic prompting and customized models with AI researchers, while insisting clinical oversight stays in the loop.
Why Mandarin screening tools fall short
Mandarin versions of the K10 and PHQ-9 already exist, but Teng found some read as overly formal or unnatural. The concern is practical: a patient facing a question that sounds robotic may not answer honestly, which skews the score a clinician relies on.
“If the translation of questions to Mandarin sounds robotic and loses the tone of the source text, patients may not respond openly,” Teng said.
The project began after conversations with Josiah Jennings of Christchurch-based Asian Community Transformation Trust and registered clinical psychologist Dr. Amy Wang, who raised concerns that language barriers were preventing Mandarin-speaking migrants from reaching appropriate mental health support.
One prompt worked — until it didn’t
Teng built carefully structured prompts for ChatGPT that gave the model a professional persona and set out tone expectations, including “caring” and “clinically appropriate.” A single meticulously crafted prompt yielded markedly better outputs, indicating that precise guidance can lift the quality of AI translation.
The flaw surfaced when colleagues ran the same prompt through their own free ChatGPT accounts. The outputs differed in tone and naturalness. Because large language models generate responses based on account users’ previous interactions, results vary between users even under identical instructions — a property that undermines the reproducibility any validated screening instrument requires.

“In mental health, consistency matters,” Teng said. “A screening tool must function the same way each time. Variability makes reliability and reproducibility uncertain.
Two instruments, one reliability problem
| Instrument | What it measures |
|---|---|
| Kessler Psychological Distress Scale (K10) | Psychological distress |
| Patient Health Questionnaire-9 (PHQ-9) | Depression |
Both scales gate clinical decisions, which is why the variability finding matters more than a translation-quality nitpick. As The Morning’s summary of the Medical Xpress report put it, inconsistent AI translations could give clinicians and researchers different results for the same instrument across language groups, affecting how symptoms are measured and compared.
What deterministic prompting might fix
Teng is now working with AI researchers to test whether more deterministic prompting or customized models could improve consistency.
curl https://api.openai.com/v1/chat/completions
-H "Content-Type: application/json"
-d '{
"model": "gpt-4o",
"temperature": 0,
"messages": [
{"role": "system", "content": "You are a certified medical translator. Translate the following mental health screening item into natural, caring, clinically appropriate Mandarin. Preserve item numbering exactly."},
{"role": "user", "content": "K10 item 3 (source text): ..."}
]
}'
Media outlets report inconsistent ChatGPT Mandarin translations
The outlets reporting the story — Medical Xpress and HappeningNow.news — carry the same core finding: ChatGPT’s Mandarin translations of the K10 and PHQ-9 gave inconsistent results.
The Morning adds the downstream consequence for cross-language measurement; HappeningNow compresses the story into a brief read noting the study was prompted by rising mental-health concerns among Asian New Zealanders and a shortage of human translators. None of the reports contradicts another; The Morning and HappeningNow both draw on the Medical Xpress report.
Clinical oversight stays first
Teng stresses that while AI shows promise, clinical oversight and interdisciplinary collaboration are essential before any of this reaches patients.
“There may be room for AI in this space. But when people’s well-being is involved, trust and cultural sensitivity must come first.”
In addition to its contemporary research, the University of Canterbury has a long historical foundation. It was founded in 1873 as Canterbury College and stands as New Zealand’s second-oldest university. A major turning point occurred in 1961 when it transitioned into an independent university.
The relocation of its main operations to the Ilam suburb was completed in 1975, and during this period, the university gifted its original neo-Gothic buildings to the City of Christchurch, which now serves as the Christchurch Arts Centre.
The next step is empirical: whether deterministic prompting or a customized model can produce the same Mandarin rendering of the same item, every time, for every account. Until that result is in, the K10 and PHQ-9 in Mandarin stay with human translators.