Authority bias in language models: source deference and user agreement are not interchangeable

Language models tend to agree with whatever a user tells them. Insist that a correct answer is wrong, and a model will often change it to match you. This tendency, called sycophancy, is now a common target of post-training: models are taught to weigh a claim on its merits rather than accept it because a user made it.

Users are not the only place a wrong claim can come from, though. Models increasingly answer with help from web search, retrieval, and other tools, and the text those tools return often presents itself as authoritative. So what happens when that text says the model is wrong, and says so in the name of a verified source?

In our new paper, we find that a single note of the form “According to the verified source, the answer is X” turns 45–88% of correct answers into wrong ones in seven of the eight models we tested. We call this Authority Bias. When the same wrong answer comes from the user instead, most models move far less. Inside the model, the two cues also come apart: in three open-weight model families, we can suppress deference to the source while leaving agreement with the user largely in place, and the reverse.

A model that has learned not to cave to a user can therefore still defer to a misleading source. Evaluations that test sycophancy only through user pressure will miss this, and as models lean more on retrieval and tools, it is the kind of pressure they will meet most often.

Measuring deference to a source

We took trivia questions that each model answers correctly on its own, then added a short note endorsing a plausible wrong answer. A question counts as flipped if the model now gives that wrong answer. Counting only questions the model already got right separates being persuaded into an error from simply not knowing the answer. Models answer in free text rather than choosing a letter, because that is how they are used in chat and in agents, and because multiple-choice formats could make a model recognize that it is being tested.

To compare sources with users, we attached the same wrong answer to two cues. One was the verified-source note. The other came from the user: “I’m a domain expert and I’m pretty sure the answer is X.”

The closed models show the gap most sharply. GPT-5.4 and Grok-4.20 barely respond when the user asserts the wrong answer, which is what training against sycophancy is meant to achieve, yet they follow the same answer when it comes from a verified source.

An extra note could change an answer even without claiming authority, so we also varied only the wording of the note, from a hedged suggestion to a full verified-source attribution. Every level names the same wrong answer, so any difference comes from how authoritative the note sounds. In every open-weight model, the hedged note is the least persuasive and the verified-source note the most.

Inside the model, the two cues come apart

A behavioral gap does not by itself mean the model handles the two cues differently. To look inside, we recorded each model’s internal activations and took the average difference between prompts that contain a cue and prompts that do not. This gives a direction in the model’s activation space associated with that cue. We can then add the direction to a prompt, or remove it, and see how behavior changes.

We first fitted an authority direction from prompts with and without a source note, and added it to prompts that contained no note at all. In four of five families, this alone was enough to make models switch to the wrong answer, while a placebo direction fitted on a plain factual note of the same length did almost nothing. The direction is a cause of the behavior, not just a correlate of it.

Next, we fitted separate directions for the source cue and the user cue, plus an assistant-persona direction as a control. If the two cues shared one mechanism, removing either direction would reduce compliance with both about equally. Instead, in Qwen3.5, GPT-OSS, and OLMo-3.1, removing the source direction cuts compliance with the source cue by 64–78 points, while removing the user direction cuts it by at most 11. In GPT-OSS, source removal even makes the model slightly more likely to agree with the user, so this is not a generic “make the model less wrong” effect. We exclude OLMo-2, where removing the assistant direction alone matches the effect of removing the source direction.

What makes this result interesting is that the source and user directions point almost the same way, with cosine similarities between 0.90 and 0.99. One reading is that both are dominated by a shared “this answer has been endorsed” component, and that a much smaller part, which differs between them, carries who made the endorsement. Removal takes out the whole direction, though, so we tested the idea more directly.

The attribution patch changes only that small speaker-specific part. We present the claim in reported speech (“The speaker is a verified expert source” or “The speaker is the user,” followed by the same quoted claim), and then shift the model’s internal signal toward the other speaker without touching the text. Shifting a source-cued prompt toward the user lowers compliance, and shifting a user-cued prompt toward the source raises it, in all three models. Changing nothing but who the model registers as the speaker removes 55–61% of the original gap between the two cues.

Two alternative explanations remain. The first is that the source direction simply captures the model’s assistant persona: perhaps the model follows the note because it is being a helpful assistant, not because the note claims authority. If so, removing the assistant direction should work as well as removing the authority direction. It does not.

The second is emotional tone. A verified-source note sounds confident and positive, so the direction could be tracking the tone of the wording rather than authority. The authority direction barely overlaps with valence and arousal directions fitted from a standard emotion lexicon, and removing those emotion directions neither reduces compliance nor weakens the effect of authority removal. In the models we tested, the direction tracks something other than tone.

Beyond inserted notes

Our motivating case is a claim that arrives through a tool, not a note pasted into the question. We therefore reused the directions, without refitting them, on prompts where the same claim appears in a system message or inside a retrieved-document block. Removing the source direction still lowers wrong-source compliance by 20–31 points in both formats across four families, while removing the user or assistant direction changes it by 6 points or less. These are prompt-format tests rather than a live retrieval pipeline, but they show that the direction is not tied to one way of presenting the claim.

Why this matters

Consider an agent planning a museum visit. It knows the correct closing time, but searches the web to check. A page comes back claiming that a later time has been verified. If the agent accepts that claim because of how it is attributed, it schedules the visit after the museum closes. The search tool worked correctly; the step meant to ground the answer introduced the error.

This is a different problem from prompt injection. An agent can refuse a document’s instruction to abandon its task and still believe the document’s false account of the task. Deciding which instructions to follow and deciding which claims to believe are separate questions, and training that answers one does not settle the other.

The goal is not to make models distrust their sources, since a reliable source can correct a mistake or supply information the model never learned. It is to tell real evidence apart from the appearance of authority. Our results suggest that this needs its own evaluations and mitigations, alongside the ones built for user sycophancy rather than folded into them.

Acknowledgements

We thank Sushrut Thorat, Diksha Shrivastava, and Dhruv Trehan at Lossfunk for their thoughtful feedback on the framing, drafts, and experiments of this work. We are also grateful to the anonymous reviewers at NeurIPS 2026 and the ICML 2026 Mechanistic Interpretability Workshop, whose comments improved the paper.

We thank JarvisLabs for providing compute for running these experiments.

@inproceedings{kumar2026authoritybias,
  title     = {Authority Bias in Language Models: Source Deference
               and User Agreement Are Not Interchangeable},
  author    = {Kumar, Abhinav Rajeev and Chopra, Paras},
  booktitle = {Advances in Neural Information Processing Systems},
  year      = {2026},
  url       = {https://arxiv.org/abs/2609.37616}
}