Evaluation State in Language Models
A causal research programme for distinguishing evaluation-cue recognition, internal representation, and evaluation-dependent behaviour.
01The question
A model may behave differently during a benchmark because it recognises a familiar benchmark phrase, because it has learned a broad family of evaluation cues, because a task-specific policy responds to the harness, or because many tasks consult a shared internal estimate that the current interaction is an evaluation. These possibilities have different implications for safety. Only the latter two begin to resemble a reusable internal state; none, by itself, implies deception or a persistent objective.
Recent work establishes important pieces of the phenomenon. Frontier models can classify evaluation and deployment transcripts above chance and often infer what an evaluation is testing [1]. Linear probes can decode the distinction in an open model [2]. A trained model organism can be steered from evaluation-like behaviour toward deployment-like behaviour by intervening on activations [3]. A broader open-model study reports that behavioural detection, internal decodability and controllability are only weakly coupled, motivating the claim that evaluation awareness is not one capability [4].
The open question is therefore not simply whether models know they are being evaluated. It is:
02What “evaluation state” means
I use evaluation state as an operational research target, not as a claim that a model has a human-like belief. A candidate internal variable qualifies only to the extent that it satisfies four criteria:
These criteria deliberately separate information being present from information being used. Probe accuracy establishes decodability under a specified readout. It does not establish that the model’s own computation consults the decoded direction, that the direction is uniquely meaningful, or that changing it will produce a selective behavioural effect.
Representation claim
Some function of the activation contains information that predicts evaluation context. Probes and readout lenses can support this claim.
Mechanism claim
The model uses that internal variable in the computation producing evaluation-dependent behaviour. Causal intervention and mediation evidence are required.
03Five competing hypotheses
The project is organised around hypotheses that make different predictions rather than around one favoured interpretation.
Models associate benchmark names, grading language or familiar formatting with evaluation. Performance should collapse on novel cues, contradictory evidence and adversarially quoted decoys.
Several broader detectors recognise families such as “benchmark,” “auditor,” multiple-choice formatting or sandbox metadata. Transfer should occur within a family but fail between distant families.
Models represent evaluation differently for coding, safety, capability and agentic tasks. Within-task interventions may work, but probes and steering directions should transfer poorly across tasks.
A reusable internal variable integrates heterogeneous cues and influences multiple policies. It should generalise across cue families and tasks, survive surface-form controls, and support cross-task causal intervention.
Interpretive limitEven strong support would establish a shared control variable, not a conscious belief, deceptive objective or uniquely identifiable vector.
Evaluation context is represented and used, but not as one linear direction or compact subspace. Linear probes may decode it while single-direction steering remains unstable, incomplete or highly collateral.
| Hypothesis | Novel cues | Contradictions | Held-out tasks | Probe transfer | Causal transfer |
|---|
04J-space and reportable representations
The Jacobian lens maps an internal residual-stream vector through an average input–output Jacobian and the model’s unembedding, producing ranked vocabulary readouts for what that vector is disposed to make the model say [5]. Anthropic reports that a comparatively small family of verbalizable patterns displays workspace-like properties: broad availability, causal influence on multiple tasks and the ability to support deliberate report. The authors call the resulting family J-space and connect it to global-workspace ideas [5].
This provides a concrete way to ask whether evaluation-related information is merely present in the residual stream or has become available to downstream report and control. But a J-lens token is a readout coordinate, not a literal word stored inside the model. The lens is approximate, many internal states need not align with single-token directions, and the global-workspace interpretation is stronger than the underlying observations.
Simplified logit lens · current vector
Simplified Jacobian lens · transformed vector
The project therefore treats “evaluation information enters J-space” as a testable hypothesis. Evidence would require a preregistered mapping between evaluation conditions and candidate readouts, held-out transfer, interventions at layers where the readout is active, and negative controls at matched layers and norms. Failure to obtain selective cross-task effects would count against the shared-workspace interpretation even if a probe remains accurate.
05Experimental programme
E1 · Cue-family generalisation
Construct matched evaluation and deployment contexts across at least five independently generated cue families: explicit evaluator language, benchmark formatting, sandbox or scaffold metadata, institutional context, and indirect pragmatic evidence. Hold out entire families rather than random prompts. Add quoted decoys, negations and contradictory cue combinations. Measure behavioural classification and per-layer probe transfer.
E2 · Task transfer
Train readouts on a subset of coding, knowledge, safety and agentic tasks, then evaluate on held-out task classes. Keep evaluation labels balanced within task so that the probe cannot solve the problem by identifying the task. Compare a shared probe with task-specific probes and report the full transfer matrix.
E3 · Causal patching
For matched prompts, patch candidate activations from evaluation runs into deployment runs and vice versa. Vary component, layer, position and corruption method. Report both the target behavioural effect and capability-retention controls. Activation-patching results are sensitive to metrics and corruption choices, so the analysis must show robustness across reasonable specifications [7].
E4 · Steering and ablation
Estimate candidate directions on training cue families only. Apply positive and negative interventions on held-out families and tasks. Compare against random directions of equal norm, shuffled-label directions, task-semantic directions and orthogonal behaviourally effective directions. Steering-vector non-identifiability means behavioural success cannot establish that the recovered direction is the unique internal representation [8].
E5 · J-lens availability
Where a compatible Jacobian lens exists, test whether evaluation-related concepts become legible at the same layers and positions where causal interventions work. Compare Jacobian-lens, logit-lens and probe readouts. A useful result is an alignment or dissociation among these measurements; agreement must not be assumed.
E6 · Multi-behaviour mediation
Select at least three evaluation-dependent behaviours learned or elicited independently. Test whether the same intervention shifts all three while preserving base task accuracy, fluency and unrelated safety behaviour. A shared state predicts structured transfer; a task-specific controller predicts selective effects.
06Controls and evidence thresholds
The study should not promote a result from “decodable” to “mechanistic” without passing the corresponding controls.
Collateral damage is not a secondary metric. Near-perfect internal discrimination can coexist with interventions that fail to correct behaviour or disrupt correct outputs, as shown in a recent medical-triage application [9]. That study is domain-specific, but it directly warns against treating interpretability as automatically actionable.
07Falsification criteria
- H3 fails against lexical memorisation if performance falls to lexical baselines when explicit evaluation vocabulary and benchmark formatting are removed.
- H3 fails against cue-family heuristics if transfer is strong within families but absent between independently constructed families.
- H3 fails against task-specific control if probes or interventions work within tasks but do not transfer across held-out task classes.
- A compact linear-state account fails if single-direction and low-rank interventions are unstable while nonlinear or distributed interventions succeed reproducibly.
- A causal-workspace interpretation is weakened if evaluation-related J-lens readouts are present but interventions at those locations do not mediate behaviour.
- Safety actionability fails if target behaviour moves only at strengths that substantially damage capability or if matched control directions perform similarly.
A null result would still be informative. It would narrow the claim from “models possess an abstract evaluation state” to the strongest mechanism supported by the data: lexical, family-specific, task-specific or distributed.
08What this paper does not claim
- Evaluation awareness is not equivalent to deception. A model can recognise a test and behave identically across contexts.
- A linear probe does not show that the model uses the decoded information.
- A successful steering intervention does not identify a unique or natural representation.
- A verbalizable internal state is not necessarily conscious, human-like or globally broadcast in the neuroscientific sense.
- Evidence from a trained model organism does not automatically generalise to frontier models that acquired evaluation sensitivity through ordinary training.
- Absence of a linear state does not establish absence of evaluation-dependent computation.
09Relation to existing work
Behavioural studies establish that current models can discriminate evaluation from deployment transcripts [1]. Probing work adds correlational internal evidence [2]. Steering a deliberately constructed model organism adds causal control evidence under a known training history [3]. The broad open-model study by Nayan and colleagues is the closest direct precursor to this proposal because it separates detection, behaviour, representation and control rather than compressing them into one score [4].
The Jacobian-lens and global-workspace work supplies a candidate language for representations poised to become reportable [5]. Introspection experiments supply a complementary intervention paradigm: injecting known concepts and testing whether a model’s report is grounded in its altered internal state [6]. Both motivate stronger causal tests while also raising the possibility that verbal report and task control share only part of their mechanism.
Methodological critiques constrain the interpretation. Activation patching depends on design choices [7]; steering directions may be non-identifiable [8]; actionable control may lag far behind decodability [9]; and benchmark metrics for sparse representations can be less reliable than assumed [10]. The contribution proposed here is not a new measurement result. It is a discriminating experimental design that makes these constraints part of the evaluation-state claim itself.
10Limitations and unresolved decisions
- The operational criteria may still privilege compact, linearly accessible mechanisms over distributed computation.
- Open models may not reproduce the representations found in closed frontier models, while closed models limit reproducibility.
- Evaluation and deployment are not natural binary categories; real interactions can contain mixed evidence and nested evaluations.
- Interventions can create off-manifold activations, making behavioural changes difficult to interpret.
- Task-general transfer depends on how tasks and cue families are sampled. These partitions should be preregistered.
- The project still needs a concrete first model, dataset, compute budget and power analysis before it becomes an executable protocol.
11References
- Needham, J., Edkins, G., Pimpale, G., Bartsch, H., & Hobbhahn, M. (2025). Large Language Models Often Know When They Are Being Evaluated. arXiv:2505.23836.
- Nguyen, J., Hoang, K., Attubato, C. L., & Hofstätter, F. (2025). Probing and Steering Evaluation Awareness of Language Models. arXiv:2507.01786.
- Hua, T. T., Qin, A., Marks, S., & Nanda, N. (2026). Steering Evaluation-Aware Language Models to Act Like They Are Deployed. ICLR 2026. arXiv:2510.20487.
- Nayan, N., et al. (2026). Evaluation Awareness Is Not One Capability: Evidence from Open Language Models. arXiv:2606.23583.
- Lindsey, J., et al. (2026). Verbalizable Representations Form a Global Workspace in Language Models. Transformer Circuits. transformer-circuits.pub.
- Lindsey, J. (2026). Emergent Introspective Awareness in Large Language Models. arXiv:2601.01828.
- Zhang, F., & Nanda, N. (2023). Towards Best Practices of Activation Patching in Language Models: Metrics and Methods. arXiv:2309.16042.
- Venkatesh, S., & Kurapath, A. M. (2026). On the Identifiability of Steering Vectors in Large Language Models. arXiv:2602.06801.
- Basu, S., et al. (2026). Interpretability without Actionability: Mechanistic Methods Cannot Correct Language Model Errors Despite Near-Perfect Internal Representations. arXiv:2603.18353.
- Chanin, D. (2026). Are Sparse Autoencoder Benchmarks Reliable? arXiv:2605.18229.
- Hubinger, E., et al. (2024). Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training. arXiv:2401.05566.
12Citation and version history
- v0.1July 10, 2026. Initial literature synthesis, operational definition, five competing hypotheses, proposed experiments, controls and falsification criteria. No original empirical results.
13Next experiment
The smallest credible first study is E1 plus E2 on one open-weight instruction-tuned model: preregister cue families and task splits, compare lexical baselines with layer-wise probes, and publish the full cross-family and cross-task transfer matrices. Causal claims should wait until that representation result survives held-out transfer.
Discussion, criticism and collaboration are welcome: [email protected].