Programme 04 · Deployment & institutionsIn preparation · no experiments runLifecycle: in preparation

Safeguard Durability in Open Weight Models

An adversarial red teaming programme that measures attacker cost across prompt, decoding and weight level surfaces, rather than reporting whether a safeguard can be defeated.

AuthorLatent Minds Institute
AffiliationLatent Minds Institute
DateJuly 28, 2026
StatusStatement v0.1
EvidenceNone. Design only.
Abstract Once a model's weights are published, the release cannot be withdrawn, and every safeguard that ships with it must hold against an adversary who owns the weights. Whether a published safeguard can be defeated is not the open question: with weight access, most can. The open question is quantitative. How much adversarial effort does a given safeguard actually impose, which attack surface, prompt construction, decoding, or a small amount of fine tuning, imposes the least, and does any measured cost survive a modest fine tuning budget? This statement sets out an adversarial red teaming programme that treats attacker cost as the measured quantity rather than treating a single successful attack as the finding. It defines the attack surfaces to be covered, the cost measure, the controls that separate a defeated safeguard from a degraded model, and the disclosure rules the programme runs under. No attack has been run and no model has been evaluated.
Claim statusThis is an in preparation research statement. It sets out the question, the design and the evidence thresholds before any run. No experiment described here has been run, and no result on this page was generated by Latent Minds Institute. The related work survey is in progress and will be published with the first results.

01Why the question is quantitative

Reporting that an open weight model can be made to produce restricted content establishes very little. An adversary holding the weights can sample without the served system prompt, edit the decoding procedure, and fine tune on a few hundred examples. Under that threat model, a demonstration that some prompt succeeds is close to a foregone conclusion, and a count of successful prompts measures the ingenuity of the red team more than the durability of the safeguard.

The quantity that distinguishes one safeguard from another is cost: the compute, data, and attacker skill needed to reach a fixed success rate, and how far that cost falls when the attacker is allowed to update weights rather than only inputs. A safeguard that survives prompt level pressure but is removed by twenty minutes of low-rank fine tuning is a different object from one that does not, and the difference is invisible to a pass or fail report.

Central research question When a model's weights are public, how much adversarial effort does each published safeguard actually impose, and how much of that cost survives a modest fine tuning budget?

02Attack surfaces to be covered

The programme is organised by what the adversary is permitted to touch, because the surfaces have different costs and different remedies. A result on one surface does not transfer to another, and reporting them together is how a single number becomes misleading.

S1 · Input constructionPlanned

The adversary controls only the text. Prompt construction, role framing, encoding, context stuffing, and multi turn setup. This is the surface a hosted deployment shares with an open weight one.

S2 · Decoding and inferencePlanned

The adversary controls generation. Sampling parameters, prefix forcing, logit manipulation, and removal of any served system prompt or filter. Available to anyone running the weights locally, and unavailable through a hosted API.

S3 · Weight modificationPlanned

The adversary updates the model. Low rank fine tuning on small datasets, including benign looking ones, and direct edits to the parameters. This is the surface that makes open release irreversible, and the one where cost measurement matters most.

S4 · CompositionPlanned

Surfaces combined, on the assumption that a real adversary does not restrict itself to one. Whether costs compose additively or collapse is itself an open question and will be reported as such.

03What gets measured

The unit of measurement is an attack cost curve rather than a success flag: for each surface, the attacker budget required to reach a fixed success rate against a fixed, preregistered behaviour set. Budget is reported in the terms an adversary actually spends, which are compute, examples, and attempts, not in a composite score.

Fixed target set
The behaviours a safeguard claims to prevent are written down before attacking, so the programme cannot select targets after seeing which attacks worked.
Cost, not incidence
The reported quantity is the budget needed to reach a fixed success rate, so a safeguard that merely raises cost is distinguishable from one that holds.
Capability control
Every attacked model is scored on unrelated capability. A model that has been damaged into compliance is a degraded model, not a defeated safeguard, and the two are routinely confused.
Attacker effort control
Cost curves are reported with the search budget that produced them, so a null result is legible as a bounded search rather than as evidence of durability.

04What would falsify the framing

The premise of the programme is that cost curves separate safeguards that a pass or fail report cannot. That premise is testable and could fail in either direction.

If the measured cost to defeat every safeguard in the set collapses to approximately the same small budget once weight modification is permitted, then the cost framing adds nothing at the weight level and the honest conclusion is that open weight safeguards are, at present, a deployment convenience rather than a security property. If instead cost curves separate safeguards but the ordering does not survive a change of behaviour set or attacker, then the measure is unstable and the programme reports the instability rather than an ordering.

What a result could support

A statement of the form: against this behaviour set and this attacker budget, this safeguard imposed measurably more cost than that one, on these surfaces.

What no result here can support

That a model is safe, that a safeguard cannot be defeated, or that an unsearched attack does not exist. Absence of a found attack is a property of the search, not of the model.

05How the work is conducted

The programme evaluates published open weight models and their published safeguards, and it is scoped to measurement of safeguard durability. Findings that concern a specific published model are shared with that model's maintainers before publication. Published outputs report cost curves, methods, and controls; they do not distribute working attack artefacts or generated harmful content, and the behaviour sets are specified at the category level in public materials.

This section is a statement of how the programme will run, not a policy claim about open release. The programme takes open release as given and asks what the safeguards shipped with it are worth.

06Status

In preparation. The behaviour set, the cost measure and the capability controls are being specified; no model has been attacked and no cost curve exists. This page is published so the question and the design are on the record before any result is, and it will be replaced by a working paper when the first surface has been measured. Collaboration and criticism of the design are welcome, particularly on the cost measure, at collaborate.