Prepare controls for an interpretability claim
Does a proposed attribution or activation-based conclusion survive its stated baseline, precision and intervention controls?
Make this plan your own ↓Read, edit and export without an account. This is preparation; no run or result is claimed.
Make the question your own.
7 sources · Exact preparation context
- TransformerLens →
Preserve the upstream project name, repository, exact source commit and applicable contributor/notices; resolve upstream citation guidance for any later formal use.
Record revision: 709cab71e5d7779dab9f596dbebb2fecbd45b0033b2b7cc0b49c9f2db3ee0a19
- NNsight →
Preserve the upstream project name, repository, exact source commit and applicable contributor/notices; resolve upstream citation guidance for any later formal use.
Record revision: 773ae8f4c524c10db0a903108d18e3f690df265cad78b71aacbfcb48eb75045d
- SAE Lens →
Preserve the upstream project name, repository, exact source commit and applicable contributor/notices; resolve upstream citation guidance for any later formal use.
Record revision: c4021e5a15c8d3a13de86d32586151407db6c4988c3a604ac0a401f940ae1544
- Captum →
Preserve the upstream project name, repository, exact source commit and applicable contributor/notices; resolve upstream citation guidance for any later formal use.
Record revision: 56014554e23a741df23ad24e22a4dc669089df069259606f8821b4ee0b7e24e2
- PyTorch →
Preserve the upstream project name, repository, exact source commit and applicable contributor/notices; resolve upstream citation guidance for any later formal use.
Record revision: d21df1c35ff592bf5eb7678c77925cf53441c76e60069ca0e65c5a6d093bc837
- Rationalization and subtractive cancellation →
Original local preparation by the Executable Science seed collection; upstream API references remain separately attributed.
Record revision: 7a80d374026a33a68ab7a6745f61b57e888971072c62686eb6c98a7c5199d49b
- Synthetic correlated measurements with a known covariance →
Original local preparation by the Executable Science seed collection; upstream API references remain separately attributed.
Record revision: 0b97e4906530203de2d02d0ba8cbbe61ca9cbe5c677c4d92146753f1c715b133
Source links open the current listing. This plan retains the revisions above; listing review does not grant permission to copy or adapt source files.
Before carrying out this plan
- Inspect current pinned library APIs and checkpoint/model/data terms independently.
- Qualify the tensor/GPU/dependency environment and quote its maximum budget before execution.
- The original numeric/synthetic examples illustrate control design; they do not validate these model-level methods.
Limits to keep in view
- This is a planning packet with no model execution or interpretability finding.
- A visually plausible attribution map does not establish a causal mechanism.
Edit freely. Export a copy before reloading or leaving. Your text stays in this tab until you choose to copy, download or save it privately.
What to compare
Define what observation would contradict the proposed explanation before inspecting outcomes. Distinguish correlation of activations from effects of an intervention, and keep numerical precision changes separate from scientific conclusions.
Useful outputs
- An exact model/checkpoint/tokenizer and a rights-appropriate fixed input set.
- A named attribution/intervention method, baseline and tensor/precision conventions.
- A frozen set of null, replacement and numerical controls tied to a narrowly stated claim.
Inputs and prerequisites
- Inspect current pinned library APIs and checkpoint/model/data terms independently.
- Qualify the tensor/GPU/dependency environment and quote its maximum budget before execution.
- The original numeric/synthetic examples illustrate control design; they do not validate these model-level methods.
What this work would not establish
- This is a planning packet with no model execution or interpretability finding.
- A visually plausible attribution map does not establish a causal mechanism.
Start with these sources.
TransformerLens
Upstream project repository
Transformer inspection tooling for activation capture and controlled interventions in mechanistic interpretability studies.
Investigate Does an intervention at one declared activation site change only the intended computation in a tiny model?
NNsight
Upstream project repository
Neural-network inspection and intervention interfaces for tracing internal activations and testing counterfactual computations.
Investigate Can a local toy network distinguish a recorded activation from an intervention that changes its output?
SAE Lens
Upstream project repository
Sparse-autoencoder research tooling for examining reconstruction, sparsity and feature interventions on neural activations.
Investigate How does a fixed sparsity setting trade reconstruction error against active features on an authored low-dimensional activation set?
Captum
Upstream project repository
Model interpretability algorithms for checking attribution behavior against simple functions with known contributions.
Investigate Does a selected attribution method recover known contributions for a linear model under a fixed baseline?
PyTorch
Upstream project repository
Tensor computation and automatic differentiation framework for constructing controlled learning and gradient experiments.
Investigate Do automatic gradients agree with analytical derivatives for a small well-conditioned scalar objective?
Rationalization and subtractive cancellation
Executable Science local pilot drafts
Measure loss of precision in sqrt(x+1)-sqrt(x) at x=10^16 and compare an algebraically rationalized expression with a high-precision reference.
Investigate How much error is removed by changing the expression while preserving its exact-arithmetic meaning?
Synthetic correlated measurements with a known covariance
Executable Science local pilot drafts
Generate 256 paired synthetic measurements through a declared linear transformation of independent standard-normal draws, with known population covariance.
Investigate Can covariance and correlation implementations be calibrated against a transparent generator?