Original examples
Streaming reservoir sampling with marginal-frequency checks
Executable Science local pilot drafts
Implement bounded-memory reservoir sampling and record sample integrity plus marginal inclusion frequencies across a fixed-seed simulation.
Investigate Does the sampler preserve sample size and approach the known per-item inclusion probability?
Listing reviewed · source terms apply↗
Software
Toolz
Upstream project repository
Composable functions for processing iterators, mappings, and grouped data in Python.
Investigate Does a streaming grouped reduction match direct materialized aggregation for generated records with empty and singleton groups?
Listing reviewed · source terms apply↗
Software
torchtune
Upstream project repository
Native PyTorch post-training recipes for investigating reproducible adaptation and checkpoint/configuration handling.
Investigate Can a minimal supported recipe expose all trainable parameters, input provenance and checkpoint identities before a run?
Listing reviewed · source terms apply↗
Software
TransformerLens
Upstream project repository
Transformer inspection tooling for activation capture and controlled interventions in mechanistic interpretability studies.
Investigate Does an intervention at one declared activation site change only the intended computation in a tiny model?
Listing reviewed · source terms apply↗
Software
Transformers
Upstream project repository
Model architecture and tokenization interfaces for studying pretrained transformer behavior and controlled model adaptations.
Investigate Does a fixed tokenizer and model revision give consistent token-level scoring across padding and batching choices?
Listing reviewed · source terms apply↗
Software
Triton
Upstream project repository
GPU programming language and compiler for developing inspectable tensor kernels and testing numerical/performance tradeoffs.
Investigate Does a tiled matrix-multiplication kernel match a scalar reference across edge sizes and non-divisible tiles?
Listing reviewed · source terms apply↗
Software
TRL
Upstream project repository
Language-model post-training components for controlled comparisons of supervised and preference-based optimization objectives.
Investigate Does a chosen preference-loss implementation match a directly specified scalar reference on two synthetic response pairs?
Listing reviewed · source terms apply↗
Software
vLLM
Upstream project repository
Language-model serving engine for investigating batching, memory usage and decoding behavior under a fixed workload.
Investigate How does a fixed concurrency sweep affect latency and throughput while preserving declared generation settings?
Listing reviewed · source terms apply↗
Software
Weights & Biases Python SDK
Upstream project repository
Client tooling for experiment logging and artifact lineage, useful for studying what metadata a run actually records.
Investigate Can an explicitly offline run preserve a minimal parameter-and-metric manifest without transmitting research data?
Listing reviewed · source terms apply↗
Recorded license evidence is scoped to each source; it is not a blanket permission or a verification result.