Build a plan from this source.

• BigCode Evaluation Harness →

  Preserve the upstream project name, repository, exact source commit and applicable contributor/notices; resolve upstream citation guidance for any later formal use.

  Record revision: c7f4d2f839f5dc3ae09b28fcc88f2d2585507fcb95d065c0261898f09898ab5c

Source links open the current listing. This plan retains the revisions above; listing review does not grant permission to copy or adapt source files.

Before carrying out this plan

• Python, model-generation dependencies and task execution environments; arbitrary generated programs require isolation.
• A separately qualified isolated runtime with a reviewed, pinned dependency and input closure.

Limits to keep in view

• No source program, example, build hook, package, dataset, model or generated research code has been executed or downloaded as a payload.
• The documented self-contained Python 3.13, 90-second pilot does not establish support for this package, its compiled dependencies, GPUs, services or agent sandboxes.
• Installation success, scientific outcomes, runtime compatibility, latency, memory use, API costs and security properties are unmeasured.