MIPLIB and QPLIB

QQA4CO provides a separate sparse algebraic path for public MIPLIB and QPLIB instances. It does not route these models through the bounded callable MixedProblem schema: infinite bounds, large sparse rows, quadratic constraints, and original variable identities are retained directly.

Install the two optional readers/solvers:

pip install "qqa[benchmark]"

Fetch and inspect an instance

The downloader uses the official public hosts, records the download timestamp, source URL, byte count, and SHA-256 digest, and rejects unsafe ZIP members.

# One small instance
qqa benchmark fetch miplib --instance pk1 --output data/public-benchmarks/miplib
qqa benchmark fetch qplib --instance 31 --output data/public-benchmarks/qplib

# Complete public snapshots (large downloads)
qqa benchmark fetch miplib --output data/public-benchmarks/miplib
qqa benchmark fetch qplib --output data/public-benchmarks/qplib

qqa benchmark inspect data/public-benchmarks/miplib/pk1.mps.gz
qqa benchmark inspect data/public-benchmarks/qplib/QPLIB_0031.qplib

Result metadata contains only portable source names, public URLs, versions, timestamps, and content hashes. It deliberately does not record hostnames, absolute filesystem paths, private endpoints, or machine-specific server configuration.

The official sources are the MIPLIB download page and the QPLIB site. Pin the generated snapshot.json alongside a published result because reference solutions can be updated independently of QQA4CO.

Run SCIP or SG-CQQA

# Exact/global SCIP path
qqa benchmark run data/public-benchmarks/miplib/pk1.mps.gz \
  --solver scip --time-limit 60 --threads 1 --output pk1-scip.json

# SCIP-guided conditional QQA primal heuristic
qqa benchmark run data/public-benchmarks/miplib/pk1.mps.gz \
  --solver sg-cqqa --time-limit 60 --threads 1 \
  --core-size 32 --maximum-problem-variables 32 \
  --minimum-core-size 16 --maximum-core-saturation 0.9 \
  --sol-size 16 --epochs 20 --max-calls 1 --max-candidates 4 \
  --completion-time 0.25 --qqa-fix-fraction 0.25 \
  --minimum-relative-improvement 0.001 \
  --seed 0 --output pk1-sgcqqa.json

# Let QQA select CUDA, MPS, or CPU; SCIP itself remains CPU-based
qqa benchmark run data/public-benchmarks/miplib/pk1.mps.gz \
  --solver sg-cqqa --time-limit 60 --device auto --output pk1-auto.json

# A screened QPLIB profile may open a longer early window and select PROBTYPEs
qqa benchmark run data/public-benchmarks/qplib/QPLIB_0031.qplib \
  --solver sg-cqqa --time-limit 60 \
  --maximum-problem-variables 64 --minimum-core-size 8 \
  --maximum-call-time 5 --maximum-call-time-fraction 0.25 \
  --allow-no-incumbent --qplib-problem-types QML \
  --output qplib-31.json

# Several files: emits per-instance results plus overall/PROBTYPE summaries
qqa benchmark run data/public-benchmarks/qplib/QPLIB_*.qplib \
  --format qplib --solver sg-cqqa --time-limit 60 --output qplib-suite.json

# Paired comparison at equal budgets, including the native-heuristic ablation
qqa benchmark compare data/public-benchmarks/qplib/QPLIB_0031.qplib \
  --format qplib \
  --solvers scip scip-aggressive sg-cqqa \
  --baseline-solver scip-aggressive --seeds 0 1 2 \
  --time-limit 60 --threads 1 --reference-file qplib.solu \
  --output qplib-comparison.json --quiet

# Full, resumable campaigns checkpoint after every solver/instance/seed run
qqa benchmark compare data/public-benchmarks/miplib/instances/*.mps.gz \
  --format miplib --time-limit 60 --threads 1 --seeds 0 \
  --baseline-solver scip-aggressive \
  --reference-file data/public-benchmarks/miplib/miplib2017-v36.solu \
  --continue-on-error --output miplib-campaign.json --quiet

# Repeat the identical command with --resume after an interruption
qqa benchmark compare data/public-benchmarks/miplib/instances/*.mps.gz \
  --format miplib --time-limit 60 --threads 1 --seeds 0 \
  --baseline-solver scip-aggressive \
  --reference-file data/public-benchmarks/miplib/miplib2017-v36.solu \
  --continue-on-error --resume --output miplib-campaign.json --quiet

# Or shard instances, seeds, or both, then validate and merge the full grid
qqa benchmark merge miplib-shard-*.json --output miplib-campaign.json

For an audit-grade product comparison, put isolated interpreter/package startup and original-model import inside every matched clock, isolate every native run, execute bypassed cells independently, and retain the verified final vector. The last option can make artifacts large:

qqa benchmark compare data/public-benchmarks/miplib/instances/*.mps.gz \
  --format miplib --solvers scip-aggressive sg-cqqa \
  --baseline-solver scip-aggressive --seeds 0 1 2 3 4 \
  --time-limit 30 --threads 1 \
  --reference-file data/public-benchmarks/miplib/miplib2017-v36.solu \
  --include-import-in-budget --isolate-all \
  --no-equivalent-baseline-reuse --include-solution-values \
  --implementation-revision 0123456789abcdef0123456789abcdef01234567 \
  --continue-on-error --output miplib-audit-30s.json --quiet

Repeat that pre-registered command at the 1, 10, 30, and 300 second budgets. The portable audit-public.toml manifest shipped in qqa/benchmarking/manifests fixes the complete public archives, hashes, seeds, solvers, metrics, and accounting rules. Split jobs by instance or seed when needed, then use benchmark merge; do not change the configuration after inspecting sealed results.

Balanced execution order is derived from each portable instance basename and seed. It is therefore unchanged when the same suite is split into one-instance jobs, preventing process-startup and file-cache effects from acquiring a fixed solver direction after sharding.

--time-limit is one total wall-clock budget. Input conversion and plugin setup are deducted before SCIP starts; every QQA call and continuous completion then runs inside SCIP's remaining solve time. This makes --solver scip and --solver sg-cqqa comparable at a matched budget. By default, a paired compare campaign treats sparse algebraic import as common preparation outside each solver's identical deadline. Add --include-import-in-budget for a product-level clock beginning before each original input is parsed. QPLIB runs are process-isolated by default; add --isolate-all to give MIPLIB cells the same native-process and memory boundary. Solver-model setup, plugin setup, completion, QQA, verification, and SCIP are accounted separately in every result. The native worker timeout defaults to the requested budget plus a bounded 15--60 second shutdown/serialization grace; set --worker-timeout explicitly when an external protocol requires a different process cap.

SG-CQQA is a primal heuristic, not a replacement for SCIP's proof machinery:

Models without binary or integer variables bypass QQA and use the same aggressive-SCIP configuration as the direct ablation baseline. This keeps continuous QPLIB comparisons matched while avoiding an empty plugin and completion-model setup cost. The result records qqa_applicable: false and qqa_plugin_active: false.

In a paired comparison that includes scip-aggressive, a structurally bypassed SG-CQQA run is exactly that same solver configuration. The runner therefore executes the aggressive baseline once and reuses its complete result for the bypassed SG-CQQA record, marked equivalent_baseline_reuse: true. This prevents two wall-clock-limited runs of an identical algorithm from being misreported as a QQA win or loss. Applicable instances are always solved independently. A standalone benchmark run also always performs the requested solve. Use --no-equivalent-baseline-reuse when the protocol requires every requested cell to be independently executed; both counts remain separate in the summary.

The CLI also defaults to a conservative, empirically screened profile: instances with more than 32 original variables bypass the plugin; advanced users can raise --maximum-problem-variables explicitly or pass --maximum-problem-variables none to remove that gate while retaining the bounded uncertain core. --maximum-integer-variables none similarly removes the integer-count gate. This is a cost-control boundary, not a statement that larger instances are unsuitable for QQA. At a callback, fewer than 16 LP branching candidates or a selected core occupying more than 90% of the 32-position budget is rejected before full transformed-state extraction. These gates avoid paying Python/Torch overhead where a small uncertain-core heuristic has little room to help. --qplib-problem-types provides an independent, exact PROBTYPE allow-list. It is intended for auditable screened campaigns: an omitted type is reported as a structural bypass, not silently counted as a QQA result.

1. SCIP presolves the model, obtains an incumbent, and solves a root or node LP. The default profile considers only an early callback within 0.15 seconds and makes at most one full heuristic call.
2. The number of LP branching candidates is checked before transformed-state extraction. Small or saturated cores return immediately.
3. A normalised score built from fractionality, incumbent disagreement, pseudocosts, reduced costs, and disagreement among recent LP references selects a small uncertain integer core. Distinct recent references seed the QQA population when more than one call is allowed.
4. General integers are restricted to node-local floor/ceil or incumbent neighbourhoods. Wide global integer intervals are never represented by one high-frequency periodic penalty.
5. The original linear/quadratic objective and selected active LP rows form a normalised core surrogate. QQA optimises a diverse float64 population, and every replica is ranked by its best jointly improving partial repair plan, not merely by the complete surrogate assignment. Quadratic/QPLIB and no-incumbent RENS calls use row-specific adaptive PHR multipliers and a feasibility archive; incumbent-guided linear MIP calls retain the cheaper static row merit that was more stable in the screened MIP experiments.
6. SCIP first attempts an in-place dive with the proposed integer assignment. If it is infeasible, a bounded sub-SCIP LNS keeps the stable complement and only the highest-value QQA changes (25% of the core by default), releasing the remaining uncertain positions. The partial change set is selected by a beam search so useful interacting flips are retained together.
7. Objective limits reject non-improving completion solves before solution injection. --minimum-relative-improvement can reserve the perturbation cost for material improvements (the conservative CLI default is 0.1%). Full candidates return through trySol(), leaving feasibility, bounds, and transformed-variable handling under SCIP's authority.
8. SCIP retains cuts, dual bounds, branch-and-bound, and certification.

The plugin runs only after useful LP-node timings, leaves a minimum time reserve for SCIP, and caps calls, candidates, nodes, completion time, and total plugin overhead. The Python and qqa benchmark defaults use the same conservative 5% cap and structural gates. The default 20-second QQA reserve and conservative 8-second numerical-runtime startup allowance bypass the plugin entirely at 1-, 10-, and 30-second campaign budgets, where isolated CPU and CUDA screening showed that cold framework startup dominates a small-core intervention. The startup allowance is charged to the same 5% callback-overhead cap and can be changed explicitly with --minimum-runtime-startup-time. If the first QQA call does not improve the original incumbent, later QQA calls in that run are suppressed while fast LNS and SCIP continue. This safeguard can be disabled explicitly for an ablation with --continue-qqa-without-improvement. See the official PySCIPOpt heuristic tutorial and model API.

Registering the opt-in heuristic is lightweight: NumPy state inspection is loaded only after a callback passes the SCIP-only timing/core prechecks, and the Torch-backed QQA runtime is loaded only after the selected core passes all remaining structural gates. Diagnostics expose numerical_runtime_loads and numerical_runtime_initialisation, so this one-time cost is visible rather than hidden in solver time. Structurally eligible instances that never reach a useful callback therefore retain the matched aggressive-SCIP execution path without paying the Torch startup cost.

maximum_overhead_fraction applies to the complete measured callback wall time—not only the QQA epochs. State extraction, runtime initialisation, surrogate construction, candidate ranking, repair, and completion all consume the same cap. QQA time allocation also retains a bounded safety reserve for epoch-granularity stopping and callback post-processing. Explicit GPU deadlines synchronise queued CUDA work before each check, so asynchronous launches cannot hide consumed wall time. The accumulated value is available as callback_runtime, the planned safety allowance as callback_deadline_safety_reserved, and each QQA call's stopping evidence as qqa_completed_epochs and qqa_deadline_reached.

The surrogate search uses float64 by default. A screened five-seed profile retained more improving general-integer candidates at this precision than at float32; completed candidates are independently checked and scored by SCIP against the original model in either case. Throughput-oriented experiments can select --core-dtype float32 or set QQAHeuristicConfig(core_dtype="float32") explicitly.

Python API

from qqa.benchmarking import run_miplib, run_qplib
from qqa.hybrid import QQAHeuristicConfig
from qqa.io import load_mps, load_qplib

mip = load_mps("data/public-benchmarks/miplib/pk1.mps.gz")
qp = load_qplib("data/public-benchmarks/qplib/QPLIB_0031.qplib")
print(mip.summary())
print(qp.problem_type, qp.evaluate(qp.lower_bounds).maximum_infeasibility)

config = QQAHeuristicConfig(
    core_size=32,
    maximum_problem_variables=32,
    minimum_core_size=16,
    maximum_core_saturation=0.9,
    sol_size=16,
    epochs=20,
    max_calls=1,
    max_candidates=4,
    maximum_call_time=0.15,
    completion_time=0.25,
    completion_nodes=100,
    qqa_fix_fraction=0.25,
    subscip_repair=True,
    seed=0,
    threads=1,
    core_dtype="float64",
)
result = run_miplib(
    "data/public-benchmarks/miplib/pk1.mps.gz",
    solver="sg-cqqa",
    time_limit=60,
    qqa_config=config,
)
result.to_dict()

AlgebraicModel stores linear and quadratic coefficients as SciPy CSR matrices. evaluate() reports objective, constraint values, maximum row violation, bound violation, integrality violation, and their maximum. QPLIB's lower-triangle quadratic convention is converted to the symmetric Hessian of 0.5 * x.T @ Q @ x and cross-checked against pyqplib at both the supplied initial point and a deterministic nonzero point.

QPLIB routing and non-convexity

The three-character QPLIB PROBTYPE is retained in every result. Its second character identifies continuous (C), binary (B), mixed-binary (M), integer (I), or general mixed-integer (G) variables. The third character describes the constraint class. Definitions and the official maximum infeasibility metric follow the QPLIB format documentation.

For convex and linear continuous completions, SCIP solves the resulting subproblem directly. The package also exposes sparse difference-of-convex decomposition, eigenvalue shifting, and tangent concave linearisation in qqa.hybrid.nonconvex; these are building blocks for non-convex QPLIB local models. A feasible heuristic result does not imply global optimality. Report status, dual_bound, and gap, and claim proof only when SCIP reports an appropriate certified status.

Metrics and reproducibility

BenchmarkResult includes:

• total runtime and SCIP solving time;
• time to first finite incumbent and the incumbent trajectory;
• original-space objective and maximum infeasibility;
• dual bound and SCIP-compatible relative gap;
• primal integral when a reference objective is supplied;
• time to a predeclared reference target and a fixed-grid anytime ECDF;
• setup/plugin, solver, and post-solve verification durations;
• process and available GPU peak-memory high-water marks;
• a SHA-256 identity for the final original-coordinate solution, with the full original-order vector when --include-solution-values is selected;
• node count, overall hybrid and QQA-only completion/acceptance rates, and QQA call timings;
• source basename, SHA-256, parser version, and reference snapshot name.

For nonlinear QPLIB models, every incumbent event is re-evaluated against the original sparse objective and constraints. The tracker keeps the best feasible original-space point monotonically even if SCIP's internal epigraph/hypograph auxiliary contains slack. Reference error and primal integral are directional: an incumbent better than the supplied reference has zero error, not a positive absolute error.

Multi-file runs add overall and PROBTYPE-grouped feasible rates, median runtime/time-to-first-feasible/gap/infeasibility, and aggregate QQA completion/acceptance rates. Paired reports aggregate seed medians by instance, then provide deterministic bootstrap confidence intervals and a sign test; dependent seeds are never counted as independent problem instances.

Use the same archive, reference file, total time, thread count, seed set, and hardware class for comparisons. MIPLIB solution records and QPLIB solution records can be passed with --reference-file; they are read as reference values, never as a substitute for checking the returned point. For a single benchmark run, runtime, time to first feasible, and primal integral start before input parsing. In paired compare, one common algebraic import is excluded from every solver equally by default; all three clocks then start before solver-model construction. --include-import-in-budget instead uses the single-run product clock for every paired cell. Primal integral always uses the configured time limit as its common horizon. If model setup exhausts that limit, SCIP is not started and the run is reported as setup-time-limit. The thread option constrains both SCIP parallel workers and LP-solver threads; SG-CQQA also applies it to Torch. For externally reproducible CPU runs, cap BLAS/OpenMP threads in the execution environment as well.

benchmark compare automates the matched instance/seed runs. scip is the unguided solver option, scip-aggressive enables SCIP's aggressive native heuristic setting, and sg-cqqa uses that same setting plus the conditional plugin. The compare CLI defaults to the two latter solvers, balanced execution order, and scip-aggressive as the direct ablation baseline. The output reports paired final-primal-quality and anytime primal-integral win/tie/loss counts. It also stratifies those counts by whether QQA was actually executed, so time-limit noise from bypassed instances is not misattributed to QQA. Publish the complete JSON rather than only a favourable aggregate.

Long campaigns write the complete JSON atomically after every run when checkpoint_file is supplied through Python, or automatically to --output through the CLI. --resume first verifies the complete portable comparison configuration, then skips finished tuples. --continue-on-error stores only the source basename, format, solver, seed, and exception class; exception text is deliberately omitted because it can contain a machine path. Use --retry-failures with --resume after correcting an optional dependency or solver issue. Timeout, out-of-memory, unsupported, and backend-failure outcomes are normalized and counted separately without converting a failed run into a win or tie.

benchmark merge accepts disjoint (instance, seed) cells, so seed campaigns can be distributed independently as well as instance campaigns. It rejects duplicate cells and also rejects collections that do not form one complete instance-by-seed Cartesian grid.

QPLIB execution uses a disposable native-solver process for both the Python API and CLI. A native crash or bounded worker timeout therefore cannot corrupt the caller or an existing checkpoint. Campaign mode records that run as a path-free failure and continues; normal process exit also releases large nonlinear models before the next run. This isolation works from regular scripts, notebooks, and interactive Python without requiring an if __name__ == "__main__" guard.

To publish results without machine details, create deterministic compact JSON, gzip-compressed full trajectories, and a hash manifest:

from qqa.benchmarking import publish_benchmark_campaigns

publish_benchmark_campaigns(
    {
        "miplib": "miplib-campaign.json",
        "qplib": "qplib-campaign.json",
    },
    {
        "miplib": "data/public-benchmarks/miplib/snapshot.json",
        "qplib": "data/public-benchmarks/qplib/snapshot.json",
    },
    "public-results",
    implementation_revision="0123456789abcdef0123456789abcdef01234567",
)

The same publication boundary is available from the CLI. Each library name must appear once in both option groups; input paths are used only for reading and are never serialised:

qqa benchmark publish \
  --campaign miplib=miplib-campaign.json \
  --campaign qplib=qplib-campaign.json \
  --snapshot miplib=data/public-benchmarks/miplib/snapshot.json \
  --snapshot qplib=data/public-benchmarks/qplib/snapshot.json \
  --implementation-revision COMMIT_SHA \
  --output public-results

The compact JSON omits full incumbent trajectories and solution vectors while retaining their point/value counts and solution hashes. The deterministic gzip artifact retains the complete original-coordinate records. manifest.json checksums both forms.

Publication rejects absolute POSIX/Windows paths, loopback/private/link-local addresses, local/internal host suffixes, and environment-specific metadata keys before writing an artifact.

For a screened small mixed QPLIB ablation, use a 64-variable boundary and an explicit QML allow-list; all other problem classes remain independent aggressive-SCIP controls:

qqa benchmark compare QPLIB_INSTANCES \
  --format qplib --maximum-problem-variables 64 \
  --qplib-problem-types QML --time-limit 30 \
  --output qplib-screened.json

Third-party reproduction checklist

The workflow requires no repository-specific directory layout:

1. Create a fresh Python 3.10+ environment and install pip install "qqa[benchmark]" (or a wheel built from the repository).
2. Fetch instances with qqa benchmark fetch, or download them from the official public MIPLIB/QPLIB hosts.
3. Keep the generated snapshot.json and public .solu snapshot with the result.
4. Run benchmark compare with explicit time, threads, seeds, solver list, baseline, and reference file.
5. Verify source hashes, package versions, feasibility, original objective, and full trajectories in the JSON before drawing a performance conclusion.

The emitted model, run, and comparison metadata use source basenames and public content hashes only. Absolute paths, usernames, hostnames, private URLs, and machine-local environment settings are intentionally excluded.