Horizon RunMap — Technical Reviewer Brief

Lucas Ribeiro · Technical review entry · Existing public evidence; no new experiment

What Horizon is

Horizon RunMap is an independently conceived and developed experimental Mixture-of-Experts inference runtime. The implemented architecture combines selective compression of routed experts, custom compact-weight GPU execution, complete immutable host backing and budgeted expert residency. Its public evidence concerns specific recorded models, executors and configurations. The runtime source remains private by author decision.

The object to evaluate is this implemented execution architecture and the evidence recorded for it. Full GPU residency and partial GPU residency are operating regimes within that design. When the available budget accommodates the relevant experts, execution uses resident compact weights. With partial residency, a cache miss requires bringing the needed expert representation to the GPU under the declared cache contract. The public campaign identifies which executor was used at each operating point.

This document is a compact route through the existing publication. The English homepage retains the project narrative, engineering notes, audit description, limitations and contact. Detailed prompt outputs are available in separate evidence explorers. The technical report and architecture notes provide more depth; the source-bound machine records are claims.json and architecture.json.

Engineering question

How can storage precision, execution precision, residency and data movement be coordinated without requiring the entire MoE parameter set to occupy GPU memory in expanded form?

Storage precision describes the representation retained for weights. Execution precision describes the values used by the arithmetic path. Residency describes which expert copies are currently available on the GPU. Movement describes the transfers needed to make missing experts available. These decisions interact, but they are separate dimensions: an FP16 activation or execution value does not mean a quantized expert has recovered the original checkpoint's BF16 values.

Memory, time to first token, post-first-token decode rate, request runtime, transfer volume and output behavior are also separate observations. A useful review keeps each result attached to its own definition, denominator, configuration and execution boundary. A lower memory observation does not by itself establish better output quality, and a decode rate does not describe the wait before the first token.

Architectural contract

The published paths selectively represent routed expert weights as INT4 qpacked data, with FP16 scales in the selected campaign. GPU expert copies remain compact. Custom W4A16 execution combines low-bit expert weights with FP16 activations; the recorded executor determines the arithmetic details. Original checkpoint dtype, quantized source format, scale dtype, activation dtype and accumulation dtype are distinct fields.

The non-routed path is kept separate. In the measured paths, non-routed checkpoint BF16 values are materialized as FP16. This is a higher-precision path relative to INT4 expert storage; it is not a claim that the original BF16 values remain unchanged. Low-bit source dequantization cannot recover information already lost through quantization.

For the documented resident-qpacked-cache-v1 contract, complete process-owned host backing is the immutable authority for all routed experts. GPU entries are disposable qpacked copies. A miss moves data from pageable host backing through bounded pinned staging into an unpublished GPU spare. Transfer completion precedes atomic publication of a complete generation. Readers retain a complete generation, and consumer retirement prevents premature reuse.

Eviction discards the GPU copy without writing expert data back to the host source. The complete RAM source remains intact. Staging is bounded, and the documented contract permits at most one in-flight miss transaction. These ownership and publication rules explain the cache's intended semantics; they do not turn required zero counters into observed measurements for an uninspected executor.

The original ADR scope is Qwen P0. A shared architectural description does not automatically certify every historical executor against every invariant. The selected campaign's PRODUCT-core scalar, Grouped T3 and top4-fused-v1 identities therefore remain explicit. The older offload-qpacked mechanism, which materializes expanded FP16/BF16 execution slots from low-bit sources, retains a different meaning from this compact GPU cache contract. See the public technical source map.

What is measured

Three separately published experiments answer different questions. Their validation status, evidence level, publication status and independent reproduction status are separate fields. No new generation, benchmark or scoring change accompanies this reviewer entry.

Residency and executor campaign

The selected E3 campaign contains 13 configurations, 39 workers and 468 measured responses across Qwen1.5-MoE-A2.7B-Chat, OLMoE-1B-7B-0924-Instruct and OLMoE-1B-7B-0125-Instruct. Qwen uses capacities 20, 30 and 39 per routed layer; each OLMoE checkpoint uses 30, 40, 50, 60 and 64. The campaign has repeated blocks, fixed prompt order, excluded warmups, greedy selection and EOS suppression. The residency report and individual result files retain the complete recorded contract.

Pooled decode is the sum of generated tokens after the first token divided by the sum of internal decode seconds. Overall output rate instead uses all generated tokens divided by internal response runtime. TTFT and response time are arithmetic request means. The table preserves the recorded decode aggregation; it is not a ranking against other runtimes. Capacities are per routed layer, not totals across the model.

The hardware context is a shared Windows desktop with an RTX 5070. VRAM is a device-wide whole-worker driver peak, including preparation, warmup and shared desktop activity. RSS is a separate whole-worker process peak. H2D is a measured-request sum, not a transfer rate. Isolated prefill is UNKNOWN, and OLMoE HTTP observation is UNSUPPORTED. Block spread is descriptive, not a confidence interval. Campaign HTML entry.

IFEval instruction following

The complete OLMoE 0924 IFEval population contains 541 prompts and 834 evaluated instructions. Termination: 467 natural EOS and 74 length-limit endings.

Passing a prompt requires all its evaluated instructions to pass. Strict and loose metrics follow the frozen evaluator's distinct rules; loose scoring includes its specified presentation adjustments. All retained responses remain in the population, including length-limit endings. The edition is E1 standalone_benchmark, separately admitted for public publication; that admission does not rewrite the original experiment contract or promote it to E3/E4.

IFEval measures the specified instruction-following checks. It does not establish general factual-answer correctness or paired BF16 preservation. External model-card values on the site are contextual references, not matched baseline arms. All 541 responses and the audit entry provide the scorer, inputs and verification links.

Earlier paired BF16 × Horizon study

Across 120 paired prompts, BF16 conformed in 29/120 cases and Horizon in 28/120. There were 8 conformity losses and 7 gains, 16 identical output-token sequences and 3 equivalent JSON pairs under the frozen numeric tolerance. Natural EOS: BF16 59/120, Horizon 54/120.

Conformity means a completed, non-empty response containing one strict JSON object and ending naturally at EOS. Answer correctness was not graded. Shared failures do not count as regressions attributed to Horizon; changed wording or JSON values alone do not establish worse quality. The frozen scorer distinguishes identical token sequences, equivalent JSON values, directional conformity changes and other divergences.

The contract uses OLMoE-1B-7B-0924-Instruct, matched prompt order and tokenizer input hashes, greedy decoding and up to 128 new tokens. Horizon uses c64 full residency, Grouped T3, INT4 expert sources and FP16 activations. The reference uses original BF16 checkpoint values through Transformers/Accelerate with the recorded CPU/GPU placement. This E3 study is separate from IFEval and from the performance campaign. Inspect all 120 pairs and audit material.

Important experimental boundaries

• OLMoE c30–c60 use PRODUCT-core scalar; c64 uses Grouped T3. Comparing these points is not a residency-only ablation because executor changes simultaneously.
• IFEval and the earlier 120-pair study are separate experiments with different populations, scoring rules and questions. Their scores must not be merged into a single quality result.
• Frontier models motivate the memory problem; they are not the population of the published campaign. The displayed frontier context does not validate Horizon deployment at that scale.
• Runtime-to-runtime superiority is not claimed. A comparison with another runtime would require its own matched model, hardware, precision, workload, output policy and timing boundaries.
• Full independent runtime reproduction is not currently possible from the public material. The source remains private by author decision. CPU verification and score recomputation do not rerun generation.
• Broader hardware, concurrency, checkpoint coverage, longer outputs and general quality preservation require their own evidence. Missing observations remain UNKNOWN; unavailable platform or dependency capabilities remain UNSUPPORTED.

What Horizon claims

The following are the existing homepage claims, with the same scope:

• Architecture implemented. Selective compression, custom W4A16 kernels and controlled expert residency integrated in a runtime.
• Compact expert execution measured. Memory, transfers and timing recorded for the published OLMoE and Qwen configurations.
• Partial and full residency measured. Recorded operating points, with each configuration’s executor identified.
• Behavior under selective compression measured. OLMoE 0924 instruction following in IFEval and output conformity in a separate BF16 comparison.

Each measurement belongs to its stated model, precision, executor, hardware and workload. The contribution is the implemented architecture and the evidence recorded for it. The public results do not replace a separate product comparison or establish behavior beyond the declared run contracts.

What Horizon does not claim

The existing homepage non-claims are:

• The fastest MoE runtime.
• State-of-the-art throughput.
• Universal superiority over llama.cpp, vLLM or KTransformers.
• Validated scaling to frontier-size MoE models.

The structured public claims additionally exclude general BF16 quality equivalence, independently reproduced runtime generation and world-first quantization, offloading or caching. These techniques have established prior art. Neither an unresolved property in a comparison table nor a verifier's successful exit establishes architectural novelty.

Closest prior art

The source-scoped comparison matrix, reviewed on 2026-09-16, describes shared techniques and architectural distinctions without a throughput ranking. Its properties refer to the cited paper or implementation path; UNKNOWN means that the consulted source does not settle a property, not that a project lacks it.

Eliseev and Mazur's Mixtral offloading work is a close architectural precedent: mixed low-bit experts, host backing, GPU expert caching, per-layer LRU and speculative loading. These overlap with Horizon's broad techniques. The comparison instead examines representation, ownership, transfer direction and publication semantics; it does not claim the earlier system is inferior.

MoE-Infinity documents host/SSD offload, activation-aware caching, prefetching and compact execution paths. Its documentation distinguishes the released implementation from the paper version. KTransformers documents heterogeneous CPU/GPU expert computation and quantized paths; Horizon's documented host role here is immutable authority and transfer coordination, with expert execution on GPU.

llama.cpp placement controls expose CPU MoE and GPU layer/tensor choices. Those options alone do not define a dynamic expert cache's publication or writeback semantics. Horizon makes complete immutable RAM authority, compact copies, bounded staging, H2D-only misses and discard eviction explicit in its cache contract. Related implementations may share individual properties; the publication does not claim that this set is unprecedented.

Auditability and code availability

The performance verifier checks published hashes and consistency of curated aggregate copies. That release does not contain the complete raw-run bundle needed to recompute every performance observation. The paired-study and IFEval verifiers recompute their declared classifications or scores from retained responses on CPU. They validate retained artifacts and scoring, not a fresh independent execution of the private runtime.

The paired publication preserves the original campaign's sanitizer failure in provenance. Its successor recovered the original responses in CPU post-processing with unchanged scoring; it did not rerun generation. Each public edition links to its own manifest and audit method. Hashes establish byte correspondence with the declared inventory, not independent reproduction or universal correctness.

Code availability

The runtime source code remains private by author decision. The public site provides experimental artifacts, technical documentation and verifiers within their declared scopes. Full independent runtime reproduction is not currently possible from the public material. Independent generation reproduction has not been attempted for the three published editions. Engineering and contact.

Evidence links

• Technical report (HTML)
• Technical report (Markdown)
• Technical report (PDF)
• Campaign data (JSON)
• Residency report (Markdown)
• Technical notes (Markdown)
• Audit method (Markdown)
• Evidence audit (Markdown)
• Manifest and SHA-256 hashes
• Aggregate and hash verifier (Node.js)
• Performance campaign HTML entry
• IFEval explorer
• IFEval raw data and audit links
• Paired-study explorer and audit material
• Paired-study evidence manifest and hashes
• IFEval evidence manifest and hashes
• claims.json
• architecture.json

The HTML entries are the discovery layer; JSON, Markdown, manifests, hashes and verifier files are the evidence layer. Use each edition's audit instructions and keep its scope attached to any conclusion.