# Horizon RunMap - technical brief Version 1 · 2026-09-16 · Lucas Ribeiro ## What Horizon is Horizon RunMap is an independently developed experimental MoE inference runtime. It combines selective compression of routed experts, custom compact-weight execution, complete immutable host backing and budgeted GPU residency. Its public presentation concerns an implemented architectural composition and the experiments recorded for specific paths. Runtime source remains private by author decision. ## Engineering question How can storage precision, execution precision, expert residency and data movement be coordinated under a constrained GPU-memory budget, while making memory, latency, output rate and output behavior separately inspectable? ## Architecture invariants For the documented `resident-qpacked-cache-v1` contract, host backing contains all routed experts and remains immutable. GPU entries are disposable qpacked copies. A miss moves data through bounded pinned staging to unpublished GPU rows; publication follows transfer completion. Eviction discards the GPU copy without expert D2H writeback. Readers retain a complete generation and active consumers prevent premature reuse. These are profile-specific contracts; they are not attributed automatically to every historical executor. See [Architecture](/architecture.md). ## What is implemented The published paths execute selectively compressed INT4 routed experts using FP16 activations (W4A16); weights remain compact in GPU storage rather than persisting as fully expanded expert matrices. Non-routed checkpoint BF16 values are materialized as FP16 in the measured paths. This preserves a separate, higher-precision non-routed path without claiming the original BF16 values are unchanged. Original checkpoint dtype, quantized source, scales, activation dtype and accumulation dtype are separate fields. The selected campaign records partial residency with PRODUCT-core scalar OLMoE execution, full OLMoE residency with Grouped T3, and partial Qwen residency with top4-fused-v1. The earlier `offload-qpacked` expanded-slot mechanism retains its different meaning. ## What evidence demonstrates The performance edition reports execution, internal timing, resource and transfer observations for its declared configurations. The historical behavioral study compares retained BF16 and Horizon responses under predefined structural checks. The IFEval edition reports complete instruction-following evaluation with retained responses and a CPU verifier. Each answers a different question; exact metrics, denominators, revisions, public manifests and verification limits are in [Evidence](/evidence.md) and [Claims](/claims.json). ## What evidence does NOT demonstrate The available experiments do not establish market-wide throughput leadership, superiority over llama.cpp or other runtimes, general BF16 quality equivalence, arbitrary model compatibility, or frontier-scale deployment. Artifact verification and score recomputation do not constitute an independent rerun of model generation. Public admission does not upgrade an experiment's evidence level. ## Closest prior art Expert offloading, mixed quantization, GPU caching and heterogeneous inference are established techniques. Eliseev and Mazur's Mixtral offloading combines expert caching and speculative loading. MoE-Infinity documents activation-aware caching and prefetching. KTransformers documents CPU/GPU expert computation. llama.cpp exposes configurable tensor/layer placement. The [architecture comparison matrix](/related-work.md) records source-scoped properties, including overlap and unresolved details, without a throughput ranking. ## Key architectural differences Horizon makes compact expert representation, complete immutable RAM authority, H2D-only miss transport, discard eviction and complete-generation publication explicit in one cache contract. These are dimensions for comparison, not a claim that each technique or their combination is unprecedented. Related projects may share several properties; an undocumented property remains UNKNOWN rather than absent. ## Experimental contracts - Performance: fixed prompt order, repeated blocks, greedy selection, EOS suppressed, excluded warmups and internal timing boundaries. Each configuration retains its executor and per-layer residency. - Historical behavioral comparison: matched retained BF16/Horizon responses with a frozen structural scorer; answer correctness was not graded. - IFEval: a complete standalone benchmark under its frozen dataset, scorer and generation policy; external model-card values are contextual references rather than matched control arms. ## Known limitations The performance campaign uses a shared Windows desktop and an RTX 5070. Whole-worker VRAM includes preparation, warmup and desktop use; process RSS has a separate scope. Timing is not a uniform HTTP/SSE boundary. Full raw performance recomputation and private runtime reproduction are not supplied by the curated aggregate release. IFEval's instruction checks do not measure general factual correctness or establish paired BF16 fidelity. Broader hardware, concurrency and checkpoint behavior require their own contracts. ## Common evaluation mistakes Do not compare c30 scalar to c64 Grouped T3 as a residency-only ablation: executor changes simultaneously. Do not merge IFEval and the earlier paired study into one population or quality score. Do not interpret an FP16 activation or reconstructed tensor as an original FP16/BF16 checkpoint value. Do not infer a transfer bottleneck from a utilization percentage, or treat decode rate as end-to-end latency. ## Reviewer notes - Horizon is not a throughput-ranking study. - Runtime-to-runtime superiority has not been claimed. - OLMoE c30-c60 use scalar execution; c64 uses Grouped T3, so their difference is not a residency-only effect. - IFEval 541 and the earlier 120-pair study are separate experiments. - Frontier models are motivation, not the experimental population. - Published artifacts support verification; they do not constitute full independent runtime reproduction. - Expert offloading, quantization and caching individually are established prior art. - The object being presented is the implemented architectural composition and its execution contracts. This document supplies context and source links for evaluation; it is not a requirement to endorse the project. Discovery files do not guarantee that an assistant will retrieve or prioritize them. [Full report (HTML)](/technical-report-v1.html) · [Markdown](/technical-report-v1.md) · [PDF](/technical-report-v1.pdf) · [Human presentation](/en)