Horizon RunMap — Technical Reviewer Brief

Lucas Ribeiro · Current technical review entry · 2026-10-01

What Horizon is

Horizon RunMap is a Mixture-of-Experts inference architecture conceived and implemented independently by Lucas Ribeiro. It combines selective compression of routed experts, custom compact-weight GPU execution and controlled residency in a working runtime. The public app provides an external path to run the architecture and IFEval on supported computers.

The published RTX 5070 campaign records execution, memory and speed. Its OLMoE 0924 full-residency Grouped T3 result recorded 61.26 post-first-token decode tokens/s at 5.99 GiB whole-worker peak VRAM (source result). The separately published IFEval edition includes the complete 541-prompt evaluation and all retained responses. The public app also completed the full IFEval process on another computer with an RTX 4060. Download the app · Explore the evidence.

The object to evaluate is this implemented execution architecture and the evidence recorded for it. Full GPU residency and partial GPU residency are operating regimes within that design. When the available budget accommodates the relevant experts, execution uses resident compact weights. With partial residency, a cache miss requires bringing the needed expert representation to the GPU under the declared cache contract. The public campaign identifies which executor was used at each operating point.

This document is a route through the current publication. The English homepage tells the project story. The current technical report and architecture notes give more depth; claims.json and architecture.json provide structured records.

Engineering question

How can storage precision, execution precision, residency and data movement be coordinated without requiring the entire MoE parameter set to occupy GPU memory in expanded form?

Storage precision describes the representation retained for weights. Execution precision describes the values used by the arithmetic path. Residency describes which expert copies are currently available on the GPU. Movement describes the transfers needed to make missing experts available. These decisions interact as separate dimensions.

Memory, time to first token, post-first-token decode rate, request runtime, transfer volume and output behavior are also separate observations. A useful review keeps each result attached to its definition, configuration and execution boundary, and reads memory alongside output quality and decode rate alongside first-token wait.

Architectural contract

The published paths selectively represent routed expert weights as INT4 qpacked data, with FP16 scales in the selected campaign. GPU expert copies remain compact. Custom W4A16 execution combines low-bit expert weights with FP16 activations; the recorded executor determines the arithmetic details. Original checkpoint dtype, quantized source format, scale dtype, activation dtype and accumulation dtype are distinct fields.

The non-routed path is kept separate. In the measured paths, non-routed checkpoint BF16 values are materialized as FP16, a higher-precision execution path relative to INT4 expert storage. The recorded precision fields distinguish original checkpoint values from values derived after low-bit source quantization.

For the documented resident-qpacked-cache-v1 contract, complete process-owned host backing is the immutable authority for all routed experts. GPU entries are disposable qpacked copies. A miss moves data from pageable host backing through bounded pinned staging into an unpublished GPU spare. Transfer completion precedes atomic publication of a complete generation. Readers retain a complete generation, and consumer retirement prevents premature reuse.

Eviction discards the GPU copy without writing expert data back to the host source. The complete RAM source remains intact. Staging is bounded, and the documented contract permits at most one in-flight miss transaction. These ownership and publication rules explain the cache's intended semantics; they do not turn required zero counters into observed measurements for an uninspected executor.

ADR 0009 originally defined a Qwen P0 cache scope; each later experimental executor retains its own recorded contract. The selected campaign identifies PRODUCT-core scalar, Grouped T3 and top4-fused-v1 separately. The older offload-qpacked mechanism materializes expanded FP16/BF16 execution slots from low-bit sources, while this cache contract keeps GPU expert copies compact. See the public technical source map.

What is measured

Three published experimental editions answer different questions. The public desktop app and the completed RTX 4060 evaluation add a product and external-machine milestone. The tables below retain the original edition contracts and scores.

Residency and executor campaign

The selected E3 campaign contains 13 configurations, 39 workers and 468 measured responses across Qwen1.5-MoE-A2.7B-Chat, OLMoE-1B-7B-0924-Instruct and OLMoE-1B-7B-0125-Instruct. Qwen uses capacities 20, 30 and 39 per routed layer; each OLMoE checkpoint uses 30, 40, 50, 60 and 64. The campaign has repeated blocks, fixed prompt order, excluded warmups, greedy selection and EOS suppression. The residency report and individual result files retain the complete recorded contract.

Pooled decode is the sum of generated tokens after the first token divided by the sum of internal decode seconds. Overall output rate instead uses all generated tokens divided by internal response runtime. TTFT and response time are arithmetic request means. The table presents observations across the campaign's configurations; capacities are per routed layer.

The hardware context is a shared Windows desktop with an RTX 5070. VRAM is a device-wide whole-worker driver peak, including preparation, warmup and shared desktop activity. RSS is a separate whole-worker process peak; H2D sums measured requests. Isolated prefill is UNKNOWN, and OLMoE HTTP observation is UNSUPPORTED. Block spread describes the observed range. Campaign HTML entry.

IFEval instruction following

The complete OLMoE 0924 IFEval population contains 541 prompts and 834 evaluated instructions. Termination: 467 natural EOS and 74 length-limit endings.

Passing a prompt requires all its evaluated instructions to pass. Strict and loose metrics follow the frozen evaluator's distinct rules; loose scoring includes its specified presentation adjustments. All retained responses remain in the population, including length-limit endings. This E1 standalone_benchmark edition was separately admitted for public publication under its original experiment contract.

IFEval measures the specified instruction-following checks; factual-answer correctness and paired BF16 preservation are separate evaluation questions. External model-card values on the site provide context. All 541 responses and the audit entry provide the scorer, inputs and verification links.

Earlier paired BF16 × Horizon study

Across 120 paired prompts, BF16 conformed in 29/120 cases and Horizon in 28/120. There were 8 conformity losses and 7 gains, 16 identical output-token sequences and 3 equivalent JSON pairs under the frozen numeric tolerance. Natural EOS: BF16 59/120, Horizon 54/120.

Conformity means a completed, non-empty response containing one strict JSON object and ending naturally at EOS. Answer correctness was not graded. Shared failures do not count as regressions attributed to Horizon; changed wording or JSON values alone do not establish worse quality. The frozen scorer distinguishes identical token sequences, equivalent JSON values, directional conformity changes and other divergences.

The contract uses OLMoE-1B-7B-0924-Instruct, matched prompt order and tokenizer input hashes, greedy decoding and up to 128 new tokens. Horizon uses c64 full residency, Grouped T3, INT4 expert sources and FP16 activations. The reference uses original BF16 checkpoint values through Transformers/Accelerate with the recorded CPU/GPU placement. This E3 study is separate from IFEval and from the performance campaign. Inspect all 120 pairs and audit material.

Reading the recorded results

• Compare OLMoE c30–c60 within the PRODUCT-core scalar family. The c64 result uses Grouped T3 and answers a different executor question.
• Read IFEval and the earlier 120-pair study as separate experiments with different populations, scoring rules and questions.
• The RTX 5070 campaign provides the published performance values. The app's completed IFEval execution on the other RTX 4060 computer is an additional project milestone.
• Keep each result with its model, executor, hardware and workload. Missing observations remain UNKNOWN; unavailable platform or dependency capabilities remain UNSUPPORTED.

Implemented system and public access

The following capabilities are implemented or measured in the recorded paths:

• Architecture implemented. Selective compression, custom W4A16 kernels and controlled expert residency integrated in a runtime.
• Compact expert execution measured. Memory, transfers and timing recorded for the published OLMoE and Qwen configurations.
• Partial and full residency measured. Recorded operating points, with each configuration’s executor identified.
• Behavior under selective compression measured. OLMoE 0924 instruction following in IFEval and output conformity in a separate BF16 comparison.

• Public desktop app for preparing the pinned model and running a complete IFEval evaluation on supported computers. App and download.
• Complete app execution of IFEval on another computer with an RTX 4060. Project presentation.

The contribution is the integrated execution architecture, its implementation, the recorded results and the public evaluation path.

Scope of interpretation

The published evidence covers the named configurations and checks. Performance, instruction following and the historical structural comparison are separate measurements. Prior work on quantization, offloading and caching provides context for evaluating Horizon's particular composition and implementation. The related-work matrix supports that comparison.

Closest prior art

The source-scoped comparison matrix, reviewed on 2026-09-16, describes shared techniques and architectural distinctions without a throughput ranking. Its properties refer to the cited paper or implementation path; UNKNOWN means that the consulted source does not settle a property, not that a project lacks it.

Eliseev and Mazur's Mixtral offloading work is a close architectural precedent: mixed low-bit experts, host backing, GPU expert caching, per-layer LRU and speculative loading. These overlaps make representation, ownership, transfer direction and publication semantics useful dimensions for examining Horizon's implementation.

MoE-Infinity documents host/SSD offload, activation-aware caching, prefetching and compact execution paths. Its documentation distinguishes the released implementation from the paper version. KTransformers documents heterogeneous CPU/GPU expert computation and quantized paths; Horizon's documented host role here is immutable authority and transfer coordination, with expert execution on GPU.

llama.cpp placement controls expose CPU MoE and GPU layer/tensor choices. Horizon's cache contract explicitly documents complete immutable RAM authority, compact copies, bounded staging, H2D-only misses, discard eviction and complete-generation publication. The matrix compares these properties at the documented implementation level.

Access and verification

The public app lets others run the architecture and the IFEval workflow on supported setups. The performance verifier checks published hashes and consistency of curated aggregate copies. The paired-study and IFEval verifiers recompute their declared classifications or scores from retained responses on CPU. Each release states its own verification method.

The paired publication preserves the original campaign's sanitizer failure in provenance. Its successor recovered the original responses in CPU post-processing with unchanged scoring; it did not rerun generation. Each public edition links to its own manifest and audit method. Hashes establish byte correspondence with the declared inventory, not independent reproduction or universal correctness.

The runtime source code remains private by author decision. The app, technical documentation, retained outputs and verifiers provide public ways to run and inspect the project. Engineering and contact.

Evidence links

• Current technical report (HTML)
• Current technical report (Markdown)
• Current technical report (PDF)
• Historical report v1
• Public evaluation app
• Campaign data (JSON)
• Residency report (Markdown)
• Technical notes (Markdown)
• Audit method (Markdown)
• Evidence audit (Markdown)
• Manifest and SHA-256 hashes
• Aggregate and hash verifier (Node.js)
• Performance campaign HTML entry
• IFEval explorer
• IFEval raw data and audit links
• Paired-study explorer and audit material
• Paired-study evidence manifest and hashes
• IFEval evidence manifest and hashes
• claims.json
• architecture.json

The HTML entries are the discovery layer; JSON, Markdown, manifests, hashes and verifier files are the evidence layer. Use each edition's audit instructions and keep its scope attached to any conclusion.