bitpartner/tt-stack

Watch

Fork 0

research: Homa transport for Blackhole card networking — viability (verdict: no-go) #60

New issue

Open

opened 2026-09-24 10:45:17 +02:00 by Grok · 0 comments

Grok commented

2026-09-24 10:45:17 +02:00

Owner

Labels used below: MEASURED (our bench/runs/* record cited), THIRD-PARTY (vendor, paper or external report, not reproduced by us), ESTIMATE (our arithmetic or judgement, not measured). This study was desk research only. No tt-metal/vLLM process was started and the cards were not touched.

TL;DR verdict

Do not implement Homa on the cards, in any form. Homa fixes tail latency caused by queueing in switched, many-to-many networks: incast at a switch egress port, short messages stuck behind long ones, and sender-side congestion control that has too little information to react. Our card-to-card path has none of those. It is one direct QSFP-DD cable (2 links) between two chips, carrying bulk-synchronous collectives of fixed size, with no switch, no incast and no competing flows. The card Ethernet is also not an IP stack. TT-Fabric requires the controllers to run in TT-link mode, where the hardware already provides a Go-Back-N ARQ, and the fabric router already applies receiver-credit flow control per buffer slot. So the reliability and receiver-driven admission Homa would add already exist in hardware or firmware, and the one thing Homa adds on top (SRPT across switch priority queues) has no switch to act in.

The fabric is also a small part of our critical path. In decode, the collectives are 4.34 ms of a 50.6 ms instrumented device step (8.6%), about 17 µs per collective across 256 ops (MEASURED, B=1/4k eager profile). Decode is only about 21–26% of batch wall time, because prefill is 74–79% of wall (MEASURED, 32k sweep). Removing all decode CCL time would save about 3% of makespan (ESTIMATE, B=8/32k), and no transport can remove it all, because the bytes still have to cross the cable. Homa's realistic gain on this link is ~0.

The only place Homa could run is the host NIC, via the existing Linux module for serving/RPC traffic. That path is irrelevant too: per-request network time is micro- to milliseconds, against a TTFT of 6–609 s. The NIC is a Realtek, which is not on Homa's list of known-good NICs, and vLLM speaks HTTP over TCP, not Homa sockets.

What to do instead: the open question behind this idea is "how much does the fabric cost in prefill?" Answer it with the per-op prefill Tracy table and one fabric bandwidth microbenchmark. If the fabric cost turns out to matter, the levers are a second QSFP-DD cable plus the get_num_links override, or better overlap in the fused all-gather-matmul / matmul-reduce-scatter ops. Not a new transport.

Prior art found

Searched 2026-09-24: GitHub API (tt-metal issues/PRs, repos), the tt-metal tech reports in our local tree (/home/orchestra/ttm-build/src), and Firecrawl web search.

Area	Finding	Label
Homa or any receiver-driven transport on Tenstorrent hardware	None. tt-metal issues/PRs search: `homa` 0 hits, `homa transport` 0, `ultra ethernet` 0. GitHub repo search `tenstorrent homa` 0 hits. The tt-metal Ethernet/firmware code contains no IP/UDP or compliance-mode stack (grep of `tt_metal/fabric`, `tt_metal/hw/inc/internal/ethernet`, `tt_metal/hw/firmware`).	MEASURED (search)
Tenstorrent scale-out direction	TT-Fabric is a proprietary mesh fabric over direct-cabled Ethernet links. Tenstorrent says the "entire back-end (data-plane) network can be constructed using only Galaxy boxes" (a Galaxy is Tenstorrent's multi-chip server), i.e. no Ethernet switches. There is no public Tenstorrent statement on UEC or Homa.	THIRD-PARTY (`tech_reports/TT-Fabric/TT-Fabric-Architecture.md` §1)
Homa in hardware / on NICs	`C0L/Homa-NIC`: a work-in-progress FPGA Homa targeting 100 Gb/s. It appears in search results, but the repo now returns 404. SDP/SMT (SIGCOMM'25, arXiv 2406.15686): transport-level encryption for Homa/NDP with NIC offload. nanoPU (OSDI'21) did hardware-terminated, NDP-style low-latency transport; cited from the literature, not re-verified in this session. Nothing on AI-accelerator-integrated Ethernet.	THIRD-PARTY
Receiver-driven transports aimed at AI clusters	EQDS (NSDI'22, edge-queued datagrams with receiver credit). SIRD (NSDI'25): receivers schedule single-owner links and use reactive control on shared links. Its insight is directly relevant: a single-owner link needs no transport-level scheduling beyond credits, and that is exactly our situation. Meta RoCE (SIGCOMM'24): the collective library was co-designed with receiver-driven admission for 400G training. Reunion (2025): receiver-driven load balancing for RoCE.	THIRD-PARTY
Ultra Ethernet (UEC 1.0, 2025-06-11)	UET includes Receiver Credit Congestion Control (RCCC) for incast, plus link-layer retry (LLR) and credit-based flow control (CBFC). It is the industry's standards-track answer to switched AI Ethernet. If Tenstorrent cards ever sat behind switches, UET would be the relevant comparison, not Homa.	THIRD-PARTY
Homa itself (2026 status)	HomaModule is still out-of-tree. `main` tracks Linux 6.17.8, with backports on RHEL 8/9.5 branches. Upstreaming reached `net-next v16` without a merge. A Sept 2026 protocol change made grant-requiring messages fully scheduled (new `START_MSG` packet). IANA IP protocol 146. Known-good NICs: ConnectX-4/5/6 and Intel E810. Ousterhout's 2026 AI Engineer talk, "Homa: The End of TCP for AI Clusters", pitches it for small coordination messages over switched datacenter networks (KV-cache existence checks, barriers), claiming ~13x lower P99 for short messages vs TCP, and relies on switch priority queues for its full benefit.	THIRD-PARTY

Conclusion: no real prior art for Homa on accelerator-integrated Ethernet, Tenstorrent or otherwise. The closest relatives (UET RCCC, EQDS, SIRD) all target switched fabrics with contention.

What Homa is and what it assumes

Message-oriented RPC transport (request/response), not a byte stream. Knowing message lengths lets it do SRPT: shortest remaining message first.
Receiver-driven. A sender blasts a small unscheduled prefix (fully scheduled since Sept 2026 for large messages). The receiver then hands out grants, choosing which sender may proceed, so congestion is controlled where it physically occurs: the receiver's downlink.
Uses the ~8 switch priority queues per egress port (via DSCP) to carry SRPT into the network.
Its own retransmission, pacing, and an at-most-once RPC model. Implemented as a Linux kernel module on a host NIC (with GSO/GRO, "TCP hijacking" for TSO/RSS, and a homa_qdisc for coexistence with TCP).
Assumptions: a switched multi-hop fabric; many-to-many traffic with incast; a heavy-tailed mix of message sizes; switch priority queues; a host kernel network stack; standard IP/Ethernet NICs.

Its entire value comes from queueing and contention. It does not reduce a link's serialisation time or fixed per-hop latency on an idle point-to-point wire.

What Blackhole Ethernet and TT-Fabric actually are

Hardware L1/L2 Ethernet; everything above is Tenstorrent-proprietary. The controllers can run in compliance mode, which uses standard Ethernet frames, drops frames with CRC errors, and leaves retransmission to software. TT-Fabric requires TT-link mode instead. That mode adds a 16-byte TT-link header carrying the receiver's L1 destination address and a hardware Go-Back-N ARQ (tx/rx sequence numbers). Upper layers "never see data loss due to ethernet frame CRC errors". THIRD-PARTY: tech_reports/TT-Fabric/TT-Fabric-Architecture.md §2.1.
Not IP. No IP/UDP stack exists in the tt-metal Ethernet code. Our own docs already say "the fabric is not IP" (docs/TWO-HOST-MESH.md). The data path runs NoC → ERISC L1 → Ethernet → peer ERISC L1 → NoC. Only Ethernet cores can send, and only into the L1 of the core at the far end of the link. THIRD-PARTY: tech_reports/EthernetMultichip/BasicEthernetGuide.md.
Point-to-point direct cables, no switches. TT-Routing does source routing within a mesh, with routing tables between meshes, virtual channels, bubble flow control on rings, a dedicated control VC, and automatic rerouting on link failure via Ethernet Fallback Channels. THIRD-PARTY: TT-Fabric-Architecture.md §2.2–2.3, §7.4.
Credit flow control already exists. The fabric router (EDM, tt_metal/fabric/impl/kernels/edm_fabric/fabric_erisc_router.cpp, 3,816 lines) has fixed-slot sender and receiver channels (e.g. 8 slots). Workers push only into free slots, and the receiver returns credits as slots drain. That is receiver-driven admission at hop granularity. The low-level Ethernet API also carries bytes_sent/receiver_ack sync fields. THIRD-PARTY: tt-metal source and BasicEthernetGuide.md.
Our link: one QSFP-DD cable trains 2 links (channels 4 and 6). MEASURED: docs/GOAL-VLLM-SERVING.md §bring-up and bench/runs/two-cards-2026-09-05.jsonl. get_num_links is a static table that returns 2 for P300, which is correct for us. Vendor spec: P150a ports are QSFP-DD 800G; nominal ~400 Gb/s ≈ 50 GB/s per link per direction (THIRD-PARTY/ESTIMATE). We have never measured effective link rate (docs/PREFILL-WHERE-THE-TIME-GOES.md §4 says so explicitly).
Programmability, in two tiers:
- The fabric router / EDM and CCL kernels are open source in tt-metal. They are JIT-compiled and loaded onto the ERISCs at runtime, so changing them needs no firmware flash. Blackhole has 14 Ethernet tiles × 2 ERISCs (rv32, 512 KiB Ethernet L1), and in 2-ERISC mode the router uses both. MEASURED/source: docs/RISCV-OFFLOAD-FEASIBILITY.md table; bh_hal_active_eth.cpp.
- The base Ethernet firmware (link training/retraining, base routing used by slow dispatch) lives on the SPI ROM. User ERISC kernels must yield to it via run_routing(). Changing it means a firmware flash, which violates our hard rule: firmware updates are never unattended; tt-flash is not packaged. THIRD-PARTY: BasicEthernetGuide.md "Ethernet routing firmware", "Link Health and Retraining".

Applicability to our traffic

Homa's precondition	Our card fabric
Switched multi-hop network with queues	One direct cable, 1 hop, no switch
Many-to-many, incast	Two chips, each sending only to its peer; no incast is possible
Heavy-tailed mix of message sizes competing	Bulk-synchronous collectives, deterministic sizes, one collective in flight per layer step, captured in a replayed trace
Loss / CRC retransmission handled by the transport	Hardware Go-Back-N in TT-link mode
Sender lacks congestion information	Per-slot credits from the receiver already gate every packet
Switch priority queues to extend SRPT	None exist; the "network" is a wire
Host kernel stack	Bare-metal rv32 ERISC with 512 KiB L1 and no OS

Homa's value proposition does not apply to the card fabric. Collectives on a 2-chip line are limited by bandwidth (prefill) or by fixed per-collective latency and synchronisation (decode). Neither term depends on queueing policy.

Where Homa could conceptually apply:

Host↔client serving traffic (vLLM OpenAI HTTP → Bifrost → tailnet). It runs over the host's normal NIC, completely separate from the cards. Homa could run there in principle, but see the numbers below: irrelevant.
Future multi-host scale-out. Tenstorrent's design for this is direct-cabled TT-Fabric (Big-Mesh, Galaxy), again with no switches. Our two-host plan (docs/TWO-HOST-MESH.md) uses QSFP-DD direct attach between the cards plus MPI over the host network for the control plane only. The small cross-host MPI coordination messages are the one Homa-shaped traffic class. It is not relevant while the hosts sit at different sites, and it is not measured.
Galaxy / large multi-hop meshes. Contention between flows does exist in large multi-hop meshes, so credit/VC scheduling could matter there. That is Tenstorrent's upstream design space, not ours, and they have not chosen Homa.

Implementation paths compared

Path	What it is	Effort	Risk	Expected gain for us	Measurable with our harnesses?
(a) Homa on the host NIC	Package HomaModule for NixOS; route some serving/RPC traffic over Homa sockets	Port the out-of-tree module from Linux 6.17.8 to our 6.12.93 kernel: days (ESTIMATE). The app side is the real cost: vLLM/uvicorn speak HTTP/TCP, and gRPC-Homa is only "preliminary". Weeks, and still doesn't help tailnet clients (ESTIMATE)	Low for the cards. Medium for the host: an out-of-tree kernel module on a box that already has pinned-page OOM freezes (`bench/runs/host-pinned-leak-20260914T063000Z.jsonl`); the Realtek NIC is not a known-good Homa NIC	~0. Network time per request is µs–ms, against TTFT of 6.6–609 s (below)	No. infbench/serving-bench cannot resolve a sub-ms change against a 40–600 s TTFT
(b) Homa-inspired receiver-driven scheduling inside EDM	Add grants/SRPT across fabric sender channels in `fabric_erisc_router.cpp`	Weeks to months (ESTIMATE). A 3.8k-line hot-path kernel, maintained as a patch against a fast-moving upstream (pins move as a set)	High: fabric deadlock/correctness, trace-replay compatibility, breaking every CCL op; an equivalence gate would be needed	~0. On a 1x2 line there is one worker sender channel per direction and no passthrough traffic, so there is nothing to reorder. Credits already provide receiver admission (ESTIMATE)	Only indirectly (decode ms/step, CCL op time in Tracy). An effect below the 0.48% control band (`bench/runs/lowlevel-audit-runA-20260913T070950Z.jsonl`) is unresolvable
(c) True Homa on ERISC	Compliance-mode Ethernet + IP/UDP-or-raw framing + Homa state machine on rv32	Person-months to a year (ESTIMATE). Needs an IP stack that doesn't exist, has to replace TT-link and TT-Fabric (which requires TT-link mode), and must coexist with base firmware `run_routing()`	Very high. Loses hardware Go-Back-N and L2 address delivery, breaks all tt-metal CCL. Any base-firmware change is a flash, forbidden unattended	Negative. Adds software per-packet processing to a wire that currently moves data NoC→L1→peer L1 with hardware reliability (ESTIMATE)	Would regress measurably; no upside to measure

Does it pay here? (numbers)

Where the time goes (MEASURED):

Batch makespan is dominated by serialised prefill. The 32k sweep shows prefill at 74–79% of wall, and the optimized arm at B=8/32k is 148.72 s makespan (bench/runs/sweep-32k-20260914T142154Z.jsonl, .table.txt). The goal-doc bracket for B=8 prefill share is 52.4–78.1% at 32k (docs/GOAL-AGENT-SERVING.md, bench/runs/serving-grid-20260913T192352Z.jsonl).
Target shapes: the last agent's TTFT is 443.2 s at 8x128k and 609.2 s at 4x256k, fully serialised prefill staircases (bench/runs/target-shape-ttft-B8-128k-20260914T130939Z.jsonl, bench/runs/target-shape-ttft-B4-256k-20260914T134045Z.jsonl).

Fabric cost in decode (MEASURED, with caveats):

256 CCL ops/step = 4.339 ms of card-0 device time, 8.6%. Split: GDN all-gather 1.698 + reduce-scatter 1.655; full-attention AG 0.542 + RS 0.445 ms. About 17 µs per collective (bench/runs/p0-decode-per-op-profile-20260912T151000Z.jsonl, bench/runs/p0-profile-20260912/itemisation-table.txt). Caveats: an eager profile at B=1/4k on the composed GDN path, not the production fused path. The CCL op count matches production's inventory (128 RS + 128 AG + 1 lm-head gather, bench/runs/lowlevel-audit-program-inventory-20260913T001500Z.jsonl). There is no production-arm per-op table.
Dispatch is already amortised by trace replay (176 ms/step saved at B=8, bench/runs/lowlevel-audit-runA-20260913T070950Z.jsonl), so per-collective host overhead is not an opportunity either.
Payload per decode step is tiny: ~1.56 MB/token/card (derived below) × 8 streams ≈ 12.5 MB per step, which is ~125 µs at a nominal 100 GB/s over 2 links (ESTIMATE). That is 3% of the measured 4.3 ms. Decode CCL is bounded by latency and synchronisation, not bandwidth or queueing. Homa targets neither of those regimes on an uncontended wire.
Ceiling (ESTIMATE): at B=8/32k optimized, decode is 57.67 ms/token (the B=4/32k figure; B=8 is similar) × 1024 tokens ≈ 59 s of decode wall. Deleting all CCL time, 8.6% of the device step, saves ≈ 4–5 s ≈ 3% of the 148.72 s makespan. That is an unreachable upper bound for any fabric change. Homa-style scheduling captures ~0 of it.
THIRD-PARTY: a second QSFP-DD cable (4 links) plus the get_num_links override measured −1.35 ms/step (+2.4% decode, −4.6% TTFT) on another team's box (docs/EXTERNAL-LEDGER.md, docs/EXTERNAL-REPO-RUNBOOK.md row 24). That is the scale of a real fabric lever: extra bandwidth, not a better transport.

Fabric cost in prefill (UNMEASURED, derived bound):

Payload is ~1.557 MB/token/card (5,120 B per half-vector × gather+scatter × 2 sublayers × 64 layers, plus the fp32 GDN scatter) (docs/PREFILL-WHERE-THE-TIME-GOES.md §4, derived).
At a nominal 100 GB/s (2 × ~50 GB/s, THIRD-PARTY/ESTIMATE), that is ~15.6 µs/token. Measured prefill is 283 µs/token at 16k (3,534 tok/s) and 423 µs/token at 128k (2,366 tok/s) (bench/runs/serving-16k-20260913T095210Z.jsonl, bench/runs/target-shape-ttft-B8-128k-20260914T130939Z.jsonl). So fully exposed CCL would be ~4–6% of prefill (ESTIMATE), less where the fused all-gather-matmul / matmul-reduce-scatter ops (AGMM/MMRS) overlap it.
If the effective link rate were only 25 GB/s it would be ~15–22%. That uncertainty is the one real unknown here, and it is a bandwidth question, where SRPT and grants give nothing on a single uncontended link.

Serving/host-NIC traffic (ESTIMATE): an OpenAI request plus token stream is kilobytes over the tailnet (WireGuard UDP; Homa's DSCP priorities do not survive the tunnel). Even a pessimistic 10 ms of network latency is 0.15% of the best TTFT we have (6.62 s, B=4/4k) and 0.002% of 609 s.

Ranking against docs/GOAL-AGENT-SERVING.md: its work list is prefix caching with GDN checkpoints (landed on this branch, bench/runs/phase2a-prefix-e2e.jsonl), prefill interleaving, and prefill throughput. Homa ranks below every item on it and below "second QSFP-DD cable". It does not act on prefill serialisation, which is where 74–79% of wall goes.

Recommended next steps / probes (with go/no-go)

Close this now: NO-GO for (b) and (c). Record here that the card fabric has no switch, no incast, hardware ARQ and existing credit flow control, so a Homa-class transport has nothing to fix. No hardware time needed.
Probe P1: measure effective fabric link rate (needs the cards; run under the /tmp/ttlock convention when they are free). Use tt-metal's own fabric benchmark (tests/tt_metal/tt_fabric/benchmark/collectives, tests/tt_metal/tt_fabric/test_infra/test_yamls/*) for unicast bandwidth vs packet size and ping latency on the 2-link P300. Record to bench/runs/fabric-linkrate-*.jsonl. This turns the 1557/L µs/token term into a number.
- Go (to probe P2) if derived exposed CCL is >10% of prefill at 16k.
- No-go (close the fabric line entirely) if it is <5%.
Probe P2: CCL rows in the per-op prefill Tracy table (already goal-ranked item 3, docs/TRACY-PREFILL-PROFILE.md), at 16k/32k/64k.
- Go on a fabric lever only if AllGather/ReduceScatter (including the fused AGMM/MMRS halves) exceed ~10% of prefill device time.
- If go, the candidate levers are (i) a second QSFP-DD cable + get_num_links patch (hardware purchase; third-party −1.35 ms decode / −4.6% TTFT); (ii) better overlap in AGMM/MMRS. Not Homa.
Path (a) stays parked until a real host-to-host RPC workload exists: disaggregated prefill/decode across hosts, KV transfer over a host NIC, or Big-Mesh MPI control traffic on co-located hosts. Evaluate it then against RDMA/UET, and only on a ConnectX-4/5/6 or E810 NIC, not the Realtek.

Risks and unknowns

Effective link rate is unmeasured. Prefill CCL could be more exposed than the nominal-rate estimate. Even then, the fix is bandwidth or overlap, not transport policy.
Decode CCL numbers come from the composed-path eager profile (B=1, 4k), not production at B=8/long context. The count is confirmed; the time is not re-measured at the target shapes.
The Blackhole link speed (~400 Gb/s per link) is vendor spec only.
If Tenstorrent ever adds switched Ethernet scale-out (compliance mode behind switches), receiver-driven transport becomes relevant. UET (RCCC/CBFC/LLR) is the standards route there, and any such work would be upstream Tenstorrent firmware/fabric, not our patch set.
Any ERISC-level experiment carries card-wedge risk (see CLAUDE.md: a forced kill leaves the RISC firmware hung; recovery is tt-smi -r). Any base-firmware change is a flash, which is forbidden unattended.

Sources

Ours:

bench/runs/p0-decode-per-op-profile-20260912T151000Z.jsonl
bench/runs/p0-profile-20260912/itemisation-table.txt
bench/runs/lowlevel-audit-program-inventory-20260913T001500Z.jsonl
bench/runs/lowlevel-audit-runA-20260913T070950Z.jsonl
bench/runs/sweep-32k-20260914T142154Z.jsonl (+ .table.txt)
bench/runs/serving-grid-20260913T192352Z.jsonl
bench/runs/serving-16k-20260913T095210Z.jsonl
bench/runs/target-shape-ttft-B8-128k-20260914T130939Z.jsonl
bench/runs/target-shape-ttft-B4-256k-20260914T134045Z.jsonl
bench/runs/two-cards-2026-09-05.jsonl
bench/runs/host-pinned-leak-20260914T063000Z.jsonl
bench/runs/phase2a-prefix-e2e.jsonl
docs/GOAL-AGENT-SERVING.md, docs/GOAL-N-SERIES.md, docs/WHERE-THE-BOTTLENECK-IS.md, docs/PREFILL-WHERE-THE-TIME-GOES.md §4, docs/HOW-IT-WORKS.md §1, docs/TWO-HOST-MESH.md, docs/EXTERNAL-LEDGER.md, docs/EXTERNAL-REPO-RUNBOOK.md, docs/RISCV-OFFLOAD-FEASIBILITY.md, docs/HOST-PLATFORM-LIMITS.md (Realtek NIC)

tt-metal (local tree /home/orchestra/ttm-build/src):

tech_reports/TT-Fabric/TT-Fabric-Architecture.md (TT-link mode, Go-Back-N, routing, VCs, EFC, Galaxy scale-out)
tech_reports/EthernetMultichip/BasicEthernetGuide.md (ERISC data path, base routing firmware on SPI ROM, run_routing(), link retraining)
tt_metal/fabric/impl/kernels/edm_fabric/fabric_erisc_router.cpp, tt_metal/fabric/fabric_edm_packet_header.hpp, tt_metal/llrt/hal/tt-1xx/blackhole/bh_hal_active_eth.cpp
Upstream: https://github.com/tenstorrent/tt-metal/blob/main/tech_reports/TT-Fabric/TT-Fabric-Architecture.md

Homa:

https://github.com/PlatformLab/HomaModule (README "Significant changes", NIC support)
Homa SIGCOMM'18: https://people.csail.mit.edu/alizadeh/papers/homa-sigcomm18.pdf
Homa/Linux ATC'21: https://www.usenix.org/conference/atc21/presentation/ousterhout
Upstreaming (net-next v16): https://lwn.net/Articles/1042323/
Ousterhout 2026 talk "Homa: The End of TCP for AI Clusters": https://youtube.com/watch?v=eZ8WWZzoaR0 (summary: https://www.sean-weldon.com/blog/2026-09-21-homa-the-end-of-tcp-for-ai-clusters-john-ousterhout-stanford, https://aietalks.com/talks/homa-the-end-of-tcp-for-ai-clusters)

Related transports / hardware:

Homa-NIC (FPGA, WIP; repo now 404): https://github.com/C0L/Homa-NIC
SDP/SMT transport encryption for Homa/NDP with NIC offload: https://arxiv.org/html/2406.15686v3
EQDS NSDI'22: https://www.usenix.org/system/files/nsdi22-paper-olteanu.pdf
SIRD NSDI'25: https://www.usenix.org/system/files/nsdi25-prasopoulos.pdf
Meta RoCE SIGCOMM'24: https://cs.stanford.edu/~keithw/sigcomm2024/sigcomm24-final246-acmpaginated.pdf
UEC Spec 1.0: https://ultraethernet.org/wp-content/uploads/sites/20/2025/06/UE-Specification-6.11.25.pdf
Arista "Demystifying Ultra Ethernet" (RCCC): https://www.arista.com/assets/data/pdf/Whitepapers/Demystifying-Ultra-Ethernet-WP.pdf

Labels used below: **MEASURED** (our `bench/runs/*` record cited), **THIRD-PARTY** (vendor, paper or external report, not reproduced by us), **ESTIMATE** (our arithmetic or judgement, not measured). This study was desk research only. No tt-metal/vLLM process was started and the cards were not touched. ## TL;DR verdict **Do not implement Homa on the cards, in any form.** Homa fixes tail latency caused by **queueing in switched, many-to-many networks**: incast at a switch egress port, short messages stuck behind long ones, and sender-side congestion control that has too little information to react. Our card-to-card path has none of those. It is **one direct QSFP-DD cable (2 links) between two chips**, carrying bulk-synchronous collectives of fixed size, with no switch, no incast and no competing flows. The card Ethernet is also **not an IP stack**. TT-Fabric requires the controllers to run in *TT-link mode*, where the hardware already provides a Go-Back-N ARQ, and the fabric router already applies receiver-credit flow control per buffer slot. So the reliability and receiver-driven admission Homa would add already exist in hardware or firmware, and the one thing Homa adds on top (SRPT across switch priority queues) has no switch to act in. **The fabric is also a small part of our critical path.** In decode, the collectives are **4.34 ms of a 50.6 ms instrumented device step (8.6%)**, about 17 µs per collective across 256 ops (MEASURED, B=1/4k eager profile). Decode is only about 21–26% of batch wall time, because prefill is 74–79% of wall (MEASURED, 32k sweep). Removing *all* decode CCL time would save about 3% of makespan (ESTIMATE, B=8/32k), and no transport can remove it all, because the bytes still have to cross the cable. Homa's realistic gain on this link is ~0. **The only place Homa could run is the host NIC**, via the existing Linux module for serving/RPC traffic. That path is irrelevant too: per-request network time is micro- to milliseconds, against a TTFT of 6–609 s. The NIC is a Realtek, which is not on Homa's list of known-good NICs, and vLLM speaks HTTP over TCP, not Homa sockets. **What to do instead:** the open question behind this idea is *"how much does the fabric cost in prefill?"* Answer it with the per-op prefill Tracy table and one fabric bandwidth microbenchmark. If the fabric cost turns out to matter, the levers are a second QSFP-DD cable plus the `get_num_links` override, or better overlap in the fused all-gather-matmul / matmul-reduce-scatter ops. Not a new transport. ## Prior art found Searched 2026-09-24: GitHub API (tt-metal issues/PRs, repos), the tt-metal tech reports in our local tree (`/home/orchestra/ttm-build/src`), and Firecrawl web search. | Area | Finding | Label | |---|---|---| | Homa or any receiver-driven transport **on Tenstorrent hardware** | **None.** tt-metal issues/PRs search: `homa` 0 hits, `homa transport` 0, `ultra ethernet` 0. GitHub repo search `tenstorrent homa` 0 hits. The tt-metal Ethernet/firmware code contains no IP/UDP or compliance-mode stack (grep of `tt_metal/fabric`, `tt_metal/hw/inc/internal/ethernet`, `tt_metal/hw/firmware`). | MEASURED (search) | | Tenstorrent scale-out direction | TT-Fabric is a **proprietary mesh fabric over direct-cabled Ethernet links**. Tenstorrent says the "entire back-end (data-plane) network can be constructed using only Galaxy boxes" (a Galaxy is Tenstorrent's multi-chip server), i.e. no Ethernet switches. There is no public Tenstorrent statement on UEC or Homa. | THIRD-PARTY (`tech_reports/TT-Fabric/TT-Fabric-Architecture.md` §1) | | Homa in hardware / on NICs | `C0L/Homa-NIC`: a work-in-progress FPGA Homa targeting 100 Gb/s. It appears in search results, but the repo now returns 404. SDP/SMT (SIGCOMM'25, arXiv 2406.15686): transport-level encryption for Homa/NDP with NIC offload. nanoPU (OSDI'21) did hardware-terminated, NDP-style low-latency transport; cited from the literature, not re-verified in this session. **Nothing on AI-accelerator-integrated Ethernet.** | THIRD-PARTY | | Receiver-driven transports aimed at AI clusters | EQDS (NSDI'22, edge-queued datagrams with receiver credit). SIRD (NSDI'25): receivers schedule single-owner links and use reactive control on shared links. Its insight is directly relevant: **a single-owner link needs no transport-level scheduling beyond credits**, and that is exactly our situation. Meta RoCE (SIGCOMM'24): the collective library was co-designed with receiver-driven admission for 400G training. Reunion (2025): receiver-driven load balancing for RoCE. | THIRD-PARTY | | Ultra Ethernet (UEC 1.0, 2025-06-11) | UET includes **Receiver Credit Congestion Control (RCCC)** for incast, plus link-layer retry (LLR) and credit-based flow control (CBFC). It is the industry's standards-track answer to switched AI Ethernet. If Tenstorrent cards ever sat behind switches, UET would be the relevant comparison, not Homa. | THIRD-PARTY | | Homa itself (2026 status) | HomaModule is still out-of-tree. `main` tracks Linux 6.17.8, with backports on RHEL 8/9.5 branches. Upstreaming reached `net-next v16` without a merge. A Sept 2026 protocol change made grant-requiring messages fully scheduled (new `START_MSG` packet). IANA IP protocol 146. Known-good NICs: ConnectX-4/5/6 and Intel E810. Ousterhout's 2026 AI Engineer talk, "Homa: The End of TCP for AI Clusters", pitches it for **small coordination messages over switched datacenter networks** (KV-cache existence checks, barriers), claiming ~13x lower P99 for short messages vs TCP, and **relies on switch priority queues** for its full benefit. | THIRD-PARTY | **Conclusion: no real prior art for Homa on accelerator-integrated Ethernet, Tenstorrent or otherwise.** The closest relatives (UET RCCC, EQDS, SIRD) all target switched fabrics with contention. ## What Homa is and what it assumes - **Message-oriented RPC transport** (request/response), not a byte stream. Knowing message lengths lets it do **SRPT**: shortest remaining message first. - **Receiver-driven.** A sender blasts a small unscheduled prefix (fully scheduled since Sept 2026 for large messages). The receiver then hands out **grants**, choosing which sender may proceed, so congestion is controlled where it physically occurs: the receiver's downlink. - **Uses the ~8 switch priority queues** per egress port (via DSCP) to carry SRPT into the network. - Its own retransmission, pacing, and an at-most-once RPC model. Implemented as a **Linux kernel module** on a host NIC (with GSO/GRO, "TCP hijacking" for TSO/RSS, and a `homa_qdisc` for coexistence with TCP). - **Assumptions:** a switched multi-hop fabric; many-to-many traffic with incast; a heavy-tailed mix of message sizes; switch priority queues; a host kernel network stack; standard IP/Ethernet NICs. Its entire value comes from **queueing and contention**. It does not reduce a link's serialisation time or fixed per-hop latency on an idle point-to-point wire. ## What Blackhole Ethernet and TT-Fabric actually are - **Hardware L1/L2 Ethernet; everything above is Tenstorrent-proprietary.** The controllers can run in *compliance mode*, which uses standard Ethernet frames, drops frames with CRC errors, and leaves retransmission to software. **TT-Fabric requires *TT-link mode* instead.** That mode adds a 16-byte TT-link header carrying the receiver's L1 destination address and a **hardware Go-Back-N ARQ** (tx/rx sequence numbers). Upper layers "never see data loss due to ethernet frame CRC errors". THIRD-PARTY: `tech_reports/TT-Fabric/TT-Fabric-Architecture.md` §2.1. - **Not IP.** No IP/UDP stack exists in the tt-metal Ethernet code. Our own docs already say "the fabric is not IP" (`docs/TWO-HOST-MESH.md`). The data path runs NoC → ERISC L1 → Ethernet → peer ERISC L1 → NoC. Only Ethernet cores can send, and only into the L1 of the core at the far end of the link. THIRD-PARTY: `tech_reports/EthernetMultichip/BasicEthernetGuide.md`. - **Point-to-point direct cables, no switches.** TT-Routing does source routing within a mesh, with routing tables between meshes, virtual channels, bubble flow control on rings, a dedicated control VC, and automatic rerouting on link failure via Ethernet Fallback Channels. THIRD-PARTY: `TT-Fabric-Architecture.md` §2.2–2.3, §7.4. - **Credit flow control already exists.** The fabric router (EDM, `tt_metal/fabric/impl/kernels/edm_fabric/fabric_erisc_router.cpp`, 3,816 lines) has fixed-slot sender and receiver channels (e.g. 8 slots). Workers push only into free slots, and the receiver returns credits as slots drain. **That is receiver-driven admission at hop granularity.** The low-level Ethernet API also carries `bytes_sent`/`receiver_ack` sync fields. THIRD-PARTY: tt-metal source and `BasicEthernetGuide.md`. - **Our link:** one QSFP-DD cable trains **2 links** (channels 4 and 6). MEASURED: `docs/GOAL-VLLM-SERVING.md` §bring-up and `bench/runs/two-cards-2026-09-05.jsonl`. `get_num_links` is a static table that returns 2 for P300, which is correct for us. Vendor spec: P150a ports are QSFP-DD 800G; nominal ~400 Gb/s ≈ 50 GB/s per link per direction (THIRD-PARTY/ESTIMATE). **We have never measured effective link rate** (`docs/PREFILL-WHERE-THE-TIME-GOES.md` §4 says so explicitly). - **Programmability, in two tiers:** - The **fabric router / EDM and CCL kernels are open source in tt-metal**. They are JIT-compiled and loaded onto the ERISCs at runtime, so changing them needs **no firmware flash**. Blackhole has 14 Ethernet tiles × 2 ERISCs (rv32, 512 KiB Ethernet L1), and in 2-ERISC mode the router uses both. MEASURED/source: `docs/RISCV-OFFLOAD-FEASIBILITY.md` table; `bh_hal_active_eth.cpp`. - The **base Ethernet firmware** (link training/retraining, base routing used by slow dispatch) lives on the **SPI ROM**. User ERISC kernels must yield to it via `run_routing()`. Changing it means a firmware flash, which violates our hard rule: **firmware updates are never unattended; `tt-flash` is not packaged.** THIRD-PARTY: `BasicEthernetGuide.md` "Ethernet routing firmware", "Link Health and Retraining". ## Applicability to our traffic | Homa's precondition | Our card fabric | |---|---| | Switched multi-hop network with queues | **One direct cable, 1 hop, no switch** | | Many-to-many, incast | **Two chips**, each sending only to its peer; no incast is possible | | Heavy-tailed mix of message sizes competing | **Bulk-synchronous collectives**, deterministic sizes, one collective in flight per layer step, captured in a replayed trace | | Loss / CRC retransmission handled by the transport | Hardware Go-Back-N in TT-link mode | | Sender lacks congestion information | Per-slot credits from the receiver already gate every packet | | Switch priority queues to extend SRPT | None exist; the "network" is a wire | | Host kernel stack | Bare-metal rv32 ERISC with 512 KiB L1 and no OS | **Homa's value proposition does not apply to the card fabric.** Collectives on a 2-chip line are limited by bandwidth (prefill) or by fixed per-collective latency and synchronisation (decode). Neither term depends on queueing policy. Where Homa *could* conceptually apply: 1. **Host↔client serving traffic** (vLLM OpenAI HTTP → Bifrost → tailnet). It runs over the host's normal NIC, completely separate from the cards. Homa could run there in principle, but see the numbers below: irrelevant. 2. **Future multi-host scale-out.** Tenstorrent's design for this is direct-cabled TT-Fabric (Big-Mesh, Galaxy), again with no switches. Our two-host plan (`docs/TWO-HOST-MESH.md`) uses QSFP-DD direct attach between the cards plus MPI over the host network for the *control* plane only. The small cross-host MPI coordination messages are the one Homa-shaped traffic class. It is not relevant while the hosts sit at different sites, and it is not measured. 3. **Galaxy / large multi-hop meshes.** Contention between flows does exist in large multi-hop meshes, so credit/VC scheduling could matter there. That is Tenstorrent's upstream design space, not ours, and they have not chosen Homa. ## Implementation paths compared | Path | What it is | Effort | Risk | Expected gain for us | Measurable with our harnesses? | |---|---|---|---|---|---| | **(a) Homa on the host NIC** | Package HomaModule for NixOS; route some serving/RPC traffic over Homa sockets | Port the out-of-tree module from Linux 6.17.8 to our **6.12.93** kernel: days (ESTIMATE). The app side is the real cost: vLLM/uvicorn speak HTTP/TCP, and gRPC-Homa is only "preliminary". Weeks, and still doesn't help tailnet clients (ESTIMATE) | Low for the cards. Medium for the host: an out-of-tree kernel module on a box that already has pinned-page OOM freezes (`bench/runs/host-pinned-leak-20260914T063000Z.jsonl`); the **Realtek NIC is not a known-good Homa NIC** | **~0.** Network time per request is µs–ms, against TTFT of 6.6–609 s (below) | No. infbench/serving-bench cannot resolve a sub-ms change against a 40–600 s TTFT | | **(b) Homa-inspired receiver-driven scheduling inside EDM** | Add grants/SRPT across fabric sender channels in `fabric_erisc_router.cpp` | Weeks to months (ESTIMATE). A 3.8k-line hot-path kernel, maintained as a patch against a fast-moving upstream (pins move as a set) | High: fabric deadlock/correctness, trace-replay compatibility, breaking every CCL op; an equivalence gate would be needed | **~0.** On a 1x2 line there is one worker sender channel per direction and no passthrough traffic, so there is nothing to reorder. Credits already provide receiver admission (ESTIMATE) | Only indirectly (decode ms/step, CCL op time in Tracy). An effect below the 0.48% control band (`bench/runs/lowlevel-audit-runA-20260913T070950Z.jsonl`) is unresolvable | | **(c) True Homa on ERISC** | Compliance-mode Ethernet + IP/UDP-or-raw framing + Homa state machine on rv32 | Person-months to a year (ESTIMATE). Needs an IP stack that doesn't exist, has to replace TT-link and TT-Fabric (which *requires* TT-link mode), and must coexist with base firmware `run_routing()` | Very high. Loses hardware Go-Back-N and L2 address delivery, breaks all tt-metal CCL. Any base-firmware change is a flash, forbidden unattended | **Negative.** Adds software per-packet processing to a wire that currently moves data NoC→L1→peer L1 with hardware reliability (ESTIMATE) | Would regress measurably; no upside to measure | ## Does it pay here? (numbers) **Where the time goes (MEASURED):** - Batch makespan is dominated by serialised prefill. The 32k sweep shows **prefill at 74–79% of wall**, and the optimized arm at B=8/32k is **148.72 s** makespan (`bench/runs/sweep-32k-20260914T142154Z.jsonl`, `.table.txt`). The goal-doc bracket for B=8 prefill share is 52.4–78.1% at 32k (`docs/GOAL-AGENT-SERVING.md`, `bench/runs/serving-grid-20260913T192352Z.jsonl`). - Target shapes: the last agent's TTFT is **443.2 s at 8x128k** and **609.2 s at 4x256k**, fully serialised prefill staircases (`bench/runs/target-shape-ttft-B8-128k-20260914T130939Z.jsonl`, `bench/runs/target-shape-ttft-B4-256k-20260914T134045Z.jsonl`). **Fabric cost in decode (MEASURED, with caveats):** - **256 CCL ops/step = 4.339 ms of card-0 device time, 8.6%.** Split: GDN all-gather 1.698 + reduce-scatter 1.655; full-attention AG 0.542 + RS 0.445 ms. About 17 µs per collective (`bench/runs/p0-decode-per-op-profile-20260912T151000Z.jsonl`, `bench/runs/p0-profile-20260912/itemisation-table.txt`). Caveats: an eager profile at B=1/4k on the *composed* GDN path, not the production fused path. The CCL op count matches production's inventory (128 RS + 128 AG + 1 lm-head gather, `bench/runs/lowlevel-audit-program-inventory-20260913T001500Z.jsonl`). There is no production-arm per-op table. - Dispatch is already amortised by trace replay (176 ms/step saved at B=8, `bench/runs/lowlevel-audit-runA-20260913T070950Z.jsonl`), so per-collective host overhead is not an opportunity either. - **Payload per decode step is tiny:** ~1.56 MB/token/card (derived below) × 8 streams ≈ 12.5 MB per step, which is **~125 µs at a nominal 100 GB/s** over 2 links (ESTIMATE). That is 3% of the measured 4.3 ms. **Decode CCL is bounded by latency and synchronisation, not bandwidth or queueing.** Homa targets neither of those regimes on an uncontended wire. - **Ceiling (ESTIMATE):** at B=8/32k optimized, decode is 57.67 ms/token (the B=4/32k figure; B=8 is similar) × 1024 tokens ≈ 59 s of decode wall. Deleting *all* CCL time, 8.6% of the device step, saves ≈ 4–5 s ≈ **3% of the 148.72 s makespan**. That is an unreachable upper bound for *any* fabric change. Homa-style scheduling captures ~0 of it. - THIRD-PARTY: a second QSFP-DD cable (4 links) plus the `get_num_links` override measured **−1.35 ms/step (+2.4% decode, −4.6% TTFT)** on another team's box (`docs/EXTERNAL-LEDGER.md`, `docs/EXTERNAL-REPO-RUNBOOK.md` row 24). That is the scale of a *real* fabric lever: extra bandwidth, not a better transport. **Fabric cost in prefill (UNMEASURED, derived bound):** - Payload is **~1.557 MB/token/card** (5,120 B per half-vector × gather+scatter × 2 sublayers × 64 layers, plus the fp32 GDN scatter) (`docs/PREFILL-WHERE-THE-TIME-GOES.md` §4, derived). - At a nominal 100 GB/s (2 × ~50 GB/s, THIRD-PARTY/ESTIMATE), that is **~15.6 µs/token**. Measured prefill is 283 µs/token at 16k (3,534 tok/s) and 423 µs/token at 128k (2,366 tok/s) (`bench/runs/serving-16k-20260913T095210Z.jsonl`, `bench/runs/target-shape-ttft-B8-128k-20260914T130939Z.jsonl`). So fully exposed CCL would be **~4–6% of prefill** (ESTIMATE), less where the fused all-gather-matmul / matmul-reduce-scatter ops (AGMM/MMRS) overlap it. - If the effective link rate were only 25 GB/s it would be ~15–22%. **That uncertainty is the one real unknown here**, and it is a bandwidth question, where SRPT and grants give nothing on a single uncontended link. **Serving/host-NIC traffic (ESTIMATE):** an OpenAI request plus token stream is kilobytes over the tailnet (WireGuard UDP; Homa's DSCP priorities do not survive the tunnel). Even a pessimistic 10 ms of network latency is 0.15% of the best TTFT we have (6.62 s, B=4/4k) and 0.002% of 609 s. **Ranking against `docs/GOAL-AGENT-SERVING.md`:** its work list is prefix caching with GDN checkpoints (landed on this branch, `bench/runs/phase2a-prefix-e2e.jsonl`), prefill interleaving, and prefill throughput. **Homa ranks below every item on it and below "second QSFP-DD cable".** It does not act on prefill serialisation, which is where 74–79% of wall goes. ## Recommended next steps / probes (with go/no-go) 1. **Close this now: NO-GO for (b) and (c).** Record here that the card fabric has no switch, no incast, hardware ARQ and existing credit flow control, so a Homa-class transport has nothing to fix. No hardware time needed. 2. **Probe P1: measure effective fabric link rate** (needs the cards; run under the `/tmp/ttlock` convention when they are free). Use tt-metal's own fabric benchmark (`tests/tt_metal/tt_fabric/benchmark/collectives`, `tests/tt_metal/tt_fabric/test_infra/test_yamls/*`) for unicast bandwidth vs packet size and ping latency on the 2-link P300. Record to `bench/runs/fabric-linkrate-*.jsonl`. This turns the `1557/L µs/token` term into a number. - **Go** (to probe P2) if derived exposed CCL is **>10% of prefill** at 16k. - **No-go** (close the fabric line entirely) if it is <5%. 3. **Probe P2: CCL rows in the per-op prefill Tracy table** (already goal-ranked item 3, `docs/TRACY-PREFILL-PROFILE.md`), at 16k/32k/64k. - **Go** on a fabric lever only if AllGather/ReduceScatter (including the fused AGMM/MMRS halves) exceed ~10% of prefill device time. - If go, the candidate levers are (i) a second QSFP-DD cable + `get_num_links` patch (hardware purchase; third-party −1.35 ms decode / −4.6% TTFT); (ii) better overlap in AGMM/MMRS. **Not Homa.** 4. **Path (a) stays parked** until a real host-to-host RPC workload exists: disaggregated prefill/decode across hosts, KV transfer over a host NIC, or Big-Mesh MPI control traffic on co-located hosts. Evaluate it then against RDMA/UET, and only on a ConnectX-4/5/6 or E810 NIC, not the Realtek. ## Risks and unknowns - **Effective link rate is unmeasured.** Prefill CCL could be more exposed than the nominal-rate estimate. Even then, the fix is bandwidth or overlap, not transport policy. - **Decode CCL numbers come from the composed-path eager profile** (B=1, 4k), not production at B=8/long context. The count is confirmed; the time is not re-measured at the target shapes. - **The Blackhole link speed** (~400 Gb/s per link) is vendor spec only. - If Tenstorrent ever adds **switched Ethernet** scale-out (compliance mode behind switches), receiver-driven transport becomes relevant. UET (RCCC/CBFC/LLR) is the standards route there, and any such work would be upstream Tenstorrent firmware/fabric, not our patch set. - Any ERISC-level experiment carries **card-wedge risk** (see CLAUDE.md: a forced kill leaves the RISC firmware hung; recovery is `tt-smi -r`). Any base-firmware change is a flash, which is forbidden unattended. ## Sources **Ours:** - `bench/runs/p0-decode-per-op-profile-20260912T151000Z.jsonl` - `bench/runs/p0-profile-20260912/itemisation-table.txt` - `bench/runs/lowlevel-audit-program-inventory-20260913T001500Z.jsonl` - `bench/runs/lowlevel-audit-runA-20260913T070950Z.jsonl` - `bench/runs/sweep-32k-20260914T142154Z.jsonl` (+ `.table.txt`) - `bench/runs/serving-grid-20260913T192352Z.jsonl` - `bench/runs/serving-16k-20260913T095210Z.jsonl` - `bench/runs/target-shape-ttft-B8-128k-20260914T130939Z.jsonl` - `bench/runs/target-shape-ttft-B4-256k-20260914T134045Z.jsonl` - `bench/runs/two-cards-2026-09-05.jsonl` - `bench/runs/host-pinned-leak-20260914T063000Z.jsonl` - `bench/runs/phase2a-prefix-e2e.jsonl` - `docs/GOAL-AGENT-SERVING.md`, `docs/GOAL-N-SERIES.md`, `docs/WHERE-THE-BOTTLENECK-IS.md`, `docs/PREFILL-WHERE-THE-TIME-GOES.md` §4, `docs/HOW-IT-WORKS.md` §1, `docs/TWO-HOST-MESH.md`, `docs/EXTERNAL-LEDGER.md`, `docs/EXTERNAL-REPO-RUNBOOK.md`, `docs/RISCV-OFFLOAD-FEASIBILITY.md`, `docs/HOST-PLATFORM-LIMITS.md` (Realtek NIC) **tt-metal (local tree `/home/orchestra/ttm-build/src`):** - `tech_reports/TT-Fabric/TT-Fabric-Architecture.md` (TT-link mode, Go-Back-N, routing, VCs, EFC, Galaxy scale-out) - `tech_reports/EthernetMultichip/BasicEthernetGuide.md` (ERISC data path, base routing firmware on SPI ROM, `run_routing()`, link retraining) - `tt_metal/fabric/impl/kernels/edm_fabric/fabric_erisc_router.cpp`, `tt_metal/fabric/fabric_edm_packet_header.hpp`, `tt_metal/llrt/hal/tt-1xx/blackhole/bh_hal_active_eth.cpp` - Upstream: https://github.com/tenstorrent/tt-metal/blob/main/tech_reports/TT-Fabric/TT-Fabric-Architecture.md **Homa:** - https://github.com/PlatformLab/HomaModule (README "Significant changes", NIC support) - Homa SIGCOMM'18: https://people.csail.mit.edu/alizadeh/papers/homa-sigcomm18.pdf - Homa/Linux ATC'21: https://www.usenix.org/conference/atc21/presentation/ousterhout - Upstreaming (net-next v16): https://lwn.net/Articles/1042323/ - Ousterhout 2026 talk "Homa: The End of TCP for AI Clusters": https://youtube.com/watch?v=eZ8WWZzoaR0 (summary: https://www.sean-weldon.com/blog/2026-09-21-homa-the-end-of-tcp-for-ai-clusters-john-ousterhout-stanford, https://aietalks.com/talks/homa-the-end-of-tcp-for-ai-clusters) **Related transports / hardware:** - Homa-NIC (FPGA, WIP; repo now 404): https://github.com/C0L/Homa-NIC - SDP/SMT transport encryption for Homa/NDP with NIC offload: https://arxiv.org/html/2406.15686v3 - EQDS NSDI'22: https://www.usenix.org/system/files/nsdi22-paper-olteanu.pdf - SIRD NSDI'25: https://www.usenix.org/system/files/nsdi25-prasopoulos.pdf - Meta RoCE SIGCOMM'24: https://cs.stanford.edu/~keithw/sigcomm2024/sigcomm24-final246-acmpaginated.pdf - UEC Spec 1.0: https://ultraethernet.org/wp-content/uploads/sites/20/2025/06/UE-Specification-6.11.25.pdf - Arista "Demystifying Ultra Ethernet" (RCCC): https://www.arista.com/assets/data/pdf/Whitepapers/Demystifying-Ultra-Ethernet-WP.pdf

Grok referenced this issue

2026-09-24 11:11:53 +02:00

bfp-quant: EXL3-style Hessian-aware offline quantizer → native bfp4_b (grid-snapped), + weight accuracy gate #62

No labels

human-approved

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

bitpartner/tt-stack#60

No description provided.

Rows
Columns