Build ttnn from the fork so prefill survives program.cpp:1932 #13

Merged

multica-agent merged 1 commit from agent/fable/cxx-fork-build into agent/claude-auth/goal-vllm-serving

2026-09-05 19:05:04 +02:00

multica-agent commented

2026-09-04 23:03:29 +02:00

Member

Tier: T1 (new Dockerfile + doc; no change to any existing file).

vllm-tt:src = Dockerfile.vllm-tt rebuilt on our tt-metal fork (dhavli/tt-metal@3b31bad4, tag v0.79.0-dev20260903 + #53320 + #53319 + #53314). It exists because the server dies in prefill warmup at TT_THROW @ program.cpp:1932, which #53320 says needs the C++ fix in #53319 — not overlayable on the vendor wheel.

Based on agent/claude-auth/goal-vllm-serving because it applies patches/tt-metal/0005, 0006 from there. 0007 is not applied: it is #53320's Python half and is already in the fork. Its one extra line over #53320 — FIR tap slice (B, k+T, D) vs (1, k+T, D) — is carried as an asserted sed; the proper home is the fork branch.

Verifiably contains the C++ fixes (all measured against the vendor wheel, which scores 0/False on each):

python3 -c 'import ttnn; print("DRAM channel-chunk path" in ttnn.conv2d.__doc__)' → True (no hardware, no binutils)
importlib.metadata.version("ttnn") → 0.79.0.dev20260903+cfx.g3b31bad4 (pretend version; setuptools_scm would say 0.75.0rc10.dev1177)
_ttnncpp.so has the #53314 TT_FATAL literal and exports op_slicing::try_determine_slice_config
the build fails if any of the above is missing

State on cfx-llm2: vllm-tt:src loaded and import-verified (ttnn, torch, vllm, vllm_tt_plugin, models.demos.blackhole.qwen36). Not run on the cards. Docker lists it at 13.5 GB on arrival (6.54 GB on nixos) and / is at 1.2 GB free — see docs/CXX-BUILD.md "Shipping" for why and what to do about it.

Build: 14 min compile (1320 unity steps, 16 jobs), ~20 min full image, on nixos. Traps hit and the exact messages: docs/CXX-BUILD.md.

**Tier: T1** (new Dockerfile + doc; no change to any existing file). `vllm-tt:src` = `Dockerfile.vllm-tt` rebuilt on our tt-metal fork (`dhavli/tt-metal@3b31bad4`, tag `v0.79.0-dev20260903` + #53320 + #53319 + #53314). It exists because the server dies in prefill warmup at `TT_THROW @ program.cpp:1932`, which #53320 says needs the C++ fix in #53319 — not overlayable on the vendor wheel. Based on `agent/claude-auth/goal-vllm-serving` because it applies `patches/tt-metal/0005`, `0006` from there. `0007` is not applied: it is #53320's Python half and is already in the fork. Its one extra line over #53320 — FIR tap slice `(B, k+T, D)` vs `(1, k+T, D)` — is carried as an asserted `sed`; the proper home is the fork branch. **Verifiably contains the C++ fixes** (all measured against the vendor wheel, which scores 0/False on each): - `python3 -c 'import ttnn; print("DRAM channel-chunk path" in ttnn.conv2d.__doc__)'` → True (no hardware, no binutils) - `importlib.metadata.version("ttnn")` → `0.79.0.dev20260903+cfx.g3b31bad4` (pretend version; setuptools_scm would say `0.75.0rc10.dev1177`) - `_ttnncpp.so` has the #53314 TT_FATAL literal and exports `op_slicing::try_determine_slice_config` - the build fails if any of the above is missing **State on cfx-llm2:** `vllm-tt:src` loaded and import-verified (`ttnn, torch, vllm, vllm_tt_plugin, models.demos.blackhole.qwen36`). Not run on the cards. Docker lists it at 13.5 GB on arrival (6.54 GB on nixos) and `/` is at **1.2 GB free** — see `docs/CXX-BUILD.md` "Shipping" for why and what to do about it. Build: 14 min compile (1320 unity steps, 16 jobs), ~20 min full image, on nixos. Traps hit and the exact messages: `docs/CXX-BUILD.md`.

multica-agent added 1 commit

2026-09-04 23:03:29 +02:00

Build ttnn from the fork so prefill survives program.cpp:1932 b13b880fb4

The vLLM server loads all 64 layers of Qwen3.8-27B on the 2x p150a mesh and
then dies in prefill warmup at chunk=2048 with TT_THROW at
tt_metal/impl/program/program.cpp:1932, the static-CB/L1 clash that upstream
PR #53320 says needs the C++ fix in #53319. FORKS.md deferred the C++
rebuild until a fatal appeared; it appeared, and no Python overlay on the
vendor wheel can carry a change under ttnn/cpp.

Dockerfile.vllm-tt-src builds the wheel from dhavli/tt-metal at
cfx/qwen36-p300-tp2 (pinned by commit 3b31bad4, v0.79.0-dev20260903 plus
#53320/#53319/#53314) inside the tag-matched vendor dev image, mirroring
setup.py's cibuildwheel path so the result has the same shared-library
shape as the wheel it replaces, then swaps it into the release image and
reproduces Dockerfile.vllm-tt from there. It applies 0005 and 0006 but not
0007: that one is #53320's Python half and is already the fork's first
commit. The single line 0007 has over #53320 -- the FIR tap slice ending at
(B, k+T, D) rather than (1, k+T, D) -- is carried as an asserted edit so
vllm-tt:src and vllm-tt:warm run the same Python.

The build asserts the C++ fixes are in the compiled object rather than
trusting "it built": a TT_FATAL literal and an exported symbol that #53314
adds, both absent from the vendor _ttnncpp.so, plus a pretend version
naming tag and fork commit because setuptools_scm's tag_regex never matches
a dated dev tag and would otherwise report 0.75.0rc10. CXX-BUILD.md records
the traps (wrong sfpi in the :latest dev image, git 2.34.1 vs GitHub over
HTTP/2, log_debug strings compiled out in Release, grep -q under pipefail),
the hardware-free probe anyone can run on a box, the on-device probes, what
invalidates the build, and the disk cost of shipping by docker save.

Built on nixos in 14 min for the compile, ~20 min for the image; loaded on
cfx-llm2 as vllm-tt:src and verified importable without touching the cards.