DSpark drafter (ThatchCloud) on ThinkingCap #66
Labels
No labels
human-approved
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
bitpartner/tt-stack#66
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Branch:
dspark-thinkingcap. Related: #23 (the old DSpark acceptance probe), #63 (MTP on ThinkingCap), #65 (ThinkingCap baseline).Labels: MEASURED (ours, with a record), THIRD-PARTY, ESTIMATE.
ThatchCloud's latest state (THIRD-PARTY, repo
Thatch-cloud/Tenstorrent.Blackhole-Qwen3.8-27B, pushed 2026-09-24)Branch tips:
experiment/t32-score-reuse@e98dbb0;lever-n-m3native-v224;83c9562;ci/qwen-hardware-correctnessis stale (3b62f7f, 09-12).Offline combined T16/DSpark runtime, one stream, synthetic prompt: 128.65 TG / 3,320 PP at 4k. Context ladder:
Real text (
docs/real-text-2026-09-24.md):4-user "m3native" packed path (uses DFlash2, not DSpark): ~20-23.6 tok/s per user while packed, but only 24-39 tok/s aggregate delivered, because most steps fall back with fewer than 4 live users.
Costs: T16 verify ~59-63 ms, draft ~21 ms/block, commit 10-21 ms.
GDN is handled commit-only (per-token state history, DMA commit of the accepted slot, no rollback).
Their stack is their own tree: tt-metal
9f9cd4fd, target Qwen/Qwen3.8-27B, their own P150-pair mesh descriptor.The drafter is the same RadixArk
Qwen3.8-27B-DSpark@b9a5dbdfwe already hold.No ThinkingCap-specific drafter exists anywhere; the ThinkingCap card mentions only MTP k=3.
RadixArk card: acceptance length 3.43 (GB300, NVFP4 target, thinking on, temp 1); H200 speedup 2.25-3.16x at concurrency 1, 1.09-1.75x at 32.
Their B=4 path loses to our plain decode. 4 users x 256 tokens at 32k: our plain B=4 decode takes ~14.8 s (57.67 ms/step, MEASURED,
bench/runs/sweep-32k-20260914T142154Z.jsonl), their packed spec path 26.2 s (THIRD-PARTY). Their win is single-stream latency, not batched makespan.Usable on ThinkingCap? Verdict: usable in principle; the value is unmeasured; port as an oracle first
What the drafter consumes
fc.What runs at TP=2
What it needs from our 0.79 tree
Against MTP (#63): ESTIMATE, same verifier, so the comparison is draft cost against acceptance
Order: MTP first (cheaper draft, KV-free). DSpark is worth building only if the oracle shows ThinkingCap acceptance clearly above MTP's, and only for B≤4 / ≤32k.
#23: why its 0.10 was a harness defect (confirming and extending agy's advisory)
Plan / done so far
bench/dspark/oracle.py:shift±1,wNandperfectcontrols; think/answer split;DSparkReferenceat every length and window.bench/dspark/tap_capture.py+run-capture.sh:vllm-tt:k2+ src19 mounts, prod-argmax env,thinkingcap-e55ba7f6-prodcache);/tmp/ttlock, not preempting).Progress: device captures done (MEASURED,
bench/runs/dspark-tc-capture-20260924T195053Z.jsonl)Setup
vllm-tt:k2+ src19 mounts, prod-argmax env with host argmax, bf8 paged KV (banner), 1x2 TP=2.thinkingcap-e55ba7f6-prod;Checks (all passed)
Speeds. These are NOT production numbers; production is in #65.
Features
The host oracle is running on both arms now. Early ThinkingCap figure, agentic no-think file re-emission: K=7 gives 7.55 tokens/round, K=3 gives 3.94 (the perfect-drafter ceilings are 8 and 4). This is the best case, as expected. The other regimes follow.
Acceptance measured with the host oracle. Greedy teacher-forced replay of the device captures, full-history context, real RadixArk drafter (sha
2aff025f) in fp32.Records:
bench/runs/dspark-tc-accept-thinkingcap-20260925.jsonlbench/runs/dspark-tc-accept-qwen-20260925.jsonlTokens per verify round (1 + mean accepted), MEASURED, pooled over full rounds
Per prompt, ThinkingCap at K=7
Every thinking prompt spent its whole 512-token budget in
<think>, so the code and free_text rows measure thinking text.Controls at K=7 (ThinkingCap)
So the drafter really uses the full-history features, and #23-style windowing would have understated it heavily.
What this means
Makespan, ESTIMATE only
Next
The DSpark device port is reopened here. PR #23's "port NOT justified" is closed as superseded, because this issue's 3.95 tokens/round at K=7 beats #23's own >=3.0 threshold.
Reopened under the new rule (owner, 2026-09-25): a closure needs an on-hardware record at the stated scope; estimates, code-reads and third-party numbers close nothing. See docs/LEDGER.md "Reopened" (PR #68).