ButterflyGate

ButterflyGate is a sub-quadratic, $O(n\log n)$ token mixer that replaces self-attention in the early layers of a pretrained model.

Result

From 4,096 tokens the untrained hybrid runs 1.05× to 2.2× faster than the dense baseline. Llama-3.2-1B runs from 2.22× at 4,096 tokens down to 1.15× at 32,768. Gemma-4-E2B runs 1.05× to 1.35×, with its peak at 8,192. The lead shrinks because only the early layers are replaced.

Conditions

• Forward pass only, batch 1, bf16, real pretrained weights.
• The gate is untrained. Next-token accuracy collapses, so this is speed, not quality.
• Known confounds in the runner: no warm-up, the baseline is always timed first, and the model is reloaded for each row. Rows below 4,096 tokens are shown in grey for that reason.
• Earlier runs were called Superfly. It is the same project.

What came of it

A mechanism-agnostic benchmark runner with warm-up and peak-memory tracking, so the next run does not have these confounds.

Superfly 8B runs

Under the Superfly name, five variants on Llama-3.1-8B against the dense baseline, 512 to 16,384 tokens: forward pass, batch 1, bf16, one batch per row, PG19 validation, untrained gate. Every variant is slower than dense up to 8,192 tokens, at 0.22× to 0.95×. At 16,384 tokens the best variant reaches 14,488 tok/s against the baseline’s 11,326, which is 1.28×; the next reaches 1.19×. Only three variants have a 16,384-token row. Token accuracy is near zero, so this is speed, not quality.

Data

Timing CSVs: butterflygate/results, commit 74f7f0d. The 8B rows are in one CSV.

@misc{paul2026butterflygate,
  author = {Paul, Kanishk},
  title = {ButterflyGate},
  year = {2026},
  howpublished = {Research page, kanishkpaul.com},
  url = {https://www.kanishkpaul.com/research/butterflygate},
  note = {Code and data: https://github.com/kanishkpaul/butterflygate},
}