The number that failed the memory test
On a Tuesday morning in Seoul, a number landed in my feed that did not survive first contact with arithmetic. A relayed sell-side note โ Nomura, by way of a Web3 aggregator, by way of a forwarded screenshot โ claimed that Moonshot AI, the lab behind Kimi, was training a model with "50 to 60 trillion parameters." The headline wrote itself: a Chinese lab leapfrogging the frontier by one or two orders of magnitude, a doubling narrative, a funding story with a hero.
I ran the memory math before I ran the sentiment. At FP8, sixty trillion parameters occupy roughly 60 terabytes of VRAM. At BF16, double that โ 120 terabytes โ before a single optimizer state or activation is allocated. An H100 with 80GB of HBM3 cannot hold a rounding error of that weight tensor. You would need somewhere between 750 and 1,500 of them just to load the weights into memory, and several times that count to actually train, once you account for Adam's momentum buffers, the fp32 master weights, and the activation checkpoints that scale with batch and sequence length. That is not a product roadmap. That is a national-scale compute program.
And the same note, in its own second paragraph, insisted that "compute is the primary bottleneck." Both statements cannot be true. One of them is noise, and my prior โ after twenty-one years of watching infrastructure claims collide with physics โ is that the loud, round, marketable number is the liar. The quiet one, the constraint, is almost always the truth.
Here is why a crypto reader should care about a busted AI rumor: the arithmetic that falsifies the parameter count is the identical arithmetic that prices every decentralized compute market on-chain. When I see a DePIN project promise "frontier-scale training on idle GPUs," I am reading the same broken sentence, just written in a different language.
Context: a three-hop claim and the one signal worth keeping
Let me be precise about provenance, because in this industry the messenger is frequently the message. What I am working from is not a technical report, not a model card, not an official announcement. It is a sell-side research note (Nomura) that was summarized by a Web3-flavored aggregator (Beating AI), that was then re-parsed by an analyst, that reached me as a forwarded artifact. Three hops. Every hop introduces unit drift, rounding, and โ most dangerously โ narrative amplification. Anyone who has reverse-engineered a token's tokenomics from a Medium post knows the pattern: the further you get from the primary source, the rounder the numbers become.
The genuinely load-bearing claims in that chain were not the parameter count. They were two smaller, quieter signals that happen to align with everything I know about the industry's actual direction of travel:
First, that K3.1 โ the incremental ".1" release โ would center on Agent long-task capability: an AI that does not merely chat, but continuously executes multi-step work across extended horizons. Second, that compute is the binding constraint, a claim consistent with export controls and with the well-documented scarcity of high-end accelerators in China.
Those two signals are the story. The parameter number is decoration. And the reason the decoration matters is that it is the same decoration the crypto compute sector uses to sell tokens.
So let me do what I always do: ignore the marketing layer, trace the constraint back to first principles, and ask what the underlying structure actually permits. Because the constraint here โ verifiable, scarce, expensive compute โ is not an AI-only problem. It is the defining economic problem of decentralized physical infrastructure, and it is the problem the crypto compute market claims to have solved.
Core: dissecting the arithmetic of an impossible model
Tracing the memory limits back to the genesis of the scaling curve. The first thing to understand is that parameter count is a storage problem before it is a training problem. A dense model's weights must be resident in memory to be used. Sixty trillion parameters at FP8 is 60TB; at BF16 it is 120TB. Modern training clusters shard across hundreds or thousands of accelerators precisely because no single device holds the model. But sharding does not eliminate the requirement โ it distributes it. You still need aggregate HBM equal to the model plus optimizer plus activations.
Do the sizing. A 70-billion-parameter model โ a genuinely large, frontier-adjacent dense model โ trains comfortably on a few hundred high-end GPUs with aggressive parallelism. Scale that linearly to 60 trillion parameters and you are at roughly 850 times the compute and memory footprint. That is not an incremental upgrade; it is a discontinuity. And it requires a data center with power delivery, cooling, and interconnect fabric that Moonshot does not own โ the note itself admits they rent from Alibaba Cloud.
There is a second impossibility: sparsity. Even if we charitably reinterpret "50-60 trillion" as an MoE total parameter count rather than a dense one, the ratio breaks. DeepSeek-V3 is 671 billion total with 37 billion active โ roughly an 18x sparsity, and that was already an engineering landmark. If you hold 60 trillion total with 60 billion active, you are at a 1,000x sparsity ratio. At that ratio, routing collapses. Expert utilization becomes so sparse that gradients for most experts vanish, the router degenerates toward a few dominant paths, and training efficiency craters. There is no published precedent for a stable 1,000x MoE. So the number is not just implausible as a dense model โ it is implausible as a sparse one too.
Dissecting the unit confusion that created the ghost. Here is the Occam's razor I keep coming back to, and it is the kind of error I have seen a hundred times in cross-language technical relay. The industry's frontier training token counts currently live exactly in the 10-to-20-trillion range. Llama 3 was trained on roughly 15 trillion tokens. Headline models have pushed past that. "50 to 60 trillion" is not a parameter count; it is, almost certainly, a training-token count that was mislabeled as parameters somewhere in the translation chain. In Chinese-to-English and English-to-Chinese technical relay, the parameter/token distinction and the trillion/billion magnitude are the two most common corruption points. One bad translation, one hop of a screenshot, and a token count becomes a parameter count, and a token count becomes a parameter count, and a sober data point becomes a viral miracle.
This is not a hypothetical. I have audited whitepapers where "TPS" silently migrated from transactions-per-second to "theoretical peak scale," where "throughput" became "capacity," where a testnet figure was laundered into a mainnet claim across two blog posts. Units are the connective tissue of technical claims, and they are the first thing to rot in transmission.
The FLOPs reality check. Let me put a third stake in the ground, because the crypto audience in particular is trained to think in orders of magnitude and will appreciate the discipline. The compute required to train a model scales roughly with parameters times tokens. A 70B model on 15T tokens is on the order of 6ร10ยฒยณ FLOPs. If you genuinely trained a 60T dense model, even on a modest token budget, you would be several orders of magnitude beyond that โ into territory that requires tens of thousands of top-tier accelerators running for months, under power contracts that exceed a mid-sized city's draw. There is no version of this that is compatible with "compute is the bottleneck." The bottleneck and the mega-model are mutually exclusive. The bottleneck wins. It always wins. Physics does not take meetings.
Why this is actually a crypto story. Now the turn. Everything above is the arithmetic of scarcity โ the arithmetic of a resource so expensive and so constrained that it becomes the pivot of an entire industry's strategy. And scarcity of verifiable compute is precisely the thesis on which the entire decentralized compute sector is built. Render, Akash, io.net, Bittensor, Gensyn โ the pitch is uniform: the world's GPUs sit idle, aggregate them, route jobs, settle payment on-chain, undercut the hyperscalers.
I have spent enough time in these protocols' codebases to say this carefully: the aggregation is real, the settlement is real, and the matching is real. What is not solved โ what is structurally unsolved โ is verification. If I rent you compute to train or to serve inference, how do I prove, cryptographically, that you ran the computation I paid for and not something cheaper? The decentralized compute market is, at its core, a trust problem wearing a bandwidth costume.
Mapping the verification leak in the compute market. This is where the AI rumor and the crypto infrastructure converge into one insight, and it is the insight I want the reader to carry out of this piece. The reason the "60-trillion" number could even be floated is that no one can independently verify a closed lab's training configuration. You take it on faith, or you don't. That is a metadata leak โ not a leak of data, but a leak of trust. The claim is unverifiable by design, so the market prices the narrative instead of the fact.
The decentralized compute market has the identical hole. When a DePIN network claims a certain FLOPs capacity, when a miner claims a certain job completion, when an inference provider claims it ran the model you specified at the precision you requested โ the default answer is "trust the oracle." And a compute oracle that simply reports what a worker told it is not an oracle. It is a stenographer for the party being audited.
The layer-two bridge is just a pessimistic oracle โ and so is every "verifiable compute" layer that quietly outsources its proof to a committee of staked validators who have every incentive to rubber-stamp. I have audited settlement logic in early state-channel systems where the challenge window was technically present but economically irrational to use, which means the optimistic assumption was the only assumption in practice. If challenging a bad compute attestation costs more than the job is worth, the fraud proof is decoration. The system is optimistic in name and centralized in behavior.
Optimism is a gamble, ZK is a proof. The clean distinction maps perfectly onto compute verification. An optimistic compute market posts results and hopes no one contests within a window; a ZK compute market produces a succinct proof that the computation was executed correctly. The former is cheap and trusts human and economic actors not to misbehave; the latter is expensive and trusts mathematics. The entire trajectory of verifiable inference โ zkML, proof-of-inference, recursive proof systems โ is the migration from the first to the second. And the migration is slow for the same reason the AI number is unfalsifiable: producing a proof of a large model's forward pass is currently orders of magnitude more expensive than running the forward pass itself. We are, right now, in the optimistic era of verifiable AI, pretending to be in the ZK era.
The agentic turn and the multi-sig blind spot. Now fold in the second real signal from the source material: Agent long-task capability. This is where my recent research at a Seoul-based Layer 2 firm becomes directly relevant, because I have spent the last year analyzing how autonomous AI agents interact with smart contracts, and I have found the same structural flaw in every implementation I have examined.
When an agent executes a long task, it eventually reaches the point where it must act โ move funds, sign a transaction, call a contract, approve a spend. And the moment an agent holds a key, you have a multi-sig or an EOA controlled by a non-human, non-deterministic process. The failure mode is not that the agent is malicious. The failure mode is error compounding. A task that is 95% reliable per step, run over twenty steps, succeeds about 36% of the time. Over fifty steps, it succeeds roughly 8% of the time. Long-horizon autonomy does not degrade gracefully; it degrades exponentially. And every one of those failing steps can be a signed, irreversible, on-chain transaction.
This is the same disease as composability. Composability is a double-edged sword for security, and agentic execution is composability with a mind of its own. Each step in a DeFi agent's workflow โ approve, swap, bridge, stake, rebalance โ composes with the next. Each composition multiplies the attack surface and the error surface simultaneously. A prompt injection at step three can steer the entire remaining chain of on-chain actions. I have mapped this exact class of vulnerability: an agent that reads an untrusted input, treats it as instruction, and then executes a privileged action with no human in the loop. The signature of the bug is always the same โ a trust boundary between "data the agent reads" and "actions the agent takes" that has been erased.
Finding the edge case in the consensus mechanism. Here is the uncomfortable synthesis. The AI lab's compute scarcity and the crypto agent's trust problem are the same problem viewed from two ends. The lab cannot prove what it trained; the agent cannot prove why it acted. Both are trying to establish verifiable computation over a process that is expensive to verify and cheap to fake. The lab solves it with opacity and brand. The agent, if we are not careful, solves it with a multi-sig that a language model can drain.
Contrarian: the decentralized compute thesis is a pessimistic oracle, and the real bottleneck is verification, not FLOPs
Let me now say the thing that will annoy half the people who fund this sector. The prevailing crypto-compute narrative is that the bottleneck is supply โ that if we simply aggregate enough idle GPUs and route enough jobs, we undercut the hyperscalers and democratize frontier AI. This framing is wrong in the same way the "60-trillion-parameter" claim is wrong. It mistakes a quantity for a capability.
The binding constraint on decentralized compute is not raw FLOPS. It is verification cost and coordination latency. Training a frontier model requires tightly-coupled, low-latency, high-bandwidth interconnect between thousands of accelerators. A geographically-distributed, heterogeneously-specified, latency-jittery network of consumer GPUs cannot substitute for a coherent NVLink-and-InfiniBand fabric, no matter how many of them you aggregate. The math of distributed training is unforgiving: gradient synchronization over a WAN is dominated by communication, and communication over a mesh of home internet connections is a catastrophe for the synchronous workloads that frontier training demands. The decentralized network is wonderful for embarrassingly-parallel workloads โ rendering, some inference, batch jobs โ and structurally terrible for the workloads the "frontier AI" narrative wants to claim.
So the sector's marketing โ "train the next frontier model on the crowd" โ is a pessimistic oracle. It posts an optimistic claim and trusts that no one will challenge it, because the challenge is expensive to mount and the claimants control the metadata. The honest version of the pitch is narrower and more defensible: decentralized networks are excellent for verifiable inference at the edge, for rendering, for privacy-preserving compute where the trust model is inverted. The dishonest version is the one that borrows the AI lab's broken parameter count and recycles it as a capacity figure.
And there is a second bias worth flagging, one that connects the AI rumor back to its crypto cousin. The source chain here is a sell-side research note โ a document whose function is to recommend trades, not to establish technical truth. I have read enough crypto "research" from funds with positions to know the genre. The number gets inflated because an inflated number supports a valuation, and the valuation supports a raise. When I see a Web3 aggregator covering AI infrastructure, I note the domain mismatch: the same outlet that cannot distinguish a parameter from a token is now the relay for a claim about model architecture. The mislabeling is not incidental. It is the tell. Wherever a narrative crosses domains without paying the toll of expertise, the numbers inflate and the units rot.
The efficiency inversion nobody wants to admit. Here is the contrarian kicker. The compute bottleneck that supposedly limits China's AI ambitions has produced the opposite of weakness in one specific dimension: it has forced efficiency innovation that the well-resourced West has less pressure to pursue. Sparse MoE, aggressive quantization (FP8, INT4), distillation, and training-efficiency tricks matured fastest in the constrained ecosystem precisely because constraint is the mother of optimization. DeepSeek's cost-efficiency was not an accident of generosity; it was the arithmetic of scarcity. The constraint that caps the parameter count is the same constraint that produced the efficiency edge. The AI rumor got the sign of the effect exactly backwards: it read scarcity as a limit on capability, when scarcity was the engine of a different kind of capability.
The same inversion applies in crypto. The networks with the tightest verification budgets are the ones innovating hardest on recursive proofs, proof aggregation, and succinct verification. The teams with unlimited budget to run expensive fraud proofs have the least incentive to compress them. Constraint is where the real engineering happens. The marketing never mentions it, because "we are forced to be clever" sells fewer tokens than "we are building the biggest thing."
Takeaway: watch the proof, not the parameter
So where does this leave us, and what should you actually track?
Do not anchor to the number. "50-60 trillion parameters" is a ghost โ almost certainly a mislabeled token count, an inflated funding narrative, or an MoE total-count dressed up as a dense one. In any of those readings, the claim is a unit error or a sales pitch, and both decay on contact with a technical report. The signal that survives the arithmetic is the constraint: compute is scarce, verification is unsolved, and the agent that executes long tasks is the agent that can silently sign a catastrophic transaction.
The forward-looking judgment is this. The next two years of AI-crypto convergence will not be decided by who has the largest model or the largest aggregated GPU count. They will be decided by who solves verifiable execution first โ the ability to prove, succinctly and cheaply, that a computation was run correctly, whether that computation is a model's forward pass or an autonomous agent's chain of on-chain actions. The optimistic era is ending. The market that prices narratives instead of proofs has a finite runway, because narratives are unfalsifiable only until someone builds a verifier that makes them falsifiable.
When that verifier arrives โ a practical proof-of-inference, a cheap recursive proof for agent workflows โ it will not just validate AI claims. It will reprice every "trust the oracle" compute network and every agent that currently signs multi-sig transactions on faith. And the teams that spent the scarce years learning to be efficient, rather than the funded years learning to be loud, will be the ones holding the proofs.
The question I am left with, watching the parameter count float around the timeline like an unverified transaction: when the next round of capital chases the next impossible number, will anyone stop to trace the gas limit back to the genesis block โ or will we keep paying for the headline and calling it infrastructure?