Nikiwa vs others
Evaluating crypto AI agents is hard. Their quality depends not only on the model behind them, but on how quickly and accurately they reach live on-chain data, and general-purpose benchmarks do not measure that. So we built our own: 95 tasks that test an agent end to end on the work crypto users actually bring to it, from tracing a wallet and vetting a token to reading prediction markets and interpreting security alerts. As AI becomes part of how people research, trade and stay safe in crypto, the industry needs a shared way to tell which agents can be trusted. We hope this benchmark can serve as one, for Nikiwa and for every other crypto AI agent.
Scope: what is benchmarked
This is a benchmark of crypto security AI agents, not of language models in isolation. An agent wraps a model with tools: live blockchain data, token and market feeds, address intelligence, prediction-market and social data. Nikiwa is such an agent, built by LegalBlock for security, compliance and investigation work.
To show what that tooling is worth, three classes of systems answer the same tasks, under the same rubrics and the same judge:
Nikiwa in Default, Auto, Thinking and Pro Thinking modes: agents with different speed and approach for different purposes.
Surf AI in Instant, Task and Task Max modes: the closest specialised alternative, also equipped with crypto data tooling.
Frontier models from Anthropic, OpenAI, Google, DeepSeek and others, queried without agent tooling.
Why the scores are so far apart: a general-purpose LLM, however capable, has no way to look up a wallet's balance, a token's holders or today's Polymarket odds. On those tasks it can only reason from training data. The benchmark measures the complete system a user gets; it is not a ranking of raw model intelligence.
Results
Purpose-built crypto agents clearly outperform general-purpose models, and Nikiwa leads that group on both accuracy and speed. Its advantage is largest wherever an answer depends on fresh on-chain data; on general crypto knowledge the field stays close.
Leaderboard
General crypto Q&A is the one field where general-purpose LLMs compete: the answers are in their training data. As soon as a task needs live data (a wallet, a token, a transaction, a market) their scores collapse and the agents pull away. Nikiwa leads seven of ten fields; the specialised crypto agent leads DeFi research and TradFi & tax.
Accuracy vs. speed
Each dot is a system, placed by average time per question and overall score · up and to the left is better
Key findings
Top overall score, by Nikiwa Pro Thinking. Nikiwa Default follows at 80.0%; Nikiwa configurations take the top three places.
Nikiwa’s lead over the strongest general-purpose LLM, Claude Opus 5 (36.5%). The 21 LLMs average 31.0%.
Faster than the closest crypto AI agent: 51 s vs 242 s per question, at a higher score.
Benchmark tasks
The benchmark consists of 95 tasks across 10 fields, written the way people actually ask. The full task set stays private, so it cannot leak into training data or be tuned against.
Fields
95 tasks across 10 fields · weighted towards wallet and token analysis
Profiling an address: attribution, holdings, counterparties and risk exposure.
Identity, market read, holder concentration, liquidity and contract safety of a token.
Finding and reading Polymarket and Kalshi markets, odds and trader activity.
Positioning and performance of perpetuals traders on Hyperliquid.
Conceptual questions on wallets, transactions and protocol mechanics.
Theft labels, exposure to illicit entities, and what the evidence does and does not prove.
Protocol rankings, liquid-staking rates, yields and funding activity.
Checking X accounts for authenticity and risk signals.
Decoding a single transaction: what moved, where, and why.
Where crypto meets traditional finance: ETFs, crypto-exposed equities, tax treatment.
Sample tasks
Three of the 95 tasks · complete rubrics and scores of all 28 systems
Notes and limitations
Scores come from an LLM judge applying task-specific rubrics. Judging is identical across systems, but it is not a human review.
Fields with 3–5 tasks (general Q&A, security forensics, DeFi research, social, transaction analysis, TradFi & tax) move a lot with a single answer. Treat their averages as directional.
Time is average wall-clock time per completed question and depends on load and network conditions during the run.
A few tasks went unscored for some systems because of run errors; overall scores average scored tasks only.
The benchmark measures complete systems, including their tooling. It is not a ranking of the underlying language models.