NVIDIA’s Dory introduces a unique approach to scaling performance in small language models through its recurrent looping mechanism. This 4-billion-parameter model divides its architecture into three sections, with the middle block looping up to four times per token to enhance reasoning capabilities. For example, at four loops, Dory achieves 192 layer operations per token, significantly improving its performance on step-by-step tasks like Terminal Bench and GDPval. However, this design comes with trade-offs, including slower generation speeds and increased memory usage, as highlighted by The Stack.
Explore how Dory balances its compact size with advanced reasoning capabilities, including its standout performance gains on sequential benchmarks and its efficiency in reducing task completion steps. You’ll also gain insight into its limitations, such as static knowledge-based performance and hardware-specific constraints, as well as practical considerations like memory caching options and loop count recommendations. This explainer offers a detailed breakdown to help you assess whether Dory’s unique architecture aligns with your specific use case.
How Dory’s Recurrent Looping Works
TL;DR Key Takeaways :
- Dory’s novel looping mechanism enables strong performance in step-by-step reasoning tasks, rivaling much larger models, but comes with trade-offs in speed, memory usage, and static performance on knowledge-based tasks.
- The looping mechanism was introduced late in training, with loop counts randomly assigned between one and four during the final 1.5 trillion tokens. Performance gains plateau beyond four loops, as shown in benchmarks like MMLU.
- Dory reportedly outperforms Google’s Gemma 4 26B A4B on average across 12 benchmarks, but this lead heavily depends on a single non-hallucination score. Without it, Gemma leads in overall performance.
- Each additional loop significantly slows generation speed and increases memory usage. On certain hardware, reasoning mode with multiple loops becomes impractical due to time and resource constraints.
- Licensing for Dory is unclear, with conflicting claims between the OpenMDW license and NVIDIA’s Open Model License Agreement. Users should verify the applicable license before commercial deployment.
Dory’s architecture is divided into three sections: an 8-pair input block, a 20-pair recurrent middle block, and an 8-pair output block. The input and output blocks process each token once, while the middle block loops up to four times. At one loop, each token undergoes 72 physical layer operations, increasing to 192 operations at four loops. Despite this, Dory’s weight file remains static, with all 4 billion parameters stored in memory simultaneously.
The looping mechanism was introduced late in training. For the first three-quarters of pretraining, Dory functioned as a standard non-looped model. Looping was added during the final 1.5 trillion tokens of training, with loop counts randomly assigned between one and four. Only the last pass was trained and graded, allowing the model to decode representations at any loop count. Beyond four loops, performance gains plateau, as shown in the MMLU benchmark: 71.19 at four loops, 71.22 at six loops, and 71.17 at eight loops.
This mechanism focuses extra compute within each token’s hidden state, avoiding the need for additional reasoning tokens or new weights. At four loops, Dory uses approximately 9 billion parameters’ worth of compute per token. In comparison, Gemma 4 26B A4B activates only 3.8 billion parameters per token while keeping 25.2 billion in memory. Both models were tested under identical conditions.
Performance Gains in Step-by-Step Tasks
Dory excels in sequential reasoning tasks. On Terminal Bench version 2.1, its score nearly doubled from 12.5% at one loop to 22.2% at four loops. Similarly, on the GDPval workplace task benchmark, Dory’s Elo rating rose from 815 at one loop to 967 at four loops, approaching Qwen 3.6 35B’s 987 Elo. In benchmarks like AA LCR (long documents), IFBench (instruction following), and tau three Banking (tool use), Dory at four loops outperformed even 26B and 35B models.
Efficiency in reasoning is another strength. At four loops, Dory completed GDPval tasks in 49 turns instead of 116, showing its ability to reduce steps. Most gains are captured with just two loops. For example, two loops account for 80% of the SWE bench coding improvement and 86% of the GDPval gain. On the SWE bench, Dory scored 50.3% at one loop and 61.8% at four loops. The model card recommends two loops as the default setting, balancing performance and speed.
Here are more guides from our previous articles and guides related to NVIDIA that you may find helpful.
- Mac Studio M5 Ultra vs Nvidia DGX Spark: Compared for Local AI
- Valve Adds Initial NVIDIA GPU Driver Support to SteamOS
- Open Source Exploit Unlocks 39% More Speed on Nvidia RTX 4090
- Dynamic Routing in NVIDIA Switchboard Boosts Speed By 50%
- Alibaba Panjiu AL128 Replaces NVIDIA with 128 Linked Chips
- NVIDIA NVFP4 Boosts Prompt Processing by 46% on Blackwell
- NVIDIA DGX Spark Price Hits $4,699 Amid Memory Shortages
- Thermaltake Element V NVIDIA Edition Unveiled
- NVIDIA RTX Spark N1X Beats the RTX 5070 Laptop GPU By 4.3%
- NVIDIA Releases 4-Billion-Parameter Dory Under Open MDW
Knowledge-Based Tasks and Limitations
Dory’s performance in knowledge-based tasks remains static. On the AA Omniscience factual accuracy benchmark, scores were 10.03 at one loop, 10.08 at two loops, and 10.03 at four loops, showing no improvement with additional loops. Looping refines hidden states but does not enhance knowledge retrieval or accuracy.
In expert question tests like GPQA Diamond and Humanity’s Last Exam, Dory also falls behind larger models. For instance, Gemma 26B scores 21.13 on AA Omniscience, more than double Dory’s score. These results underscore Dory’s focus on reasoning rather than factual knowledge.
Comparisons with Larger Models
Dory reportedly outperforms Google’s Gemma 4 26B A4B on average across 12 benchmarks, with an unweighted average score of 40.60 at four loops compared to Gemma’s 36.73. However, this lead relies heavily on a single non-hallucination score, where Dory scores 79.38 compared to Gemma’s 16.36. This metric rewards models for avoiding confident wrong answers, including admitting “I don’t know.” Without this score, Gemma leads 38.6 to 37.1 across the remaining benchmarks.
The non-hallucination score contributes approximately 5.25 points to the average difference between Dory and Gemma, exceeding Dory’s total lead of 3.87 points. Across the 10 benchmarks listed on the model card, Dory achieves five wins and five losses against the 26B model, indicating competitive but not dominant performance. Recalculating the average across the other 11 tests without the non-hallucination column gives Gemma the edge.
Speed and Memory Trade-Offs
Each additional loop significantly affects generation speed. On an RTX 4090, speeds dropped from 188 tokens per second at one loop to 128 at two loops, and 78 at four loops. On an 18GB M3 Pro, Dory generated 1,316 tokens in 35.1 seconds at one loop but only 1,022 tokens in 70 seconds at four loops, reducing token usage by 22% but doubling the time.
Testing on a 16 GB M4 using NVIDIA’s llama.cpp fork revealed further challenges. Running the recommended two loops with reasoning mode enabled, the model took over 10 minutes to generate 2,604 tokens, producing tokens at 4.36 per second, making reasoning mode impractical on such hardware.
Memory usage also scales with loop count unless manually adjusted. By default, Dory uses “per loop” caching, where each pass through the middle block saves a separate copy of earlier tokens. On a 16 GB M4 Mac with a 32,000-token context, memory usage increased from 1.3 GB at one loop to 3.5 GB at four loops. Switching to “last loop” caching kept memory flat at 1.3 GB but reduced accuracy, dropping scores by 2.1 points on AA LCR and nearly 2 points on Terminal Bench. This forces earlier passes to rely on final loop representations, diverging from the model’s training setup.
Hardware and Software Support
Dory officially supports NVIDIA’s Hopper and Blackwell architectures, but independent testers have run it on an RTX 4090 and Apple Silicon Macs. vLLM added support for Dory on October 25, 2026, with stable version 0.33 released on November 2, 2026. However, vLLM servers currently lock in a single loop count per request, as dynamic loop changes remain an open pull request.
For Mac or llama.cpp users, NVIDIA’s fork or community pull requests are required. The open llama.cpp pull request notes that Metal support is untested and configurations above four loops are unsupported.
Licensing Contradictions
NVIDIA’s repository lists conflicting licenses. The main model card specifies the OpenMDW license, which allows unrestricted commercial use. However, another file claims the model is governed by the NVIDIA Open Model License Agreement. Users should confirm the applicable license before deploying Dory commercially.
Dory’s looping mechanism allows it to compete with much larger models in reasoning tasks, offering a unique balance of size and performance. However, its static knowledge-based performance, slower speeds, and memory trade-offs require careful consideration. For agent-style tasks and long documents, Dory’s efficiency makes it a compelling option, provided its constraints align with specific workloads.
Media Credit: The Stack
Disclosure: Some of our articles include affiliate links. If you buy something through one of these links, Geeky Gadgets may earn an affiliate commission. Learn about our Disclosure Policy.
