Molmo 2

The second major feature is dense video captioning, powered by Molmo2-Cap, a dataset of 104,000 unique videos with 431,000 clip-level captions that average hundreds of words each. Human annotators deliver rich spoken narrations, which are transcribed and then enriched with frame-level details from Molmo itself to capture subtle visual cues. This results in supervision that includes events, relationships, and rare details far surpassing typical large-scale caption sets. For the video understanding model, this means it learns to produce highly descriptive, searchable narratives of long clips, enabling a user to scan a summary and locate the moment a cup falls or a player scores. Unlike surface-level captions, this dense approach ensures the model internalizes the full temporal story, making it invaluable for archival retrieval, accessibility, and research that demands comprehensive scene understanding.

Beyond single-video analysis, Molmo 2 introduces robust multi-image reasoning through Molmo2-MultiImageQA and Molmo2-MultiImagePoint. These datasets contain 45,000 image sets (2–5 images) with 72,000 QA pairs and over 470,000 pointing samples, teaching the model to ground answers across multiple visual contexts. For instance, it can compare charts, reference a document image against a photograph, or identify an object seen from different angles in separate frames. This capability is critical for applications like medical imaging comparison, multi-camera surveillance, and cross-referencing textual documents with visual evidence. Additionally, Molmo 2’s inference-time SlowFast strategy allows processing key frames at high resolution while downscaling others, maintaining accuracy on long videos while using fewer vision tokens—a practical efficiency boost for deployment.

Under the hood, Molmo 2’s architecture pairs a vision encoder with a language model backbone (Qwen 3 for 8B and 4B variants; Olmo for the 7B-O fully open variant). A lightweight connector interleaves visual tokens with text, image indices, and timestamps, enabling joint reasoning over space, time, and language. Training proceeds in two stages: first, pretraining on a mixture of 60% captioning, 30% pointing, and 10% NLP data with Tulu-derived supervised fine-tuning to maintain language proficiency. Second, multimodal SFT mixes images, multi-image sets, videos, and pure text with sampling rates tuned empirically. For video, up to 128 frames are sampled at ≤2 fps, encoded with a vision transformer, and patches pooled into 3×3 windows before passing through the connector. Crucially, bi-directional attention across all visual tokens between frames boosts multi-image and video performance, and a token-weighting scheme balances diverse tasks during optimization.

Concrete use cases span robotics, where Molmo 2 counts robot grasps and verifies manipulation sequences, providing action logs with visual proof. In traffic monitoring, it tracks vehicles through occlusions and reports incident timestamps and locations. For accessibility, the video understanding model can narrate actions and locate objects, such as 'Find the window above the kitchen sink,' aiding the visually impaired. Content moderation benefits from artifact detection: given a prompt like 'Point out every instance of inconsistent lighting in this generated video,' the model returns exact frames and regions, helping developers debug AI systems. Similarly, scientific researchers analyzing animal behavior can ask 'Locate the animal that makes the plank tip downward' and receive precise spatial-temporal evidence, replacing manual annotation with scalable, verifiable AI outputs.

Molmo 2 is designed for a wide audience: AI researchers exploring grounded video learning, robotics teams needing open perception stacks, assistive technology developers, and educators teaching multimodal AI. The 8B variant offers top performance, the 4B variant maximizes efficiency, and the 7B-O variant provides a fully open pipeline from vision encoder to language model using Olmo. All variants are available on HuggingFace alongside PixMo-derived datasets, cookbooks, and a demo on the Ai2 Playground, with an API via OpenRouter. Under Apache 2.0, the codebase includes pretraining, SFT, and long-context SFT scripts, enabling fine-tuning on custom data. In summary, Molmo 2 sets a new open standard for video pointing, tracking, and dense captioning, proving that careful data curation can outperform larger models trained on noisy video corpuses—delivering trustworthy, evidence-backed answers for anyone who works with visual time-series data.