NVIDIA Launches Nemotron 3 Nano Omni Model, Unifying Vision, Audio and Language for up to 9x More Efficient AI Agents

AI agent programs at present juggle separate fashions for imaginative and prescient, speech and language — dropping time and context as they move knowledge from one mannequin to the opposite.

Unveiled at present, NVIDIA Nemotron 3 Nano Omni is an open multimodal mannequin that brings these capabilities collectively into one system, enabling brokers to ship quicker, smarter responses with superior reasoning throughout video, audio, picture and textual content. This best-in-class mannequin provides enterprises and builders a manufacturing path for extra environment friendly and correct multimodal AI brokers with full deployment flexibility and management. 

Nemotron 3 Nano Omni units a brand new effectivity frontier for open multimodal fashions with main accuracy and low price, topping six leaderboards for advanced doc intelligence, and video and audio understanding.

AI and software program corporations already adopting Nemotron 3 Nano Omni embody Aible, Utilized Scientific Intelligence (ASI), Eka Care, Foxconn, H Firm, Palantir and Pyler, with Dell Applied sciences, Docusign, Infosys, Ok-Dense, Lila, Oracle and Zefr evaluating the mannequin. 

“To construct helpful brokers, you may’t wait seconds for a mannequin to interpret a display,” mentioned Gautier Cloix, CEO of H Firm. “By constructing on Nemotron 3 Nano Omni, our brokers can quickly interpret full HD display recordings — one thing that wasn’t sensible earlier than. This isn’t only a velocity enhance: It’s a elementary shift in how our brokers understand and work together with digital environments in actual time.”

Nemotron 3 Nano Omni Permits Sooner, Leaner Multimodal Agents

Contemplate an AI agent for buyer assist processing a display recording whereas analyzing uploaded name audio and checking knowledge logs — or an agent for finance tasked with parsing PDFs, spreadsheets, charts and voice notes. At this time, most agentic programs accomplish these duties with separate fashions for imaginative and prescient, speech and language. 

This strategy will increase latency via repeated inference passes, fragments context throughout modalities, and provides price and inaccuracies over time.

By combining imaginative and prescient and audio encoders inside its 30B-A3B, hybrid mixture-of-experts structure, Nemotron 3 Nano Omni eliminates the necessity for separate notion fashions, driving inference effectivity at scale. It pairs this effectivity with sturdy multimodal notion accuracy, enabling AI programs to obtain 9x increased throughput than different open omni fashions with the identical interactivity. The result’s decrease prices and higher scalability with out sacrificing responsiveness or high quality.

In agentic programs, Nemotron 3 Nano Omni can work alongside proprietary cloud fashions or different NVIDIA Nemotron open fashions — reminiscent of Nemotron 3 Tremendous for high-frequency execution or Nemotron 3 Extremely for advanced planning — in addition to proprietary fashions from different suppliers, to energy sub-agents for agentic workflows reminiscent of laptop use, doc intelligence and audio-video reasoning.

• Laptop use brokers — Nemotron 3 Nano Omni powers the notion loop for brokers navigating graphical consumer interfaces, reasoning over onscreen content material and understanding consumer interface state over time. H Firm’s newest laptop utilization agent, powered by Nemotron 3 Nano Omni, makes use of a local enter decision of 1920×1080 pixels to obtain high-fidelity visible reasoning. In preliminary evaluations on the OSWorld benchmark, this integration confirmed a big leap in navigating advanced graphical interfaces and used Nemotron 3 Nano Omni’s capability to course of very high-resolution pictures. 
• Doc intelligence — Interprets paperwork, charts, tables, screenshots and mixed-media inputs, enabling brokers to cause throughout visible construction and textual content content material coherently. Important for enterprise evaluation and compliance workflows.
• Audio and video understanding — For customer support, analysis and monitoring workflows, Nemotron 3 Nano Omni maintains audio-video context, tying what was mentioned, proven and documented right into a single reasoning stream as a substitute of disconnected summaries.

Open and Customizable, Deployable Anyplace

Nemotron 3 Nano Omni is launched with open weights, datasets and coaching strategies — giving organizations full transparency and management over how the mannequin is custom-made and deployed. 

Builders can use instruments like NVIDIA NeMo for customization, analysis and optimization for domain-specific use instances. As a result of the Nemotron household of fashions is open, organizations can deploy them in environments that meet regulatory, sovereignty or knowledge localization necessities.

The Nemotron 3 household — together with Nano, Tremendous and Extremely fashions — has seen over 50 million downloads up to now yr. Omni extends the household’s capabilities into multimodal and agentic domains. 

The mannequin is on the market on Hugging Face, OpenRouter and construct.nvidia.com as an NVIDIA NIM microservice and via a broad ecosystem of NVIDIA Cloud Companions, inference platforms and cloud service suppliers. 

Its open, light-weight structure helps constant deployment from native programs like NVIDIA Jetson {hardware}, NVIDIA DGX Spark and DGX Station to knowledge heart and cloud environments. 

Go to the NVIDIA technical weblog for tutorials, cookbooks and deployment guides for Nemotron 3 Nano Omni use instances. Stay up to date on agentic AI, NVIDIA Nemotron and extra by subscribing to NVIDIA information, becoming a member of the neighborhood and following NVIDIA AI on LinkedIn, Instagram, X and Fb.