“`html
SANTA CLARA, CA – May 16, 2024 – NVIDIA today announced that the gpt-oss-20b open-source large language model (LLM) is now readily accessible to Windows developers, offering a significant boost to AI-accelerated request development. This release, coupled wiht Microsoft’s AI Foundry Local, allows developers to run powerful AI models directly on thier RTX AI PCs without relying on cloud connectivity.
For Windows developers, access is streamlined through Microsoft AI Foundry Local, currently in public preview. Foundry Local provides an on-device AI inferencing solution, integrating into development workflows via command line, Software Development Kits (SDKs), and Application Programming Interfaces (APIs). The system leverages ONNX Runtime, optimized with CUDA, and will soon incorporate NVIDIA TensorRT for RTX, promising further performance gains. Installation of Foundry Local followed by the command “Foundry model run gpt-oss-20b” in a terminal is all that’s required to begin utilizing the model.
the gpt-oss-20b model, developed by the OpenLLM project at Berkeley AI Research (BAIR), boasts 20 billion parameters and is licensed under the apache 2.0 licence, promoting open innovation. Performance benchmarks, as showcased by NVIDIA, demonstrate the model’s capabilities across a range of RTX AI PCs, including the ASUS ROG Zephyrus G14 (RTX 4060), the MSI Titan GT77 HX (RTX 4090), and the Dell XPS 15 (RTX 4070). The RTX 4090,for example,achieves up to 13 tokens per second with the gpt-oss-20b model.
This release marks a pivotal moment for AI development on Windows, empowering a new generation of AI-powered applications. Developers can now build applications with advanced reasoning capabilities directly on their local machines,fostering faster iteration and greater control over their data. Potential applications span creative workflows, productivity tools, and the development of complex AI agents.
Stay informed about the latest advancements with the weekly RTX AI Garage blog series, featuring community-driven AI innovations, NVIDIA NIM microservices, and AI
