Nvidia Rolls Out AI Chatbot & Assistant That Runs Directly On Your Computer
Image via Nvidia
Nvidia wants to turbocharge the way we interact with artificial intelligence with the introduction of Chat with RTX, a demo app that brings the power of smart chatbots directly to your desktop. This tool, designed for Windows PCs equipped with NVIDIA RTX, allows for local, speedy, and customized generative AI interactions, moving beyond the cloud servers to offer a more personal and immediate AI experience.
Chat with RTX is a versatile AI companion that not only responds to your questions but is also capable of summarizing YouTube videos and documents, providing answers tailored to your data—given that it’s built into your PC.
A bonus about tapping inside rather than into the cloud is that the software only requires only an RTX 30- or 40-series GPU with a minimum of 8GB of VRAM to operate. This means users can now leverage the advanced tensor cores in their GeForce RTX cards for a variety of tasks, including running their own local large language models (LLMs) with personal documents and data. Running locally on Windows RTX PCs, Chat with RTX ensures fast results while keeping the user’s data secure on the device.
Currently in its beta phase (version 0.2 as of February 13), Chat with RTX is still in the early stages of development but shows potential. The installation process is straightforward, akin to setting up Nvidia graphics drivers, though it does require a hefty 35GB of space, plus an additional 100GB for optimal performance. The software harnesses retrieval-augmented generation (RAG), NVIDIA TensorRT-LLM software, and NVIDIA RTX acceleration to bring generative AI capabilities to GeForce-powered Windows PCs, allowing users to connect local files as a dataset to open-source LLMs like Mistral or Llama 2.
Geeky specifications aside, a key feature of Chat with RTX is its ability to process queries using local files for convenient contextually relevant answers, rethinking the way users search through their saved content.
“For example, one could ask, ‘What was the restaurant my partner recommended while in Las Vegas?’ and Chat with RTX will scan local files the user points it to and provide the answer with context,” Nvidia explains.
Chat with RTX is now free to download.
