Gaia runs faster on Ryzen AI PCs, using the XDNA NPU and RDNA iGPU:
Running large language models (LLMs) on PCs locally is becoming increasingly popular worldwide. In response, AMD is introducing its own LLM application, Gaia, an open-source project for running local LLMs on any Windows machine.
Gaia is designed to run various LLM models on Windows PCs and features further performance optimizations for machines equipped with its Ryzen AI processors (including the Ryzen AI Max 395+). Gaia uses the open-source Lemonade SDK from ONNX TurnkeyML for LLM inference. Models can allegedly adapt for different purposes with Gaia, including summarization and complex reasoning tasks.
[...] MD's new open-source project works by providing LLM-specific tasks through the Lemonade SDK and serving them across multiple runtimes. Lemonade allegedly "exposes an LLM web service that communicates with the GAIA application...via an OpenAI compatible Rest API." Gaia itself acts as an AI-powered agent that retrieves and processes data. It also "vectorizes external content (e.g., GitHub, YouTube, text files) and stores it in a local vector index."
Also at Phoronix.
AMD Press Release and GitHub Repository.
(Score: 2, Interesting) by Anonymous Coward on Monday March 24, @05:24PM
What models are actually available and viable for Gaia?
How does it compare with Ollama + Open WebUI? Was fairly easy to get Deepseek and other stuff ( https://ollama.com/library [ollama.com] ) up and usable. You can do RAG and web search stuff too. However the default RAG stuff doesn't do PDFs that well (some characters seem to be misread/wrong) - get better results using the Azure AI stuff. So if you want RAG on Ollama + Open Web UI you should probably convert your stuff to a text in an AI friendly format for better results.
(Score: 4, Interesting) by corey on Monday March 24, @10:30PM (3 children)
I admit that I'm a big AI stick-in-the-mud. But I genuinely want to know why people would want to run an LLM locally, other than for educational purposes? I've seen discussions where people are running them on PCs and always wonder why. I run a flightradar24.com feed (with SDR + RPi), and understand that reciprocally, other people will question the desire to do this, so is it also just personal interest / tinkering?
(Score: 5, Informative) by EvilSS on Tuesday March 25, @12:04AM (1 child)
(Score: 2) by Freeman on Tuesday March 25, @02:02PM
The privacy thing is great, but also security for businesses. What do you want your employees to be using when working on sensitive material? chatgpt.com or something local?
Joshua 1:9 "Be strong and of a good courage; be not afraid, neither be thou dismayed: for the Lord thy God is with thee"
(Score: 2) by looorg on Wednesday March 26, @04:48PM
I think it would be confidentiality and security for the information. You can run a LLM based on information you know and control and that you don't want to feed into other peoples systems. That way you could also know references of all data in the model. You could build some very niche specialized systems.
I can imagine a few things I would perhaps like to feed a few hundred or so books and topical papers into one. To see what it can make of it. Or that you can use as a reference to later produce text then that fits the writing style of what you put in. Instead of having these giant hodgepodge of data from multiple sources and many writers making the style and conclusions perhaps contradictory or odd.