High End Desktop PC

[infernocus]

දැන් තියෙන PC එකට GPU එකක් ගහපන් 16gb වගේ හරි 24gb වගේ හරි.
AI වලට කියලා වෙනම PC හදන්න එපා. උබ වැඩේ serious යනවා නම් ඉතින් cloud තමයි

24GB is not enough to run LLMs but it is enough If you want do run diffusion models. You need a lot of VRAM if you want to run a good LLM locally. one option could be using a computer with a unified memory, like an Apple Silicon mac, or GDX Sparx or AMD AI Max chip, but all of these solutions are expensive and can only able to run quantized models, so better off with using ChatGPT or any other AI API.

 

[Adefer]

Waiting for more reviews, But I've seen a lot of LLMs work with good enough performance. There won't be Nvidia level support but tool and library support will increase.
Nvidia has GDX Sparx, but that will cost $3000, however I don't think we can buy this for MSRP, It will be a better option, but can't justify the cost.

I have a fully smart home built using ESPHome devices + Home-assistant, I need an AI to fully automate and decision making to run the home.
Things like predicting the solar output and optimizing the power usage. (As I am on fully off grid).

One simple example, If there was less solar output in the last few days, make a prediction that today's output will also be lower, so in order to fully charge off-grid batteries, AI needs to turn off hot water, and save power.

And I need this to work offline without any 3rd party API support, that will be my own HAL9000

why you need llm for this? just make some workflows + a few forecasting models and that’s enough. you can use something like gpt-mini only as a language bridge to control and give natural commands, but for the automation logic itself no need full llm. i work in building automation field and we did this at very large scale buildings ,energy optimization, load shedding, predictive maintenance etc , all done with standard workflows + ml models (forecasting, usage, etc).