සාමන්යෙන් දැනට තියෙන හොදම llm model ටික local pc එකක් රන් කරන්න . හොදම gpu එක මොකක්ද ??
Latest ads
-
Gampaha Thenya Mishel
- menuka2000
- Updated:
-
Handmade Character Soft Toy Bluey Bingo- anil1961
- Updated:
-
Colombo Dahua POE Switch 8 Ports (DH-PFS3010-8ET-65)- hashaz2012
- Updated:
-
Singapore V2Ray VPN for Tunneling
- hu KANNA
- Updated:
-
Dating Website & Online Chat Room- hotvideo
- Updated:
AI, LLM වැඩකරුවො පොඩ්ඩක් එනවද
- Thread starter lkkolla
- Start date
there's a sub for that
r/LocalLLaMA
also checkout Microsoft SLMs
r/LocalLLaMA
also checkout Microsoft SLMs
Depends.
1. Denata thiyana hodama open source LLMs thama
Medium scale ( 3B to 30B parameters)
- LLaMA 3.1
- Mistral v0.2
- qwen 2.5
Large scale (more than 30B parameters or Mixture of Models )
- Deepseek v3
- LLaMa 3.2
- Mixtral
- Qwen
2. LLM ekak run karanna ona gpu vram eka mostly depends wenne me Dewal ekka
- LLM eke parameters gana (like 3B, 7B, 8B, 70B). Parameters gana wedi weddi ona vram ekakth wedi
- anith eke model quantization. Model eke format eka anuwath use wena vram eka wenas (GGUF , GPTQ). samanyen mehema quantize karala thiyana models performance original ekata sapekshawa adui hebei overall loku wenasak ne
3. Model inference ekedi model eka load wena precision eka adu karana methods use karala. Such as LoRA and QLoRA inference. Meken godak gpu vram.eka save karanna puluwan.
3. LLM ekak run karanna GPU ekak thora gaddi mainly consider karanna ona dewal
1. GPU VRAM - Meka thamai oyage gpu eke ram capacity eka like 4gb, 8gb, 12gb, 24gb. Me vram eke thama oyage LLM eka run wenne. Oyage LLM eke parameters gana wedi nam ekata haryana oya gpu vram.eka ganna ona. Eka gpu ekak madi nam keepak aran model eka gpus athare split karala run karanna puluwan
2. Number of cuda cores - Meka nikan ape cpu eka thiyana cores wage. Me cuda cores gana wedi wenna wedi wenna oyage model eka run wena speed eka wedi.
3. Power - oyage gpu eka kochchara power ekak gannawada kiyana eka normally oya consumer vs workstation gpus compare karaoth consumer evala power eka gannawa wedi.
Summary ekak widiyata gaththoth vram eka balapanne model size ekata. Cuda cores gana balapanne model eka run wena speed ekata.
4. Recommend karana gpus
Consumer
Best - 4090
Better - 3090
Minimum 8gb wath thiyana ekak ganna neththan wedak ne 12 or 16 thibboth hodai. Mokada 7B parameters model ekak run karanna 8gb recommended at least.
Anith eka 4090 eke cuda cores 16k wage thiyanawa eka Nisa speed.
Workstation
Best - h100 80gb
best for value - A6000 48gb
Online
User friendly- vast.ai
Cheapest- massedcompute.com
Can buy a6000 for 0.36 per hour with a coupon code.
Ollama wage software ekak use karanna puluwan model ekak use karaddi Meka model eka run karana ekai use karana ekai user friendly interface ekakin denawa
1. Denata thiyana hodama open source LLMs thama
Medium scale ( 3B to 30B parameters)
- LLaMA 3.1
- Mistral v0.2
- qwen 2.5
Large scale (more than 30B parameters or Mixture of Models )
- Deepseek v3
- LLaMa 3.2
- Mixtral
- Qwen
2. LLM ekak run karanna ona gpu vram eka mostly depends wenne me Dewal ekka
- LLM eke parameters gana (like 3B, 7B, 8B, 70B). Parameters gana wedi weddi ona vram ekakth wedi
- anith eke model quantization. Model eke format eka anuwath use wena vram eka wenas (GGUF , GPTQ). samanyen mehema quantize karala thiyana models performance original ekata sapekshawa adui hebei overall loku wenasak ne
3. Model inference ekedi model eka load wena precision eka adu karana methods use karala. Such as LoRA and QLoRA inference. Meken godak gpu vram.eka save karanna puluwan.
3. LLM ekak run karanna GPU ekak thora gaddi mainly consider karanna ona dewal
1. GPU VRAM - Meka thamai oyage gpu eke ram capacity eka like 4gb, 8gb, 12gb, 24gb. Me vram eke thama oyage LLM eka run wenne. Oyage LLM eke parameters gana wedi nam ekata haryana oya gpu vram.eka ganna ona. Eka gpu ekak madi nam keepak aran model eka gpus athare split karala run karanna puluwan
2. Number of cuda cores - Meka nikan ape cpu eka thiyana cores wage. Me cuda cores gana wedi wenna wedi wenna oyage model eka run wena speed eka wedi.
3. Power - oyage gpu eka kochchara power ekak gannawada kiyana eka normally oya consumer vs workstation gpus compare karaoth consumer evala power eka gannawa wedi.
Summary ekak widiyata gaththoth vram eka balapanne model size ekata. Cuda cores gana balapanne model eka run wena speed ekata.
4. Recommend karana gpus
Consumer
Best - 4090
Better - 3090
Minimum 8gb wath thiyana ekak ganna neththan wedak ne 12 or 16 thibboth hodai. Mokada 7B parameters model ekak run karanna 8gb recommended at least.
Anith eka 4090 eke cuda cores 16k wage thiyanawa eka Nisa speed.
Workstation
Best - h100 80gb
best for value - A6000 48gb
Online
User friendly- vast.ai
Cheapest- massedcompute.com
Can buy a6000 for 0.36 per hour with a coupon code.
Ollama wage software ekak use karanna puluwan model ekak use karaddi Meka model eka run karana ekai use karana ekai user friendly interface ekakin denawa
Last edited:
