අපි Deepseek-R1 install කරමු

NEMISIS

Well-known member
  • Nov 13, 2013
    11,397
    19,531
    113
    Colombo
    7b eken loku wadak ganna baha. 70b wage ekak ape mechine wala run krnawata wada hodai itim :lol:
    If you have a mac you can do this easily, it's processor and unified memory is very good for this kind of workloads, it may not be good as running on a GPU but should be good enough. i am thinking buying a M4 Macbook next week with 48GB Ram, Just for this purpose. 70b model is about 43GB, General rule is, you need to have 1.2*model size of RAM, 48GB should be good enough.

    btw i played around with both 1.5b and 7b models, i was happy with the results given the size of the model, didn't expect GPT4 level results from this. Having a conversation with a locally running AI is something we never thought of.
     
    Last edited:

    podi.kollo

    Well-known member
  • Oct 9, 2012
    3,629
    529
    113
    In the Mirrôr
    If you have a mac you can do this easily, it's processor and unified memory is very good for this kind of workloads, it may not be good as running on a GPU but should be good enough. i am thinking buying a M4 Macbook next week with 48GB Ram, Just for this purpose. 70b model is about 43GB, General rule is, you need to have 1.2*model size of RAM, 48GB should be good enough.

    btw i played around with both 1.5b and 7b models, i was happy with the results given the size of the model, didn't expect GPT4 level results from this. Having a conversation with a locally running AI is something we never thought of.
    "Mac" is good for smaller models or quantized versions. However, it has some limitations, such as the lack of NVIDIA GPU support (it relies on Apple’s Metal API, which isn’t as optimized for large LLMs). With my M4 Pro (24GB), I’ve tried the 7B version locally. But trust me, 48GB of RAM is not enough to run the 70B model smoothly. At least 48GB or more of unified memory is required if you’re considering running the 70B model on a Mac.
     
    • Love
    Reactions: NEMISIS