AI Help - Document වලින් AI GPT Model එකක් Training කරන්නෙ කොහොමද ?

Nidarshana_k

Well-known member
  • Feb 19, 2022
    29,657
    1
    51,400
    113
    ප*යාගල
    sorry.com
    කොල්ලනි පොඩි ප්‍රශ්නයක්. මම ලග තියෙනව Document සෙට් එකක්. ( PDF/Video/Docs/Txt Etc ),
    මෙව පාවිච්චි කරල Custom AI Model හදාගන්නෙ කොහොමද? පොඩි ගයිඩ් එකක් දියන් මන් ඉතුරු ටික හොයාගන්නම්

    තව එකක් Nvidea GPU එකක් නෑ මගෙ Mac එකේ. අඩු ගාන්කට Model Training කරන්න පුලුවන් Platforms තියෙනවද

    Thanks in Advance මගෙ සුන්දරයෝ :cool:

     
    • Like
    Reactions: Cenon

    Nidarshana_k

    Well-known member
  • Feb 19, 2022
    29,657
    1
    51,400
    113
    ප*යාගල
    sorry.com

    Asmodeus

    Well-known member
  • Feb 6, 2023
    6,877
    15,667
    113
    Ursa Major
    That the AI model depends on the problem you are looking for and the dataset too. Just assume you want a yes/no answer from some data. You need to clean those data and divide training/test sets and then use an appropriate classification algorithm (your yes/no question usually needs logistic regression). and use a neural network to train and obtain your answer. Then you can save those models and use them with any software you want.
     

    Nidarshana_k

    Well-known member
  • Feb 19, 2022
    29,657
    1
    51,400
    113
    ප*යාගල
    sorry.com
    you want a yes/no answer from some data.
    No brother I want a conversational AI trained on my documents. Im thinking about going with openAI embedding model and LLM
    use a neural network to train and obtain your answer. Then you can save those models and use them with any software you want.
    Great idea but my client is cheap ass guy he doesn't want to spend much :(
     
    • Like
    Reactions: Asmodeus

    gulp

    Active member
  • Mar 17, 2015
    130
    133
    43
    1. Ingestion

    docs -> converted embeddings -> vector db

    2. User query -> converted embeddings -> cosine similarity search in vector db -> pick top N results -> results converted to text + original user query -> LLM -> Answer