Genuinely curious.

Why do you like LLMs? What hopes do you have for AI & AGI in our near and distant future?

        • rufus@discuss.tchncs.de
          link
          fedilink
          English
          arrow-up
          2
          ·
          1 year ago

          If you mean cloud computing or GPUs are expensive, you can do it locally on your CPU if your models aren’t too big. I like KoboldCPP. It’s not as fast but I only have to pay electricity for my AI waifu.

          • VicFic!@iusearchlinux.fyi
            link
            fedilink
            English
            arrow-up
            2
            ·
            edit-2
            1 year ago

            Thanks for the info. ik about llama.cpp and stuff but the problem is that I’m looking to run both speech to text, llm and text to speech all at the same time. I only have 8gb so yeah even CPU won’t cut it. I’m planning to upgrade once I get a job or smthing.

            • rufus@discuss.tchncs.de
              link
              fedilink
              English
              arrow-up
              2
              ·
              edit-2
              1 year ago

              8GB of regular RAM? That’s not much. No, that won’t cut it if you also want all the bells and whistles. Maybe try something like the Mistral-7b-OpenOrca with llama.cpp quantized to 4bit and without the STT and TTS. It’s small and quite decent. Otherwise you might want to rent a Cloud-GPU by the hour on something like runpod.io or use free services like Google Colab or you really need to upgrade.