a Mixture-of-Experts (MoE) gonzo engineer, threat modeling for security and privacy.
my posts and opinions are mine and not those of my employers, clients, or sensible people.
ontologically defined as #amateurRadio, #ransomware, #infosec, #parenting #quids, #aikido, #photography, #obsidian, #PKM, #omnifocus, #radio, #mobile, #bestPractices, #reiki, #johnnyDecimal, #AI, and whatever we’re calling the next Internet.
i am candid about #mentalHealth, sometimes to my detriment.
PHL & IOW, US
@gabek @mitch @obsidianmd i don’t either, i have other ways of doing what the paid version supports. i use cloud foundation models and local; my backends for embeddings are always ollama, lmstudio, and/or anythingLLM.
#anythingLLM has an easily deployed docker release and desktop application. it’s not as able in managing and cross-threading conversations as LM (really Msty does it best) but #aLLM has a nice setup for agents and RAG.
@INeedMana if they offer an OpenAI-ish compatible API (e.g. https://blahblah/v1) you can add it as an OpenAI service with a new endpoint and your own API creds inside AI Providers but IDK about the various other extensions.
the mesh-AI one uses fabric though and you can configure fabric to use mistral’s API just fine i reckon?
@mitch @obsidianmd this extension is great and i wish others used it instead of reimplementing things https://github.com/pfrankov/obsidian-ai-providers
@mitch @obsidianmd does that use the AI Providers extension? LocalGPT is an extension that does and doesn’t have that “premium tier” thing going on.
do you configure the embeddings for local vectors and also local inference? can you compare it to like the RAG function in AnythingLLM or open-webui or something similar?
i use a variety of methods haven’t got a favorite yet.
OH MY GOD @obsidianmd it knows about #johnnyDecimal ??? look at Edge Sources in Settings! 🤯
@gabek @mitch @obsidianmd some of the small models i like using with obsidian vaults locally are deepseek+llama distills and MoE models for every occasion. fiction and creative, classification and vision. there’s a few 8x merged models that are extremely fun for d&d.
i have a speech operated adventure like #Zork that uses a 6x MoE that can be really surreal.
there’s a phi2-ee model on hf that is small and fast at electrical eng work, i use that for a radio and electronics project vault!