ijeff@lemdro.id to Technology@lemmy.worldEnglish · 2 years agoLarge Language Models up to 4x Faster on RTX With TensorRT-LLM for Windowsblogs.nvidia.comexternal-linkmessage-square1linkfedilinkarrow-up162arrow-down14file-textcross-posted to: aistuff@lemdro.id
arrow-up158arrow-down1external-linkLarge Language Models up to 4x Faster on RTX With TensorRT-LLM for Windowsblogs.nvidia.comijeff@lemdro.id to Technology@lemmy.worldEnglish · 2 years agomessage-square1linkfedilinkfile-textcross-posted to: aistuff@lemdro.id
minus-squarekorewa@reddthat.comlinkfedilinkEnglisharrow-up3·2 years agoDang I need to try these for now only the stable diffusion extension for automatic 1111 is available. I wonder if it will accelerate 30b models that doesn’t fit all in the gpu vram. If it only accelerates 13b then it was already fast enough
Dang I need to try these for now only the stable diffusion extension for automatic 1111 is available.
I wonder if it will accelerate 30b models that doesn’t fit all in the gpu vram.
If it only accelerates 13b then it was already fast enough