🇦🇺𝕄𝕦𝕟𝕥𝕖𝕕𝕔𝕣𝕠𝕔𝕕𝕚𝕝𝕖@lemm.ee to LocalLLaMA@sh.itjust.worksEnglish · 10 days agoHow much gpu do i need to run a 90b modelmessage-squaremessage-square14fedilinkarrow-up11arrow-down10file-text
arrow-up11arrow-down1message-squareHow much gpu do i need to run a 90b model🇦🇺𝕄𝕦𝕟𝕥𝕖𝕕𝕔𝕣𝕠𝕔𝕕𝕚𝕝𝕖@lemm.ee to LocalLLaMA@sh.itjust.worksEnglish · 10 days agomessage-square14fedilinkfile-text
minus-squarered@lemmy.ziplinkfedilinkEnglisharrow-up0·9 days agoall you need is 2 3090s and some good CPU and lot of ram, then you should be able to get like 10tps
all you need is 2 3090s and some good CPU and lot of ram, then you should be able to get like 10tps