🇦🇺𝕄𝕦𝕟𝕥𝕖𝕕𝕔𝕣𝕠𝕔𝕕𝕚𝕝𝕖@lemm.ee to

LocalLLaMA@sh.itjust.worksEnglish · 10 days ago

How much gpu do i need to run a 90b model

1

How much gpu do i need to run a 90b model

🇦🇺𝕄𝕦𝕟𝕥𝕖𝕕𝕔𝕣𝕠𝕔𝕕𝕚𝕝𝕖@lemm.ee to

LocalLLaMA@sh.itjust.worksEnglish · 10 days ago

Do i need industry grade gpu’s or can i scrape by getring decent tps with a consumer level gpu.

Chat

Sylovik@lemmy.world
link
fedilink
English
arrow-up
0·
9 days ago
You may try Harbor. The description claims to provide an OpenAI-compatible API.

LocalLLaMA@sh.itjust.works

localllama@sh.itjust.works

You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !localllama@sh.itjust.works

Community to discuss about LLaMA, the large language model created by Meta AI.

This is intended to be a replacement for r/LocalLLaMA on Reddit.

Visibility: Public

This community can be federated to other instances and be posted/commented in by their users.

1 user / day
2 users / week
47 users / month
47 users / 6 months
0 local subscribers
2.42K subscribers
52 Posts
27 Comments
Modlog