There has been an overwhelming amount of new models hitting HuggingFace. I wanted to kick off a thread and see what open-source LLM has been your new daily driver?
Personally, I am using many Mistral/Mixtral models and a few random OpenHermes fine-tunes for flavor. I was also pleasantly surprised by some of the DeepSeek models. Those were fun to test.
I believe 2024 is the year open-source LLMs will catchup with GPT-3.5 and GPT-4. We鈥檙e already most of the way there. Curious to hear what new contenders are on the block and how others feel about their performance/precision compared to other state-of-the-art (closed) source models.
deleted by creator
What sort of tokens per second are you seeing with your hardware? Mind sharing some notes on what you鈥檙e running there? Super curious!
I would also be interested in Code-Pilot Models that are reaching for same performance like GitHub or Microsofts paid Models.
Currently I use TabbyML but the available Models are by far inferior.
deleted by creator
I was pleasantly surprised by many models of the Deepseek family. Verbose, but in a good way? At least that was my experience. Love to see it mentioned here.
Personally I find myself renting GPU and running Goliath 120b. Smaller models could do what I鈥檓 doing if I spent more time optimizing my prompts. But every day I鈥檓 doing different tasks, and Goliath 120b will just handle whatever I throw at it, no matter how sloppy I am. I鈥檝e also been playing with LLAVA and Hermes vision models to describe images to me. However, when I really need alt-text for an image I can鈥檛 see, I still find myself resorting to GPT4; the open source options just aren鈥檛 as accurate or detailed.