Exciting news for those who want to experiment with Mixture of Experts (MoE) models but find training and fine-tuning too expensive! With MyShell, we are thrilled to introduce JetMoE, a Llama-2-level model trained for under 0.1 million $. With 8B total and 2.2B active

I still remember walking into the 7th floor of Sutardja Dai Hall and seeing the PR2 robot do magic. It’s awesome to see foundation model and robotics getting together!

Massachusetts Institute of Technology (MIT) Princeton University NVIDIA Lepton AI Paper: arxiv.org/abs/2402.19481
Code: github.com/mit-han-lab/di…
Blog: hanlab.mit.edu/blog/distrifus…
Website: hanlab.mit.edu/projects/distr…

A great collaboration with Muyang Li Tianle Cai Jiaxin Cao Qinsheng Zhang Han Cai bddppq Ming-Yu Liu Kai Li and Song Han

My colleague ashu garg and I have spent the past few months exploring the world of generative AI inference. We believe it's a massive opportunity for startups in 2024!

foundationcapital.com/why-2024-will-…

We’ve just added Lepton AI to our independent performance benchmark for serverless LLM APIs - and it's showing some impressive results!

Lepton is now leading on Tokens per Second throughput for Mistral AI's Mixtral 8x7B Instruct. It is the only Mixtral hosting provider in our

All right, fully open-source code, Apache license for anyone and any company to use freely: github.com/leptonai/searc…

Our goal: enable creators and enterprises to build AI applications as easy as possible, like this search application.

Happy Friday and have fun!

🚀Introducing The LLM Inference Provider Leaderboard leaderboard.withmartian.com - a live-updated, unbiased eval of API Inference products.

Featuring: Abacus.AI, Anyscale, DeepInfra, Decart, Fireworks AI, Lepton AI, Together AI, Perplexity, Replicate, as

Building an AI app has never been easier. Over the weekend, we built a demo for conversational search with <500 lines of python, and it's live at search.lepton.run. Give it a shot! Code to be open sourced soon as we clean up all '# temp scaffolds' stuff. (1/x)

There is a war on LLM benchmarks, but benchmark on API is hard if not impossible (will explain more). For product use case, how about having your own LLM? We're launching Lepton AI's LLM engine for an easy, dedicated, high-performance runtime. Type your HuggingFace model id and

Thank you to the incredible team Lepton AI for harnessing the potential of Medusa in practical LLM applications and offering a variety of finely-tuned Medusa heads to the community!

Writing prompts is tricky and tedius. We at Lepton AI recently worked with our friends at HippoML to release api for PromptLLM - better prompts for AIGC🖼️

Remember the song 'down by the bay'? Read on to guess which image is generated by AI prompt :)