At lunch today we were talking how it would make sense if
7B models were 7 cents,
13B models were 13 cents,
and of course 8B models were 8 cents (per 1M tokens).
It just makes sense, so we set our prices!
deepinfra.com/pricing

2x the speed, 24/7, 8x7b Mixtral in fp16 is now 24c/24c per million tokens. Enjoy!

The small 8b Llama3 is just $0.10 per 1M tokens. Get started here deepinfra.com/meta-llama/Met…

Also we just dropped pricing on most of our 7b, 13b and 70b models to $0.10, $0.18, $0.64 per million input tokens. We will always have the best prices. deepinfra.com/pricing

Official Mixtral-8x22b-Instruct model just got released and is now on DeepInfra. This is the best open LLM and we are hosting at the best price of $0.65 / 1M tokens. deepinfra.com/mistralai/Mixt…

Just added the first instruction finetunned version of mixtral 8x22B deepinfra.com/HuggingFaceH4/… still not official from mixtral but not bad

We set the bar when we launched the first mixtral, and we're going to do it again! The new mixtral, with 65K context, at ... 65c / 1 million tokens! This new model is almost 3 times larger. deepinfra.com/mistralai/Mixt…

databricks/dbrx-instruct is now on @deepinfra. $0.6 per 1M tokens - the best price of all providers. Also up to 130 tps. Try it our here: deepinfra.com/databricks/dbr…

You can now host your custom LLMs at DeepInfra. It's managed LLM hosing service. Pay-per GPU/h $2/A100, $4/H100. It's super simple. Read more here deepinfra.com/blog/custom-ll…

Checkout the new Mistral 0.2 model. We are running it in fp8 on H100s and it does 160tps. deepinfra.com/mistralai/Mist…

Jensen is like a rock star, leather jacket, big crowds, and giving autographs :)

A couple exciting updates! First, we quantitatively evaluated the improvement from combining Quiet-STaR with chain-of-thought (i.e. letting the model think before each CoT token). We found it improves zero-shot CoT accuracy on GSM8K by over 7%!

Also we just shipped the highly requested multimodal Llava1.5. Give it a try deepinfra.com/llava-hf/llava…

Guided JSON response is now available DeepInfra API. Read more about it here
deepinfra.com/blog/json-mode. Restricting the output to JSON had almost no performance penalty and is FREE.

We’re committed to building AI that improves lives & unlocks a better future for humanity. We are proud to sign the Open Letter: Build AI for a Better Future along with @SVAngel, OpenAI, Meta, Google, Microsoft & many others. openletter.svangel.com