A nice blog post about Keras 3 from NSF Unidata
unidata.ucar.edu/blogs/news/ent…
The post's conclusion:

'For deep learning training I (Thomas) will be using a Keras 3 API exclusively. It more closely resembles the scikit-learn api and I find it to be easier to explain.'

Why Google Deepmind's Mixture-of-Depths paper, and more generally dynamic compute methods, matter:

Most of the compute is WASTED because not all tokens are equally hard to predict

Gemma in Keras: how to build a chatbot and fine-tune it to speak like a pirate 🏴‍☠️🦜. This was a fun demo to make! It runs with Keras on JAX with the new keras.distribute.ModelParallel API.
Colab: bit.ly/gemma-pirate-d…
Video: youtu.be/AzQBFmPDtTI?si…

Not one but two Gemma competitions are currently live on Kaggle. And we have Keras starter notebooks for both:

kaggle.com/code/awsaf49/p…

kaggle.com/code/awsaf49/k…

Have fun with Gemma! (and check out the prizes: $250,000 in total!)

Keras 3.0.5 and Keras-nlp 0.8.1 now come pre-installed in Kaggle notebooks -- so you can run Gemma without any extra install steps 🚀

François Chollet it's happening!

This was a lot of fun to demo.
Colab here: bit.ly/gemma-pirate-d…
Fav quote from pirate Gemma: 'It's nice that ye like math!'⚔️💣⚔️

My notes reading the Gemma paper:
- arch similar to llama
- 6T tokens for the 7B model!!!
- huge vocab size
- GeGLU for FFN, I wish they ablated the dim used there, people tend to use 4x while I like to be closer to 2.5-3x
- surprised they use Sandwich-Norm, I think Normformer

Keras has added a new distributed training API, supporting full model parallelism for Gemma, and large models in general. It is backed by the XLA compiler in JAX. Code sample here: kaggle.com/code/nilaychau…

Google is back baby! Taking the first spot for open models on the Hugging Face LLM leaderboard for its sizes (2B & 7B): huggingface.co/collections/go…

A great implementation of Neural Radiance Fields (NeRFs) with Keras, running on JAX. And a nice backstory 😃.
github.com/ariG23498/nerf…

News from small and efficient ML models (the whole package is 1.0MB) 😃. Impressive accuracy!

This is really nice!

🎥 Check out the details of our recent Keras integration with François Chollet and meg.ai 🇨🇦 @ #home!