2023: “I hope Generative AI is not going to kill us all!”

2024: “I hope Generative AI is going to go from proof of concept experiment at my company to small production deployment in the next 12-18 months!”

Prediction: In the next phase of AI, some gains will come from *scaling up* dataset & LLM size- and many will now come from *scaling down*.

Bigger dataset/model != better anymore.

Scaling up: In some frontier areas like multimodal where we're likely far from data scale…

Prediction: In the next phase of AI, some gains will come from *scaling up* dataset & LLM size- and many will now come from *scaling down*.

Bigger dataset/model != better anymore.

Scaling up: In some frontier areas like multimodal where we're likely far from data scale…

Excited to announce the acceptance of our paper titled 'Training Language Model Agents without Modifying Language Models' (title change to “Offline Training of Language Model Agents with Functions as Learnable Weights” in the revised version.) at #icml2024 ICML Conference

1/N

It was awesome to get to chat with this group about some meaty topics in enterprise AI!

One of the biggest themes was: how should enterprises actually get value out of AI products? Three quick thoughts:
- Evaluation: Need to be able to quantify success on real, trusted metrics…

LLMs cannot “recursively self improve”

This falls out from the conceptual matrix described in section 2.1 of our paper below. Any LLM can only approximate this matrix, so it has rows missing. For “improvement” it needs to fill out missing rows (1/n)

arxiv.org/abs/2402.03175

We’re getting ready to start Lightspeed's premier event for enterprise IT and innovation leaders, Velocity NYC. Follow along for highlights from our lineup of speakers at the forefront of AI, including:

→ Alex Ratner, CEO and Co-Founder of Snorkel AI
→ Arvind Jain, CEO and…

AI value is likely coming to the enterprise in two major waves:
- Wave I: based on public data; low stakes deployment with human/system error buffers; marginal unit ROI. Ex: internal code co-pilot.
- Wave II: based on enterprise-specific data & expertise; high stakes…

This is an exceptional and succinct articulation of LLM use cases and implications by Vishal. Crazy he called it in 22’(!!)

Incredibly excited to see OSS LLMs leap forward w/ Meta's Llama 3, and for Snorkel AI to be part of the 'Llama ecosystem.'

Thanks to Joe Spisak for the great mention!

Watch the full video here: buff.ly/49YQfJv

#Llama3 #SnorkelAI

Prediction: we'll soon abstract away from the distinction of *how* LLMs are adapted.

Developers will instead focus exclusively on *what* labeled data they adapt their LLMs with.

Whether this data is injected via a prompt, fine-tuning, alignment, etc will become a low-level…

Prediction: we'll soon abstract away from the distinction of *how* LLMs are adapted.

Developers will instead focus exclusively on *what* labeled data they adapt their LLMs with.

Whether this data is injected via a prompt, fine-tuning, alignment, etc will become a low-level…

As long as AI systems are trained to reproduce human-generated data (e.g. text) and have no search/planning/reasoning capability, performance will saturate below or around human level.

Furthermore, the amount of trials needed to reach that level will be far larger than the…

In a world with Llama3/Phi3/Arctic/etc (and more coming!), base LLMs are now a commodity.

AI is now all about the inputs & outputs that customize an LLM for unique use cases:
- Inputs: Labeled, curated data to prompt/tune/align
- Outputs: How you map from model -> product/UX

Yes! the same way all tech companies write their own code, all AI companies will train, optimize, run their own models (instead of out-sourcing AI to other companies through APIs).

1/ With LLMs like Llama 3 & Phi 3, enterprises are no longer blocked on the models.

The game is now about one thing: developing the data to tune & evaluate these LLMs for real business use cases.

Excited to support this w/ the new Snorkel AI release! venturebeat.com/data-infrastru… 🧵