Frontier LLMs come every 1.5-2 years
I have a theory that the LLM frontier moves every 1.5-2 years.
Let me qualify. I mean that a major leap of the whatever is the best model (currently GPT-4) happens with that interval. Incremental improvements don't count: e.g. Claude 3 Opus is claimed to be maybe slightly better than GPT-4, but it is not a major leap. Neither is GPT-4 turbo, which is perhaps very slightly better.
Here are the announcement dates for the major LLM releases from OpenAI:
- GPT-3: May 2020
- ChatGPT: Nov 2022 (based on GPT-3)
- GPT-4: Mar 2023
- GPT-5: Oct 2024 according to Metaculus prediction markets; Polymarket puts 41% on Q3'24 and 80% on any time before the end of 2024.
The data hints at something like this: 3->4 took almost 3 years, and 4->5 will take 1.5-2 years if you believe the prediction markets (ChatGPT wasn't a major version). So this hypothesis is quite thinly supported.
It would be pretty surprising if this timeline continued to hold. It is highly variable whether each company feels ready to announce something at 1.5 years, and how far along a model is at that time. And model, compute and data requirements keep getting larger, which would make it harder to keep this cycle going. But on the other hand, more and more money is pouring in right now. And 18 months is the typical VC cycle: startups plan their growth and runway for 1.5 years, and the best time to raise money is when you've just had a major successful launch.
The cycle does not seem to apply to Anthropic's releases though:
Nor Meta's Llama:
- Llama: Feb 2023
- Llama 2: Jul 2023
- Llama 3: Jul 2024 according to Metaculus, with most probability mass in 2024.
I would explain the difference mostly by Anthropic and Meta doing more catch-up growth (Anthropic was founded in 2021, after GPT-3 was released). But I'm sure there are other factors, and anyway this analysis is coarse.