Sam Altman, CEO of OpenAI, dropped a 💣 at a recent MIT event, declaring that the era of gigantic #Artificial Intelligence models like GPT-4 is coming to an end. He believes that future progress in AI needs new ideas, not just bigger models. So why is that revolutionary? Well, this is how OpenAI's LLMs (the models that 'feed' chatbots like ChatGPT & Google Bard) grew exponentially over the years: ➡️GPT-2 (2019): 1.5 billion parameters ➡️GPT-3 (2020): 175 billion parameters ➡️GPT-4: (2023): amount undisclosed – but likely trillions of parameters That kind of parameter growth is no longer tenable, feels Altman. Why?: ➡️RETURNS: scaling up model size comes with diminishing returns. ➡️PHYSICAL LIMITS: there's a limit to how many & how quickly data centers can be built. ➡️COST: ChatGPT cost over over 100 million dollars to develop. What is he NOT saying? That access to data is becoming damned hard & expensive. So if you have a model that keeps needing more data to become better, that's a problem. Why is it becoming harder and more expensive to access data? 🎨Copyright conundrums: Getty Images, individual artists like Sarah Andersen, Kelly McKernan & Karloa Otiz are suing AI companies over unauthorized use of their content. Universal Music asked Spotify & Apple Music to stop AI companies from accessing their songs for training. 🔐Privacy matters & regulation: Italy banned ChatGPT over privacy concerns (now back after changes). Germany, France, Ireland, Canada, and Spain remain suspicious. Samsung even warned employees not to use AI tools like ChatGPT for security reasons. 💸Data monetization: Twitter, Reddit, Stack Overflow & others want AI companies to pay up for training on their data. Contrary to most artists, Grimes is allowing anyone to use her voice for AI-generated songs … for a 50% profit share. 🕸️Web3's impact: If Web3 fulfills its promise, users could store data in personal vaults or cryptocurrency wallets, making it harder for LLMs to access the data they crave. 🌎Geopolitics: it’s increasingly difficult for data to cross country borders. Just think about China and TikTok. 😷Data contamination: We have this huge amount of ‘new’ - and sometimes hallucinated - data that is being generated by generative AI chatbots. What will happen if we feed that data back into their LLMs? No wonder that people like Sam Altman are looking for ways to make the models better without having to use more data. If you want to know more, check this brand new Radar podcast episode