Smashing Protons, Scaling Models: Are Physics and AI Running Out of "Rizz"?

There is this crazy thought that has been bugging me for some time. While folks in India obsess about elections and about leaders who will, in all likelihood, do nothing, I am obsessed with a different kind of nothingness.

I was talking with a physicist the other day, and she was not happy with the state of research in physics. She complained that there have been no notable breakthroughs in years. She remarked that the amount of output in terms of papers is really great, but that there is just not enough experimental data to validate the theories.

Now, full disclosure, I am not a physicist. I am just a lousy computer scientist with a very average grasp of math. But I am blessed to have some very smart physicists as friends, who are sweet enough to dumb things down. And that understanding is what gives me the audacity to write this. So, if I am making conceptual mistakes, blame it not on my education. Rather, blame it on my IQ! You have been warned!

Okay, now, the first half of the twentieth century was a depressing and yet awful time to be living in. The speed of progress in Physics was just fascinating. Fundamental rules of reality were being discovered, modified, and replacing old ideas about the universe at a breakneck pace. I am personally fascinated by this phase because this led to the discovery and development of two concepts that are really fascinating (and equally difficult to understand): Quantum Theory and the Standard Model.

Based on how I understand it, the Standard Model is an awesome mathematical model of how the universe behaves at a microscopic level. But let me digress for a bit, because particle physics is fucking awesome, at least the way experimental folks do it. So, just like kids (and even adults ;) ), physicists love smashing. The context of smashing is of course different for all three groups. Physicists figured out it would be a great idea to smash particles like protons at very high speeds. And the data from those collisions would be used to discover new particles. That is how we discovered weird-sounding shit like "muons." Anyway, by the first half of the twentieth century, there were too many "new" particles in this "particle zoo." And so, a bunch of smart folks built the Standard Model to try bringing in some order to this "particular" chaos (see what I did there?). What happened with the Standard Model was that we had a periodic-table-like structure grouping all known matter on the basis of the fundamental particles they were built of (quarks, leptons, bosons), and the forces that governed the fundamental particles (electromagnetism, the strong force, and the weak force). So far so good? Yes! In fact, this model was so good that it actually managed to predict some yet undiscovered stuff. It took the experimentalists decades of engineering to build the machines that would actually discover those particles. But at the end of it, the Standard Model was a fascinating success.

And then came Higgs. If you are as old as me, a core memory for you, growing up, was the media coverage around the Large Hadron Collider (LHC) in and around 2008. The LHC was gearing up for the first test, and the world was scared that the experiment would create a black hole. And that black hole would just eat us all. But, unless we are living in a different reality, that did not happen. The whole objective of the LHC was to find the Higgs boson, the final missing piece of the Standard Model responsible for giving other particles mass. The idea was the same: smash stuff, see stuff, use computers to wade through the stuff, learn what to do, what not to do, repeat. The Standard Model was absolutely spot on in predicting the existence of other stuff before. So, physicists were stoked. And in 2012, they discovered the existence of the Higgs boson. And (as far as I understand it), the Higgs boson behaved just like the Standard Model had predicted. All good. Right?

Nope. You see, sometimes even boring folks like scientists need some spice in their life. The experiment indeed discovered the Higgs, but it was almost too "vanilla." For a brief moment, the scientific community felt like they had conquered the universe. The Standard Model was treated like the undisputed Theory of Everything; the final boss of physics. We thought we had the ultimate blueprint.

But then, the fractures started showing. In spite of being right about the stuff we can see, my physicist friends keep telling me the Standard Model is actually woefully incomplete. It is impossible to explain gravity with it. It entirely fails to explain dark energy or dark matter, which (allegedly) make up about 95% of the Universe! Imagine claiming you have the ultimate map of the world, but it entirely leaves out the Pacific Ocean. The model that was supposed to be the "end of physics" is now fracturing under the weight of everything it can't explain. The Standard Model is good for the 5%, but we desperately need a new paradigm with more rizz for the rest.

Okay. Now, I'll tell you another story: same plot, different characters. In 2017, a group of Google researchers had a great discussion over beers (I keep telling my wife that beer is what takes science forward... she disagrees). And that discussion led to the invention of something called a Transformer architecture. That Transformer thing is what powers all the fancy Large Language Models that we use today. The invention of the Transformer was followed by an explosion of applied research, with modifications and optimizations pushing the concept to new extremes. But almost a decade later, it seems that foundational research on Transformers has actually plateaued. It would be an over-exaggeration to say we know why that works, but not how. Since these are AI systems, we provide a learning algorithm, and the machine ends up doing the learning. The problem is, we are not exactly sure what they are learning, how they are representing all of that knowledge, and how they are actually getting better at stuff that we did not really "train" them at, in the first place. But why?

Well, mostly, if not completely, it's because of money. Silicon Valley saw a Golden Goose in the Transformer architecture and assumed it was our "Standard Model"; the final, ultimate blueprint that would lead straight to Artificial General Intelligence (AGI). Instead of looking for new fundamental breakthroughs, the industry adopted a singular, brute-force religion: Scaling.

Scaling in AI is exactly what building the Large Hadron Collider was for physics. Instead of inventing a new paradigm, AI companies realized that if you just threw more data and massive clusters of GPUs at the exact same Transformer architecture, the models got better. So, VC money flooded into buying hardware to build ever-larger models. But just like the physicists who thought the Standard Model was the end of history, the tech bros are starting to see the cracks. The VCs didn't get their instant 100x returns, and the models are hitting data walls.

This is where the similarities with the Standard Model become uncanny. The Transformer architecture, while wildly successful, is not the "Theory of Everything" for intelligence. LLMs are truly amazing at generating text and solving specific problems. But predicting the next word in a sentence is probably just the 5% of "visible matter" in the universe of general intelligence. It doesn't account for actual reasoning, logic, or the "dark matter" of human consciousness.

We need to accept that the religion of "Scaling" will soon plateau. You can only build so many massive colliders, and you can only build so many massive server farms. When that plateau hits, the tech industry is going to face the exact same fracturing realization that physicists are facing now: our "ultimate model" was just a stepping stone, and the underlying architecture is inherently flawed.

What is the way forward? I don't know! For me, as a person who constantly tries to work on the fundamental questions about AI while doing applied research/engineering as a day job, I think we need to escape the shells of disciplined thought. My impression is that scientists of yesteryears were often polymaths, and that helped them balance the Murray vs. Feynman way of thinking and come out with radical new ideas. I could be wrong, but I think we are getting intellectually hollow (probably because there is so much to keep track of in your own niche domain). And to crawl out of the hollowness would require more interdisciplinary thought and more abstract systems-level thought, not just about the problem at hand, but also about the bigger picture.

The conclusion: It is not just physics. Even Computer Science (especially AI) is stuck at a stage where there have been very few "fundamental" breakthroughs in recent years. And perhaps, both disciplines are coping with the same problems: economic and intellectual!