The Murray vs Feynman approach in current AI research

(This was written in my intellectual infancy aroudn 2021)

Feynman was one of the most influential physicists of the modern age with some really weird ideas and a Nobel Prize. In his book “Feynman’s Rainbow”, Leonard Mlodinow (renowned theoretical physicist) explains how Feynman divided scientists into two categories based on how they look at the world. I am writing this piece based on my understanding of the book, the concept that the book talks about, and my understanding of the current research efforts in Artificial Intelligence.

In his book, Mlodinow says that Feynman classified mathematicians as being either ‘Greek’ or ‘Babylonian’. In his opinion, the “Greeks’’ would use the ‘full force of logical machinery’ to solve a problem. The scientist adhering to the Greek approach would thus naturally try to use a set of axioms to approach a problem and use those axioms to formulate a hypothesis and test it. In contrast to this, the “Babyloninans” recognize that there is more to the world and to scientific reasoning than merely expressing everything as emerging out of a set of some axioms. The Babylonians, as per Feynman, care only if a particular method works for a problem and not if that method fits into any bigger scheme of things. The Greeks focus on the underlying order behind a phenomenon. The Babylonians focus on the phenomenon itself. This classification is very interesting and intuitive. While both methods try to find a mathematical answer to a problem, their approaches to arriving at that mathematical system is entirely different.

This interplay of the Greek and Babylonian ways to look at the world (from a perspective of Feynman) plays out in every domain of natural and social sciences. And so, for any discipline, there are a group of scientists who will either focus on finding solutions to a particular problem, without really caring for what is the bigger picture associated with that problem or if it fits within the framework of some big picture. With them, there is always a second group of scientists who will not just focus on the solution of a particular problem, but will also try to put the problem into a framework of similar problems and then try to come up with an universal solution that might solve all such related problems simultaneously.

If we go by Mlodinow’s book, it is said that Feynman considered himself to be a Babylonian. Murray Gel-Mann is referred to as a Greek in his thinking. At the end, it all boils down to the subtle interplay between logic and intuition. Think about it. Feynman’s diagrams were radical when they were proposed. They apparently lacked the finesse and beauty of solving something through lengthy calculations (physicist colleagues would correct me if I were wrong to put it this way). But it was intuitive. And his solutions made people look at problems very differently. But his solutions were in stark contrast to how Gel-Mann approached problems. Gel-Mann was concerned with the bigger picture and how a set of axioms would be able to solve a diverse range of problems. This approach to mathematics is applicable to all disciplines and this is where things get interesting.

One of the most exciting and widely researched area of study at the present happens to be Artificial Intelligence. It has, as a separate branch of study, grown and evolved from mathematics, computer science, and other diverse disciplines such as linguistics, neuroscience, cognitive sciences among many others. New insights from such diverse fields have cumulatively changed how we understand intelligence as a phenomenon and artificial intelligence as a process. With the newest development iteration of Artificial Intelligence in the form of Deep Learning, we are achieving new milestones everyday. But there is no clear consensus about “how” should we really move forward with AI research in the near future. And in the next few paragraphs, I will try to explore how computer scientists being either Greeks or Babyloninans is impacting AI research and how this difference in approaches would impact the future.

If one goes back to the initial descriptions of AI given by the earliest computer scientists, one finds a common theme repeating time and again. The earliest computer scientists believed that it was just a matter of time before their newest invention (which was capable of doing calculations with a speed and efficiency that was never seen before) would easily do anything that humans were capable of doing. An electronic machine that ran on something called logic gates was supposed to be as ‘intelligent’ as it’s creator. And it was assumed that symbolic logic would solve everything. But research efforts of the last thirty or more years have shown that particular assumption to be wrong. If we think about it, the earliest computer scientists working on AI were probably Greek in their thinking. They believed that carefully working with combinations of a set of basic axioms (logic rules), they could explain and replicate the phenomenon of human intelligence. And so the Greeks worked on the problem for a long time until people realized that the thing was not that simple and they rather thought of simpler cases to apply smart algorithms. And slowly the Greeks were replaced by the Babylonians.

In the next iteration of Artificial Intelligence and Machine Learning, people started developing task specific smart algorithms. In other words, these people wrote programs that made the computer smarter in some things while other programs made it smarter for something else. Any hopes for an unified set of axioms that would help the machines learn ‘anything’ and get better at it seemed to be a very implausible idea.

And then through the 80s, 90s and early 2000s, people slowly developed a brand new approach for AI. They started mimicking the working of the human brain. And by the late 2010s, scientists discovered how to “train” these algorithms (called artificial neural networks) in a cheap and effective way to solve a bunch of problems that were earlier done through different approaches. And then when these algorithms (Deep Learning algorithms) started doing miracles, the Greeks came to the scene again. The fact that a neural network can in theory learn ‘anything’ was proven in the late 1900s. But when it was seen that the same neural network design could be cleverly tweaked to learn anything, the Greeks started dominating the field again. People started using the same principles of algorithmic learning on a variety of problems. And that is exactly when the problem started.

As we have seen throughout the article, the Greeks and the Babylonians have a completely different way of approaching problems. But then in the late 2010s, when scientists realized that neural networks were more effective for doing AI research over the many tools that they have had in their disposal for years, there was a weird evolution in the community; the Greeks started adopting Babylonian traits!

Over the last five or six years, AI researchers have used some common systems with some task-specific minor revisions to the neural architectures for solving a bunch of diverse problems. As a result, we have been expanding on a design philosophy that is at a low-level inspired from real human brains. But at a higher-level, these algorithms have become sophisticated engineering marvels. They seem to be super-efficient at what they do. But unfortunately it seems that these algorithms have no idea about ‘what’ they are doing or ‘why’. For instance, since 2017 a Google algorithm called “Transformer” has gotten better at machine translation than most algorithms that we have ever designed. But when one looks at the kind of knowledge that the algorithm learns about a language, it doesn’t seem that it really “learns” much. Instead, the smart algorithm uses giant machines to discover simple associative rules that help it to “learn” how to translate between sentences.

Over a video call a few months ago, a colleague and friend in a German university remarked how the new algorithms were getting more “muscular” but not really intelligent. And I believe that other AI researchers would agree with him. Although things like explainable AI are being heavily researched now and efforts to unravel the sort of linguistic information being learnt by language models or translation systems is underway, you wouldn’t probably deny that we have shifted more from a Greek way of doing things to a more Babylonian way. This is logical. With a field of study that is just beginning to make impact on the lives of everyone on the planet, nobody really knows which way to go. And it makes sense to use intuition to see if certain ‘hacks’ make the systems better.

So what does it mean? Will this cycle of a Greek-to-Babylonian-to-Greek cycle continue until we understand AI better? Or is it wise to stick to opposing poles (like Feynman and Gel-Mann) and try to see whose approach wins?Probably nobody has an answer.

A Greek perspective to look at the problem of AI has interesting philosophical consequences. One, if all ‘intelligent’ activity could be done by a single machine (like a machine version of a theory of everything), then it would be awesome. We would then have a mathematical idea about what makes the machines be good at vision and language. Here I am skipping the metaphysical and avoiding speculating on things like consciousness. But in the case that we have a mathematical model that makes the machines as good as humans in things like language, we could probably go a step further and try to see if we (humans) also follow that mathematical model to some extent. Based on that, we could probably link that mathematical model to other theories of physics and explain how the laws of physics (the basic axioms) give rise to a phenomenon like language.

A Babylonian perspective would probably be not that grand. A Babylonian approach to AI would be aiming at constantly developing solutions that ‘work’ for a particular problem. So, it is very much possible that the models for image and the models for language processing would be very different in how they work at a higher-level. It is possible that such a Babylonian way would never really find a link between different such models.

Both of the approaches seem fine. But at the end of the day, I feel that AI is much more than just having a machine do stuff that was earlier only possible for humans to do. It all boils down to how we look at the problem. If, in the near future, a particular system is able to generate flawless texts and is able to do perfect translation (very unlikely but let’s just assume), can we really say that the AI problem is solved and that we have made truly intelligent machines and that now it is time to move on to something else? What if we simply used the Greek approach as a motivation, while using the Babylonian aspect of intuition to make existing systems better? Again, I don’t know.

But what I do know is that we are in the middle of some really exciting times in terms of AI research. And it is the approach of AI researchers that will define how the future will play out for us. So which ‘path’ to take? Feynman’s, Gel-Mann’s or combine the best of the both approaches like every other researcher who attempts to do inter-disciplinary work! Only time will tell.