If you ask Yann LeCun, Silicon Valley has a groupthink problem. Since leaving Meta in November, the researcher and AI luminary has taken aim at the orthodox view that large language models (LLMs) will get us to artificial general intelligence (AGI), the threshold where computers match or exceed human smarts. Everyone, he declared in a recent interview, has been “LLM-pilled.”
On January 21, San Francisco–based startup Logical Intelligence appointed LeCun to its board. Building on a theory conceived by LeCun two decades prior, the startup claims to have developed a different form of AI, better equipped to learn, reason, and self-correct.
Logical Intelligence has developed what’s known as an energy-based reasoning model (EBM). Whereas LLMs effectively predict the most likely next word in a sequence, EBMs absorb a set of parameters—say, the rules to sudoku—and complete a task within those confines. This method is supposed to eliminate mistakes and require far less compute, because there’s less trial and error.
The startup’s debut model, Kona 1.0, can solve sudoku puzzles many times faster than the world’s leading LLMs, despite the fact that it runs on just a single Nvidia H100 GPU, according to founder and CEO Eve Bodnia, in an interview with WIRED. (In this test, the LLMs are blocked from using coding capabilities that would allow them to “brute force” the puzzle.)
Logical Intelligence claims to be the first company to have built a working EBM, until now just a flight of academic fancy. The idea is for Kona to address thorny problems like optimizing energy grids or automating sophisticated manufacturing processes, in settings with no tolerance for error. “None of these tasks is associated with language. It’s anything but language,” says Bodnia.
Bodnia expects Logical Intelligence to work closely with AMI Labs, a Paris-based startup recently launched by LeCun, which is developing yet another form of AI—a so-called world model, meant to recognize physical dimensions, demonstrate persistent memory, and anticipate the outcomes of its actions. The road to AGI, Bodnia contends, begins with the layering of these different types of AI: LLMs will interface with humans in natural language, EBMs will take up reasoning tasks, while world models will help robots take action in 3D space.
Bodnia spoke to WIRED over videoconference from her office in San Francisco this week. The following interview has been edited for clarity and length.
WIRED: I should ask about Yann. Tell me about how you met, his part in steering research at Logical Intelligence, and what his role on the board will entail.
Bodnia: Yann has a lot of experience from the academic end as a professor at New York University, but he’s been exposed to real industry through Meta and other collaborators for many, many years. He has seen both worlds.
To us, he’s the only expert in energy-based models and different kinds of associated architectures. When we started working on this EBM, he was the only person I could speak to. He helps our technical team to navigate certain directions. He’s been very, very hands-on. Without Yann, I cannot imagine us scaling this fast.
Yann is outspoken about the potential limitations of LLMs and which model architectures are most likely to bump AI research forward. Where do you stand?
LLMs are a big guessing game. That’s why you need a lot of compute. You take a neural network, feed it pretty much all the garbage from the internet, and try to teach it how people communicate with each other.
When you speak, your language is intelligent to me, but not because of the language. Language is a manifestation of whatever is in your brain. My reasoning happens in some sort of abstract space that I decode into language. I feel like people are trying to reverse engineer intelligence by mimicking intelligence.
Imagine going to a lecture and the professor showing you some elementary mathematics, then instead of trying to understand the concept—like how to multiply—you watch the professor’s language [to learn which word follows the previous word.]
Why not focus on AI that is language-independent, that’s not a guessing game?
How does an EBM handle tasks differently than an LLM?
An energy-based model is able to self-correct.
The best analogy would be climbing Everest. There are a lot of different paths, and every season the paths might differ because of the weather. You have to evaluate in real time how much traffic there will be, how much oxygen you have.
If you’re an LLM climber, you don’t see the whole map. You fix in one direction at a time and keep going. If there’s a hole, you’re going to jump and die. LLMs are not allowed to deviate until they complete a task.
The EBM is a true reasoning model. When you go to the mountain, you need a combination of the data from your experience and what you learn in real time. You’re able to see in multiple directions, choose one, and if you encounter a hole, try another way. The task is always in the back of your mind.
If not language, what types of data make up the training dataset for an EBM?
It can be anything.
Our model is very small, below 200 million parameters, which allows us to train it really fast. We are not trying to make one model for everything—one brain for all. We make a smaller model for each individual business. The data will be different for each client.
The training is quite different from a traditional LLM. We give it partial data, called sparse data. The model is able to extract the full data from the sparse data. Imagine I show you how to draw a cat and you can extrapolate how to draw a dog. We see this kind of extrapolation.
How might a business deploy an EBM?
I’m especially interested in the energy sector. In real time, you have to process a lot of variables and distribute energy accordingly. Right now, it just dumps a chunk of energy, some of which is used and some of which is not. People manage it, but we can automate it.
We are also interested in pharmacology—drug discovery, cancer, and so on. All of this requires complicated data processing. We’re also talking to one of the largest chip manufacturers, and one of the largest data centers.
Do you expect Logical Intelligence to complement or compete with AMI Labs, Yann’s Paris-based startup, which is also developing an alternative model architecture?
Our company is focused on building the brain. AMI is focused on world models, where you put AI in real-world scenarios and use data to navigate the world and make predictions. I’ll let them speak for themselves, but we are committed to working together.
You’ve opted not to open-source your model, Kona. Why is that?
We’ll consider making something open source eventually. But I feel very responsible for what I’m creating, and want to make sure I understand it well enough before I put it out there in the world.
This is a real step towards AGI. You want to ask yourself how safe it is, what are the possibilities, and where are the boundaries. I just want to be a responsible parent.
The term AGI has become pretty loaded; I’m sure you don’t use it lightly. Can you expand on why you think EBMs represent a route to AGI?
I see AGI as an ecosystem of compatible AI models that serve the world and people in the most productive, safest way. That requires the ability to self-align, self-assess, plan. It also means there is no room for hallucination.
There are different stages to the AGI evolution. We are somewhere in the very baby steps.
You’re looking for funding at the moment. How will you put that money to use?
Everything needs to be scaled up. This model needs to be scaled. We want to try different use cases, but that requires different teams to work with different partners.
I also feel that part of my job is to educate people that there are different forms of AI. It doesn’t have to be text-based AI. People say, “Oh, we’re in an AI bubble.” But we’re not. We’re in an LLM bubble.