Large Language Models are not Inventions, They're Discoveries.

Dor Sarig

and

March 5, 2024

min read

LLMs like represent more of a discovery than an invention according to Jeff Bezos.

In his recent conversation with Lex Friedman, Bezos likened LLM development to Galileo's discovery of Jupiter's moons through the telescope - we now see unexpected capabilities emerge from language models, just as Galileo was surprised to find moons in the night sky.

The behavior of large language models is less like an engineered product, such as a Boeing 787, whose every behavior is predictable and designed, and more like a journey filled with unexpected findings.

These AI systems are not entirely within the realm of engineered certainty; they generate constant surprises through their capabilities, which we are only beginning to understand.Using such powerful technologies without proper visibility and control is akin to navigating uncharted waters without a compass or a map.^‍

‍

FAQs

Why does Jeff Bezos compare LLMs to a scientific discovery rather than an engineering invention?

Bezos argues that LLMs resemble discoveries because their capabilities emerge unexpectedly, much like Galileo discovering Jupiter's moons through a telescope. Unlike engineered systems built to spec, language models produce constant surprises that even their developers did not anticipate or design — making them fundamentally different from predictable engineered products.

How is a large language model different from an engineered product like a Boeing 787 in terms of predictability?

A Boeing 787 is a fully engineered product whose every behavior is predictable and intentionally designed. Large language models, by contrast, exhibit emergent capabilities that were not explicitly programmed, generating outcomes their creators did not foresee. This unpredictability places LLMs outside the realm of traditional engineered certainty.

What analogy did Jeff Bezos use in his conversation with Lex Friedman to describe LLM development?

In his conversation with Lex Friedman, Bezos compared LLM development to Galileo's discovery of Jupiter's moons through the telescope. Just as Galileo was surprised by what he observed in the night sky, developers encounter unexpected capabilities emerging from language models that were never explicitly designed or anticipated.

Why is deploying LLMs in enterprise applications without proper visibility and control considered risky?

Because LLMs behave more like discoveries than engineered systems, their outputs and capabilities cannot be fully predicted in advance. Deploying them without proper visibility and control is comparable to navigating uncharted waters without a compass or map — organizations expose themselves to unknown risks they lack the tools to detect or govern.

What does the discovery-versus-invention framing of LLMs mean for AI security strategies?

Framing LLMs as discoveries rather than inventions means their behavior cannot be fully anticipated through traditional software security models. Because emergent capabilities constantly surface, security strategies must prioritize runtime visibility, ongoing monitoring, and control mechanisms that account for the unpredictable nature of these systems throughout their entire lifecycle.