Summary in 3 sentences
This investigative journalism book covers the rise of OpenAI, originally a nonprofit focused on ushering safe and ethical AI, and how it became the corporate behemoth it is today due to Sam Altman and surrounding leadership. It examines the environmental and human labor costs of AI and how none of this empire building was “inevitable” in AI development, but rather a strategic and capitalistic choice.
How I discovered it
Recommended through Off the Grid podcast
Who should read it?
Anyone using AI, especially OpenAI products, should be aware of the backstory of the company and of the other potential trajectories of AI development that have been lost due to the world’s focus on connectionist AI.
How it changed me
Behavior, ideas, perspectives, emotional shifts
- I felt quite a bit of grief reading this book, mostly at the lost time and funds towards other types of AI research that would meaningfully shape the field—whether that’s neurosymbolic AI or computer vision/interaction-based AI.
- Personally did not care to know all the tea on Sam Altman. The board configuration and his firing (and un-firing) is fascinating though.
- In many ways the story of OpenAI is an intensified tale of greed and lack of transparency that is common in so many tech companies. Changing priorities, getting blindsided by leadership, messaging of ethical choices erased over time in favor of profits and investors.
- The book made me curious if OpenAI hadn’t been leading the charge in AI development—if it were an already large company like Google instead, would we have been better off? Would there have been more transparency? My gut says no. I wonder if perhaps because this tale is from a small startup turned behemoth, we can better heed the warning signs of the industry.
Top quotes
On the loss of AI research diversity
The number of independent researchers not affiliated with or receiving funding from the tech industry has rapidly dwindled, diminishing the diversity of ideas in the field not tied to short-term commercial benefit.
On the inevitability, determinism argument
This was a favorite argument in Silicon Valley—the inevitability card. If we don’t do it, somebody else will… technologies are not inevitable. The ability to advance them is driven by a collective belief that they are worth advancing.
On the renaming of automata studies to artificial intelligence
Cade Metz, a longtime chronicler of AI, calls this rebranding the original sin of the field: So much of the hype and peril that now surround the technology flow from McCarthy’s fateful decision to hitch it to this alluring yet elusive concept of “intelligence.”
On symbolism vs connectionism AI
The first camp, known as the symbolists, believed that intelligence comes from knowing. Humans know more than animals and can use that knowledge to understand and act on the world. Achieving AI must then involve encoding symbolic representations of the world’s knowledge into machines, creating so-called expert systems. The second camp, called the connectionists, believed that intelligence comes from learning. Humans have a greater capacity to learn than animals and can use that ability to acquire and advance different skills. Developing AI should focus instead on creating so-called machine learning systems, such as by mimicking the ways our brains process signals and information.
Marcus advocated instead for combining connectionism and symbolism, a strain of research known as neurosymbolic AI. Expert systems can be programmed to understand causal relationships and excel at reasoning, shoring up the shortcomings of deep learning. Deep learning can rapidly update the system with data or represent things that are difficult to codify in rules, plugging the gaps of expert systems.
On body-based learning for AGI
Language, the theory goes, is the primary medium through which humans communicate, meaning all of the world’s knowledge must at some point be documented in text. It follows then that AGI should be able to emerge from training an algorithm on massive amounts of language and nothing else. This idea is in contrast to the “grounding” hypothesis, which asserts that the physical world and our ability as humans to perceive and interact with it is just as crucial an ingredient to our intelligence. AGI would then only be able to emerge from the combination of language and perception, like computer-vision, as well as interaction, such as through a physical or virtual agent taking actions in the real world.
On environmental impact
According to the International Energy Agency, each ChatGPT query is estimated to need on average about ten times more electricity than a typical search on Google.
every AI-generated image could consume enough energy to charge a smartphone by roughly 25 percent.
“They are extractivist projects that come to the Global South to use cheap water, tax-free land, and very poorly paid jobs. And then they don’t contribute to our country; they don’t improve our internet access”