OpenAI's alignment problem

I.

Less than two months ago, on stage at the Code Conference, I asked Helen Toner how she thought about the awesome power that she’d been entrusted with as a board member at OpenAI. Toner has the power under the company’s charter to halt OpenAI’s efforts to build an artificial general intelligence. If the circumstances presented themselves, would she really stop the company’s work and redirect employees to working on other projects?

At the time, Toner demurred. I had worded my question inelegantly, suggesting that she might be able to shut down the company entirely. The moment passed, and I never got my answer — until this weekend, when the board Toner serves on effectively ended OpenAI as we know it. (She declined to comment when I emailed her.)

By now I assume you have caught up on the seismic events of the past three days at OpenAI: the shock firing on Friday of CEO Sam Altman, followed by company president Greg Brockman quitting in solidarity; a weekend spent negotiating their possible returns; ex-Twitch CEO Emmett Shear being installed by the board as OpenAI's new interim CEO; and minority investor Microsoft swooping in to create a new advanced research division for Altman and Brockman to run.

By mid-afternoon Monday, more than 95 percent of OpenAI employees had signed a letter threatening to quit unless Altman and Brockman are reinstated. It seems all but certain that there will be more twists to come.

I found this turn of events as stunning as anyone, not least because I had just interviewed Altman on Wednesday for Hard Fork. I had run into him a few days before OpenAI’s developer conference, and he suggested that we have a conversation about AI’s long-term future. We set it up for last week, and my co-host Kevin Roose and I asked about everything that has been on our minds lately: about copyright, about open-source development, about building AI responsibly and avoiding worst-case scenarios. (We’ve just posted that interview, along with a transcript, here.)

The days that follow revealed fundamental tensions within OpenAI about its pace of development and the many commitments of its CEO. But in one fundamental respect, the story remains as confusing today as it did on Friday when the board made its bombshell announcement: why, exactly, did OpenAI’s board fire Sam Altman?

The official explanations have proliferated. The ...