Inadequate Equilibria: Summary and Review

Inadequate Equilibria: Where and How Civilizations Get Stuck by Eliezer Yudkowsky

Read online at the book website

Read online at LessWrong

Review

The main theme of this book is identifying when the Efficient Markets Hypothesis doesn’t apply, and when you should discard modesty and believe you actually know better than everyone else, even when that may look like hubris to outsiders.

Summary

The book reviews over at Slate Star Codex and Shtetl-Optimized are fairly good.

Notes

Central Idea: “Neglectedness”. You have underconfidence because you don’t think you can do better than everyone else, when you actually can.

Humility heuristic: Divide problems into areas that civilization is good at (stock market) and bad at (rushed doctors in medicine who haven’t read the latest research about your individual circumstances, actionable pandemic analysis by overworked public health professionals). You should have humility in areas where civilization is good at, but have confidence/arrogance in areas where civilization is bad at.

Modest Epistemology: “Who are you to suppose that you are smarter than professional economists?”

I expect that the most common benefit of inadequacy analysis will be to break a certain kind of blind trust—that is, trust arrived at by mental reasoning processes that are insensitive to whether you actually inhabit a universe that’s worthy of that trust—and open people’s eyes to the blatant brokenness of things that are easily observed to be broken. Understanding the background theory helps cancel out the elaborate arguments saying that you can’t second-guess the European Central Bank even when it’s straightforward to show how and why they’re making a mistake.

Definitions

Inefficient: Can correctly predict the future average price movement
Inexploitable: Profit cannot be made off of correct predictions
Inadequate: Unfixable by an individual effort. Requires multiple actors to simultaneously change behavior.

Causes of inadequacy:

Central command bottleneck (all important decisions can only be made by the recalcitrant committee at the central bank)
Bad Nash Equilibrium, two or more parts are simultaneously broken.

Inadequacy can still be exploitable, e.g. light boxes. You can still individually pick low hanging fruit, even if you can’t solve the problem globally/systemically. So the best opportunities are both inadequate and exploitable.

Hubris bias

People are used to adequacy, so any time you identify inadequate situations you can personally fix, it is met with derision/sneer/condescension/criticism because it looks arrogant to believe that you can do better than all of civilization.

In the ancestral environment, skill increased linearly. It was not possible for a hunter with no experience to go out alone and come back with a million deer carcasses. So, proposing that you can do this makes you look batshit crazy. But in the modern environment, if a system is inadequate, then civilization can leave trillion-dollar bills on the ground that it refuses to pick up.

Arguments from adequacy are very common, like the “index fund” argument, it’s much more common to “reason by analogy” and assume the fruit is already picked, and sneer at technologists and hedge fund managers who have overconfidence for thinking they can do much better than average. Similar to Peter Thiel’s “definite optimism”, it’s much easier to have “indefinite optimism/pessimism” and just assume the index fund of the future will be better, but we don’t really know how, we don’t have to build it, we can just sit back and the future will just come to us.

Reward vs. Success and Incentives (the study of systems)

When individual rewards don’t follow with success (in the stated goals of the system, e.g. “educate students” instead of “give them a sheepskin”), then whole civilizations can fail.

Doctors are high variance. Some doctors are amazing but other doctors are shitty. Choosing the right doctor can be an order of magnitude difference. The same applies to knowledge workers, e.g. Average is Over.

Competence under inadequate Nash equilibria

By default, “competence” means “status”. So stating you can do beter than doctors sounds arrogant because it sounds like you are asserting you are higher status than doctors, not that you’ve found a suboptimal equilibrium where doctors are wrong because of incentives.

“Status” and “Smart”

“Smart” implies status, but pointing out inadequacy is not about being “smart” or “high status”, it’s about bad incentives, and this can be overcome with not much smarts, but unfortunately people perceive a statement you can do better as arrogant because they don’t have the vocabulary for inadequate equilibria, so by default they reason you are claiming you are smarter/higher status.