The Babel Fish Problem

The Babel fish is a small, leech-like fish that, popped into your ear, lets you instantly understand anything said to you in any language. It is, the Guide notes, mind-bogglingly useful, and also so improbably perfect that some have used it as a knock-down argument against the existence of God.

I think about the Babel fish a lot, because it is exactly what people want from AI: instant, effortless understanding. The part everyone skips is the bargain. You have to put the thing in your ear. You have to let it sit at the most intimate point of the conversation and trust it entirely.

The bargain worth refusing

• Most AI asks you to put the fish in your ear and then quietly keeps a copy of everything it hears.
• Understanding should not cost you ownership of your own inner life.
• The better design keeps the fish local: your memories, your context, your machine.

That is the whole thrust of the Friendly-AI work. Keep the astonishing usefulness of the Babel fish. Lose the part where understanding you becomes somebody else’s product. You can know where your towel is and keep it, too.