Don't Trust Smart People

Some ideas about refining your ideas, starting with a chat with Jeff Schneider.

One day I was walking to lunch with Jeff Schneider, a colleague at Carnegie Mellon, talking about some bad idea that had been foisted on everybody by a brilliant guy. Jeff said, “I have a rule: Don’t trust smart people.”

Jeff’s reasoning is this: If a smart person tells you an idea, you should be skeptical. Maybe you are biased by your respect for the source, or maybe the source is a talented idea salesperson. But if you hear a brilliant idea from a dummy, the idea is standing on its own, you can trust it. Let’s call it Schneider’s Law. The irony is that Jeff Schneider is smart, so Schneider’s Law implies that we cannot trust Schneider’s Law, which means that we can, which means that we cannot, which means … OK, I’m sure you get it.

Let’s extend Schneider’s Law. Distrust smart people? Okay, and who else? Anybody else come to mind, whose ideas you should distrust?

Yourself! If there is anybody whose ideas you cannot trust, it is you. I have sold myself on so many bad ideas, it is pathetic. I definitely cannot trust my own ideas. So then, how to deal with self-delusion? Here are a few ways.

Putting ideas into words forces a degree of precision that mere thinking lacks. I don’t know how my brain expresses ideas internally, but it has some deficiencies. Once during my morning shower and I had an idea that was so great, I stepped out of the shower immediately to write it down. But the idea was so bad it could not be written down at all. It was not really an idea. Too bad, I really loved that non-idea.

When an idea is written down, it is easier to play with, to explore implications, and to refine. Sometimes that first idea is just the start, the first step down an interesting line of thinking, possibly taking you into unexplored territory.

Writing is technology-assisted thinking, but does that mean illiterate people are handicapped? Are there compensations? I have heard that oral cultures develop traditions that support memory. Would it make sense to use the spoken word, rather than the written word, to work on ideas? I cannot imagine it would be all-around better, but perhaps there would be some advantage. Perhaps using both the spoken word and the written word would be useful?

And there we have an example … the idea on the possible value of using both spoken and written words. That idea did not come to me until I had written down the preceding idea about the value of writing ideas down. Now I should wait a couple of days, and probably delete the previous paragraph, but it illustrates my point, whether it is good or bad, so maybe I will leave it in regardless.

After you write an idea down, and play with it a bit, set it aside and wait. New ideas are thrilling, but that thrill impairs judgment. Remember Schneider’s law. Do not trust that idea. Wait at least a day or two. (I hope you are not working on a deadline.) Then you can look at the idea more objectively. Usually, my reaction after a day or two is, “Well, that was stupid,” or “Sure, but everybody knows that,” or just “Who cares?”

Critics are invaluable. You need friends who will tell you when you’re being stupid or boring. You need people who will really think about it, probe it, look for the flaws and the implications. Such good friends are rare. You need to seek them out, treasure, and nurture them.

The older I get, the harder it is to find those critics. I understand why. People are inclined to listen respectfully to their elders, rather than questioning or challenging them. Too bad. I recall some occasions when I wish I had challenged a respected elder’s ideas.

One day in 1982 or so, I ran into Allen Newell in the corridor at Carnegie Mellon. Allen was a great thinker — one of the founders of AI, a Turing Award winner, and a towering figure in the AI / CS / cognitive science communities at Carnegie Mellon. Also a sweet and gregarious guy. Allen asked me about my research plans, and I gave him a quick spiel about some new area I was excited about. Allen said, “To be an expert about something, you have to think about it for ten years.”

What I wish I had said: “Wow, that sucks. Does that mean if I work on something new, I will have no results for ten years? How on earth does anybody get a PhD in just six years?” If I had said that I could have had a great interaction with one of the original sources of the ten-year rule. Instead, I said something like, “That’s interesting,” and filed it away.

(If you don’t know, the ten-year rule originated with a 1973 study of chess masters (Simon & Chase 1973). Newell had just published a related paper (Newell & Rosenbloom 1982) which might be why it was on his mind when I ran into him. The ten-year rule was later popularized in essays and books by Geoffrey Colvin (2006) and Malcolm Gladwell (2013).)

Another example involves Marvin Minsky, also a founder of AI, also a Turing Award winner. I think it was Marvin who suggested that when robots were building robots, they would become so cheap that someday, when you visited a lettuce farm, you would see a robot squatting down next to each head of lettuce, flicking off the occasional grasshopper. Brilliant! And perhaps a great target for Schneider’s Law. But Marvin was a respected elder, and I didn’t challenge this idea. Too bad.

(Elon Musk said something similar once — suggesting that when robots build robots their price will approach zero, and the economy as we know it will become meaningless. We need to ask Jeff Schneider: When two smart people like Minsky and Musk embrace the same idea, does that make it more trustworthy, or less?)

But where was I? Oh yeah, you need critics. How does one cultivate good critics? Maybe it helps to be a good critic yourself. Of course, you need a smart friend to start with. But don’t forget Schneider’s law.

This is a technique I learned on the high school debate team. Debate teams have to argue both sides of the topic. Tournaments randomly assign positions. Sometimes you take the positive, and sometimes the negative. It is an interesting practice, with roots in ancient Greece. It can open your mind and lead to deeper thinking. So, when you’re working on an idea, especially for the ideas you love the most, you should take the negative.

Here’s an example. It began when I visited Rob Howe’s robotics lab at Harvard, and met his then-student Aaron Dollar. Aaron had an exciting project that ran somewhat against the usual thinking that robots should have anthropomorphic hands. Aaron and Rob had a “simple hands” project going.

Their project meshed well with my research interests. I was driven to think hard about simple hands. I spent weeks trying to develop the case for simple hands — why simple hands are the right way to go. It was a struggle. Then, one morning, in the shower again, it occurred to me that I should take the negative. In short order I had refined my thinking — simple hands were not the right way to go, not for everybody, not for every task or every research project. This simple re-casting of my thinking led to a more refined case in favor of simple hands, and eventually to a fun research project.

This discussion of ideas reminds me of one more point. What is the value of an idea? As my dad liked to say, ideas are cheap. The value of an idea arises in the effort required to make it work. Right now, for example, we are witnessing the investment of billions of dollars in AI, specifically employing computational neural nets. But don’t forget, the original computational neural net ideas go back many decades. When I first started in AI, in 1974, neural nets were broadly considered to be a brilliant idea that failed. The value arose from the decades of dogged work to test and refine the original idea.

While I hope you will find some of the above discussion useful, a good idea is only the beginning — just a promising direction. On the other hand, even the bad ideas can be fun, making good fodder for discussions and blog posts.

Colvin, Geoffrey. “Secrets of Greatness: What It Takes to be Great.” Fortune Magazine, October 2006.

Gladwell, Malcolm. “Complexity and the ten-thousand-hour rule.” The New Yorker, August 2013.

Newell, Allen, and Paul S. Rosenbloom. “Mechanisms of skill acquisition and the law of practice.” In Cognitive skills and their acquisition, 1–55. Psychology Press, 1982.

Simon, Herbert A., and William G. Chase. “Skill in Chess: Experiments with chess-playing tasks and computer simulation of skilled performance throw light on some human perceptual and memory processes”. American Scientist 61, no. 4 (1973): pp. 394–403.