Peter Seibel - Coders at Work - Reflections on the craft of programming

Eich:We hired somebody a while ago; he was a friend of one of the superbrains we’d hired. But this was a guy who was, I think, just undergrad or a bachelor’s degree; I’m not sure if he even finished. He met a guy who was working for us and they’re both OCaml hackers, and he was doing his own OCaml hacking on the side. And he was thinking about problems that we were seeing in my static analysis. When we interviewed him, I knew he was young but you couldn’t tell. Some people thought, “Oh, yeah, he hasn’t done much. You know, we should only hire rock stars; what are we talking to him for?”

I said, “No, you guys are looking at this wrong. This is like one of our bright interns. Get them while they’re young. He’s done a bunch by himself, he’s gotten into OCaml; he knows not just the source language, but the runtime, and he’s hacked native methods and he was writing an OCaml operating system, toy operating system. But this guy is good.” And it wasn’t that I gave him any particular programming test; it was just that I’d listened to him talk about what he’d done and why he’d done it. He wasn’t just repeating pabulum about C++ patterns. We have kids like that, unfortunately. Nice people and adequate programmers, for what they were doing, Java Enterprise stuff. But we needed somebody different, and this guy was different.

So in the interview the main problem was overcoming people misreading his age or thinking he wasn’t accomplished enough. But we hired him and he’s just been a superstar. He’s done a bunch of static analysis tooling, originally on this open-source Berkeley Oink framework, and then on GCC as plugins, working with the GCC guys. Now he’s kicking our mobile effort into high gear, just doing poor man’s profiling and printfof timestamps and finding out where the costs are and whacking them.

So when I interviewed him I knew there was talent. That he came recommended from somebody bright was good, because you know bright people like each other and can judge each other—generally there’s not a dysfunctional, “Hire my friend, who’s really not bright.” They want to work with bright people. Maybe this sounds like I’m cheating, but that’s one way I recognize talent. And I think that’s why we’re hiring superhackers. I think we’re hiring up all the Valgrind hackers. Some of those guys can do anything; they don’t fuck around.

Seibel:So is that something you often do in interviews: get them to talk about their own projects?

Eich:I do. I don’t give people puzzles to solve. We have people who do that here. To the extent that we have to do that and we’re using that to filter applicants, I worry.

Seibel:Is that even a good first-pass filter?

Eich:I’m skeptical. Google does that in spades, and they hire a bunch of very bright puzzle-solvers. But some of them, their street smarts are not necessarily there, and mature judgment. So I’m skeptical of it. I think we have to do it to some extent because you can end up getting someone who talks well, but actually isn’t effective at programming, and so you want to see them think on their feet, you want to see if they’ve solved a problem before. So we give them fairly practical problems. Not esoteric puzzles or math-y things, but more like programming problems.

Check their C++ knowledge, because C++ is hairy. So it’s sort of a sanity check, not enough to say, “Let’s hire him.” But if they pass it, that’s good; if they don’t, we worry. To say, “Let’s hire them,” we have to see something else, and that’s the spark that involves particulars, like what they’ve done and their approach and what languages they’ve used.

Maybe I’m also sympathetic to the odd duck. I don’t mind people who are a little different. I don’t want to hire somebody who’s hard to work with, but we need talent. We need people who think differently.

When I was an undergrad I was really affected by Pirsig’s Zen and the Art of Motorcycle Maintenance . And I had been going through Plato and the early philosophers. I was, at that point, inclined more towards idealism in some philosophical sense. I thought little-endian byte order was superior to bigendian, because after all, the least significant digits are in the lowest address—there was some kind of harmony or geometry in that. But try reading a hex dump. Practical things matter; particulars matter. The famous School of Athens painting with Aristotle pointing down and Plato pointing up—I’m more on the pointing-down side now. As I get older I get more and more skeptical and more and more interested in what works.

When I’m interviewing people, when I’m looking for talent, it’s very hard for me to not stick with particulars and practicalities. OK, so this guy knew OCaml—it meant he was smart, but should we hire him? Well no, but he also did things on his own and he thought on his feet when I talked to him, and he was already thinking about compilation or analysis problems that we were going to hire him to work on. But maybe the important thing there, the real story, was the network we were going through, the guy we hired, he was his friend.

Seibel:Do you still enjoy programming?

Eich:Yeah. It’s a bit like an addiction; it’s a little problematic. It’s not just the programming part of getting the code to run; to me now it’s more and more finding the right idea that has the New Jersey philosophy of a 90/10 trade-off—a sweet, sound theoretical core that isn’t going to solve all your problems but when you fall on the 10 percent that loses, you don’t go to hell. You can actually win this way and the code stays small enough and simple enough and there’s some dance between theory and implementation. I like that. That still appeals to me; it still is fun; it keeps me up at night thinking about it when I should be sleeping.

Seibel:Are there parts of it that you don’t enjoy as much anymore?

Eich:I don’t know. C++. We’re able to use most of its features—there are too many of them. It’s probably got a better type system than Java. But we’re still screwing around with ’70s debuggers and linkers, and it’s stupid. I don’t know why we put up with it.

Impatience and hatred of primitive tools drives me to try to be a better programmer. Our code is riddled with assertions now and they are fatal. That’s important to us. But this is something that has helped me, especially when I’m doing one of these allegedly sound, 90/10, sweet trade-off moves on the code that doesn’t quite satisfy all the invariants. I forget something; an assertion will botch and then it’s like, bing, I know what to fix.

Also I’m even now learning about my own weaknesses, where I’ll optimize something too much. I’ll have made some kind of happy pony land in my head, where I forgot some important problem. That’s always a challenge because programmers have to be optimists. We’re supposed to be paranoid, neurotic, Woody Allen types who are always worried about things, but really you wouldn’t get anywhere in programming if you were truly paranoid.

Seibel:Do you feel at all that programming is a young person’s game?

Eich:I think young people have enormous advantages, just physiological advantages to do with the brain. What they don’t have is the wisdom! You get crustier and maybe you get slower but you do learn some painful lessons that you try to pass on to the next generation. I see them ignoring me and learning them the hard way, and I shake my fist!

But, apart from that, if you stay well-read and keep at it, your output doesn’t necessarily have to be voluminous. While producing a lot of code is still important, what has interested me—and this is something that we talked about at Netscape when we talked about their track for principal engineer—is somebody who isn’t management but still has enough leadership or influence to cause other programmers to write code like they would write without them having to do it, because you don’t have enough hours in the day or fingers.