Peter Seibel - Coders at Work - Reflections on the craft of programming

Seibel:So you want types mostly so the compiler can optimize better?

Fitzpatrick:No. I also want it to blow up at compile time to tell me like, “You’re doing something stupid.” Then sometimes I don’t care and I want it to coerce for me at runtime and do whatever. I don’t want to be too optimistic about Perl 6, but they’re preaching a lot of things I want to see. But I don’t think it’ll ever come out.

Seibel:Do you like C++?

Fitzpatrick:I don’t mind it. The syntax is terrible and totally inconsistent and the error messages, at least from GCC, are ridiculous. You can get 40 pages of error spew because you forgot some semicolon. But—like anything else—you quickly memorize all the patterns. You don’t even read the words; you just see the structure and think, “Oh, yeah, I probably forgot to close the namespace in a header file.” I think the new C++ spec, even though it adds so much complexity, has a lot of stuff that’ll make it less painful to type—as far as number of keystrokes. The auto variables and the for loops. It’s more like Python style. And the lambdas. It’s enough that I could delude myself into thinking I’m writing in Python, even though it’s C++.

Seibel:And you use C++ for efficiency.

Fitzpatrick:Yeah, pretty much. I mostly use it at Google. Anything that’s halfway performant is in C++ there. I also write a ton of Java at Google.

Seibel:From what I understand, Google has a C++-centric culture because that’s what they used originally and they’ve built a whole bunch of software infrastructure around it. While you can’t undo all that history, there’s probably a lot of code written at Google in C++ where it’s not really necessary for performance.

Fitzpatrick:Especially because, over time, Java has gotten faster and the JVM has gotten a lot smarter. The thing that annoys me about Java is that everyone has such a strong aversion to JNI stuff. Sometimes a library is in C++. The Python people—in the outside community and inside Google—don’t care. They’re like, “Oh, we’ll, SWIG-wrap it.” They get on their way and they’re happy. Python gets support for something right away if it’s in C++ because they’re not religious about what the source language is.

Java people are like, “Must be pure Java. We cannot use JNI because then if the JVM crashes, we don’t know why.” The problem with that is you end up writing everything twice, once for C++ and Python and all the other languages, and then once for Java. So if they could come up with a good embedding story or get over this fear of JNI, then I wouldn’t mind it.

Seibel:What about explicit memory management versus garbage collection? People still argue about that. Do you have a strong opinion one way or the other?

Fitzpatrick:No, not really. I’m amused to watch other people’s strong opinions when generally they’re not backed up by anything. I personally don’t find it that annoying to manage memory, at least in C++ with like scoped pointers. I can write in C++ for days and never actually say “new” or “delete”. It seems to just all kind of work.

I rewrote memcached inside Google to work with Google infrastructure and to add it to App Engine. That was all written in C++ because I needed a very exclusive control of memory to reduce fragmentation. So I really appreciated having it there.

Seibel:The original memcached was in C. Did you redo it in C++ because C++ is more accepted within Google, or were there other advantages?

Fitzpatrick:I started to take the existing one and port it but it turned out to be more work. Memcached isn’t that much code to begin with, so it was a lot quicker to just rewrite in C++. It was like half as much code, rewriting it in C++.

Seibel:Do you think that was because of C++ or just because you were smarter this time around?

Fitzpatrick:It could be. Once, when I was 11 or 12, we were on a trip around the US and I wrote that game Mastermind on a TI-85 calculator. I’m writing this program—a couple hundred lines—on this tiny little screen trying to remember where I am. I ended up deleting the damn thing twice. So I wrote the thing three times. But then it got so easy. That’s a good point—the second time around it was a lot easier.

Seibel:You’ve done a lot of your work in Perl, which is a pretty high-level language. How low do you think programmers need to go—do programmers still need to know assembly and how chips work?

Fitzpatrick:I don’t know. I see people that are really smart—I would say they’re good programmers—but say they only know Java. The way they think about solving things is always within the space they know. They don’t think end-to-end as much. I think it’s really important to know the whole stack even if you don’t operate within the whole stack.

When I was doing stuff on LiveJournal, I was thinking about things from JavaScript to how things were interacting in the kernel. I was reading Linux kernel code about epoll and I was like, “Well, what if we have all these long TCP connections and JavaScript is polling with these open TCP connections that are going to this load balancer?” I was trying to think of how much memory is in each structure here. That’s still somewhat high-level, but then we were thinking about things like, we’re getting so many interrupts on the Ethernet card—do we switch to this NAPI thing in the kernel where rather than the NIC sending an interrupt on every incoming packet it coalesces them to boundaries that were equivalent to 100 megabits speed even though it was a gigabit NIC. We were collecting numbers to see at what point this made sense and freed up the processor.

We were getting a lot of wins for really low-level stuff. I had somebody recently tell me about something: “Java takes care of that; we don’t have to deal with that.” I was like, “No, Java can’t take care of this because I know what kernel version you’re using and the kernel doesn’t support it. Your virtual machine may be hiding that from you and giving you some abstraction that makes it look like that’s efficient, but it’s only efficient when you’re running it on this kernel.” I get frustrated if people don’t understand at least the surface of the whole stack.

In practice, nothing works. There are all these beautiful abstractions that are backed by shit. The implementation of libraries that look like they could be beautiful are shit. And so if you’re the one responsible for the cost of buying servers, or reliability—if you’re on call for pages—it helps to actually know what’s going on under the covers and not trust everyone else’s libraries, and code, and interfaces.

I almost don’t think I would be a programmer today if I was starting off. It’s just too ugly. This is why I’m so excited about things like App Engine. Someone described Google’s App Engine as this generation’s BASIC. Because this generation, everything is networked. When I was programming, it was one language, and it was on my own machine, and the deploy was up enter, or RUN enter. Kids today don’t want to write something stupid like a “bounce a ball” app on their own machine. They want a web site to interact with.

I still have people mailing me who are like, “Hey, I have this idea—I want to make Wikipedia meets YouTube, meets—” Everyone wants to do a web site where their favorite four web sites aren’t quite right and they want to make one that looks kind of like that.

The fact that App Engine gives you one button, “Put this on the Web,” and you write in one language, arguably a pretty easy-to-learn one, Python, is perfect. It’s a great intro to programming—there are so many layers and layers of bullshit that it gets rid of.