Peter Seibel - Coders at Work - Reflections on the craft of programming

Seibel:But you do think that refactoring can work, if you take every seventh interval to do it? You don’t have to end up needing a big rewrite?

Crockford:I think it can work. Throw it out and start over should only be considered in the cases where you didn’t do that or you did it badly or something went wrong and you’ve got a code base that has become unworkable. And you can make a reasonable judgment that it will be faster to replace it than to fix it.

Seibel:What about the risk that you don’t fully understand what the code you want to rewrite actually does. Because any piece of code contains bits of embedded knowledge—little bits of cruft that are hard-won functionality that you don’t think of when you say, “Oh, we can just rewrite this.”

Crockford:That is a real problem. One of the reasons that we’re in the mess that we’re in is that the Web is so poorly specified. The specifications were incomplete and were largely misinterpreted and many of those misinterpretations have become part of the canon. So these systems are way more complicated than they should be due to those historical reasons. Working at that level, yeah, I have huge sympathy for that, that there is a lot of undocumented knowledge that is reflected in the code base.

Microsoft has a similar problem with their operating systems, in that they shipped crap for too many years and then they had to remain compatible with all the bad stuff that was based on the bad stuff that they had done. And so the constraints that puts on the design of their next system are just horrendous. So it’s really hard going forward with that. Ultimately they may find that they can’t go forward anymore either.

Those sort of specification errors are really, really hard. And we have them in the Ajax world. Most of our problems in the Ajax world are due to the differences at the browser level. Doing the cross-browser stuff is much harder than it should be because the Web is not specified well and because the implementations are so variable.

We’ve gotten a lot better at that over the last few years, particularly with the advent of the Ajax libraries. Most of them do a very good job—not a complete job yet—but a pretty good job of boosting the level of programming that you do. So we’re not having to deal directly with the browser guts; we have a sort of virtualized application layer that we can work on that is fairly resilient and pretty portable. We have one group here at Yahoo! which is primarily responsible for dealing with the pain that the browser causes. And when they do their job right, it makes it easier for all the other developers here. So that’s good.

Seibel:On the other hand, rewrites don’t always work out. You just mentioned the second-system effect and in one of your talks, you described seeing it in action as “heartbreaking.” When was that?

Crockford:It was at Electric Communities. We got together the smartest team of programmers I’ve ever seen assembled anywhere. And we had enough money and we were going to reimplement the stuff that Chip and Randy had already done and knew exactly how to do it. Except it was grander.

Seibel:So that was basically Habitat, redone.

Crockford:Yeah, we were going to redo Habitat except it was now going to be globally distributed. And it turned out to be really hard. We actually got it built but it was painful. Not something I’d like to do again.

Seibel:Would the advice you gave before—to be very disciplined about only reimplementing what you already understand—would that have been sufficient to head off the disaster?

Crockford:I think it could have helped. We didn’t think about it in stages properly. We didn’t have an incremental approach. Had we taken an incremental approach, I would have started with two parallel efforts. One, work out a secure distributed platform which doesn’t do anything but has the infrastructure for doing the messaging and the object management. Two, let’s rebuild Habitat. Knowing what we know, with modern languages, let’s just rebuild it.

Then the second phase would be, OK let’s merge them together. Can we hoist this one on top of that one and still have a working system? OK, now distribute it.

Had we taken that kind of incremental approach, I think we would have been very successful with it. But we tried to do all of those in one step and that was too hard.

Seibel:And you think you were led into trying to do it in one step because you knew big chunks of it.

Crockford:Because we were so smart and we had so much experience. We had it wired. Couldn’t miss. Programmers are optimistic. And we have to be because if we weren’t optimists we couldn’t do this work. Which is why we fall prey to things like second systems, why we can’t schedule our projects, why this stuff is so hard.

Seibel:Is programming getting easier? In the future will more people be able to do something that we would recognize as programming?

Crockford:My interest in programming is helping other people to do programming, designing a language or a programming tool specifically so that it’s more accessible to more people—the thing that got Smalltalk started. Smalltalk went in a different direction, but the initial direction was really attractive to me. How do we build a language specifically for children or how do we build a language specifically for people who don’t think of themselves as programmers?

Seibel:Is that because you think should everybody learn to program, at least a little bit?

Crockford:I think you have to. The world has been pretty much taken over by computers now and in order to defend yourself, or to be a full citizen, you have to have some understanding of how these things work.

Seibel:Some folks would also argue that learning to program teaches a way of thinking that’s important, like reading and math are different ways of thinking, and both important.

Crockford:I used to think so. I had these amazing insights when I started programming: everything became orderly, and I saw structures and things I had never seen before. I thought, “Wow, this is amazing. Everybody should learn how to do this,” because suddenly I was feeling a lot smarter. Except pretty quickly I’d find, talking to other programmers, that somehow they didn’t get it. Programmers are capable of completely misunderstanding the world in exactly the same way everybody else does. I was really sad to figure that out.

Seibel:Do you still enjoy programming as much as you always did?

Crockford:Oh, yeah.

Seibel:Do you think that programming is at all biased toward being young?

Crockford:I used to think so. A few years ago I had sleep apnea, but I didn’t know it. I thought I was just getting tired and old, and I got to the point where it was so difficult to concentrate that I couldn’t program anymore because I just couldn’t keep enough stuff in my head. A lot of programming is you keep stuff in your head until you can get it written down and structured properly. And I just couldn’t do it.

I had lost that ability and I thought it was just because I was getting older. Fortunately, I got better and it came back and so I’m programming again. I’m doing it well and maybe a little bit better now because I’ve learned how not to depend so much on my memory. I’m better at documenting my code now than I used to be because I’m less confident that I’ll remember next week why I did this. In fact, sometimes I’ll be going through my stuff and I’m amazed at stuff that I had written: I don’t remember having done it and it’s either really either awful or brilliant. I had no idea I was capable of that.