Peter Seibel - Coders at Work - Reflections on the craft of programming

Seibel:Overall, do you think that the software industry is a brilliant engine of innovation or a horrible mess?

Crockford:I’m trying to think of a nice way to say, “Horrible mess.” I’d think generally software has gotten better. Not at the same pace that Moore lets the hardware got better. We track way, way slow compared to him, so it takes us 20 years to double our efficiency in software development. But we have seen improvement. Most of our improvement is due to the fact that we don’t have to make it fit anymore. We don’t have to make it fast anymore. So that should have liberated us to just making it good. But we don’t spend enough time doing that, I think.

Seibel:So if we are, however nicely you put it, a horrible mess, what could we do to not be such a mess?

Crockford:That’s what I’m trying to figure out. A lot of it I think has to do with the way that we create standards. The reason why things are working as well as they are now is because the Net works; all the benefits that came, came from being able to tie everything together and have that happen pretty reliably.

But you don’t have to scratch it very deep to find places where we got that wrong, where we could’ve got it better. The dilemma is, how do we fix this stuff in place? Anytime we change a software standard, it’s an act of violence. It is disruptive. It will cause stuff to fail. It will cause cost and harm to people. So we need to be really careful when we revise the standards because there is that cost. We have to make sure that we’re adding so much value to offset that cost. From what I see of the way that standards are being manipulated right now, that’s not occurring. Standard changes are being motivated by “we want to do it” or “because it’d be neat” or some other motivation which is not necessarily closely related to creating a lot of value for the world. So I’m struggling with that. How do we get better at that?

Seibel:You seem to lean toward specifying less. That, obviously, is a way to avoid over specifying things and standardizing things that you’re going to regret later. But if less is specified in standards, then people have to make more stuff up and you’re going to have a big pile of de facto standards as people try to settle on OK ways of getting stuff done. Is making standards simpler really going to fix the problem, if the complexity just pops up elsewhere?

Crockford:What we really need to be doing is getting better at predicting what we’re really going to need in the future. Maybe we have to wait for time travel before we finally start getting this stuff right. In the meantime, I look on that experimentation and proliferation of possible approaches as a positive thing in that maybe the right approach to take to standardization is to figure out which of those are the best thought out, which are the most maintainable, which are the most growable, and pick that. Rather than a standards committee trying to guess the best way to do it, we pick from examples in the marketplace what is actually demonstrably the best way to do it.

Seibel:But you feel like overall we’re making some progress?

Crockford:Progress isn’t always forward. Sometimes we’re leaping forward and sometimes we’re leaping backwards. When we leaped to the PC, we lost a whole lot of stuff. In the timesharing era, we had social systems online. A timesharing system was a marketplace. It was a community and everybody who was a part of that system could exchange email, they could exchange files, they could chat, they could play games. They were doing all this stuff and all that got lost when we went to PCs. It took another 20 years or so to get that back.

We also took a huge step backwards in terms of security. Timesharing systems were starting to understand how to defend the system and the users of the system from each other. When we went to PCs, you owned your machine and everything running in that machine had the same privileges, the same rights to do whatever it had to do, and it turned out that not all the software running in your machine is acting in your interest. We’re still struggling with that. We’ve seen lots of improvements going into the PC operating systems, but we’re still not at the point where some of the more forward-looking timesharing systems were way, way back.

Seibel:Which ones are you thinking of?

Crockford:MULTICS was doing some really interesting stuff in cooperative processes, and having multiple address spaces which were able to communicate with each other but couldn’t get into each other’s stuff. That’s the basic baseline you need in order to start doing cooperative computing. And we’re now trying to figure out how to get that into the browser. It’s a long time between MULTICS and here. We’re starting now to catch up to insights that were being acted on way back then.

Seibel:I’ve noticed a similar thing with languages—PCs were programmed in assembly because even C was too high-level and only now are we getting back to languages with some of the power of languages like Smalltalk and Lisp that existed when PCs came out. I wonder if programmers are as aware of the relatively short history of our field as they could be, or do we keep reinventing the wheel?

Crockford:I think we’re tragically unaware of our history, and I’m often really disappointed to see that people who are now practicing this craft having no intellectual curiosity about where this stuff came from and just assume that some committee got it right and presented them with a set of tools or languages, and all they have to do is use it properly.

There are amazing stories about where this stuff came from, and what influenced what, and who did what, and what is now considered a mistake, and what should be considered a mistake, but hasn’t yet. I think of myself sometimes as an archaeologist of software technology and over the years I’ve accumulated this collection of underappreciated technology, things that I think were really, really good, which are significantly in advance of the state of the art of what we’re doing now. I keep hoping that we will somehow rediscover this stuff and learn to appreciate it and benefit from having done it, but it’s a really slow process. I see folks are really entrenched in the way that things are working right now and it’s really difficult to move.

Seibel:What are some of those technologies?

Crockford:Lisp and Smalltalk, you’ve just mentioned. That’s brilliant stuff, and we’re now finally seeing those ideas being factored into modern languages, so we’re doing a lot of work now in JavaScript and trying to modernize that. It turns out JavaScript already anticipated a bunch of that stuff; it’s got functions which are lexically scoped and first-class, which was brilliant. Now we’re trying to figure out how to get more of the goodness of Smalltalk and Scheme into this language without breaking it. You could argue that we’d be better off if we just threw out everything we’re working on now and go back to Smalltalk and Scheme, and we probably would be better off than we are right now, but that doesn’t appear to be an option.

As we’re getting more and more into mash-ups, we want to have code from all over the place—stuff that we’ll never, ever test with—actually run out in the field. Which is a new kind of programming. We’ve never done that before. I think that’s the future of programming and we’re getting to it in JavaScript first and it’s working here because the language, in spite of all the stuff it got wrong, got this other stuff right.