Peter Seibel - Coders at Work - Reflections on the craft of programming

The funny thing is, thinking back, I don’t think all these modern gizmos actually make you any more productive. Hierarchical file systems—how do they make you more productive? Most of software development goes on in your head anyway. I think having worked with that simpler system imposes a kind of disciplined way of thinking. If you haven’t got a directory system and you have to put all the files in one directory, you have to be fairly disciplined. If you haven’t got a revision control system, you have to be fairly disciplined. Given that you apply that discipline to what you’re doing it doesn’t seem to me to be any better to have hierarchical file systems and revision control. They don’t solve the fundamental problem of solving your problem. They probably make it easier for groups of people to work together. For individuals I don’t see any difference.

Also, I think today we’re kind of overburdened by choice. I mean, I just had Fortran. I don’t think we even had shell scripts. We just had batch files so you could run things, a compiler, and Fortran. And assembler possibly, if you really needed it. So there wasn’t this agony of choice. Being a young programmer today must be awful—you can choose 20 different programming languages, dozens of framework and operating systemsand you’re paralyzed by choice. There was no paralysis of choice then. You just start doing it because the decision as to which language and things is just made—there’s no thinking about what you should do, you just go and do it.

Seibel:Another difference these days is that you can no longer understand the whole system from top to bottom. So not only do you have lots of choices to make, they’re all about which black boxes you want to use without necessarily fully understanding how they work.

Armstrong:Yeah—if these big black boxes don’t work properly, and you have to modify them, I reckon it’s easier just to start from scratch and just write everything yourself. The thing that really hasn’t worked is software reuse. It’s appallingly bad.

Seibel:Yet you’re the architect not only of Erlang but of an application framework, the Open Telecom Platform. Is it reusable?

Armstrong:To an extent it’s reusable. But the same problem will occur. If that framework exactly solves your problem—if some programmer who doesn’t know anything about the design criteria for OTP looks at it in a few years’ time and says, “Oh, that’s great; that’s exactly what I want to do,” then it’s fine and you get this measure of reusability. If it’s not , then you have a problem.

Fairly recently I’ve seen people say, “This is really kind of artificial, we’re twisting the code to fit into this OTP framework.” So I say, “Well, rewrite the OTP framework.” They don’t feel they can change the framework. But the framework’s just another program. It’s really rather easy. And I go into it and then it does what they want. They look at it and they say, “Yeah, well, that’s easy.” They accept that it’s easy. But they say, “Well, our project management doesn’t want us messing around with the framework.” Well, give it a different name then or something.

Seibel:But do you think it’s really feasible to really open up all those black boxes, look inside, see how they work, and decide how to tweak them to one’s own needs?

Armstrong:Over the years I’ve kind of made a generic mistake and the generic mistake is to not open the black box. To mentally think, this black box is so impenetrable and so difficult that I won’t open it. I’ve opened up one or two black boxes: I wanted to do a windowing system, a graphics system for Erlang, and I thought, “Well, let’s run this on X Windows.” What is X Windows? It’s a socket with a protocol on top of it. So you just open the socket and squirt these messages down it. Why do you need libraries? Erlang is message based. The whole idea is you send messages to things and they do things. Well, that’s the idea in X Windows—you’ve got a window, send it a message, it does something. If you do something in the window it sends you a message back. So that’s very much like Erlang. The way of programming X Windows, however, is through callback libraries—this happens and call this. That’s not the Erlang way of thinking. The Erlang way of thinking is, send a message to something and do something. So, hang on, let’s get rid of all these libraries in between—let’s talk directly to the socket.

And guess what? It’s really easy. The X protocol’s got, I don’t know, 100 messages, 80 messages or something. Turns out you only need about 20 of them to do anything useful. And these 20 messages you just map onto Erlang terms and do a little bit of magic and then you can start sending messages to windows directly and they do things. And it’s efficient as well. It’s not very pretty because I haven’t put much effort into graphics and artistic criteria—there’s a lot of work there to make it look beautiful. But it’s not actually difficult.

Another one is this typesetting system I did where the abstraction boundary I opened up is Postscript. As you get to that boundary you think, “I don’t want to go through the boundary,” because what’s underneath is—you imagine—enormously complicated. But again, it turns out to be very easy. It’s a programming language. It’s a good programming language. The abstraction boundary is easy to go through and once you’ve gone through, there’s a lot of benefit.

For my Erlang book, my publisher said, “We’ve got tools to make diagrams.” But the thing I don’t like about diagramming tools is it’s really difficult to get an arrow to meet exactly. And your hand hurts. I thought, “The amount of time to write a program that spits out Postscript and then say, ‘I want a circle there and the arrow goes exactly there,’ and get the program right, isn’t long.” It takes a few hours. Doing diagrams with programs takes about the same time as doing them in a WYSIWYG thing. Only there are two benefits. Your hand doesn’t hurt at the end and even when you blow the thing up to a magnification of 10,000, the arrow points exactly right.

I can’t say beginner programmers should open up all these abstractions. But what I am saying is you should certainly consider the possibility of opening them. Not completely reject the idea. It’s worthwhile seeing if the direct route is quicker than the packaged route. In general I think if you buy software, or if you use other people’s software, you have to reckon with an extremely long time to tailor it—it doesn’t do exactly what you want, it does something subtly different. And that difference can take a very long time to solve.

Seibel:So you started out saying software reuse is “appallingly bad,” but opening up every black box and fiddling with it all hardly seems like movement toward reusing software.

Armstrong:I think the lack of reusability comes in object-oriented languages, not in functional languages. Because the problem with object-oriented languages is they’ve got all this implicit environment that they carry around with them. You wanted a banana but what you got was a gorilla holding the banana and the entire jungle.

If you have referentially transparent code, if you have pure functions—all the data comes in its input arguments and everything goes out and leaves no state behind—it’s incredibly reusable. You can just reuse it here, there, and everywhere. When you want to use it in a different project, you just cut and paste this code into your new project.

Programmers have been conned into using all these different programming languages and they’ve been conned into not using easy ways to connect programs together. The Unix pipe mechanism—A pipe B pipe C—is trivially easy to connect things together. Is that how programmers connect things together? No. They use APIs and they link them into the same memory space, which is appallingly difficult and isn’t cross-language. If the language is in the same family it’s OK—if they’re imperative languages, that’s fine. But suppose one is Prolog and the other is C. They have a completely different view of the world, how you handle memory. So you can’t just link them together like that. You can’t reuse things. There must be big commercial interests for whom it is very desirable that stuff won’t work together. It creates thousands of jobs for consultants. And thousands of tools to solve problems that shouldn’t exist. Problems that were solved years ago.