Peter Seibel - Coders at Work - Reflections on the craft of programming

Seibel:Would you then push changes all the way down through the process? Or did people start short-circuiting the process, going directly to the programmers and saying, “All right, we figured out the customer needs X ”?

Allen:Yeah. One could never really write specs that were going to be adequate and useful at a detail level over the years of the life cycle. That was a problem. And now we have another process, of course—just do it and throw it away, kind of.

Seibel:Well, it was Brooks, in his famous book, who said, “Build one to throw away because you’re going to.”

Allen:Yes. And in fact, that is true—I very much believe that. But lots of times, that has led, in my opinion, to not thinking at all before you start building.

I always like to have a picture—a model. Often a flowchart and some specification about the interfaces. We were heavy users, of course, of flowcharts at that time because one didn’t get access to the machine that often and that was a very nice model for thinking through how parts of the system would interact, what would be done, and specifying what would be done where, and the functionality of the various components. I don’t know what the analog of that is now.

Seibel:Even with flowcharting, there are formal flowcharts produced as documentation and then there are the flowcharts you draw on a blackboard to try and understand something. Was it more of the former or the latter?

Allen:In some cases, it would be formal flowcharts. Often in the kernels of things there were some very complex pieces and one would do that. Otherwise, it was just informal and a way of working at solving a problem. Blackboards would be covered and become the record for the month or whatever period of time.

Seibel:So the big project you led was the PTRAN compiler project, which was when you first started working on explicit concurrency, as opposed to the implicit concurrency in the CPU pipelines and so forth. When you started that, that was a new thing, both for you and for IBM.

Allen:It was new for IBM, but we were very, very late coming into it. The great work that initiated it from a real pragmatic point of view was at Illinois starting in ’69 and ’70.

Seibel:And what language did the PTRAN compiler compile? Was it straight Fortran with no added constructs for parallelism?

Allen:That’s right, that’s where we started from. What I wanted to do was to do the same thing we’d done for optimization: The user writes a sequential code in the language in a way that’s natural for the application and then have the compiler do the optimization and mapping it to the machine and taking advantage of concurrency.

In PTRAN, the idea was still to take the “dusty decks,” as we called them, referring to an existing code base, and to automatically take advantage of the hardware’s parallel components.

Seibel:So, essentially, that was targeting what today we would think of as symmetric multiprocessors?

Allen:Yeah, could be. There are many, many models of parallelism, which is one of the difficulties. I think that that can be greatly simplified. But multicore is one of the things that really are particularly interesting, for me at least. But there are many models of parallelism.

We actually built it from existing work, particularly Dave Kuck’s. Some work from NYU. We hired a group of newly minted PhDs from these places that already had built up a lot of expertise. We had a quite a lot of significant results, both on the practical and on the theoretical side—we worked on both at the same time. I’m a very strong believer that one wants to be able to take the practice into identifiable algorithms, and theory, and ways of thinking about how to solve the problems and also to take the algorithms into practice to see how really valuable they are, and how they apply. I think our field is best done when it works on both sides on the same projects.

Seibel:On the PTRAN project, you were leading a team. Were you still coding at all by then?

Allen:I wasn’t doing the coding, but I was very close to it. As an example, when the Static Single Assignment work was done, I didn’t see how it could be implemented in any reasonable time. I mean, it was a very good algorithm, but I didn’t see an implementation that was bounded in time and space in some real way. So I had that challenge out there. I had to see that code. I needed it. It had to be implemented. It couldn’t be just a paper—a very nice paper, a famous paper—that shows graphs and complexity bounds.

If we can’t implement it in a real system, that challenge would still be out there. It wasn’t going to be as useful as I wanted it to be. Finally, one of my people had an encoding. And I walked through that, every piece of that code, and looked at the data structures that were used. And it was astounding. I said, “This is it. It works.”

Seibel:So you were looking at all of the pieces that were going to go into the whole system?

Allen:Yes, yes, yes.

Seibel:You also were managing all these people? Or were you a technical architect with someone else managing the group?

Allen:No, I was the research manager for the group. There were about 10 or 12 core people and we divided the work up so that each person really had ownership of a piece of it.

Seibel:People have been debating at least since Gerald Weinberg’s book The Psychology of Computer Programming whether it’s better for people to “own” code, so they take responsibility for it, or to have people work more collaboratively so you avoid having silos that only one person understands. It sounds like you thought dividing up the ownership was the way to go?

Allen:We worked collaboratively, but the collaboration was about the state of the system, of the implementation. And some people were very good at the implementation, and so they’d own a piece—some piece of the optimizer or the intra-procedural analysis was definitely one or two people. But also, there were a number of people that were doing a lot of the theory work, or the abstract work of writing the papers, and writing a lot of the papers and algorithms. And it was the bringing together of those two special parts that I think really made the group so strong.

This was a period of a lot of work going on in the analysis and transformations for parallelism. So what I tried to do was, for each of the people that were doing theory to get them to write some code, express it in code as a part of the system. And the people who were just doing the other part, well I’d try to get them to write things up so they were more generally available.

Seibel:A lot of programmers will do anything to avoid becoming a manager. Was managing something that you also enjoyed?

Allen:Well, Research in the earlier times did not distinguish—doing management was not a promotion, not a salary raise. It was just somebody had to manage this piece of work, and “OK, don’t you want to do it?” Or, “You’re the obvious one to manage this piece of work.” And it was technical management; there wasn’t much people management involved. But in Research people are RSMs, Research Staff Members, the day they enter, and for the rest of their career mostly. All my colleagues, that’s what we are. So one moved in and out of management without any stigma attached.

Seibel:So presumably the people who got chosen to manage were the ones who were actually good at it. How did you pick up those skills?