Thomas Limoncelli - Time Management for System Administrators

make can also be useful for keeping files on various servers up-to-date. For example, let's suppose the aliases file in our example needs to be the same on both of our email servers. We decide that we'll update the file on this server, and push it to server2. That recipe might look like this: push.aliases.done: $(PDIR)/aliases scp $(PDIR)/aliases server2:$(PDIR)/aliases touch $@

We push the file to server2 using scp , then touch a file called push.aliases.done . Since this file is created after the successful copy of the file, we can build recipes so that the push is only done if it's absolutely needed. We can also force the file to be recopied by simply deleting the push.aliases.done file and typing make. Traditionally, there is a recipe called clean that deletes all the *.done files and other machine-generated files.

There is nothing special about files that end with .done . It is simply customary to name-flag or timestamp files with .done at the end.

Here's a complete example. There are two files that need indexing if they change: aliases and access . If either of them has been reindexed, postfix is told to reload. They also are both pushed to server2 if they change. Finally, the command cd /etc && make is sent to server2 if and only if one or more of the files has been pushed to it.

By carefully constructing the recipes with proper dependencies, and touching *.done files where required, make will do the absolute minimal amount of work to bring the system up-to-date: # # Makefile for server1 # NEWALISES=/usr/sbin/newaliases PDIR=/etc/postfix POSTMAP=/usr/local/postfix/sbin/postmap # # High-level "commands" # all: aliases.done access.done reload_if_needed.done push push: push.done reload: postfix reload stop: postfix stop start: postfix start reload_if_needed.done: aliases.done access.done postfix reload touch reload_if_needed.done clean: rm -f \ $(PDIR)/aliases.pag $(PDIR)/aliases.dir \ $(PDIR)/access.dir $(PDIR)/access.pag \ push.aliases.done push.access.done reload_if_needed.done # # Recipes for particular files that need indexing/regeneration # # When aliases changes, generate the .pag and .dir files aliases.done: $(PDIR)/aliases.pag $(PDIR)/aliases.dir $(PDIR)/aliases.pag $(PDIR)/aliases.dir: $(PDIR)/aliases $(NEWALIASES) # When access changes, generate the .pag and .dir files access.done: $(PDIR)/access.dir $(PDIR)/access.pag $(PDIR)/access.dir $(PDIR)/access.pag: $(PDIR)/access $(POSTMAP) $(PDIR)/access # # pushes # push.done: push.aliases.done push.access.done ssh server2 "cd /etc && make" touch $@ push.aliases.done: aliases.done scp $(PDIR)/aliases server2:$(PDIR)/aliases touch $@ push.access.done: access.done scp $(PDIR)/access server2:$(PDIR)/access touch $@

This Makefile is a good starting point for you to use on your systems. It is rather sophisticated because we do things to make sure Postfix isn't reloaded unless absolutely necessary.

With a Makefile like this, you no longer have to remember a multitude of commands and which ones should be used for which updated files. You never have to worry about forgetting to type a command. Many complicated procedures are reduced to:

Edit the appropriate file.

Type make.

make can be the ultimate tool for bringing together many smaller automated processes. Once, I had to merge the processes and procedures for three large networks into one. Each network had a different way of managing its aliases, hosts, and other administrative files. As I learned the procedures for each network, I constructed a Makefile specific to that network's master server. The high-level recipe names were the same in all three networks, but the commands they ran to accomplish the work were specific to each network.

The strategy was to create a new master server that would eventually replace all the legacy servers. Initially, the new master's Makefile simply initiated a make on the three legacy masters via rsh (this was long before ssh ). I then migrated recipes to the new master one at a time. For example, first I decided that the new master would be the single source for the aliases file. I merged the aliases files of the three legacy networks and put the result on the new master. Once it was tested there, I added recipes on the new master to push that merged file to the legacy masters as if it were their own. I continued this process for each file or database.

Since each change was small and specific, I could test it incrementally. After literally hundreds of small changes, all the servers were "singing from the same songbook." At that point, it was easy to eliminate the legacy masters and let the new master be the authoritative master for all clients.

Warning

Any file that is automatically pushed to other servers should always have a comment at the top of the file warning other system administrators where the file came from and where to edit it.

Here's the warning I use: # THIS FILE IS MAINTAINED ON: server1.example.com # Edit it with this command: xed file.txt # If you edit this file on any other machine, # it will be overwritten. BE CAREFUL!

Since the previous note mentioned xed , I should explain what it is. There are many programs called xed , but this one can be found on http://www.nightcoder.com/code/xed. This program calls whatever editor you usually use ($EDITOR can be set to vi, pico, emacs, and so on) after locking the file. It is a must for any site that has multiple system administrators working on the same machine. If you are using RCS to track changes to a file, it does all the "check in" and "check out" work for you. This gives you infinite undo and a logfile of who changed what. If you find that a system has been acting funny for the last month, just check the log to see who changed the file a month ago and, well, be nice—we all make mistakes.

Hard Things Done Once

When we find ourselves doing something very difficult, automating the task records what we've done. When we do it in the future, it will be easier. This is how we build up our little bag of tricks.

Encapsulating a Difficult Command

Sometimes it takes hours to work out exactly the right command required to do something. For example, there is a program that creates ISO images, the kind you burn onto CD-ROMs. Its manual page describes hundreds of options, but to make an image readable by Windows, Unix, and Mac systems, the command is simply: $ mkisofs -D -l -J -r -L -f -P " Author Name " -V " disk label " -copyright copyright.txt -o disk.iso /directory/of/files

Sure, you can do it from a GUI, but where's the fun (or ability to script) in that?

This command also lets you do things not found in most GUIs, such as the ability to specify a copyright note, author name, and so on.

This is a good example of something to work into a .BAT file (DOS) or a Unix/Linux shell script.

Here's a shell script called makeimage1 that uses this: #!/bin/bash mkisofs -D -l -J -r -L -f -P "Limoncelli" -V 'date -u +%m%d' $*

The 'date -u +%m%d' sets the volume name to the current date.

One of the things that held me back from writing good scripts was that I didn't know how to process command-line parameters. Here are instructions for copying all the command-line arguments into a script.

The $* in the makeimage1 script means "any items on the command line." So, if you typed: $ makeimage1 cdrom/

then the $* would be replaced by cdrom/.

Since $* works for multiple arguments, you can also do: $ makeimage1 cdrom/ dir1/ dir2/