Thomas Limoncelli - Time Management for System Administrators

www-draft.example.com

The next release of our web site was developed here.www-qa.example.com

The draft site would be copied here for QA to check over. Once the copy was made, the files were immediately made read-only. If QA approved this site, we needed to be able to verify that these exact bits were copied to the live site.www.example.com

This was the live site that external people would see.

The web designers would ask the system administrators to copy their draft to www-qa.example.com. When the QA group approved the site, they would tell the system administrators to make the site go live.

Each of these two functions was automated:

readyforqa

Copied the draft site to the QA site. golive

Copied the QA site to the live site.

Marketing demanded a way to make emergency updates when the QA department wasn't available. We created this command:

emergency-draft-to-live

Copied the draft site directly to the live site after asking "Are you sure?" a few times.

These three scripts comprised Layer 2, which I mentioned earlier. Layer 1 was a script that did the actual copying of one site's data to another site, making a backup along the way, and setting files to read-only (changing the ownership of the files, too). Layer 1 had to be done as root because it was changing ownership of files and accessing machines via secured channels.

sudo was programmed as described in Table 13-1.

Table 13-1. Web update permission table

Web developers

QA

Marketing

Readyforqa

X

X

Golive

X

Emergency-draft-to-live

X

We actually went through the effort of having management sign off on this chart, with real signatures, to make sure they understood that they were agreeing to what they thought they were agreeing to. The political process to get this approved was the difficult part. It took weeks. Presenting the information to management in the chart form made it a lot easier for a decision to be made. They could understand and update the chart themselves until they were happy with it. Translating the final chart into a sudo configuration file was the easy part.

Per Layer 3, we decided to make an easier way for people to access these commands. We considered a web interface, but, in this case, the users were satisfied with a menu program that presented them with a list of options that ran the appropriate command.

The menu ran without any additional privileges (i.e., not under sudo ), but called the Layer 2 programs using sudo as needed .

Summary

Automation is great because it saves you time. It also permits you to push work to other, less-technical people.

There are four types of problems that SAs typically deal with:

Simple things done once

Hard things done once

Simple things done often

Hard things done often

"Hard things done once" and "Simple things done often" are the right things to try to automate. "Hard things done often," while tempting, is usually better served by off-the-shelf packages (commercial or free).

To automate a process, first be sure you can do the steps manually. Then document each step, and make sure that you can automate each step. Then bring all of the steps together.

You can save a lot of typing time by making aliases. This is true for command-line systems as well as for applications, such as SSH. Set the alias as close to the actual application as possible. For example, setting the alias in the SSH configuration file means all systems that leverage SSH will use the alias.

The Unix/Linux make command is extremely powerful. It is not just for programmers. You can use it to automate system administration tasks. On Unix/Linux systems, especially servers, standardize on having a Makefile in /etc that automates common tasks such as reindexing aliases, cloning data, and so on.

Bash and /bin/sh shell languages are more sophisticated and powerful than you may realize. The examples in this chapter reveal how to parse command-line options and even how to write a small malware detector!

When writing a long command line, test each part as you write it.

When writing code for other people, the user interface becomes more important. There are tricks and techniques to creating useful user interfaces. You can avoid the issue by shifting all data entry to a program like MS Excel or by providing a menu system or web interface that lets people access higher-privileged systems.

When writing code to let users do privileged operations, be extremely careful. Build on security tools that already exist and have good credibility, such as sudo . Use a permission table to explain to management who will have access to what. It is their job to manage risk and your job to help them understand the issues. Get approval before you deploy the system.

As you move through your career, you will find yourself automating more and more tasks. It is a good idea to learn a programming language suited for system administration functions, such as Perl, Python, Ruby, or Shell, as well as operating-specific techniques like those featured in the O'Reilly Cookbook series mentioned previously in this book.

Appendix A. Epilogue

Congratulations. You've made it all the way to the end of the book. So now what?

First, I recommend you reread the book. We learn through repetition. If you reread (or skim) the book while the topics are fresh in your mind, it will have a strong impact.

Second, practice makes perfect. The more you practice the techniques in this book, the better you will get at them. Suddenly, you'll find yourself knowing the techniques so well that you'll be able to customize them and improve them in ways that make sense for your particular lifestyle or situation. One reader found that he was better able to manage his daily to do lists when he swapped the sides of his organizer where he put his schedule and to do items. Who knew? Whatever floats your boat! I just recommend you try my way first to get a sense of the system.

Third, accept slippage. Sometimes you will lapse into your old habits. That's OK, as long as you recognize it and get back to using the techniques as soon as you can. It might be helpful to reread the appropriate chapter for some inspiration.

Fourth, you might consider reading some traditional time management books, ones not written for system administrators in particular. This book focuses on the things specific to the system administration lifestyle and leaves a lot of general topics to the other books that cover them very well. I recommend Getting Things Done by David Allen (http://www.davidco.com).

What to Do with All Your "New" Free Time?

The techniques in this book may save you hours, if not days, each week. If you save a little more than an hour per day, you can get the same amount of work done in a four-day workweek .

So, what will you do with all this free time ?

Please don't squander it. I beg you. When I first started applying time management techniques to my life, I used all the new free time I gained on my then-current addiction: reading more Usenet NetNews. I guess the contemporary equivalent is to spend it reading RSS feeds, blogs, web sites, and such. Many such things are time wasters. Please don't use your new-found free time to pack more time wasters into your life.

I have a better idea.

Use this new-found free time to fight injustice.

The most common injustice that I see every day is the way corporations steal our lives away from our families. We wake up one day to find that our children have grown up hardly knowing us, or that our significant others are leaving because they hardly see us. "How did the time pass so quickly?", we wonder to ourselves.