Oskar Andreasson - Iptables Tutorial 1.2.2

#!/bin/bash

Into the line below:

#!/bin/bash -x

As you will see, this changes your output from perhaps a couple of lines, to copious amounts of data on the output. The code shows you every single command line that is executed, and with the values of all the variables et cetera, so that you don't have to try and figure out exactly what the code is doing. Simply put, each line that gets executed is output to your screen as well. One thing that may be nice to see, is that all of the lines that bash outputs are prefixed by a + sign. This makes it a little bit easier to discern error or warning messages from the actual script, rather than just one big mesh of output.

The -x option is also very interesting for debugging a couple of other rather common problems that you may run into with a little bit more complex rulesets. The first of them is to find out exactly what happens with what you thought was a simple loop, such as an for, if or while statement? For example, let's look at an example.

#!/bin/bash

iptables="/sbin/iptables"

$iptables -N output_int_iface

cat /etc/configs/machines | while read host; do

$iptables -N output-$host

$iptables -A output_int_iface -p tcp -d $host -j output-$host

cat /etc/configs/${host}/ports | while read row2; do

$iptables -A output-$host -p tcp --dport $row2 -d $host -j ACCEPT

done

done

This set of rules may look simple enough, but we continue to run into a problem with it. We get the following error messages that we know come from the above code by using the simple echo debugging method.

work3:~# ./test.sh

Bad argument `output-'

Try `iptables -h' or 'iptables --help' for more information.

cat: /etc/configs//ports: No such file or directory

So we turn on the -x option to bash and look at the output. The output is shown below, and as you can see there is something very weird going on in it. There are a couple of commands where the $host and $row2 variables are replaced by nothing. Looking closer, we see that it is only the last iteration of code that causes the trouble. Either we have done a programmatical error, or there is something strange with the data. In this case, it is a simple error with the data, which contains a single extra linebreak at the end of the file. This causes the loop to iterate one last time, which it shouldn't. Simply remove the trailing linebreak of the file, and the problem is solved. This may not be a very elegant solution, but for private work it should be enough. Otherwise, you could add code that looks to see that there is actually some data in the $host and $row2 variables.

work3:~# ./test.sh

+ iptables=/sbin/iptables

+ /sbin/iptables -N output_int_iface

+ cat /etc/configs/machines

+ read host

+ /sbin/iptables -N output-sto-as-101

+ /sbin/iptables -A output_int_iface -p tcp -d sto-as-101 -j output-sto-as-101

+ cat /etc/configs/sto-as-101/ports

+ read row2

+ /sbin/iptables -A output-sto-as-101 -p tcp --dport 21 -d sto-as-101 -j ACCEPT

+ read row2

+ /sbin/iptables -A output-sto-as-101 -p tcp --dport 22 -d sto-as-101 -j ACCEPT

+ read row2

+ /sbin/iptables -A output-sto-as-101 -p tcp --dport 23 -d sto-as-101 -j ACCEPT

+ read row2

+ read host

+ /sbin/iptables -N output-sto-as-102

+ /sbin/iptables -A output_int_iface -p tcp -d sto-as-102 -j output-sto-as-102

+ cat /etc/configs/sto-as-102/ports

+ read row2

+ /sbin/iptables -A output-sto-as-102 -p tcp --dport 21 -d sto-as-102 -j ACCEPT

+ read row2

+ /sbin/iptables -A output-sto-as-102 -p tcp --dport 22 -d sto-as-102 -j ACCEPT

+ read row2

+ /sbin/iptables -A output-sto-as-102 -p tcp --dport 23 -d sto-as-102 -j ACCEPT

+ read row2

+ read host

+ /sbin/iptables -N output-sto-as-103

+ /sbin/iptables -A output_int_iface -p tcp -d sto-as-103 -j output-sto-as-103

+ cat /etc/configs/sto-as-103/ports

+ read row2

+ /sbin/iptables -A output-sto-as-103 -p tcp --dport 21 -d sto-as-103 -j ACCEPT

+ read row2

+ /sbin/iptables -A output-sto-as-103 -p tcp --dport 22 -d sto-as-103 -j ACCEPT

+ read row2

+ /sbin/iptables -A output-sto-as-103 -p tcp --dport 23 -d sto-as-103 -j ACCEPT

+ read row2

+ read host

+ /sbin/iptables -N output-

+ /sbin/iptables -A output_int_iface -p tcp -d -j output-

Bad argument `output-'

Try `iptables -h' or 'iptables --help' for more information.

+ cat /etc/configs//ports

cat: /etc/configs//ports: No such file or directory

+ read row2

+ read host

The third and final problem you run into that can be partially solved with the help of the -x option is if you are executing the firewall script via SSH, and the console hangs in the middle of executing the script, and the console simply won't come back, nor are you able to connect via SSH again. In 99.9% of the cases, this means there is some kind of problem inside the script with a couple of the rules. By turning on the -x option, you will see exactly at which line the script locks dead, hopefully at least. There are a couple of circumstances where this is not true, unfortunately. For example, what if the script sets up a rule that blocks incoming traffic, but since the ssh/telnet server sends the echo first as outgoing traffic, netfilter will remember the connection, and hence allow the incoming traffic anyways if you have a rule above that handles connection states.

As you can see, it can become quite complex to debug your ruleset to its full extent in the end. However, it is not impossible at all. You may also have noticed, if you have worked remotely on your firewalls via SSH, for example, that the firewall may hang when you load bad rulesets. There is one more thing that can be done to save the day in these circumstances. Cron is an excellent way of saving your day. For example, say you are working on a firewall 50 kilometers away, you add some rules, delete some others, and then delete and insert the new updated ruleset. The firewall locks dead, and you can't reach it. The only way of fixing this is to go to the firewall's physical location and fix the problem from there, unless you have taken precautions that is!

One of the best precautions you may take against a locked down firewall is to simply use cron to add a script that is run every 5 minutes or so that resets the firewall, and then remove that cron line once you are sure the installation works fine. The cron line may look something like the one below and be entered with the command crontab -e.

*/5 * * * * /etc/init.d/rc.flush-iptables.sh stop

Make absolutely sure, that the line will actually work and do what you expect it to do before you start doing something you expect will or may freeze the server you are working on.

Another tool that is constantly used to debug your scripts is the syslog facility. This is the facility that logs all log-messages created by a ton of different programs. In fact, almost all large programs support syslog logging, including the kernel. All messages sent to syslog have two basic variables set to them that are very important to remember, the facility and the log level/priority.

The facility tells the syslog server from which facility the log entry came from, and where to log it. There are several specified facilities, but the one in question right now is the Kern facility, or kernel facility as it may also be called. All netfilter generated messages are sent using this facility.

The log level tells syslog how high priority the log messages have. There are several priorities that can be used, listed below.

debug

info

notice

warning

err

crit

alert

emerg

Depending on these priorities, we can send them to different log files using the syslog.conf. For example, to send all messages from the kern facility with warning priority to a file called /var/log/kernwarnings, we could do as shown below. The line should go into the /etc/syslog.conf.