Olaf Kirch - Linux Network Administrator Guide, Second Edition

To distinguish articles and recognize duplicates, Usenet articles have to carry a message ID (specified in the Message-Id: header field), which combines the posting site's name and a serial number into ‹ serial @ site ›. For each article processed, the news system logs this ID into a history file, against which all newly arrived articles are checked.

The flow between any two sites may be limited by two criteria. For one, an article is assigned a distribution (in the Distribution: header field), which may be used to confine it to a certain group of sites. On the other hand, the newsgroups exchanged may be limited by both the sending and receiving systems. The set of newsgroups and distributions allowed to be transmitted to a site are usually kept in the sys file.

The sheer number of articles usually requires that improvements be made to the above scheme. On UUCP networks, systems collect articles over a period of time and combine them into a single file, which is compressed and sent to the remote site. This is called batching .

An alternative technique is the ihave/sendme protocol that prevents duplicate articles from being transferred, thus saving net bandwidth. Instead of putting all articles in batch files and sending them along, only the message IDs of articles are combined into a giant "ihave" message and sent to the remote site. The remote site reads this message, compares it to its history file, and returns the list of articles it wants in a "sendme" message. Only the requested articles are sent.

Of course, ihave/sendme makes sense only if it involves two big sites that receive news from several independent feeds each, and that poll each other often enough for an efficient flow of news.

Sites that are on the Internet generally rely on TCP/IP-based software that uses the Network News Transfer Protocol (NNTP). NNTP is described in RFC-977; it is responsible for the transfer of news between news servers and provides Usenet access to single users on remote hosts.

NNTP knows three different ways to transfer news. One is a real-time version of ihave/sendme, also referred to as pushing news. The second technique is called pulling news, in which the client requests a list of articles in a given newsgroup or hierarchy that have arrived at the server's site after a specified date, and chooses those it cannot find in its history file. The third technique is for interactive newsreading and allows you or your newsreader to retrieve articles from specified newgroups, as well as post articles with incomplete header information.

At each site, news is kept in a directory hierarchy below /var/spool/news , each article in a separate file, and each newsgroup in a separate directory. The directory name is made up of the newsgroup name, with the components being the path components. Thus, comp.os.linux.misc articles are kept in /var/spool/news/comp/os/linux/misc . The articles in a newsgroup are assigned numbers in the order they arrive. This number serves as the file's name. The range of numbers of articles currently online is kept in a file called active , which at the same time serves as a list of newsgroups your site knows.

Since disk space is a finite resource, you have to start throwing away articles after some time. [123] Some people claim that Usenet is a conspiracy by modem and hard disk vendors. This is called expiring . Usually, articles from certain groups and hierarchies are expired at a fixed number of days after they arrive. This may be overridden by the poster by specifying a date of expiration in the Expires: field of the article header.

You now have enough information to choose what to read next. UUCP users should read about C-News in Chapter 21. If you're using a TCP/IP network, read about NNTP in Chapter 22. If you need to transfer moderate amounts of news over TCP/IP, the server described in that chapter may be enough for you. To install a heavy-duty news server that can handle huge volumes of material, go on to read about InterNet News in Chapter 23.

One of the most popular software packages for Netnews is C News. It was designed for sites that carry news over UUCP links. This chapter will discuss the central concepts of C News, basic installation, and maintenance tasks.

C News stores its configuration files in /etc/news , and most of its binaries are kept below the /usr/lib/news/ directory. Articles are kept below /var/spool/news . You should make sure that virtually all files in these directories are owned by user news or group news . Most problems arise from files being inaccessible to C News. Use su to become the user news before you touch anything in the directory. The only exception is the setnewsids command, which is used to set the real user ID of some news programs. It must be owned by root and have the setuid bit set.

In this chapter, we describe all C News configuration files in detail and show you what you have to do to keep your site running.

Articles can be fed to C News in several ways. When a local user posts an article, the newsreader usually hands it to the inews command, which completes the header information. News from remote sites, be it a single article or a whole batch, is given to the rnews command, which stores it in the /var/spool/news/in.coming directory, from where it will be picked up at a later time by newsrun. With any of these two techniques, however, the article will eventually be handed to the relaynews command.

For each article, the relaynews command first checks if the article has already been seen at the local site by looking up the message ID in the history file. Duplicate articles are dropped. Then relaynews looks at the Newsgroups: header line to find out if the local site requests articles from any of these groups. If it does, and the newsgroup is listed in the active file, relaynews tries to store the article in the corresponding directory in the news spool area. If this directory does not exist, it is created. The article's message ID is then logged to the history file. Otherwise, relaynews drops the article.

Sometimes relaynews fails to store an incoming article because a group to which it has been posted is not listed in your active file. In this case, the article is moved to the junk group. [124] There may be a difference between the groups that exist at your site and those that your site is willing to receive. For example, the subscription list might specify comp.all, which should send all newsgroups below the comp hierarchy, but at your site you might not list several of the comp newsgroups in the active file. Articles posted to those groups will be moved to junk. relaynews also checks for stale or misdated articles and reject them. Incoming batches that fail for any other reason are moved to /var/spool/news/in.coming/bad , and an error message is logged.

After this, the article is relayed to all other sites that request news from these groups, using the transport specified for each particular site. To make sure an article isn't sent to a site that has already seen it, each destination site is checked against the article's Path: header field, which contains the list of sites the article has traversed so far, written in the UUCP-style bang-path source-routing style described in Chapter 17, Electronic Mail. If the destination site's name does not appear in this list, the article is sent to it.

C News is commonly used to relay news between UUCP sites, although it is also possible to use it in an NNTP environment. To deliver news to a remote UUCP site, either in single articles or whole batches, uux is used to execute the rnews command on the remote site and feed the article or batch to it on standard input. Refer to Chapter 16, Managing Taylor UUCP, for more information on UUCP.