[leafnode-list] Not picking up some new groups?
matthias.andree at gmx.de
Tue Jan 27 13:47:40 CET 2004
John Carlyle- Clarke schrieb am 2004-01-27:
> Looking in /var/spool/news/leaf.node I am puzzled by what is there
> (although I don't know if it is correct or not!). There is an
> ":active.read" file dated Jan 22, and a "groupinfo" file dated today.
> There is a file for each server, also dated today, which seems to
> contain the interesting groups for each server, rather than a full
> list (although I notice a couple of groups seem to appear in all these
> files - even on servers that don't carry them). There is also a
> "last:servername.0" file for each server, dated Jan 22, size 0.
The mtime of your newsgroups.borland.com, determines which date is sent
in NEWGROUPS. Was any action (restore from backup, manual edit) taken
that might have changed the mtime outside fetchnews?
I just checked if the server time of newsgroups.borland.com is exact, it
is. Is your system clock exact? (ntpdate or ntpd when going online; ntpd
or chrony when "always-on") If it is ahead, fetchnews may miss new
> If I telnet into the Borland server (newsgroups.borland.com) on and do
> a newgroups, I get this:
> 200 Borland NNTP service: Posting allowed, 5.7c2,, S0 newgroups
While this is not the right syntax, giving the proper syntax as leafnode
will do yields the same result from the server.
> Although I see some new groups on some servers, I see nothing on the borland server at all. Here is a sample output:
> [jcc at pc69 news]$ gzip -dc news.all.1.gz | grep -E "found [1-9][0-9]* new newsgroups" | more
> Jan 26 08:01:37 pc69 fetchnews: news.gmane.org: found 1 new newsgroups
> Jan 27 05:01:46 pc69 fetchnews: news.gmane.org: found 1 new newsgroups
> I am running version 2.0.0.alpha20040119a. On Jan 22 I added an only_groups_pcre line to one of the servers in my config. Here are my server definitions.
This is unrelated.
I am attaching a new README-leaf.node file that is to appear in the next
snapshot release, I hope you find it useful. If you (anyone) have (has)
further questions/suggestions as to the file, let me know.
Encrypt your mail: my GnuPG key ID is 0x052E7D95
-------------- next part --------------
This file describes the files in the leaf.node directory.
(C) Copyright 2004 Matthias Andree. GNU GPL v2
1. Persistent files
tracks the last time when the active file was completely
refreshed, either through fetchnews -f or because
"timeout_active" days have passed since the last refresh.
fetchnews checks the mtime of this file, which, in regular
operation, should never be older than "timeout_active" days.
this file is always 0 bytes.
contains the list of newsgroups that leafnode knows about
and is both large and preciuos
The syntax is, in Pseudo-C:
groupname TAB mode TAB high TAB low TAB timestamp TAB description
groupname and description are self-documenting, timestamp is the
UNIX time when the group was added to leafnode, and mode can be
n (no posting), m (moderated), y (posting allowed).
low and high are the lowest and highest article numbers (natural
numbers) in leafnode's group store. These need to be tracked
explicitly because the "high" number must be monotonically
tracks when the last _full_ fetch of the active from SERVER:PORT
was completed. A PORT of 0 means looking up "nntp/tcp" from
These files are empty.
these files store the "high number" of the upstream server that
fetchnews has last seen, for the interesting groups. A missing
:PORT means looking up "nntp/tcp" from /etc/services.
These files contain lines with the following syntax:
group.name SPACE high
high is a natural number.
The mtime of this file is used to build the NEWGROUPS request.
2. Intermittent files
these files correspond to SERVER:PORT and SERVER and exist while
fetchnews fetches news from the server in question, these
contain checkpoint information that is read back next time
fetchnews runs, in order to not re-check XOVER information that
has successfully been fetched before.
temporary file that exists for the few seconds while the
groupinfo is written back to disk. It is renamed to groupinfo
only when successfully written.
is present while a leafnode process that can modify groupinfo is
running (everything except leafnode-version, newsq).
It contains the process ID (PID) of the process that set the
lock file on the first line and the host name on the second
line. The second line is there to avoid cleaning "stale" locks
from other hosts, should the spooldir be shared by some hosts
(via NFS, for instance).
It can be removed manually when the process ID recorded in the
file does not belong to a leafnode related process (i. e. after
crash and reboot).
this file is used to work around NFS non-atomicity in the
open(2) O_EXCL mode. It can be safely deleted at any time,
although leafnode takes care not to leave one behind if it can
More information about the leafnode-list