[leafnode-list] Re: trying to decide which is better
matthias.andree at gmx.de
Thu Nov 20 23:02:26 CET 2008
Eric S. Johansson schrieb am 2008-11-20:
> Matthias Andree wrote:
> > If IMAP (rather than NNTPd) is an option, have a look at Dovecot and if
> > it meets your requirements.
> we're already using dovecot but, I don't think it serves the need for
> a internal conference server, at lease, as far as I know.
Not directly - you'd likely have to add a mailing list underneath.
> > Leafnode uses what INN would call tradspool - files 1 2 3 4 5... in a
> > group directory. So, that would add some 80,000 files over the course of
> > the years. Quick file access can be achieved with modern file systems,
> > such as "dirhash" on UFS, "dir_index" on ext3, and most tree-structured
> > file systems should also be able to deal with this.
> I think in my circumstances a group/[00-ff]/1 2 3... mechanism might
> serve my needs better. If my hands weren't so crappy and C so hostile
> to speech recognition, I would try to fix it myself.
Unfortunately, exactly that is nontrivial to fix, since several places
of leafnode's hardcode the directory naming structure; that would have
to be consolidated first. I've been thinking about a better code
structure and perhaps C++ or something, but I have too little spare time
at the moment to make sweeping changes...
> (I've been living with RSI for the past 15 years)
That's a pity. When I had initial symptoms (aching forearms) while
writing up my diploma thesis (think master's), I went into the next
store and bought an ergonomic keyboard, reduced my writing efforts for a
couple of days, and insisted that my employer buy an ergonomic keyboard
for me, too. No symptoms for the past years -- so seems it was just in
> I've turned on dir_index and it does make things slightly faster but
> EXT3 still runs out of steam at 32,000 files in the directory. I'm
> probably going to have to create a loopback filesystem using JFS.
Is leafnode in some place the problem or that large directory?
(dir_index must have been on before the directory creation, else you
need to unmount and e2fsck -Df /dev/foobar to create and populate the
index, ext3 will silently fall back to linear searching if the hash
> > I'm happy to help with patches addressing non-scalability - I'd say do
> > some test runs on a test machine and see if it gets slow with "many
> > messages", let's profile and fix the slow parts.
> well I am going to use leaf node for my first tests of the injection
> code. I will let you know where things break (unfortunately, I'm
> really good at breaking things)
But OTOH, breaking things in interesting ways makes for the spice in
software development, right? ;-)
> > In what context exactly? I'm not quite sure what you mean.
> > If you need to replace the ID or References, you need to that outside
> > leafnode. If you POST, leafnode suggests a Message-ID that the client
> > can either pick up and use, or the client can generate its own, leafnode
> > doesn't care beyond some basic syntax and duplicate checks.
> I'm generating my own ID and references. When I post, I get this error message
> nntplib.NNTPTemporaryError: 441 Invalid header "Message-ID:
> silly.test.1227217025392 at silly.com", article not posted
Ah, I see. Try adding angle brackets, that should do the job. Example:
Message-ID: <silly.test.1227217025392 at silly.com>
This also holds for other headers (notably References:).
More information about the leafnode-list