Re: du

[prev] [thread] [next] [lurker] [Date index for 2005/10/02]

From: Chet Hosey
Subject: Re: du
Date: 00:34 on 02 Oct 2005
On Sat, Oct 01, 2005 at 09:09:44AM -0500, Peter da Silva wrote:
> On Oct 1, 2005, at 8:44 AM, Chet Hosey wrote:
> >  What's rediculous is that we're so entrenched in UNIX tradition that 
> >you have Linux kernel developers refusing to permit non-standard VFS 
> >additions which would provide additional benefits to users of a 
> >certain developmental filesystem, because it's not UNIX-y enough.
> 
> Having experienced some of the horrible results of doing things that 
> aren't "UNIX-y enough", even on ostensibly UNIX-oriented file systems 
> like NFS, I'm inclined to assume that whatever it is you want to do is 
> going to be another source of hate when you get it done.
> 
> The *only* network file system I've used that gave me proper UNIX 
> semantics on remote file systems was OpenNET on Microsoft Xenix. Oh, 
> the irony.

This isn't a network file system -- it's ReiserFS4, which brings its
own hatred but attempts to bring things around towards sanity.

The attempt is to provide filesystem transactions using a new interface.
Some of the kernel people seem to want an API to be developed which could
be used across all filesystems, whether they implement transactions or
not, while the Reiser folks wants to wait before designing a standardized
interface for something that isn't complete yet in their own codebase,
let alone usable with other filesystems.

> >and find has to continue to do stupid tricks in order to feel more 
> >responsive.
> 
> Hacks to make up for poor caching algorithms have nothing to do with 
> API semantics.

Not specifically, but when those whose goal is to make filesystems
more efficient at common operations (and more featureful/flexible) are
sidetracked by the need to develop APIs for functionality that isn't
yet fully described it's still a stumbling block.

Okay, for some more hate:

Since ReiserFS4 tries to merge operations whenever possible to reduce
the actual amount of writing required by a flush to disk, and is
a data-journalling filesystem which sometimes generates "duplicate"
information before freeing old disk contents, it's difficult to predict
how much additional space will be required for a flush.

ReiserFS 4 reserves some filesystem space for its operational needs to
make it impossible to have more data queued to be flushed. This is so
the filesystem doesn't promise a write, then find out that it doesn't
have the free space to make the transaction. This is a good thing.

The problem is that it reserves a fixed percentage of filesystem size. I
believe it's 5%. This means that with a 250 GB volume, 12.5 GB will be
reserved *just in case* a transaction which requires that much space
hits the pipe.

How realistic is it that you'll have 12.5 GB of transaction data waiting
to be flushed to disk? Let me change the size, let me specify at mount
time or creation time or by using twiddly little utilities or by modifying
something in /proc, but why oh why does one of the most responsive and
interesting filesystems insist on wasting such a large amount of size?
Why choose a value which assumes that I might have more RAM than will
fit in a typical IA32 motherboard? I know this stuff isn't being swapped
out -- it's kernel cache data!

The official response thus far is that there are bigger fish to fry (which
is true, development is active and improving). The official attempt to
make everyone feel better is to say that there's some space efficiency
gained by being able to pack small files and tails into chunks. However,
that doesn't begin to justify such brain damage. It's enough to make me
avoid playing with new filesystems by spending time in the real world.

It's made worse by the fact that I like ReiserFS 3 despite its warts,
and as a result I have high hopes for the next revision.
There's stuff above here

Generated at 16:00 on 04 Oct 2005 by mariachi 0.52