[AUUG-Talk]: XFS Experiences?
nathanr at nathanr.net
Wed Jan 5 08:20:23 EST 2005
On 04/01/2005, at 2:26 AM, David J N Begley wrote:
> Collective brains trust of AUUG - does anyone have any experience with
> XFS (file system) on either Linux or IRIX boxen? Whilst I am more
> in negative (or even just mildly annoying) experiences, if anyone feels
> sufficiently motivated to sing XFS' praises I wouldn't mind hearing
> those too.
Ok. We've been running XFS in production on an internal IMAP mail
server for about 2-3 years now (since SGI shipped the XFS patches, well
before it was in the mainline kernel). At the time the decision was
1. XFS performance was better than any other filesystem in almost every
2. JFS (IBM) corrupts itself quite regularly (I retested this about 12
months ago and was suprised to see it hadn't improved).
3. Support for ACLs (important when we ran it on a Samba server, and at
the time no other filesystem supported ACLs - now others do)
4. Stability. We've never had a problem.
(BTW - we've had other servers running XFS, but the IMAP server seems
to map closest to your scenario).
> I am in the process of preparing an I/O performance simulation in
> order to
> pitch ext3fs, ReiserFS and XFS against one another for "real world"
> splutter, choke) Squid access patterns; on the thorny issue of
> however, I am left (at present) only examining documented behaviours
> design priorities for each file system.
Are you talking about reiserfs v3 or v4? v4 I would believe might come
somewhere close to XFS (its only been out for a handful of months), but
v3 I'd be very suprised. v4 is somewhat of a political football in the
Linux community - I won't rehash it here, but it'll be a long road into
mainstream kernel acceptance for v4, and may have a fair few features
stripped before Linus will accept it. On the other hand, XFS is in both
2.4 and 2.6 kernels.
Which distribution are you running? It can be slightly painful to
install using XFS because the Red Hat (either Fedora or RHEL) installer
doesn't have XFS on its GUI partitioner, you need third party CDs for
Debian, SuSE Linux Enterprise Server supports it, but a bug in their
kernel they shipped with SLES9 means that has a kernel "oops" as soon
as you try to mount an XFS partition.
> Before anyone asks, RAID mirroring (or similar) approaches are not
> (only RAID0/striping). Whilst the hardware will technically be on a
> I have to assume that the Squid server will not have any knowledge of
> impending loss of power (internal politics - long boring story).
Ok, in other news, if you are running a softraid setup (I realise you
aren't, but I want to get this on the record for others that read this,
particularly in mailing list archives), you need to have your /boot as
ext2 or ext3. Then your root mount and every other mount can be
whatever you like (in our case XFS).
If power outages are a possibility, then I'd recommend you use a
filesystem that has been around for a while, and fairly proven (ie. not
reiser4). XFS did the job for us, as we had to go through this a few
years ago before we started buying UPSs. But that was a few years ago,
and I'm unsure how hard the server was being hit at the time.
> Of course, search the Web and everyone's got a "nightmare" story
> regarding any
> of these three file systems (though fewer for ext3fs, admittedly);
> the "ordered journalling" option of ext3fs and newer ReiserFS
> reduces the window (between data and metadata updates) for failures to
> havoc. Alas, XFS' behaviour is more akin to ext3fs' "writeback
I think you are talking reiser4, which as I say, is likely to be
outside the mainstream kernel for a while. Read the stories on reiser4
inclusion from a few months back on http://www.kerneltrap.org/ (easier
than reading LKML). If you're talking reiser3, then I'd be particularly
interested in the results of your benchmarking.
> So... has anyone used XFS (IRIX or Linux since this appears to be an
> of XFS' design that is not specific to Linux) on a "busy" (I/O) system
> has accidentally lost power? If so, what are your experiences with
> data loss
> (either individual files, parts of files or entire file systems)?
Hmm. Well, we've never lost data, but I haven't run an XFS system
without a UPS for a couple of years now. But from a couple of years
ago, we didn't lose data. Couldn't tell you what I/O load the machine
was under at the instant the power went off two years ago either. ;-)
What I can tell you is that our XFS IMAP server is used by 30+ people
all day. IMAP is a fairly chatty protocol that is annoyingly expensive
on I/O (for a reasonably simple job - browsing email), and might be
comparable on I/O to your squid server. A fast reliable filesystem
(XFS) combined with U160 SCSI disks (softraid1) and an IMAP server
supporting Maildirs (courier-imapd) was the answer, and is humming
along quite well. FWIW, I've had both Debian and Red Hat based servers
running XFS, but said IMAP server is Debian.
Hope this helps.
More information about the Talk