Linus Torvalds [Fri, 23 Nov 2007 20:25:38 +0000 (15:25 -0500)]
Linux-2.3.7.. Let's be careful out there..
The new and much improved fully page-cache based filesystem code is now
apparently stable, and works wonderfully well performancewise. We fixed
all known issues with the IO subsystem: it scales well in SMP, and it
avoids unnecessary copies and unnecessary temporary buffers for write-out.
The shared mapping code in particular is much cleaner and also a _lot_
faster.
In short, it's perfect. And we want as many people as possible out there
testing out the new cool code, and bask in the success stories..
HOWEVER. _Just_ in case something goes wrong [ extremely unlikely of
course. Sure. Sue me ], we want to indeminfy ourselves. There just might
be a bug hiding there somewhere, and it might eat your filesystem while
laughing in glee over you being naive and testing new code. So you have
been warned.
In particular, there's some indication that it might have problems on
sparc still (and/or other architectures), possibly due to the ext2fs byte
order cleanups that have also been done in order to reach the
afore-mentioned state of perfection.
I'd be especially interested in people running databases on top of Linux:
Solid server in particular is very fsync-happy, and that's one of the
operations that have been speeded up by orders of magnitude.
Linus Torvalds [Fri, 23 Nov 2007 20:25:32 +0000 (15:25 -0500)]
Linux 2.3.7pre6
Anybody who is interested in FS performance should take a look at the
latest pre-patch of 2.3.7 (only pre-6 and possibly later: do NOT get any
earlier versions. pre-5 still causes file corruption, pre-6 looks good so
far).
Careful, though: I fixed the problem that caused some corruption less than
an hour ago, and while my tests indicate it all works fine, this is a very
fundamental change. The difference to earlier kernels is:
- ext2 (and some other block device filesystems that have been taught
about it) uses write-through from the page cache instead of having a
separate buffer cache and the page cache to maintain dirty state. This
means much less memory pressure in certain situations, and it also
means that we can avoid unnecessary copies.
- the page cache has been threaded, so on SMP you can actually get
noticeable speedups from processes that do concurrent file accesses.
- lower-latency read paths, especially the cached case.
Both of these are big, and fundamental changes. So don't mistake me when I
say it is experimental: Ingo, David and I have been spending the last
weeks (especialy Ingo, who deserves a _lot_ of credit for this all: I
designed much of it, but Ingo made it a reality. Thanks Ingo) on making it
do the right thing and be stable, but if you worry about not having
backups you might not want to play with it even so. It took us this long
just to make it work reliably enough that we can't find any obvious
problems..
The interesting areas are things like
- writes to shared mappings now go blindingly fast. We're talking mondo
cleanups here. We used to do really badly on this, now we do really
well.
- does bdflush still do the right thing? There may be a _lot_ of tweaking
to do to get everything working at full capacity.
- can people confirm that it is stable for everybody?
- if anybody has 8-way machines etc, scalability is interesting. It
should scale to 8-way no problem. We used to scale to 1-way, barely.
Numbers?
- fsync(). It doesn't work right now, but it should be easy to make it
work well on big files etc - something we've never been able to do
before (we used to lack the indexing from file to dirty blocks: now we
have access to that quite automatically thanks to having the
inode->page index in place, and the dirty blocks are right there)
and I'd really appreciate comments from people, as long as people are
aware that it _looks_ stable but we don't guarantee anything at this
point.
Linus Torvalds [Fri, 23 Nov 2007 20:25:25 +0000 (15:25 -0500)]
Linux 2.3.7pre1
I'd like to point out that the current pre-2.3.7 series is fairly
experimental. As amply demonstrated by the filename (the "dangerous" part
in the filename hopefully made some people go "Hmm..").
We're working on re-architecting (or rather, cleaning up so that it works
like it really was supposed to) the page cache writing, and as a result a
number of filesystems are probably going to be broken for a while unless
we get people jumping in to help.
Right now 2.3.7-1 (aka "dangerous") is not stable even with ext2, in that
swapping doesn't work. Ingo just sent me patches to fix that, and I'm
hoping to remove the "dangerous" part from 2.3.7-2, but even then a number
of filesystems will be broken.
We _may_ end up just re-introducing the "update_vm_cache()" code for
filesystems that really don't need the added performance, but it would
actually be preferable if people really wanted to make them perform well
with the new direct write-through cache code.
Linus Torvalds [Fri, 23 Nov 2007 20:25:11 +0000 (15:25 -0500)]
pre-2.3.4..
There's a pre-2.3.4-1 out there in "testing" on ftp.kernel.org, which has
the new scalable network code (well, the first cut of it, anyway). It also
updates ISDN and PPC to newer versions. Please test it out and give
feedback..
Linus Torvalds [Fri, 23 Nov 2007 20:25:09 +0000 (15:25 -0500)]
Linux-2.3.3 and a short hiatus..
There's a Linux-2.3.3 out there on ftp.kernel.org, this one hopefully
fixes pretty much all the waitqueue changes (and I'll disable waitqueue
debugging in 2.3.4 unless something comes up).
And yes, before anybody tells me, I know I forgot to increment the version
number. So "uname" is goign to report 2.3.2 unless you fix that by hand.
I'm also leaving for a very quick trip to Finland in another two hours, so
don't bother emailing me - please discuss isues on the kernel list, and
I'll catch up when I get back on Friday (yes, I'll spen as much time in
airplanes as I do on the ground - fun, fun).
Have fun,
Linus Torvalds [Fri, 23 Nov 2007 20:25:02 +0000 (15:25 -0500)]
Linux 2.3.1pre3
As to 2.3.x, we're beginning with a long overdue waitqueue cleanup, which
means that a lot of small details need to get fixed in a variety of files.
A working pre-patch of this is to be found as pre-patch-2.3.1-3, but not
all drivers have been fixed - and help is appreciated (even drivers that
_have_ been fixed have not necessarily actually been tested due to lack of
hardware).
Linus Torvalds [Fri, 23 Nov 2007 20:18:57 +0000 (15:18 -0500)]
Linux 2.2.8
Most of 2.2.8 by far is just architecture updates: arm, ppc and m68k stand
out as having been pretty much synchronized to their respective devel
trees, but there are some fixes to alpha and x86 too.
The one major fix in 2.2.8 is the SMP fix for disable_irq(), courtesy of
Andrea Arcangeli (I disagreed in details and did it differently in the
end, but all the heavy lifting was done by Andrea). This is the thing that
caused silenth deaths for some people with certain network adapters (3c509
and 8390-based cards in particular: the latter covers ne2000 clones which
are fairly common).
There are lots of smaller things (driver updates, filesystem cleanups and
some networking fixes), but the SMP irq thing is the one to kill for if
you happened to have any of the affected cards.
Linus Torvalds [Fri, 23 Nov 2007 20:18:41 +0000 (15:18 -0500)]
There's a pre-3 patch on ftp.kernel.org in the kernel/testing directory,
and I'd really like people to give it a good testing: especially if you've
seen slow network connections to some clients (ie Windows). David worked
in the compatibility patches to work around some of the Windows TCP stack
"features" (and Apple too, for that matter), and we want to get this well
tested. It's all fairly straightforward, but let's be careful out there..
Linus Torvalds [Fri, 23 Nov 2007 20:18:31 +0000 (15:18 -0500)]
Linux 2.2.5 - and a vacation
I made Linux-2.2.5 yesterday (as some people already have noticed: due to
popular demand I try to delay the announcement for some time in order to
let the thing percolate to mirror sites, in case anybody wondered).
The 2.2.5 release is meant to be a final cleanup release before I leave
for a two-week vacation. So please take these release notes to also mean
that it is probably a good idea to hold off emailing me stuff directly,
unless it is a major bug that you really think I should look at
immediately. I would suggest people discuss problems on the mailing list
and on the newsgroups, where other competent people are, rather than
expecting me to do much about it.
Also, note that there have been various indications that egcs potentially
miscompiles the kernel, or at least makes some problems worse. We don't
know whether that is due to one or more kernel bugs, compiler problems, or
just combinations of "features" in both. I would suggest that if you have
problems you at least verify whether the problems still exist with
gcc-2.7.2.
That said, I bet that both the kernel people and the egcs people would be
really happy the more people look into this - if somebody feels motivated
enough and sees problems with egcs, it would be extremely powerful to try
to pinpoint the particular file that seems to bring on the problems. I'm
afraid it needs a known failure mode and lots of legwork to find out what
triggers it, though.
- compiles with accounting.
- add support for Microgate SyncLink and Synchronous HDLC
- stallion driver update
- alpha EV6 and SMP fix for bootup with newer compilers
- ptrace fix for sparc/i386
- small sparc updates
- floppy driver could oops at bootup under certain setups
- random driver updates (bw-qcam, sound driver error codes, etc oneliners)
- FIOASYNC ioctl fix
- network locking fixes
- SMP "struct user" and signal sending fixes
Linus Torvalds [Fri, 23 Nov 2007 20:18:26 +0000 (15:18 -0500)]
Linux 2.2.4
As of 2.2.4, I should be synchronized with the Sparc[64] and PPC ports,
which is the major reason why the patch is pretty huge. Apart from the
architecture synchronizations, 2.2.4 does:
- dumping core over NFS could do bad things. Core-dumping cleaned up and
fixed.
- various small TCP/IP buglets fixed. Linux got confused by hosts that
didn't report any mss, and had problems with zero-sized fragments, etc.