LinuxWorld Convention & Expo Summer 2001
A few words about the new look if you've seem some of my
prior reports of Linux Events, this report
The reasons are as follows:
- I finally retired my old Ricoh 4200 digital camera (for sale,
Email me for
details ), and my new camera (Canon S10) takes 1600x1200 pictures and has
enough memory that I use that size by default. The problem is that I used
to resize my pictures so that they weren't too big and could be viewed by
anyone on a normal sized screen (my desktop at home is 2048x1536, at at
work it's 3600x1440 (dualhead), a bit beyond "normal"), however this doesn't
work anymore: I didn't want to sacrifice the picture size for the lowest
common denominator, so it became clear that a viewer with multiple sizes was
I first started to use IDS, but it was way
too slow on my machines and would kill the P120-64Megs of Ram that you're
getting this page from. Thankfully
Raphael Moll, gave me a
pre-release of his excellent RIG, which re-implements IDS in PHP and C++,
and is noticeably faster. Without this, I would not have been able to
make the picture library available
to more than 2 or 3 concurrent users
- After having read a few
reports from the excellent folks at
Linux Weekly News, I realized that the one
long page format isn't that bad afterall, especially if there isn't loads
of things to say for each portion of the event.
- The one page format is faster to write for me
Writing these pages isn't my full time job. Actually, it's not my job at
all . I unfortunately can't spend the 30 or more hours I've spent to
write some of my past reports, and half the time, when I was done, it was 7
to 10 days after the event, when it was "old and boring news" in web time.
- Having recently come back from the
Ottawa Linux Showcase, I didn't
have the time to spend 5 days at the show, but that said, it didn't seem much
worth it either, so that was ok.
Linuxworld this year
I haven't been going to the linuxworlds in New York since they've typically been
too close to or conflicted with other events (like the excellent
linux.conf.au in Autralia this year)
Last year, if you remember, Linuxworld was
still in San Jose in the convention center. As it was a great success, it
grew beyond the available booth space, some talks had to be given
across the street for space reasons and the president of IDG
announced that this year, we'd
have to go to Moscone in San Francisco.
What he didn't know (and neither did we) is that we'd have an economy
downturn, that many startups would close in the following year and that overall,
companies didn't have that much money or many money at all to spend and go to
linuxworld (I addition to the free booth space at the .org pavilion, I heard
that at least one company only went there because they were given the booth
space for free.
As a result, we only used a small portion of Moscone, making the whole use of
it a waste, especially because that portion of Moscone didn't have conference
rooms that were big enough and we had to go to the Marriott hotel, a few minutes
away, to go to any conference or keynote, and it got real old real fast (I
think, the second day, I walked back and forth a total of 6 times and it
made hoping to the conference floor for 10mn between two conferences not
I obviously do not have access to IDG's financials, but my guess is that they
may not have broken even, or if they did it was probably not by that much.
Of course, it was a bit sad to see this and compare it to last year's show, but
to most visitors, it probably still looked like a decent show. Also, this isn't
really linux specific, most tradeshows have been hit similarly because companies
can't spend the unreasonable amounts of money they used to spend in the past.
Since the first two days of Linuwrold were really a couple of days of
tutorials and the conferences and showfloor really opened on tuesday, I'll
refer to tuesday as the first day and wednesday the second. Thursday was the
last day, and developer conferences ended on wednesday, so it didn't really
seem worth going.
Day 1 Keynote: Linux Throughout the Enterprise - Ready or Not?
I skipped the first days of the show (sunday and monday), which only
had tutorials, and after only a few hours of sleep after finishing my
linux10 report, I got up at some insane hour (06:25),
to catch the Caltrain to San Francisco (it's actually slower than a car,
but if you factor in parking and aggravation in traffic jams, it's a livable
alternative) and catch the first keynote.
Shane Robinson, CTO of Compaq, gave a keynote on the state of affairs with the
internet and linux, and gave a little insight to what Compaq was doing.
He explained that we had had a decade of technological advancement and it was a
decade of tremendous growth for the internet
A few numbers and points he gave were:
He then gave some examples of linux uses and mentioned Korean Air, which uses
linux for their flight scheduling and ticket managemen and went on to list
the current challenges he saw with linux:
- 7 Trillion Emails sent in the US.
- At the apogee of napster, 1.4B songs downloaded in one month, 8.5% PCs
tapped in and napster was responsible for 5% net traffic in the US.
- Rebirth of open source: projects like sendmail, linux, or apache, that
then became extremely widespread (rebirth is compared to the sixties and
seventies when open source was quite common)
- Because of the new demands from internet users, there have been big
challenges on the server side in reliability expectations, hard drive
storage needs and response time.
- Most enterprises are shooting for less than 5mn of downtime per year.
- Security breaches cost $15B annually and they need to be addressed
in near real time.
Shane then explained that Compaq has contributed to making linux more ready for
big enterprises in the following ways:
- Providing better support and services for big enterprise customers
- helping with integration and migration
- adopting and adhering to new standards like hotplug PCI, which is one of the
things compaq works on
- scalability to 4 and more CPUs, which has been addressed to some extent
by the 2.4 kernel (although, some work is still needed for good performance
beyond 8 CPUs). Process limits and threads have similarly been worked on.
- broader hardware support (like the hotplug PCI work)
- applications and development software
- management software
- enterprise applications
- other random work like mobile IPV6 on the Ipaq
He gave an impressive demo of an Ipaq running linux with a camera and a
wireless LAN card running and roaming with mobile IPv6 between two wireless
networks while broadcasting video back to the server.
Then, we had a demo of a Compaq Proliant server running oracle application
clusters, and adding/removing a node from the running cluster while it is
running the database.
Conference: 2.4 Kernel Scalability Features
Larry Woodman, technical director of kernel engineering at mission critical
linux gave a talk on 2.4 kernel scalability features.
The improvements on he listed were:
- CPU scalability:
- less use of the global lock_kernel
- more subsystem based spinlocks
- spinlocks embedded in data structures
- semaphores used to serialize address space access
- more of a spinlock hierarchy established
Lock granularity has a tradeoff though: the more fined grained you get, the
less lock contention you have, but the more overhead you get because of all
the locks you need to go through of any single operation.
The hard part is to find the sweet spot in the number and location of locks,
and obviously this depends on how many CPUs you have in your target
platform. Right now, for Larry, 4 CPUs is the sweet spot in linux 2.4
- Multithreading improvements:
- multiple threads can access address space specific data structures
- Single mmap_sem semaphore was replaced with multiple reader/single
writer semaphore (down_read/down_write operations)
- A reader lock is now acquired for reading per address space structures
- exclusive write lock is acquired when altering per address space
- 32bit UIDs and GIDs
- 64bit virtual address space
- architectural limit of the virtual address space was expanded to 64 bits
on alpha and IA64
- IA64 implements 51 bits (16 disjoins 47 bit regions)
- alpha implements 43 bits (2 disjoint 42 bit regions)
- future alpha is expanted do 48 bits (2 disjoint 47 bit regions)
- Unified filesystem cache
- Single pagecache was unified from previous pagecache read/buffermem
- eliminates copying buffers from buffermem to pagecache on file read ops
- reduces memory consumption by eliminating double buffered copies of FS
- eliminates overhead of searching two levels of data cache
- Distribution of interrupts
- more hardware interrupt service routines can be processed simulteanously
on multiple CPUs (irq_attempt())
- Software interrupts (softIRQs) can be processed simultaneously on all
CPUs with hardware interrupts enabled
- More SMP spin locks are maintained with device specific data structures
- Increased number of threads and tasks
- maximum allowable number of tasks/address spaces as well as threads
- Configuration of maximum number of threas is runtime tunable via /proc
filesystem (no more NR_TASKS)
- Scheduler optimizations minimize overhead of context switching between
- IA64 Port and architecture optimizations
- ia64 specific TLB optimizations
- large rotating register file
- ia64 SMP specifics
- ia64 IO specifics
- 64 bit virtual address spac
- NUMA support under development
- Alpha Port and architecture optimizations
- 64 bit virtual address space (EV67 is 43 bits: half user, half kernel;
EV68 supports 48 bits: also half user, half kernel)
- 2TB (41 bits) physical address limit for EV67
- 64TB (46 bits) physical address limit for EV68
- accurate SMP compatible processor time optimizations
- NUMA support under development
- Bigmem for IA32
- 1GB physical memory limitation in the linux kernel
- 4GB physical meory limitation for 32bit systems
- 4GB physical memory optimizations in the linux kernel
- 64GB physical memory using PAE on IA32
- Disks and filesystems
- File size increase
- File offset was increased from 31 bits to 44 bits in the VFS layer
- Filesystem size and file size are increased to 16TB
- buffermem and pagecache were merged in 2.4 to unify fs cache
- Raw IO
- Journaling filesystems (ext3, reiserfs, xfs, jfs)
- logical volume manager included in the main kernell
- Working NFS V3 (udp only in the stock kernel)
- Other miscellaneous
- fast memory transferts for Pentium, AMD, Cyrix, Winchip and a few others
- network rewrite to optimal performance and better use of SMP systems
- kernel based http daemon (khttpd)
- Iptables/Netfilter replacement for ipchains
- Tunable memory parameters (/proc/sys/vm/)
Conference: JFS for linux
Steve Best, senior software engineer at IBM gave us an update on JFS
The goal is to merge JFS in the official 2.5.x tree when it opens up and
backport it in 2.4, which would hopefully make it available in linux 2.4 in a
JFS is proven FS technology (it's been in AIX for 10 years), and is been rebuilt
from the ground up in 1995. Because of the shared code base, it is capable of
mounting partitions from OS/2
The features of JFS are as follows:
- Support for file sizes up to 512 terabytes (0.5K blocks) to 4 petabytes (4K
- Max aggreagate is from 4PB to 32PB and is currently limited by linux I/O
structures not being 64 bits yet.
- Journaling of metadata only (asynchronous for added speed)
- Filenames in unicode (UTF16)
- B+Tree with extents (variable-length sequence of aggregate blocks allocated
as one unit)
- Built to scale: in memory and on disk data structures are designed to be
scalable, and code is designed to be run on SMP systems with 8 or more CPUs
- Dynamic Inode allocation
- Support for symlinks in inodes
- Directories can store up to 8 entries directly inside the inode, and then
B+trees keyed on name are used
- Support for both sparse and dense files (chosen at mkfs time)
- Ability to increase filesystem size and option to integrate with LVM
Evolution of the code and current status:
- Announced and shipped in 2000/02/02 at Linuxworld in NYC
- 41 code drops so far
- JFS patch and tar files supports 2.2.14 onward and 2.4.x from a single
- Completely independent of any kernel changes
- Beta 1 in 2000/12
- Release 1.0.0 in 2001/06 (current is 1.0.3)
Steve has good hopes that it will be included in distributions soon as it's
easy to add (it doesn't require a massive kernel patch like XFS does)
Feature Presentation: Golden Penguin Bowl
Like previous years, Nick
Petreley organized the penguin bowl
I didn't get to take many pictures or write down many of the questions since I
was a judge for the event, and I couldn't be on both sides of the stage, nor
could I really take pictures and take notes
The short version is that the Nerds won and that Nick has to debug his gameshow
If you want a few of the questions, here are some (I'll let you research the
- What is the number of Neo's apartement in the Matrix?
- How about the room Trinity was in?
- In the Alien series, what was Repley's first name?
- And for 10,000 points: what is CPIP in RFC 1149?
Video, Graphics, DVR/PVR -> Scaling linux to 1024 CPUs the easy way
I was then eagerly looking forward the PVR on linux talk but the speaker never
showed up, so after waiting for about half an hour, I gave up and happened to
recognize Larry McVoy's voice coming from an adjacent room, so I went to join
his talk in progress, because even though I didn't know what the topic was
about, Larry is always most entertaining to listen to
(for those who don't know him, in addition to being entertaining, and not afraid
to voice his opinions, he is actually quite knowledgeable on many topics).
It turns out he was giving the "Scaling linux to 1024 CPUs the easy way" talk,
which from anyone else might have been boring, but not with him as a speaker
He explained how companies have in the past modified their kernels to have very
granular locks were all wrong (he's worked at both Sun and SGI)
Not only is it hard to make a kernel very modular, but as you add more locks
to remove the contention on each individual lock, you make the kernel more
bloated and complex, and especially you slow down the common case as those locks
have to be taken every single time a function is called (also, on a hardware
level, on most architectures, taking a lock is expensive and causes bus
Larry's solution to this problem is not to try to scale an OS to many CPUs, but
to run multiple instances of the OS on the hardware and setup some ways for them
to communicate (a principle somewhat similar to RT Linux where linux runs as the
lowest priority task under an RTOS, except that in this case you'd have several
That was a definitely a novel approach, and Larry is looking for coders if
you're interested in helping.
At first, it was almost hard to believe: a linuxworld night without a party,
but really, it made sense: many companies didn't even have the money to have a
presence at all at the show, and the ones that did come didn't have the money
to blow away on big parties like the two previous years.
There were a one or two private parties (more small receptions I guess), but
easy to get to (most of the people "in the know" that I talked to ended up doing
nothing that evening)
Day 2 Keynote: The New New Old War
Rob Scheschareg introduced Professor Lawrence Lessig, Professor of Law at
He gave a talk about the lawyer that he "produces" and what they've been doing
Professor Lessig first explained the differences between free and controlled
media, as far as the content is concerned and the physical layer is concerned.
For the internet, the physical layer is controlled, but the content is either
free or controlled.
By regulation, phone companies are not allowed to discriminate on the content
that travels on phone lines.
However, on cable companies, there is no such law and they are allowed to decide
what travels on their cables, discriminate traffic, favor some streams or
protocols over other ones.
He then obviously mentioned the DMCA and the fact that content providers and
the government are now able to scare people off from writing code that doesn't
fit with their agenda (the Dmitry
case was an obvious example)
He explained that they have the upper hand now, and either ignoring them because
we have better things to do, or in disgust, or trying to work around it by
building free networks or products that can't be stopped, just isn't the right
way to go. We need to fight the actual laws: it's not alright that
guns can be sold whether they are used for defence or used to kill a policeman
but that one can be put in jail for 25 years for writing code that is a threat
to companies that are behind the DMCA
Napster had to garantee that 100% of the content being traded through their
system didn't violate copyrights and had to shut down. Xerox doesn't have to
garantee that their copy machines aren't used for copyright violation.
Professor Lessig concluded that we have to do something about the situation,
either by talking to politicians, by marching, or at the very least, by
supporting people who are doing so by supporting the
IDG Linus Torvalds Community award
The money this year goes to the free standards group for all the work they've
Scott Mcneil thanked a few people, including Daniel Quinlan for working on bases
for the LSB for close to 9 years, Alan Cox (for the logo) and Bruce Perens (who
started the LSB)
We should also thank IDG for giving money back to the community with the two
money awards they gave.
Advances in video capture and digital video under linux
Frank Hum gave an overview of the video support available under linux, from
tuner cards with BTTV chipsets to video formats with encoding like DivX.
For digital video, he recommends recent CPUs (1GHz or better),
and you also get good use of the SSE or 3Dnow instructions.
For video editing, you have mainactor for $100 or Broadcast 2000 which is open
He also mentionned a program called
VCR lets you use a linux machine
as a PVR and encode programs to Dvix with avifile (poor man's Tivo, except that
it ends up costing more than a Tivo
Feature Presentation: The state of Open Source
Larry introduced the speakers of his panel, the state of open source:
Linus Torvalds, Brian Behlendorf, Jeremy Allison and Dirk Honel
Some of the questions and points they addressed were:
After the panel, the OSDL enterprise achievement award went to Heinz Mauelshagen
for his work with LVM (logical volume manager).
- What was the biggest thing in open source that happened in the last year?
- Linus has been impressed with KDE, which he switched to this past year after
it went past the eye candy stage to a stage where you have office
Brian has been impressed with how big companies, including microsoft
have responded to open source
For Jeremy, the 2.4 kernel actually is the most important thing we've had
in the last year, bring linux to the enterprise level.
Dirk does see the applications as the important part. The fact that LSB 1.0
was released is the most important part for him so that we can start solving
the incompatibilities between distributions.
- How about microsoft calling the GPL a threat to the capitalism?
- Jeremy answered: "so what? and the problem is?"
Dirk thinks that the GPL being a threat is actually a good thing as it
drives competitors to innovate.
Linus explained that for him Linux got where it is thanks to the GPL license
as it keeps people who use it honest if they contribute and use it
commercially (with a BSD style license, you can use the code and you don't
have to contribute back)
- How do you look at new software that needs to be created?
- There were two views on the matter: you write whatever software you need
as you need it (Linus and Jeremy) but Dirk thinks that we should look at
the needs of other people and try to respond to it.
- When does open source make sense?
- Who knows who wrote the paperclip in MS office? If it where open source,
you could go to his house and shoot him
Opinions differed a bit, some thought that everything should be open source
within 50 years, but Dirk and Linus believed that highly specificialized
applications (like a software for a medical company) but that comodity
software (office) definitely benefits from being OSS.
- How about single signons?
- The whole panel was worried about that because it would give microsoft too
much control and information that it really shouldn't have. However, they
are once again going to leverage their monopoly on the desktop to take
over another market and put infrastructure in MSIE 6.0 to allow web sites
to make all commercial transactions through microsoft's servers
Linus however thinks that microsoft will not be able to tax every single
transaction over the web because governments would step in to do the taxing
Well, what is there to say, it was smaller, not near as exiting as previous
years, but there were still a few cool things to see. IBM had probably the most
impressive presence throughout the show. If you still don't believe that they
are committed to linux, believe it.
You want to see all the booth pictures? Go
You have a clue? You need a job?
Transgaming makes your windows games run under wine
You really want to support their efforts and subscribe to their service (I will, even though I don't even play games)
Obviously, I'm biased since VA Linux designed this server (and sold it to
Appro when it got out of the hardware business)
but this it's really a sweet
box: Dual Athlon 1.2G, 4 hot swappable SCA drives + CD Rom all in a 1U
You can touch, but you can't take home
Too bad the trip you can win is only for two
After working some time in the OSDN booth (eh, I had to earn my long sleeved
shirt ), I went to sample the few parties that were happening (most
private): Linux Journal had a hostpitality suite, then VA had a private
presentation on sourceforge onsite and dinner, HP had pre-movie party with
snacks, and then came what I had been waiting for for a while: J.T.S. Moore's
Revolution OS movie showing at the Metreon.
I won't spoil it for you, but basically it shows the evolution of linux from
when it started, through windows refund day, linuxworlds, Red Hat and VA's IPOs.
After the movie, we discussed with the director, J.T.S. Moore, and Bruce
Perens, featured in the movie
This event unfortunately conflicted with EFF's party a few blocks away where
Dmitry spoke, as well as Stallman and a representative from the EFF, but I
got to see most of the crowd just before the speeches were over.
Yes, we were having fun
Come on, buy the few linux10 shirts we have left
'cause the money is going to the EFF
That's all folks
Overall the show was still pretty good, and outside of the location, which was
less than ideal, IDG did again a good job.
Thanks for coming, and see you next year.
A big thank you goes to Raphael
Moll for not only giving me a prerelease of RIG (Ralf's Image Gallery), and
fixing it up to the last minute so that I could release this report.
With IDS, I just couldn't have made all the pictures available, but with RIG,
I can, they're all here (note that
if you browse anything else than the default size pictures or the 640 pixel
wide resized pictures, they're going to have to be generated on the fly, and
it could be a tad slower)
And here's the now usual little footer note:
If you'd like to use some pictures or text, please Email me first.
'Till next time
Link to Home Page
2001/09/05 (04:26): Version 1.0. Enjoy!
2001/09/05 (11:11): Version 1.1. Added a few words about IBM
2001/09/05 (13:45): Version 1.2. IPCP is really CPIP. Thanks Ron