Marc's Public Blog


All | Aquariums | Arduino | Btrfs | Cars | Cats | Clubbing | Dining | Diving | Electronics | Exercising | Flying | Hiking | Linux | Linuxha | Museums | Public | Rc | Sciencemuseums | Snow | Solar | Trips




More pages: March 2018 February 2018 January 2018 December 2017 November 2017 October 2017 September 2017 August 2017 July 2017 June 2017 May 2017 April 2017 March 2017 February 2017 January 2017 December 2016 November 2016 October 2016 September 2016 August 2016 July 2016 June 2016 May 2016 April 2016 March 2016 February 2016 January 2016 December 2015 November 2015 October 2015 September 2015 August 2015 July 2015 June 2015 May 2015 April 2015 March 2015 February 2015 January 2015 December 2014 November 2014 October 2014 September 2014 August 2014 July 2014 June 2014 May 2014 April 2014 March 2014 February 2014 January 2014 December 2013 November 2013 October 2013 September 2013 August 2013 July 2013 June 2013 May 2013 April 2013 March 2013 February 2013 January 2013 December 2012 November 2012 October 2012 September 2012 August 2012 July 2012 June 2012 May 2012 April 2012 March 2012 February 2012 January 2012 December 2011 November 2011 October 2011 September 2011 August 2011 July 2011 June 2011 May 2011 April 2011 March 2011 February 2011 January 2011 December 2010 November 2010 October 2010 September 2010 August 2010 July 2010 June 2010 May 2010 April 2010 March 2010 February 2010 January 2010 December 2009 November 2009 October 2009 September 2009 August 2009 July 2009 June 2009 May 2009 April 2009 March 2009 February 2009 January 2009 December 2008 November 2008 October 2008 September 2008 August 2008 July 2008 June 2008 May 2008 April 2008 March 2008 February 2008 January 2008 December 2007 November 2007 October 2007 September 2007 August 2007 July 2007 June 2007 May 2007 April 2007 March 2007 February 2007 January 2007 December 2006 November 2006 October 2006 September 2006 August 2006 July 2006 June 2006 May 2006 April 2006 March 2006 February 2006 January 2006 December 2005 November 2005 October 2005 September 2005 August 2005 July 2005 June 2005 May 2005 April 2005 March 2005 February 2005 January 2005 December 2004 November 2004 October 2004 September 2004 August 2004 July 2004 June 2004 May 2004 April 2004 March 2004 February 2004 January 2004 October 2003 August 2003 July 2003 May 2003 April 2003 March 2003 January 2003 November 2002 October 2002 July 2002 May 2002 April 2002 March 2002 February 2002 November 2001 October 2001 September 2001 August 2001 July 2001 June 2001 May 2001 April 2001 March 2001 February 2001 January 2001 December 2000 November 2000 October 2000 September 2000 August 2000 July 2000 June 2000 April 1999 March 1999 September 1997 August 1997 July 1996 September 1993 July 1991 December 1988 December 1985 January 1980



2012/08/29 Linuxcon 2012 in San Diego
π 2012-08-29 00:00 in Linux
I was invited this year again to speak at Linuxcon 2012. This year, I gave a talk about sleep monitoring with arduino which went quite well.

The conf had plenty of talks to see, and great parties. Just a few pictures below to give a taste, and I also have a lot of pictures of my visiting of San Diego.

A few pictures from the keynotes:



kernel panel
kernel panel


A few slides from talks:



And we had some nice parties. The first night was a rooftop party:





The second evening, we had a party on an almost-island, including a small cruise to get there:







The 3rd evening, we had a beach Hawaiian party on Coronado Island, which was quite good:

the long bridge that gets you there
the long bridge that gets you there






A few random picts from the conf:




It was a good conf and a great time.

See more images for Linuxcon 2012 in San Diego
2012/08/28 San Diego's Midway Aircraft Carrier
π 2012-08-28 00:00 in Flying
Since I was on the San Diego Waterfront, it was obvious that I would have to visit the Midway Aircraft Carrier.

It's the best aircraft carrier I've ever visited. They show you a lot more than the average boat, you see 4 levels of decks, and the inside of the city on the sea.







you can go to church
you can go to church

multiple dining rooms
multiple dining rooms

multiple medical facilities
multiple medical facilities

ER room
ER room

dentist
dentist


After seeing le down levels I had never seen before, it was time to go on the main deck:










San Diego view from the aircraft carrier
San Diego view from the aircraft carrier

I had a bit fewer than 3 hours, which wasn't long enough to see it all without rushing. It's definitely the best aircraft carrier I've visited.

2012/08/23 3rd 2012 Droid n Friends at Thunderhill
π 2012-08-23 00:00 in Cars
It was about time for another Droid n Friends day. Unfortunately Attila wasn't able to join us due to having injured his foot, but the rest of us still went to have fun :)

Flying there was "interesting" since the plane I was about to take failed preflight and I had to switch planes at the last minute. Interestingly, by then the overcast had actually gotten worse and the hole I was going to aim for had closed, so I just left IFR, which was pretty painless.



I got there a bit late, 08:35, but just in time to get ready for my session at 09:00 and go.


Dave got to drive one of Andy's cars. I'm told he didn't hate it too much :)
Dave got to drive one of Andy's cars. I'm told he didn't hate it too much :)




Stephen taking his NSX for the first time
Stephen taking his NSX for the first time

Gassing up before the next session.
Gassing up before the next session.

The flight home was uneventful:


haha, that was a good one
haha, that was a good one


Here's a session I did:

And for the last session, I put my GoPro in the McLaren that Dave was driving to give rides:

2012/08/16 Dan Chan the Magic Man
π 2012-08-16 00:00 in Public
Jennifer got tickets to see Dan Chan the Magic Man, and Kat the Acrobat.

Since I grew up seeing magic shows, I can be a tough public, and I thought they did a really good job, their show was enjoyable.

They have a new deal where they work with restaurants to perform there while people get dinner, and that worked out pretty well. If you're thinking about going, do get tickets from him directly though they're cheaper than the online offers and he gets to keep the money that would otherwise be lost to those ticket sites.

Food wasn't cheap, but good :)
Food wasn't cheap, but good :)

The balloon trick was fun
The balloon trick was fun

Dan also did close up before and after the show, in addition to his show
Dan also did close up before and after the show, in addition to his show

His web site is http://www.danchanmagic.com/ if you want to go see him or hire him.

See more images for Dan Chan the Magic Man
2012/08/15 Quick Visit From my Dad and Patricia
π 2012-08-15 00:00 in Family
My dad came over for a few days to visit. Saturday we went to San Francisco, and Sunday we went for a tour of Google, and my dad wanted to try our nearby minigolf after the pictures I posted some time past, so we did.

Not that I'm supposed to be good at golfing since I don't golf, but I won again ;)






2012/08/15 The tale of SSDs: Crucial C300 early Death, Samsung 830 extreme random IO slowness, and settling with OCZ Vertex 4
π 2012-08-15 00:00 in Linux
For indexing purposes, I didn't change the title, but I don't recommend the OCZ Vertex 4 anymore, it does lock up under certain workloads. My new drive is the Samsung 840 Evo which performs fine where the 830 absolutely did not for me.

Please also read this very good report from Luke on SSD unreliabilities.

Crucial C300 sudden death and full data loss

So, before people tell me "you should have bought intel", I got the C300 from work and installed it because I trusted it had been pre-qualified and was good. Turns out, I was one of the people doing the pre-qualifying :) Later, I did look at intel for my drives, but they were lacking in size capacity when I was looking for 512GB and later for 1TB. Too bad because they've been shown to be the most or amongst the most reliable drives so far.

I wasn't using TRIM with the C300 at the time, because it's not necessarily recommended if you use dm-crypt ( http://asalor.blogspot.com/2011/08/trim-dm-crypt-problems.html ). The C300 likely did not appreciate that I filled it a few times and that it had to use its garbage collection algorithm to reclaim blocks (when it was getting full, it was very very slow, as in 100x slower than a spinning drive at times).

In the end, it died after fewer than 3 months of use, where it just disappeared from the SATA bus. Some magic 'plug the power but don't do data' for 20mn or so, several times, brought it back (after several days of trying) long enough for me to see it could still talk SATA and that all my partitions and data were gone. Even then, I wasn't able to really usefully talk to it anyway.

If you read up on this, and hear Crucial support recommending that you let the drive sit at the bios for 20mn once it a while so that its garbage collection can kick in after 15mn of it not being used, you'll understand that you should run away in the other direction rather than buying one of those drives: http://forums.crucial.com/t5/Solid-State-Drives-SSD/Bios-is-not-recognize-crucail-m4-SSD-64GB/td-p/63835

Maybe the C400 sucks less, but once burnt, twice shy, and the C300 was so bad, that I just don't trust Crucial to write proper firmware at this point.

Samsung 830 512GB severe random IO slowness with linux and windows

So, I then did some reading about SSDs and shopping around. I found that the Intel did not come in 512GB (it was a bit smaller), and that while Intel ruled SSDs early on, it hasn't kept up in performance and offerings. Also, they use older sanforce controllers that do not do well with software encrypted data (exactly what I had).

From online reading, the next best player was Samsung, so I got an 830 512GB drive from them, and was quite hopeful after reading reviews and how they designed and manufacture all the parts themselves.
That decision came back to bite me. I spent too many hours (between 10 and 20) with samsung tech support, doing many benchmarks, and working with the linux community, because its random IO performance was bad. How bad? Well, how does 5 times slower than my hard drive in the same laptop sound?

On time spent with the linux community debugging this with multiple filesystems, laptops, and tweaks, I'm not kidding. See a small sample:

  • http://www.spinics.net/lists/dm-crypt/msg04403.html
  • http://www.spinics.net/lists/dm-crypt/msg04406.html
  • http://www.spinics.net/lists/dm-crypt/msg04408.html
  • http://www.spinics.net/lists/dm-crypt/msg04436.html
  • http://www.spinics.net/lists/linux-btrfs/msg18009.html
  • http://www.spinics.net/lists/linux-btrfs/msg18168.html
  • http://www.spinics.net/lists/linux-btrfs/msg18204.html
  • http://www.spinics.net/lists/linux-btrfs/msg18238.html
  • http://www.spinics.net/lists/linux-btrfs/msg18240.html
  • http://permalink.gmane.org/gmane.comp.file-systems.ext4/33491
  • http://permalink.gmane.org/gmane.comp.file-systems.ext4/33637
  • There is one problem that might be linux specific: the drive is very slow for encrypted block device access without:
    blockdev --settra 8192 /dev/mapper/cryptroot

  • http://www.spinics.net/lists/dm-crypt/msg04408.html
  • http://www.spinics.net/lists/dm-crypt/msg04436.html
  • But past that, I found no other problems, and replicated the problem with ntfs, and even after putting the SSD in a windows desktop with Windows 7 64bit. Samsung's old Benchmark registered so low on their own Random IO test that the bar graph showing speed was a single line at the '0' mark.
    Samsung support had no idea why I saw this. They suggested I get another drive in case mine was bad.
    The new drive had identical problems and in the end I had to return both. Yes, the drives had the latest firmware. The test below shows lower block IO because it was on SATA2, not SATA3 like in my laptop but I was only worried about Random IO.

    I have no idea why those drives were so unexplainably slow on random IO reads (not even writes) when block IO was fast (500MB/s). I don't know how they work for others, but they failed to perform for me on 3 machines, 4 filesystems, and 3 operating systems.

    That was a lot of time wasted, and I'm not impressed with Samsung anymore :(

    Samsung 840 Evo 1TB (bought in November 2013)

    As explained below, the OCZ drive was fast and didn't have the random IO performance issue that the Samsung 830s had for me, but the Vertex ended up locking up every few days after 3-6 months of use, needing replacement. When I got tired of that, I eventually upgraded to a 1TB Samsung 840 Evo.

    Non surprisingly, it's the fasted drive of them all, but it's over a year newer. With an upgrade to linux 3.11 in the meantime, encrypted btrfs on that drive is almost as fast as native raw speeds.

    OCZ 512GB Vertex 4 vs Samsung 830 512GB vs Samsung 840 EVO 1TB linux tests with btrfs, ext4, and dmcrypt

    Update: the drive is fast, it never really lost data (except apparently the last few blocks written before locking up), but I had 3 drives eventually go into a mode where they lock up and my laptop needs to be power cycled under certain workloads (mencoder is one for me). They always recovered, only lost the last bits of data being written when the drive locked up, but that's just no good. I replaced the drive 3 times and eventually switched to a 1TB Samsung 840 Evo.

    Old text from July 2012:
    After more research still, this drive came highly recommended too. It's the fastest from OCZ and the most expensive (they have slower cheaper ones). And yet, the most expensive one was $100 cheaper than the Samsung, and 36 times faster for random IO reads. Oh, also the OCZ has 5 year warranty instead of 2 for Samsung.

    There are only crude tests, but they are the ones that mattered to me to validate that the OCZ didn't have severe problems in my hardware like the Samsung 830 did.
    Here are optimizations I used:

  • fdisk cylinders were chosen so that they were multiple of 512 to make sure I was aligned with internal block write sizes
  • btrfs was created with mkfs.btrfs and mounted with mount -o compress=lzo,discard,nossd,space_cache,noatime
  • ext4 was created with mkfs.ext4 -O extent -b 4096 -E stride=128,stripe-width=128 and mounted with mount -o discard,noatime
  • cryptsetup luksFormat --align-payload=8192 -c aes-xts-plain /dev/sda2
  • cryptsetup luksOpen --allow-discards /dev/sda4 cryptroot
  • The OCZ was upgraded to Firmware Version 1.5, and the Samsung 830 claimed to have the latest firmware as of 2012/08 (that was version CXM03B1Q as reported by smartmontools). The Hitachi Hard drive in my laptop is a slow 5400RPM model.

    Long story short, the OCZ is super fast, worked great in my tests, and was cheaper than the Samsung and the Crucial. Win all around.

    Scanning 15K inodes with the Vertex 4 (vs Samsung and spinning hard drive)

    Summary of results:

  • Samsung 830 is so slow, it doesn't make sense
  • OCZ Vertex 4 is super fast
  • Samsung 840 Evo 1TB more than a year later is much much faster and easily beats the Vertex 4
  • for reasons that I ignore, encrypted btrfs on my HD was 5x slower than unencrypted in the du -s case.
  • dmcrypt on linux makes btrfs random IO 50% slower
  • random seeks/IO on ext4 in this du test is 2x faster than btrfs on SSD, and 3x faster on HD.
  • At the same time, I'm pretty sure btrfs got faster between kernels 3.5 and 3.11 (this is supported by btrfs now being faster than ext4 when it was slower before
  • OCZ Vertex 4 512GB | Samsung | Samsung | Hitachi HD | 830 512GB | 840 1TB | 1TB 5400 RPM --------------------------------------------------------------------------------------------------- Encrypted BTRFS: | | | reset_cache; time du -sh src | | | 514M src | | | real 0.959s | 25.741s | 0.53s | 5.252s | | | Unencrypted btrfs | | | mount -o compress=lzo,discard,nossd,space_cache,noatime | | | reset_cache; time du -sh src kernel 3.5 | kernel 3.5 | kernel 3.11| kernel 3.5 514M src | | | real 0.636s | 24.937s | 0.11s | 1.007s | | | Unencrypted ext4 | | | mkfs.ext4 -O extent -b 4096 -E stride=128,stripe-width=128)| | | mount -o discard,noatime /dev/sda2 /mnt/mnt2 | | | reset_cache; time du -sh src | | | 519M src | | | real 0.289s | 12.459s | 0.21s | 0.380s



    Block Reads on Vertex 4 and Samsung 840 EVO (vs Samsung 830 and spinning hard drive)

    Summary:

  • raw device speed is about the same for Samsung 830 and OCZ. The 840 is marginally faster
  • encrypted btrfs at 329MB/s is slower, but still is plenty fast. If anyone is doing software encryption faster than that, let me know :)
  • well, a bit over a year later, on the same laptop with the same CPU, encrypted btrfs block speed is about as bad as native block speed. Impressive.
  • unencrypted btrfs block speed is about as fast as block device speed in kernel 3.11. Very nice.
  • OCZ Vertex 4 512GB | Samsung | Samsung | Hitachi HD | 830 512GB | 840 1TB | 1TB 5400 RPM ---------------------------------------------------------------------------------------------------------------- dd if=/dev/sda3 of=/dev/null bs=1M count=2000 | | | 2097152000 bytes (2.1 GB) copied, 4.65606 s, 450 MB/s | 479MB/s | 541MB/s | 104 MB/s | | | hdparm -t /dev/sda3 | | | /dev/sda3: | | | 1430 MB in 3.00 seconds = 476.12 MB/s | 463.45 MB/s | 1112.81MB/s | 99.83 MB/s | | | Unencrypted ext4 block read: | | | reset_cache; dd if=testcopy of=/dev/null bs=1M | | | 2146631680 bytes (2.1 GB) copied, 4.21942 s, 509 MB/s | | 458MB/s | 105 MB/s | | | Unencrypted btrfs block read: | | | reset_cache; dd if=testcopy of=/dev/null bs=1M | | | 2146631680 bytes (2.1 GB) copied, 4.31774 s, 497 MB/s | 495 MB/s | 542MB/s | 90.3 MB/s | | | Encrypted btrfs block read: | | | dd if=w2k-s001.vmdk of=/dev/null bs=1M kernel 3.5 | kernel 3.5 | kernel 3.11 | kernel 3.5 2146631680 bytes (2.1 GB) copied, 6.51863 s, 329 MB/s | 253 MB/sec | 533 MB/sec | 69.1 MB/s



    Block writes on Vertex 4 and (vs spinning hard drive)

    I didn't benchmark write speed on the Samsung 830 because I couldn't get random reads to be fast, so I didn't care about writes.

    Summary:

  • my HD is slow enough (even though 95MB/s isn't actually slow) that encryption doesn't make it slower for block writes
  • on SSD however, it looks like I may hit an encryption bottleneck, speed goes down by almost 2x. Still 130MB/s is fast enough for writes, and I really mostly care about ultimate read speed, which I do have.
  • Somehow with kernel 3.11 on the Samsung 840 EVO, encrypted writes are now 80% of unencrypted speed. Very nice.
  • OCZ Vertex 4 512GB: | Samsung EVO | Hitachi 1TB | 840 1TBM | 5400 RPM --------------------------------------------------------------------------- Raw device write (write to /dev/sdxx bs=1M) | kernel 3.11 | (2.1 GB) copied, 9.35569 s, 229 MB/s | 430 MB/s | 90.9 MB/s | | File write on unencrypted ext4: | kernel 3.11 | (2.1 GB) copied, 11.3984 s, 188 MB/s | 495 MB/s | 92.0 MB/s | | File write on unencrypted btrfs: | kernel 3.11 | (2.1 GB) copied, 8.58974 s, 250 MB/s | 413 MB/s | 130 MB/s | | File write through encryption layer on btrfs | kernel 3.11 | (2.1 GB) copied, 16.4673 s, 130 MB/s | 341 MB/s | 95.6 MB/s

    SSD winner and conclusions

    SSDs are not hard drives. They are faster for sure when they work well, but don't expect them to fail like hard drives. They can fail suddenly with absolutely no warning, and chancees of recovery after that are slim to none. Recovery companies know how to deal with hard drive platters, but most aren't setup to deal with SSDs yet. Moral of the story: have backups (I did, and now I actually backup my SSD on the 2nd HDD in my laptop and can boot on the HDD if the SSD craps out while I'm on a trip).

    The OCZ Vertex 4 512GB won by far obviously. The price is competitive, the speed beats pretty much everything out there, and the 5 year warranty shows that they mean business (most only offer 1, 2, or sometimes 3 years). 5 years is a record in this field.
    Well, that was then, turns out OCZ was more confident than good, their drives did lock up, they made bad tradeoffs on other drives as explained by Luke on http://lkcl.net/reports/ssd_analysis.html, and they are now going bankrupt as of Dec 2013.
    After about a year of use, I confirmed that the OCZ drive is just unstable with linux/dmcrypt/btrfs on my laptop. I'm not sure what part causes it to hang and require a power cycle every other day or so after a few months of use, but that was quite annoying.

    I'm now hopeful that the Samsung EVO 840 will be my new "good drive" for a long longer :)

    It sucks that it took me so long to find a good drive, and so many backup/restore cycles, but I think I got it now. Wish me luck! :)

    2012/08/11 How To Succeed Without Really Trying Musical at Foothill College Theatre
    π 2012-08-11 00:00 in Public
    Jennifer recently signed up with Goldstar and since she's been having more time at home, she's found some shows that we can go see.

    "How To Succeed Without Really Trying" is an old musical that was performed at Foothill College. It was actually quite enjoyable for an old musical from a work area that is quite different from today's work culture, and the performers were quite good too.



    I need to remember that Foothill does perform good plays.

    2012/08/10 Visit of Lick Observatory up Mt Hamilton
    π 2012-08-10 00:00 in Public
    Jennifer got us a pair of tickets for talks and a visit of the observatory, we took 2 people from the group she signed up with, and I got to drive every one up to Mt Hamilton.
    It had been a while since I had done that drive. I remembered it was 'fun', but I forgot how long it was. I was confident that I would easily beat the 1H estimate I got from google maps, but actually I barely matched it despite some spirited driving up the road, and passing multiple cars (almost all were nice enough to pull over).




    a few other people were outside to do their own viewing
    a few other people were outside to do their own viewing

    We got there 15mn before sunset (i.e barely in time to get a few pictures), and went to attend the history talk about James Lick. His story is actually quite interesting, you can read up on it here. Because Jennifer was still on crutches, they thought it'd be best that we stayed in the room with the great lick 36 inch refractor. It was actually quite cool to see it operated, that also meant that we did not get to see the science talk that was being given at the same time :(






    By missing the science talk, we got to see the refractor being setup, and it's so long that the entire floor is raised and lowered so that the operator can be at the right height. It's pretty cool :)

    After looking at the constellation they finally managed to focus on after a while, we went to see the Nickel Telescope, which is a 40 inch reflecting telescope which was actually made out of spare parts.

    A few random slides they were showing on screens:





    The drive home was also fun :)


    2012/08/05 Watching the Nasa Curiosity Mars Landing at Moffett
    π 2012-08-05 00:00 in Public
    I was able to get a ticket to go watch the NASA Curiosity Landing displayed on a giant screen at Moffett. They had some displays to show the different modules on the rover, including a laser that can actually do a basic detection of rocks from 7 meters away (so it can be busy analysing something locally while shooting its laser at other rocks to see if it should go there next).

    The other interesting thing is that it weighs 900kg, or almost the weight of my miata, and that was the largest weight they had the technology to send to Mars, without it crashing on impact. The last rovers were 9 times lighter and were dropped in giant airbags. This was too heavy for airbags and had to be dropped with 2 different parachutes for different speeds, and then slowed down with trusters and a platform that lowered the rover down with cables on the ground and flew off. It was a very complicated list of things that had to go just right.
    I personally thought "no way, those guys are crazy", and they indeed called the landing "the 7 minutes of terror", but it not only went perfectly, the rover was also able to send pictures right after touch down, something that they weren't sure would work at all that night, due to radio relay line of sight issues.

    It was flawless, great job folks!










    Nasa Moffett designed and tested the chute in the wind tunnel
    Nasa Moffett designed and tested the chute in the wind tunnel


    And then we "watched" a rendering of the landing from telemetry that was received (14mn late):





    that's the flying platform lowering the rover to the ground with cables
    that's the flying platform lowering the rover to the ground with cables


    More pages: March 2018 February 2018 January 2018 December 2017 November 2017 October 2017 September 2017 August 2017 July 2017 June 2017 May 2017 April 2017 March 2017 February 2017 January 2017 December 2016 November 2016 October 2016 September 2016 August 2016 July 2016 June 2016 May 2016 April 2016 March 2016 February 2016 January 2016 December 2015 November 2015 October 2015 September 2015 August 2015 July 2015 June 2015 May 2015 April 2015 March 2015 February 2015 January 2015 December 2014 November 2014 October 2014 September 2014 August 2014 July 2014 June 2014 May 2014 April 2014 March 2014 February 2014 January 2014 December 2013 November 2013 October 2013 September 2013 August 2013 July 2013 June 2013 May 2013 April 2013 March 2013 February 2013 January 2013 December 2012 November 2012 October 2012 September 2012 August 2012 July 2012 June 2012 May 2012 April 2012 March 2012 February 2012 January 2012 December 2011 November 2011 October 2011 September 2011 August 2011 July 2011 June 2011 May 2011 April 2011 March 2011 February 2011 January 2011 December 2010 November 2010 October 2010 September 2010 August 2010 July 2010 June 2010 May 2010 April 2010 March 2010 February 2010 January 2010 December 2009 November 2009 October 2009 September 2009 August 2009 July 2009 June 2009 May 2009 April 2009 March 2009 February 2009 January 2009 December 2008 November 2008 October 2008 September 2008 August 2008 July 2008 June 2008 May 2008 April 2008 March 2008 February 2008 January 2008 December 2007 November 2007 October 2007 September 2007 August 2007 July 2007 June 2007 May 2007 April 2007 March 2007 February 2007 January 2007 December 2006 November 2006 October 2006 September 2006 August 2006 July 2006 June 2006 May 2006 April 2006 March 2006 February 2006 January 2006 December 2005 November 2005 October 2005 September 2005 August 2005 July 2005 June 2005 May 2005 April 2005 March 2005 February 2005 January 2005 December 2004 November 2004 October 2004 September 2004 August 2004 July 2004 June 2004 May 2004 April 2004 March 2004 February 2004 January 2004 October 2003 August 2003 July 2003 May 2003 April 2003 March 2003 January 2003 November 2002 October 2002 July 2002 May 2002 April 2002 March 2002 February 2002 November 2001 October 2001 September 2001 August 2001 July 2001 June 2001 May 2001 April 2001 March 2001 February 2001 January 2001 December 2000 November 2000 October 2000 September 2000 August 2000 July 2000 June 2000 April 1999 March 1999 September 1997 August 1997 July 1996 September 1993 July 1991 December 1988 December 1985 January 1980