Marc's Public Blog - Linux Hacking


All | Aquariums | Arduino | Btrfs | Cars | Cats | Clubbing | Computers | Dining | Diving | Electronics | Exercising | Festivals | Flying | Halloween | Hiking | Linux | Linuxha | Monuments | Museums | Outings | Public | Rc | Sciencemuseums | Solar | Tfsf | Trips

This page has a few of my blog entries about linux, but my main linux page is here
Picture of Linus


Table of Content for linux:

More pages: April 2023 March 2023 September 2021 May 2020 January 2020 January 2019 December 2018 March 2018 January 2018 September 2017 January 2017 October 2016 August 2016 July 2016 June 2016 March 2016 February 2016 January 2016 May 2015 March 2015 January 2015 October 2014 May 2014 April 2014 March 2014 January 2014 November 2013 September 2013 May 2013 March 2013 January 2013 December 2012 August 2012 May 2012 March 2012 January 2012 December 2011 August 2011 July 2011 January 2011 October 2010 August 2010 June 2010 April 2010 March 2010 January 2010 December 2009 November 2009 September 2009 August 2009 July 2009 May 2009 January 2009 December 2008 November 2008 October 2008 January 2008 November 2007 August 2007 July 2006 January 2006 August 2005 April 2005 November 2004 March 2004 February 2004



2016/02/29 Hacking a thinkpad slim tip adapter to output more than 90W (required to charge a Thinkpad P70)
π 2016-02-29 01:01 in Computers, Electronics, Linux
The thinkwiki power connector page explains very nicely how Lenovo power supplies come in round and slimp (rectangular) tip, and can be 35, 45, 65, 90, 135, 170, and 230W. The way the laptop knows how much power it can draw, is by measuring a resistor between the center pin and ground. So far, most of my power supplies had been 90W, and all my thinkpads, including the T540p, were ok enough with that, even if maybe they were not always charging at full speed.

However, when I got my thinkpad P70, although it also uses around 25W in typical use, which means plenty of extra wattage on a 90W power supply to charge the battery, the P70 was unfortunately designed not to charge under any condition unless it recognizes a power supply that's at least 135W (it ships with a 170W power supply, and it's a huge monster brick). I'm pretty mad at lenovo for that because they could totally have designed the P70 to charge at a slower rate from a 90W power supply, or at least when the laptop is sleeping, but no, even when it's sleeping, it will just not charge at all. Sadly, this means that the RAVPower 23000mAh Portable Charger Power Bank External Battery Pack I bougght, was able to power the laptop, but not charge it. I guess in this case, it's not a huge issue since energy would be wasted trying to charge the laptop battery instead of just powering it, but still, that was disappointing.

here, the pack is powering my laptop at 20V, 2.1A
here, the pack is powering my laptop at 20V, 2.1A

The P70 will function off a 90W power supply by just not discharging its battery, and consuming around 1.5A at 20V (i.e. 30W), so that meant that not only I had to buy some bulky 135W power supplies (less bulky than 170W thankfully), but also that I was now unable to use my travel iGo 110W 12V DC power supply, which I use to charge in cars. For AC, I did buy a Lenovo ThinkPad 135W (Slim Tip) Replacement AC Adapter for lenovo ThinkPad T440p 20AN 20AW, Lenovo ThinkPad T540p 20BE 20BF: https://www.amazon.com/gp/product/B00HOMBO0M


But for car use, I was upset about the situation, so I read up and realized I only had to change the resistor in my barrel connector to slimtip adapter and that would likely allow my thinkpad to start charging from my 110W iGo power supply. The hard part was finding where the resistor was since I had to replace a small resistor with a bigger one (the other way around, I could have trivially added a resistor in series).

So, I started with the AC Charger Power Supply Adapter Converter Cable Cord For Lenovo ThinkPad : https://www.amazon.com/gp/product/B00T013J44 and had to find where the resistor was hidden. After cutting the cable in 2, I convfirmed it was on the slimtip side, so I had to take the plug apart until I found it.



After that, it was just a matter of adding a new resistor, and in the end, I added a resistor bridge, so that I could select no resistor, 550 Ohm (90W), 1kOhm (135W), or even 1.9kOhm (170W). For my laptop, though 135W was enough, and my iGo only delivers 110W anyway.



I added a amp/volt meter (RioRand 2in1 4.5-30V 10A Digital Voltage Detector DC Current Volt Amp Meter), so I can tell how much current is flowing and whether I might be exceeding the wattage of the power supply. When my laptop is running, and the battery charging, it's now using 5.6A (just about 110W)


So problem solved. I'm not super happy at Lenovo for having stupidly designed their Thinkpad P70 not to charge from a 90W power supply under any use case, even at reduced speed, or even with the laptop sleeping or off, but since I already had the laptop, I was stuck with it, so this little resistor hack did the trick. Hope this helps someone.

Oh yeah, for car use, instead of this old obsolete iGo brick, I made my own 12V-20V DC-DC converter, and you can read more about it on this page


2016/02/22 Thinkpad P70 NVME vs M2 Sata vs 2.5inch Sata SSD
π 2016-02-22 01:01 in Computers, Linux
So far I had been using the Lenovo T540 which I got because of its 3K 15" screen, but I really hated that generation of thinkpads due to the lack of mouse buttons (hence the bluetack on the pad, used to know where I can click to emulate buttons 1, 2, and 3), and someone who decided it would be great to remove almost all LEDs, including caps lock, and hard drive activity. So when the P70 came out, I was very interested in it, not just because they fixed the LED and mouse button situation, but especially because of the first thinkpad with a 4K screen. The P50 has the same resolution in 15", but I figured if I'm going to have 4K's worth of dots (8 million pixels), I might as well have a 17" screen.

So let's be honest and say that it is a big a heavy laptop, but that was an acceptable tradeoff for me. CPU wise, it's actually no faster than the T540, but the big new thing is the 2 M2 slots in addition to capacity for 2 2.5" Sata drives, like the T540 (one goes in place of the DVD Burner using a special tray).

yes, it's big
yes, it's big

the 2 M2 slots are on the right, and the 2nd drive goes under where the SSD is visible in this picture
the 2 M2 slots are on the right, and the 2nd drive goes under where the SSD is visible in this picture

size of M2 sticks compared to the SSD (with free shot of the 96Wh battery :) )
size of M2 sticks compared to the SSD (with free shot of the 96Wh battery :) )

Comparison with the T540
Comparison with the T540

So the P70 can currently house 5TB of storage and I was interested to see how well the M2 slots worked compared to the Sata ones under linux. First thing to note is that currently M2 slots can only house 512MB of storage each vs 2TB for the 2.5" Sata slots that have more room. Sadly, Lenovo is still not selling the Drive Caddy Adapter for the P70, but the good news is that you can buy this $17 Soogood 2nd HDD SSD Hard Drive Caddy Adapter Use For Lenovo Thinkpad T440p T540p W540p from Amazon, unscrew the little tab from the DVD Drive, screw it on the drive caddy adapter, and it'll work in the P70.

To try that out, I got the following (with price by GB as of 2016/02), in increasing price per GB: - 4.6c/GB: Seagate 2TB Laptop HDD SATA 6Gb/s 32MB Cache 2.5-Inch Internal Bare Drive (ST2000LM003) - 30c/GB: SATA6G: Samsung 850 EVO 2 TB 2.5-Inch SATA III Internal SSD (MZ-75E2T0B/AM) - 33c/GB: M2/SATA Samsung 850 EVO 500 GB M.2 3.5-Inch SSD (MZ-N5E500BW) - 64c/GB: M2/NVMe: Samsung 950 PRO -Series 512GB PCIe NVMe - M.2 Internal SSD 2-Inch MZ-V5P512BW

So it's interesting to note that SSDs still cost 6.5x more than a hard drive (actually I was surprised to see how cheap the 2.5" 2TB hard drive was). Next, it's good to know that M2/Sata SSDs cost about the same price per GB than full size Sata SSDs. And last, at twice the price, the M2/NVMe SSD is no bigger than the SATA M2 version, but clearly more expensive. It's supposed to be up to 4 times faster in bulk transfer rate due to the use of 4 PCI lanes, but I was curious to see how much faster it would actually be for real use with linux.

First, is looking at support. To make things interesting, I decided to make the NVMe SSD my boot drive. The Thinkpad bios of course supports it fully, but a big gotcha is that it'll only boot if you use a GPT partition with EFI. Setting that up from scratch was a pain, but I got through it, and it's out of topic for this post.


Next, is linux support. Thankfully linux has supported NVME for a while now. I just set CONFIG_BLK_DEV_NVME=y in my kernel so that I didn't have to worry about the module being in my initrd. Device partitions show up as /dev/nvme0n1p1 and so forth.

Now, the benchmarks. For my main filesystems, I have btrfs on top of dmcrypt, created with:

cryptsetup luksFormat --align-payload=8192 -s 256 -c aes-xts-plain64 /dev/device

In the tests below, I did used hdparm, iozone, and 2 kinds of dd, ddd with data, and dd0 copying just /dev/zero:

ddd test: sync; dd if=/mnt/ram/file of=file bs=100M count=100 conv=fdatasync; dd if=file of=/dev/null bs=100M
dd0 test: sync; dd if=/dev/zero of=file bs=100M count=100 conv=fdatasync; dd if=file of=/dev/null bs=1M

Let me start with the summary conclusions first, since this is what most people probably care about:

  • NVMe is faster for big contiguous I/O, even through dmcrypt, that's good.
  • M2 NVMe and M2 sata do not seem to go through buffer cache for writes, but SATA/6GB does
  • When using buffer cache, SATA/6GB beats NVMe/M2 and SATA/M2 by a lot on block reads. Why?
  • For random I/O, NVMe is up to 50% faster, and faster with ext4 than btrfs
  • Encrypted NVMe is a bit faster than non encrypted Sata/M2
  • At $328 for 512GB M2/NVMe vs $168 for 500GB M2/Sata, unless you're doing big contiguous I/O, the 2x price difference is hard to justify, cached I/O and random I/O are not significantly faster on NVMe.
  • More detailled summaries related to the test results below:

  • NVMe only really looks faster on big block sequential read/write (indeed 4X read, 2X write). On Random I/O, it's no faster. Other overheads seem to kill its performance advantage.
  • 2TB Sata 6GB SSD is faster for read/write encrypted than NVMe is non encrypted (except hdparm). Looks like the block caching layer works better for Sata than NVMe?
  • iozone through block cache is faster on M2/SATA than M2/NVMe, very weird.
  • iozone direct I/O shows NVMe is fastest (by 50% only) and external SATA just a bit faster than M2/Sata. Clearly the block caching layer hides differences for iozone.
  • when using dmcrypt, NVMe is only marginally faster in iozone than Sata/M2 or Sata/6G
  • kernel build speed is same on SATA/M2 and NVMe/M2, encrypted or not, but SATA6G is 10% faster (probably the same effect that block cache works better on SATA/6G than SATA/M2 or NVMe/M2)
  • Thankfully on big sequential IO (ddd test), NVMe finally shows being 4x faster on read, and 2.5x faster on write.
  • ext4 does better on iozone -I (directIO) than btrfs, 2x as fast on random read for NVMe
  • So there you go. I realize that my test suite may not have been perfect, but hopefully the results are helpful to others. I'm hoping linux will get fixed, or that I can find a tunin g parameter to bridge the gab in cached IO speed between Sata 6G 2.5" and M2 slots. Below are test results in more details are you go down the page.

    Here are test results for cached IO:

    		SATA/M2 dmcrypt	 SATA/M2	NVMe dmcrypt	NVMe		SATA6GB/dmcrypt
    hdparm -t	 534MB/s	 517MB/s	1307MB/s	2140MB/s	 534MB/s
    

    ddd 10GB read 550MB/s 560MB/s 2000MB/s 2400MB/s 550MB/s ddd 10GB write 500MB/s 503MB/s 1300MB/s 1200MB/s 506MB/s

    dd with /dev/zero hits optimizations that actually penalize NVMe: dd0 10GB read 7GB/s 6.7GB/s 2.3GB/s 4.4GB/s 6.6GB/s dd0 10GB write 495MB/s 503MB/s 1.3GB/s 1.3GB/s 2.0GB/s

    iozone -e -a -s 4096 -i 0 -i 1 -i 2 (cache makes Sata6G faster than M2, weird): ioz read 4KB 1760MB/s 1790MB/s 675MB/s 766MB/s 7863MB/s ioz write 4KB 268MB/s 298MB/s 295MB/s 291MB/s 777MB/s ioz ranread4KB 5535MB/s 7392MB/s 3185MB/s 7261MB/s 7892MB/s ioz ranwrite4KB 288MB/s 329MB/s 205MB/s 381MB/s 796MB/s

    For comparison, I then used directio to bypass the caching layer:

    iozone -I -e -a -s 4096 -i 0 -i 1 -i 2 (direct IO):
    		SATA/M2 dmcrypt	 SATA/M2	 NVMe dmcrypt	 NVMe		 SATA6GB
    										 dmcrypt notcrypted
    		 btrfs		 btrfs   ext4	 btrfs		 btrfs   ext4	 btrfs     ext4
    ioz read 4KB	 255MB/s	 303MB/s 408MB/s 303MB/s	 396MB/s 562MB/s 310MB/s 379MB/s
    ioz write 4KB	 195MB/s	 238MB/s 344MB/s 257MB/s	 358MB/s 361MB/s 233MB/s 365MB/s
    ioz ranread4KB	 305MB/s	 351MB/s 482MB/s 476MB/s	 699MB/s1434MB/s 309MB/s 475MB/s
    ioz ranwrite4KB	 265MB/s	 223MB/s 392MB/s 260MB/s	 217MB/s 315MB/s 254MB/s 366MB/s

    Details of each test, per type of drive;

    ===============================================================================
    saruman M2 SATA 512GB encrypted btrfs
    

    saruman:/tmp# hdparm -t /dev/mapper/cryptroot Timing buffered disk reads: 1604 MB in 3.00 seconds = 534.03 MB/sec

    iozone -e -a -s 4096 -i 0 -i 1 -i 2: random random kB reclen write rewrite read reread read write 4096 4 257076 281089 1761711 6094733 6066754 290373 4096 512 277108 295167 1588844 8426437 11128258 318731 4096 4096 268940 308248 1760087 8062616 8377131 320802

    iozone -I -e -a -s 4096 -i 0 -i 1 -i 2: 4096 4 38626 45227 102236 111141 30936 64317 4096 512 138467 128878 134674 137832 140067 126248 4096 4096 195439 273100 255379 307300 305605 265093

    saruman:/mnt/mnt# sync; dd if=/dev/zero of=file bs=100M count=100 conv=fdatasync; dd if=file of=/dev/null bs=1M 10485760000 bytes (10 GB) copied, 21.1761 s, 495 MB/s 10485760000 bytes (10 GB) copied, 1.49373 s, 7.0 GB/s

    kernel 4.4.1 make -j8: real 17m54.712s user 126m26.620s sys 6m21.948s

    btrfs send/receive encrypted partition to non encrypted, 87GB: 10mn30

    saruman M2 SATA non encrypted btrfs ----------------------------------- saruman:/mnt/mnt4# hdparm -t /dev/sdc4 /dev/sdc4: Timing buffered disk reads: 1554 MB in 3.00 seconds = 517.93 MB/sec

    iozone -e -a -s 4096 -i 0 -i 1 -i 2: random random kB reclen write rewrite read reread read write 4096 4 285020 297565 1739944 7434579 6605595 305785 4096 512 310937 336617 1298278 3703511 5483860 299416 4096 4096 298433 317101 1790911 7392988 7434579 337476 iozone -I -e -a -s 4096 -i 0 -i 1 -i 2: 4096 4 47219 93497 118900 126065 39160 86462 4096 512 214381 213388 194307 209215 206951 209289 4096 4096 238792 221839 303768 352951 351342 223618 ext4 iozone -I -e -a -s 4096 -i 0 -i 1 -i 2: 4096 4 119524 146035 150173 159105 53090 140697 4096 512 358954 354810 366466 423088 419760 375542 4096 4096 344087 398828 408259 478337 482569 392487

    saruman:/mnt/mnt3# sync; dd if=/dev/zero of=file bs=100M count=100 conv=fdatasync; dd if=file of=/dev/null bs=1M 10485760000 bytes (10 GB) copied, 20.8537 s, 503 MB/s 10485760000 bytes (10 GB) copied, 1.55651 s, 6.7 GB/s

    kernel 4.4.1 make -j8: real 17m55.612s user 126m31.952s sys 6m27.452s

    ******************************************************************************** saruman M2 NVMe 512GB encrypted btrfs

    saruman:/tmp# hdparm -t /dev/mapper/cryptroot2 /dev/mapper/cryptroot2: Timing buffered disk reads: 3924 MB in 3.00 seconds = 1307.56 MB/sec

    iozone -e -a -s 4096 -i 0 -i 1 -i 2: random random kB reclen write rewrite read reread read write 4096 4 284205 341651 742181 6195843 6178018 341101 4096 512 314351 319573 864883 8865630 8902382 384993 4096 4096 295741 201042 675234 3281854 3185110 205974 iozone -I -e -a -s 4096 -i 0 -i 1 -i 2: 4096 4 80042 83160 122616 127129 41421 53963 4096 512 168713 153333 167710 209923 198123 168026 4096 4096 257968 258566 303023 490828 476440 260014

    saruman:/mnt/mnt2# sync; dd if=/dev/zero of=file bs=100M count=100 conv=fdatasync; dd if=file of=/dev/null bs=1M 10485760000 bytes (10 GB) copied, 8.01494 s, 1.3 GB/s 10485760000 bytes (10 GB) copied, 4.63397 s, 2.3 GB/s

    kernel 4.4.1 make -j8: real 17m57.513s user 126m58.360s sys 6m25.164s

    btrfs send/receive encrypted partition to non encrypted, 87GB: 7mn (33% faster than Sata M2)

    saruman M2 NVMe non encrypted btrfs ----------------------------------- saruman:/mnt/mnt4# hdparm -t /dev/nvme0n1p4 /dev/nvme0n1p4: Timing buffered disk reads: 6422 MB in 3.00 seconds = 2140.38 MB/sec

    iozone -e -a -s 4096 -i 0 -i 1 -i 2: random random kB reclen write rewrite read reread read write 4096 4 274808 271851 684271 2634219 2280673 177630 4096 512 290191 336755 668196 3362795 4039655 184779 4096 4096 291427 340345 766588 7355007 7261741 381768 iozone -I -e -a -s 4096 -i 0 -i 1 -i 2: 4096 4 94814 117116 147645 159695 44075 108039 4096 512 271130 267800 239969 325933 301841 271062 4096 4096 358602 275933 396591 719099 699939 217665 ext4 iozone -I -e -a -s 4096 -i 0 -i 1 -i 2: 4096 4 107281 200735 205900 263035 54832 124219 4096 512 537478 374641 525060 1173664 1139338 558807 4096 4096 361545 526137 562632 1483477 1434069 315883

    saruman:/mnt/mnt4# sync; dd if=/dev/zero of=file bs=100M count=100 conv=fdatasync; dd if=file of=/dev/null bs=1M 10485760000 bytes (10 GB) copied, 7.90653 s, 1.3 GB/s 10485760000 bytes (10 GB) copied, 2.35863 s, 4.4 GB/s

    kernel 4.4.1 make -j8: real 17m54.221s user 126m46.264s sys 6m10.592s

    ******************************************************************************** saruman Samsung Evo 850 2TB SSD encrypted btrfs

    hdparm -t Timing buffered disk reads: 1606 MB in 3.00 seconds = 534.87 MB/sec

    iozone -e -a -s 4096 -i 0 -i 1 -i 2: random random kB reclen write rewrite read reread read write 4096 4 513924 738957 8586475 9144037 8017464 697665 4096 512 772100 820362 9287391 10136778 10586522 840551 4096 4096 719099 777517 7863339 8047509 7892238 796108

    iozone -I -e -a -s 4096 -i 0 -i 1 -i 2 4096 4 76626 47961 92886 80947 36063 63788 4096 512 135087 138998 143127 150981 144393 132219 4096 4096 233245 234058 310797 311552 309855 254080

    ext4 iozone -I -e -a -s 4096 -i 0 -i 1 -i 2 (unencrypted) 4096 4 118013 141212 130575 111534 42343 138654 4096 512 336347 312192 332847 414404 424542 342707 4096 4096 365227 364638 379298 484160 475122 366536

    saruman:/tmp# sync; dd if=/dev/zero of=file bs=100M count=100 conv=fdatasync; dd if=file of=/dev/null bs=1M 10485760000 bytes (10 GB) copied, 5.23919 s, 2.0 GB/s 10485760000 bytes (10 GB) copied, 1.58287 s, 6.6 GB/s

    kernel 4.4.1 make -j8: real 16m4.305s user 104m0.816s sys 7m2.832s

    2016/02/01 ESPlant, Arduino Miniconf at Linux.Conf.AU 2016
    π 2016-02-01 01:01 in Arduino, Linux
    Another year, another arduino miniconf, this year, the LCA open hardware folks brought us the ESPlant, a new board meant to monitor plants, and based off the ESP8266, the super cheap wifi chip that can now be used with the arduino IDE and libraries thanks to a compat layer.

    Jon, opening the miniconf
    Jon, opening the miniconf

    The Melbourne Hardware Hacking team who worked on this
    The Melbourne Hardware Hacking team who worked on this





    Jon gave a cool talk on the hardware assembly machine he built to put those boards together, impressive:



    Short of having a plant nearby, I concentrated on making pretty lights on the LED strip :)


    blinkie achieved :)
    blinkie achieved :)

    I added my blinky code here: https://github.com/marcmerlin/ESPlant

    Demo:

    2016/02/01 Linux.conf.au 2016, Geelong
    π 2016-02-01 01:01 in Linux
    Another year, another LCA, this time by Geelong, Australia (located 1h from Melbourne and 1h from Ballarat, 2 previous LCA locations).

    Like every year, it was great to see everyone again.

    A few pictures from the conf:

    where we stayed, Deakin, University
    where we stayed, Deakin, University


    The robot miniconf had cool little robots:




    During the radio miniconf, we did some beacon finding with yagis:




    LCA still had a custom designed board that encodes HDMI video into open video formats on linux via USB:


    Among the talks, tridge gave a demo of his new code for quadplanes (quadcopters that turn into planes) for the next challenge they're preparing for:




    Catching up with people in the hallway track and evening parties was good as usual:



    Stephen, keeping a great WiFi network working like every year
    Stephen, keeping a great WiFi network working like every year


    nothing beats seeing tridge in person to work with him on my ardupilot crash plane into the ground bug :)
    nothing beats seeing tridge in person to work with him on my ardupilot crash plane into the ground bug :)




    After a week, it was all over again. We all had a great time, thanks all:


    I hung out with the Melbourne Open Hardware Hackers after conf close, before getting ready for our great ocean road trip the next day:


    See more images for Linux.conf.au 2016, Geelong

    More pages: April 2023 March 2023 September 2021 May 2020 January 2020 January 2019 December 2018 March 2018 January 2018 September 2017 January 2017 October 2016 August 2016 July 2016 June 2016 March 2016 February 2016 January 2016 May 2015 March 2015 January 2015 October 2014 May 2014 April 2014 March 2014 January 2014 November 2013 September 2013 May 2013 March 2013 January 2013 December 2012 August 2012 May 2012 March 2012 January 2012 December 2011 August 2011 July 2011 January 2011 October 2010 August 2010 June 2010 April 2010 March 2010 January 2010 December 2009 November 2009 September 2009 August 2009 July 2009 May 2009 January 2009 December 2008 November 2008 October 2008 January 2008 November 2007 August 2007 July 2006 January 2006 August 2005 April 2005 November 2004 March 2004 February 2004

    Contact Email