I've been going to linux.conf.au for 18 years now (since 2001), and presented a fair amount of linux talks related there, but the big change for me was the open hardware miniconf that started in 2010. Thanks to its projects every year, I got to learn a lot about microcontrollers and some about electronics.
This talk was my first non linux talk which detailled everything I learned from those miniconfs and projects I worked that stemmed from them. I presented it at LCA 2019 Christchurch.
you can find the talk pdf here: http://marc.merlins.org/linux/talks/Using_Open_Hardware/Using_Open_Hardware.pdf (you'll want this one to get all the clickable links in the slides)
you can view the talk slides in html here or below:
Talk video below:
I arrived the sunday before the conference and helped out the open hardware organizers with a bit of last minute setup. I also got to do some last minute testing and tuning of my panels:
hacked up ESP32 with level converters on breadboard to run 3x 64x32 SmartMatrix panels with SmartMatrix::GFX
64x64 P3.8 SmartMatrix::GFX panel vs 3x 64x32 SmartMatrix::GFX P4 flexible panels vs 4x 16x16 FastLED::NeoMatrix P10 panels
After finishing the code tuning and demos just in time, gave a 20mn miniconf talk on the history of linux.conf.au hardware miniconf. I went through how much I learned from those confs and what I was able to achieve as a result. I sure got to learn a lot about microcontroller and driver programming:
I wasn't able to bring my burning man 4096 neopixel matrix, it doesn't even fit in my car, but the irony is that my small 64x64 rgbpanel has the same resolution and fits easily in my backpack
The 64x64 compact display is showing the hand X-ray here
A few days later, I gave the longer version of my talk at the main conference. By then it had grown to over 160 slides in a 45mn slot, or 16 seconds per slide. Ooops...
The full talk went into details on what I learned in the hardware hacking field, a lot of it was simply electricity, U=RI, wires, pre-made components (small inline volt/amp meters, DC-DC converters, and so forth).
This year, the Open Hardware Miniconf team designed a donkeycar for us at LCA 2019 Christchurch. It's a car that navigates by itself using its onboard camera connected to a Raspberry Pi using training video data gathered and analysed offline by tensorflow. That sure was an ambitious project!
I arrived the day before to help finish up the kits for the next morning:
the cars were eager to perform :)
Andy and Jon who ended up working all night to make sure the kits would work the next morning
The next morning, we showed up to build the kit:
rPi with custom last minute hat for the donkey car
Jon gave a talk about the car design
Nice way to support 5V neopixels on 3.3V microcontrollers
I inherited 64 strips of mostly 64 neopixels per strip (some were as low as 61, and some as high as 66).
not all the same lengths
64 strips is run as 16 lines of 4 strips of 64 pixels (256 pixels), were tested 4 by 4, as the line of 256 pixels that they were with a Neopixel tester that sends test patterns:
the neopixel tester I'm using, along with a 5V 10A DC converter outputting 8A into the controller (testing full white)
The first 16 lines took a long time (almost 2 full days due to the time to measure everything including the cardboard, cutting it carefully, marking where all the LEDs will go, and then testing as I go along to make sure I'm not repeating something that will have to be undone:
A bit of test from my MatrixGFXDemo.ino code
Then comes the issue of attaching all of this. I decided very early to remove the IP67 protection as the silicon is resitant to virtually all glues, making it very hard to work with. I also had to splice broken LEDs in the strips I inherited, so it's much easier to do without the protective casing. I simply attached the bare pixels with those very stick glue dots:
After about 3 days of work, got 50% done:
Power delivery: getting the wire sizes right is important, but turns out you don't need huge wires for 64 pixels. What I did was connect power from each of them at the end and therefore spread the maxium 10A they can draw to 4 times 2.5A, which can go fine over smaller wires as pictured below. The green wire goes between the Data Out to Data In pins of the strips (3 short green wires only for 4 strips)
The 64x64 array was meant to be 2 arrays of 32x64 for ease of transport, each with their own 60A 5V power supply (when actually 40A would have been enough for each set of 2048 LEDs even if their maximum is around 80A in theory). I used thick 10 gauge wire to make sure the power bus could support 100A or so if needed, even if in real life, it'll never really much more than 20-30A:
power testing, this can replace a small sun ;)
Once 50% was verified to be working fine, the other 50% was still a lot of work, but I didn't have to stop for testing of the power, layout, and design. It took another day and a half to do the other 50% at much higher pace, getting all the little wires cut at the right length in advance, gluing all the strips in one fell swoop (still hours or work), and then all the soldering, with validation at the end:
For anyone contemplating that work, most of the work was:
cutting the cardboard backing to the right size, and marking where the strips were going to go
cutting/fixing all the strips (they were second hand, some were broken or the wrong length)
320 glue points for the strips, one by one :)
cutting lots of little wires to the right length, stripping them
only then does soldering come in
then test each set of 4 strips (64x4) with a special neopixel tester
build the 10 gauge power bus
the hardest thing has been to solder all the little wires to the 10 gauge power bus turns out. They don't like staying together due to size and thermal issues.
and all the twisted pair and wiring on both sides (seemed trivial but it was more work than I thought) to connect to the microcontroller
ESP32, why not something else like Teensy?
So, my shirt that drives 24x32 uses ESP8266. ESP8266 can do up to 4 lines of parallel output, which is not sufficient for a proper frame rate on 4096 pixels. ESP32 does allow up to 24 lines of parallel output (untested) and can easily do 16 lines of output (110 frames per second for 4096 pixels).
Teensy would would have worked too, but I've had too many problems with teensy, namely:
the hacked up build environment patched on top of some version of the arduino sdk. Recently I've had 30 second pauses before compiles even happen if I'm using the serial port (reported the bug, never heard back, never got fixed). This made sense years ago, but not anymore today that arduino supports other hardware boards "the right way"
newer sdk patch that's supposed to fix things, working even less for me (1.42 beta worked even less than 1.41 when I tried it).
serial output just stopping randomly after outputing X lines, making debugging impossible. This was a big deal for me, I reported it, but never got a fix.
no hypervisor like task that catches crashes and gives you a traceback with line numbers like you get on ESP chips (which really helps for debugging problems). This is also a big plus of ESP chips
no onboard flash usable for an SPIFFS filesystem (like the 4MB of flash on ESP32). Now, I'll have to admit that SPIFFS starts falling in performance due to how it's written when you reach 1MB or more of data, but it could be fixed with a better driver and beats no onboard filesystem at all on teensy (you need the 50-ish dollar teensy 3.5 to get an sdcard reader)
closed bootloader that prevents better debugging, and maintained by a single person who is very helpful, does a lot of work, but cannot compete with more open chips maintained by multiple people. It pains me to say this, because Paul Stoffregen does an incredible amount of work for one person, but he remains one person with a closed bootloader design and a hacked up SDK (sorry to say it). Compare with ESP32 which has near real time support on https://gitter.im/espressif/arduino-esp, plus https://github.com/espressif/arduino-esp32/ .
and not that it really matters to me, but despite all these issues, teensy costs at least 2 to 3 times the price of ESP32.
Teensy 3.6 is 32 bit 180/240 MHz ARM Cortex-M4 single core vs 32-bit LX6 microprocessor, operating at 160 or 240 MHz dual core
Teensy 3.6 is 1MB of flash vs 4MB of flash for ESP32 (which can be segmented for OTAs via Wifi and SPIFFS filesystem)
Teensy 3.6 has 256KB vs usually 4MB of SRAM on ESP32, huge difference
ESP32 is dual core (although that adds complexity), adds Wifi and BT vs built in sdcard on teensy
Teensy has more pins but requires an expensive breakout board to use them all
I think teensy 3.1 (now 3.5/3.6) was the best chip around for many years, but honestly ESP32 seems like a better solution for most needs, especially debuggability. This is not to say that Teensy 3.6 is a bad offer, it does a few things better than ESP32, but at a much higher cost, and its SDK and problems explained above, make it a less desirable solution for me.
Why not use those premade 32x32, 32x64, or even 64x64 panels?
This is a very good question. First, they don't exist for neopixels, they exist for a different lower tech solution that requires a lot of work to drive.
It's a lot easier to get a lot of pixels for not much work on those panels: https://www.adafruit.com/product/2276 , but they are more bulky, don't bend at all, and driving them is a lot more work than neopixels. Sadly, because they require row scanning (like an old CRT TV), they also don't look good if you move them, move your head, or take a picture.
Turns out that those panels can be made bendable too now: https://www.adafruit.com/product/3803 but still, they don't seem to come in bigger pitch sizes, and still have the persistence of vision problem I just described.
In the case of a big display, neopixels that usually require too much space (high pitch), actually come out ahead if you want your display to be 1m^2. RGB panels come out ahead for smaller displays with high resolution, you can go as low as 3mm pitch, which beats all existing neopixels: https://www.adafruit.com/product/2279 .
If you want to go higher sizes with those RGB panels, it gets more complicated without an FPGA and arduinos typically can't run more than 32x64, meaning you need 2 arduinos to run 64x64, unless you get this https://www.adafruit.com/product/3649 but then it's apparently only supported by a special controller for rPI.
My point here is that neopixels cost more, but they're a lot easier to drive, despite the timing issues you start running into when you're driving a lot (the more you drive per line, the lower the refresh rate, putting a reasonable limit for a single MCU around 10,000 LEDs if you're ok with a 35Hz refresh rate).
I personally wish for Neopixel matrices that ship in 32x32 or higher, potentially with the option to inject a new data line every 256 or 512 pixels (so you can drive them as one big slower array, or cut the data line in the middle and inject parallel data lines for faster refresh rates.
ESP32 8 or 16 Parallel output and driver
First, 4096 pixels without parallel output, I would only get 7fps, which is quite slow.
With Sam's driver: https://github.com/samguyer/FastLED , you can use the RMT driver in ESP32 which allows for 8 parallel outputs. This can be used for more than 8 pins, I used it with 16 and the driver can switch RMT back and forth between the first set of 8 pins and the 2nd set of 8 pins.
Yves' driver obviously gives you better FPS, but taxes the CPU a lot more by doing all the bit banging. Also, in my testing, it did not work reliably until I added level shifters, while Sam's driver actually produced better waveforms that worked at 3.3V without level shifters.
The 2 drivers don't get setup the same though, see those differences:
Which driver is best for you? I'd say it depends but if you are ok with up to 8 parallel pins, use Sam's driver with RMT, and if you want more pins (up to about 24), use Yves' driver.
Wiring and Level Shifters
I first did it wrong by wiring directly to ESP32. This was doubly a mistake because there is sadly no standard pin numbers between ESP32 boards, meaning that I had to re-wire my plugs if I changed chips:
My other problem was that while 3.3V output worked ok enough, when using Yves Basin's 16 line parallel output code, the software built waveforms didn't work well enough at 3.3V. I had to add level shifters, which also nicely added a level of indirection between my cat5 twisted pair cables and the pin numbers on the chip:
Later, I changed one more thing which was to reomve the bidirectional level shifters that were unnecessary and caused issues at boot on ESP32 by messing with some IO pins. Turns out I had go make sure GPIO2 and GPIO12 were low at boot or flashing and reboots would fail (hence the resistors in the picture). However, I ended up replacing them with simpler 74HC245 unidirectional level shifters which don't mess with I/O pins and removed the need for the resistors
Thankfully I was able to leverage the weeks/months of work I put on https://github.com/marcmerlin/FastLED_NeoMatrix and then demos I wrote for it, or shamelessly borrowed from more talented programmers :)
I then spent a lot of time on my https://github.com/marcmerlin/NeoMatrix-FastLED-IR code that ran my Neopixel shirt and adapted it so that its demos would work on a 64x64 matrix while skipping the handling of neopixel strips that are on my pants and arms. I then did a recording of the entire set of demos, including 64x64 animated gifs I found and liked, and ended up with 41mn:
The build was a lot of work, no fun at all: over 4 days of solid work... If you do this, strongly consider getting pre-built matrices that are ideally at least 32x32. Sadly most of the ones for sale today are 16x16 which still means getting 16 of them for about $500, laying them out and soldering them. It's not trivial work either if you re-inject power in them in more than one place, but clearly less work than laying 64 strips by hand like I did.
Get power right. I had some experience there, so I did my math beforehand and verified as I went along. It's not so hard to change a power supply, but it sucks if you have to replace all your power wires you spent so long to cut and solder.
Software is key of course. Running 16 strips in parallel requires some work from a small embedded CPU. Doing Infrared at the same time is not trivial. You can look at my code on how I got it to work, including this bug I found: https://github.com/espressif/arduino-esp32/issues/1781
The RMT driver on ESP32 is great doing doing DMA to 8 lines and either doing infrared without interrupts (sadly I found no IRRemote compatible RMT driver for arduino), or for outputting 8 lines of neopixels at once without big banging from the CPU (this is the FastLED Neopixel driver that Sam Guyer wrote). 8 lines only gives 55fps for 4096 LEDs, while 16 lines gives a nicer 110fps and leaves the RMT driver free for IR Receiving (putting aside that there is no driver at the moment).
I couldn't have done this without plenty of great work from others, be it the FastLED authors and contributors, Yves who offered his suport since he did a bigger build than mine, and his 16 line parallel driver, Jason Coon and others for the Aurora SmartMatrix demos I was able to use, and Mark Estes for even more LEDMatrix demos he wrote and that I was able to use too. Thanks all.
Oh yeah, I built this for Burning Man and despite being hard to transport due to its size, it made it there ok and survived the playa dust for a week:
due to lack of skill and lack of time, I used my protoboard and taped it on the reard of the display. Not professional, but it works
running matrix demo
animated GIFs are fun
somehow my protoboard and ESP32 survived the playa dust for the week
in Arduino, Clubbing, Electronics
After I figured that my previous shirt was getting a bit stale, I wanted to design EDM Party Shirt v2, which both had to be easier to wash (the previous one had LED strips all over it that were too much work to remove, making the shirt not really washable), and I also wanted it to be more special.
I had been playing with Adafruit::NeoMatrix and figured it would be cool to have a NeoMatrix on my shirt (also it would be easier to remove a whole panel to wash the shirt).
So, in the great tradition of yak shaving, I first decided that I should be able to use FastLED instead of Adafruit::Neopixel. This was mainly at the time because FastLED allows parallel output and therefore allows updating a screen by segmenting it in smaller pieces that are updated in parallel. With Adafruit::NeoMatrix, my 768 pixel panel would have taken about 30ms to refresh, or only 33 frames per second, which is not a lot. With FastLED, I was able to trivially segment it in 3 and get 10ms/100fps, which is a whole lot better for animations.
Turns out, switching to FastLED also gave me access to LEDMatrix, an alternate library which while lacking Adafruit::GFX primitives, offers another nice API including better text and basic sprite support.
So, I started a port of Adafruit::NeoMatrix to FastLED and wrote FastLED::NeoMatrix. It was actually a non trivial port, which took much longer than I thought, but I got it working in the end:
by then I was already able to display with 3 data lines, one for each 32x8 panel
In the process, I managed to plug the wrong power into the panel and blew a pixel. Thankfully I was able to borrow another pixel from a strip and soldering it in place to fix it:
Once the port was done, came time to get the panels together. Wiring was a bit challenging because I had to ensure that power was injected at a mostly equal value in 9 different points (3 per panel) due to voltage sag at higher amps. I then quickly found out that with 3 data lines at high frequency, there is cross talk between the wires, so I switched to a CAT-5e cable where each data line is twisted with its ground and the remaining 2 wires are used to bring VCC (5V). Technically my 3 panels can use up to 60A at 5V at full power. Obviously many things will melt before that, and I'm using a power converter that tops out at 10A. Even 10A is too much for the wiring, but thankfully my displays don't use that much brightness (it would be blinding to people):
two 5Ah 16V lipos give about 160Wh, just enough to run 2 panels 10 to 12H
the fancy energy meter keeps track of how much time I used and how many Ah came out of the batteries. Useful to know how much time I have left. The blue meter is amps at 5V (after the power converter) and the red one amps at 16V (straight from the battery)
on the right, I have a backup ESP8266 if the main one fails, a backup BEC (5V converter) that's technically 10A capable but not meant to do more than 5A continous, and bottom right is my older 3A BEC with in line amp meter
the new tobsun 5V converter is bulky but does 10A without issues
For the back of the shirt, I got some fabric and created an inside pocket to hold the panel:
it then mirrors the front
All this crap goes into a fanny pack, and gets wired to my shirt, pants, and glasses:
the cool part is that my back also has a panel inside the shirt that uses the shirt as a diffuser
I've been a fan of Neopixels and Adafruit's Neomatrix library since last year. Why is Neomatrix special? It allows you to create Neopixel matrix displays and treat them like a panel display on which you can apply Adafruit::GFX primitives.
The problem is that https://github.com/adafruit/Adafruit_NeoMatrix only supports the https://github.com/adafruit/Adafruit_Neopixel backend which in turn only supports neopixels, and only in a serialized fashion. Serialized means that if you have 1024 pixels (32x32 matrix), it takes about 35 to 40ms to send a single frame, which means you can only update at a 25fps rate, which is not that good.
https://github.com/FastLED/FastLED fixes this by offering support for a lot of different kinds of LEDs, but also for offering parallel output on multiple MCUs, allowing you to get up to 16x faster updates by segmenting the strip to update sequentially. It also offers much nicer primitives like automatic fading, and the ability to have multiple independent strips of different lengths that can display different patterns.
Neomatrix is aweseome since it offers access to the GFX library primitives (circles, lines, rectangles, and fonts), but since it didn't work with a FastLED backend, I ported it to FastLED.
You can get the code here: https://github.com/marcmerlin/FastLED_NeoMatrix
You can also see my NeoMatrix Shirt on this page
Thanks to Adafruit::GFX, it's easy to display fonts and bitmaps
Sure, it's here: https://github.com/marcmerlin/Neopixel-IR (but go to the end of this page for more details).
So, I have 3 versions of the shirt, let's go through them depending on how complex you'd like to go:
Multi Color LED strip, non controllable
It used a 12V controllable dimmer: https://www.amazon.com/gp/product/B00MHKCTLE which can simply change the brightness of the LEDs, which are of a set color. I bought the LED strip in Akihabara Japan, but I'm guessing something similar can be found elsewhere.
they are a flat trip that doesn't really like being bent
all the LEDs are in series, so if the chain fails somewhere, nothing lights up beyond the point of failure (which is why I have extra wires soldered on to make redundant paths and get around breakage
I have had some sections of 3 LEDs where one color died
All of the controllers I tried have a dimming function, but it fails to work when I use the fade dimming patterns, which means they are full bright, suck battery and blind others. I've therefore had to add a 2nd voltage regulator where I can just dim the voltage sufficiently so that the current is then also capped by each color string and therefore the brightness is controlled (you cannot limit the global current fed to the LED controller because you don't know whether it's all fed into a single color, or shared amongst 3).
Of course the last thing is that all the LEDs for a color are in series in a single circuit, so you light up all the reds, or all the blues or greens, or none. You can't turn on one LED individuallly. This is where the next solution comes in.
ESP8266/Arduino Controlled Neopixels
To fix the issue of not being able to have custom multi color LED patterns, where each LED/pixel can be controlled separately, the solution is neopixels (aka WS2811/WS2812/WS2812B). Each LED/pixel is controllable separately, which is done by a computer, in this case the ESP8266 can run my own programs to control all the LEDs.
The ESP8266 is a 3.3V based microcontroller for low power use (good when battery powered) while the neopixels are 5V powered, but thankfully the neopixels can use 3.3V signalling from the ESP8266 while being 5V powered to light up properly. The one thing to keep in mind is that neopixels use some power even if they are not lit. In my case 200 neopixels, take about 120mA when they are off, and 4.5A when they are all lit in white on full brightness.
Neopixels are almost infinitely versatile, instead of having them in single line strips like on my shirt and pants, you can also make display matrices out of them:
All the displays, 4 made with neopixels and soldered by hand
Yeah, the 24x24 one took forever to lay out, glue and solder (6H+)
More than 2 years ago, I spent many hours writing an interrupt written driver for LED matrices that require fast line scanning while you setup the right rows, and even faster per color refreshes where you turn the color on and off depending on how bright you want each color component to be to yield different color mixes. This is explained in more details on my page on Driver for direct driving single to 3 color LED Matrices with software PWM.
Then came in Neopixels which can be individually addressed without fast row scanning, and the Adadfruit Neomatrix library. Those made creating color matrices much more trivial (except for the part where you have to actually build and solder that matrix):
2 row scanned matrices on the upper left, 7x7 neomatrix, another row scanned 32x16 Adafruit panel.
on the bottom row, self built 16x8 neomatrix, and another 12x12 neomatrix
I ended up taking my old Adafruit GFX demo I wrote fro my interrupt driven scan matrices, porting it to Neomatrix (which was trivial) and then extending it. You can find my MatrixGFXDemo code here,
Soon afer, I dug up my Adafruit 16X32 RGB LED Matrix Panel, setup an old Arduino Mega to control it and have enough RAM left to run some fun code (on an Uno memory would have been too tight), it turns out that thanks to Adafruit GFX support, it took very little time to get my same demo working on the RGB Panel.
Here is a video of the demo I wrote on all those different displays:
Here are some things the demo does:
Init displays the same 8x8 pattern as many times as it will fit
The code will then draw lines, rectangles and circles that match the size of the display:
If the display is at least 16x8, it will display the resolution in various ways appropriate to the display size:
As explained on this page, I wrote a pretty extensive driver-set for an ESP32 based board with lots of IO, but hardly anyone has that board (fewer than 100 made), so I ported what I could of the code to the WROVER.
Sadly the WROVER lacks a touch screen, so you'll have to wire at least a rotary encoder or a joystick. In the demo above I also wired 2 neopixels and an IR receiver to mirror the hardware on the IoTuz.
Source code: https://github.com/marcmerlin/IoTuz
See this video for a demo: