Vince's thoughts

Aller au contenu | Aller au menu | Aller à la recherche

Tag - Geek

Fil des billets

lundi 5 juin 2017

Introducing the WAHOOcorder



Front panel

tl;dr

What ?

  • A set-top box modified to add Wi-Fi

What's special about it ?

  • Bidirectional interactivity from a browser
  • Advanced features such as user-firendly search interface
  • Bidirectional integration with existing operator website
  • 5 EUR (5 USD) modification

Demo Video

Jump straight to the features to skip the setup presentation.


Note: A French-audio version is also available here.

What is a WAHOOcorder ?

Well, it started its life as a VOOcorder, a Cisco set-top box distributed since 2009 by the Belgian cable operator VOO. The VOOcorder is controlled by an IR remote or via its front panel. It has an integrated cable modem connecting it to the VOO back-end for TV Guide and VOD, but that's about it regarding interactivity, and in particular it has no Wi-Fi. Due to its age, it's now considered legacy and has been superseded by a new set-top box called .évasion, which works with a companion app and website that lets you zap or schedule recordings from your smartphone.

I thought it would be cool to add similar features to the good ol' VOOcorder by giving it Wi-Fi connectivity.

And so the Wireless-Activated Hacked VOOcorder was born, or WAHOOcorder in short.

Small Logo

What's inside.

As I don't have access to the STB firmware code, and I didn't want to perform irreversible modifications, I designed the additional features as a layer around the standard hardware.

Here are the added low level features:

  • Brains: ESP8266. What else ?
  • Control: The VOOcorder can only be controlled using a IR remote
  • User feedback: Apart from the video overlay, the main user feedback is the front panel
  • Monitoring: The VOOcorder also outputs debug information which can be retrieved from a serial port

All these features are presented in a bidirectional Web UI and API, presented both as a stand alone app as well as injected in the official VOO website.

Let's go into the details.

Brains.

A simple AIThinker ESP-12F module is powered from the set-top box's front USB connector. The 5V is regulated to 3.3V by a AMS1117 followed a 220µF capacitor. Everything is assembled in "dead bug" style on the ESP-12F module, and is located inside the plastic front panel so that the WiFi antenna is out of the shielded case.

Assembly Front

Assembly Back

A small heatsink was later added because the ESP can get hot when enclosed. That heatsink also touches the STB metal case, further dissipating heat. As it is enclosed, the ESP is configured for OTA upload, but to avoid spurious flashing, a (hidden) button has to be pressed for the chip to reset in OTA mode.

Control.

By simply sniffing the signal using a version of IRrecvDumpV2 on the IR receiver, I identified a protocol similar (but not identical) to the NEC protocol . In particlar, it is mirrored (high in idle state, then 9000µs low, then 4400µs high, then 600µs low, then 1600 high, ...), and strangely enough, the first byte (address) are not followed by a byte with all the bits inverted ("not address"), but the first two bits are just repeated without being inverted, so if address is 0b10000000 , the next byte should be 0b01111111 but it is 0b10111111...

Anyway, I decided to store the 32 bits as recorded and play them back untouched and it works perfectly. The codes for the remote keys are :

 0x80BFE11E for key 0
 0x80BF49B6 for key 1
 0x80BFC936 for key 2
 0x80BF33CC for key 3
 0x80BF718E for key 4
 0x80BFF10E for key 5
 0x80BF13EC for key 6
 0x80BF51AE for key 7
 0x80BFD12E for key 8
 0x80BF23DC for key 9
 0x80BF3BC4 for key POWER
 0x80BFF30C for key MENU
 0x80BF41BE for key EXIT
 0x80BF01FE for key VOL+
 0x80BF817E for key VOL-
 0x80BF39C6 for key MUTE
 0x80BF31CE for key BACK
 0x80BFBB44 for key FAV
 0x80BFA15E for key CH+
 0x80BF619E for key CH-
 0x80BF11EE for key INFO
 0x80BF53AC for key UP
 0x80BF4BB4 for key DOWN
 0x80BF9966 for key LEFT
 0x80BF837C for key RIGHT
 0x80BF738C for key OK
 0x80BFA35C for key GUIDE
 0x80BF5BA4 for key TV
 0x80BF19E6 for key RADIO
 0x80BFA956 for key HDD
 0x80BF6B94 for key VOD
 0x80BFB34C for key REW
 0x80BF8976 for key PLAY
 0x80BF0BF4 for key FFWD
 0x80BFB14E for key SKIP_BACK
 0x80BFE916 for key STOP
 0x80BFC33C for key PAUSE
 0x80BFC13E for key REC
 0x80BF916E for key RED
 0x80BF21DE for key GREEN
 0x80BF9B64 for key YELLOW
 0x80BF6996 for key BLUE

A GPIO of the ESP is connected in parallel with the IR receiver via a signal diode, and driven low to short the receiver to GND and emulate an incoming signal. This also leaves the original IR remote fully functional.

Remote

User feedback.

This was by far the hardest part. The front panel leds are driven by an Elan ePVQ6200 chip for which I couldn't find a datasheet, but I could find one for the ePVP6200 which looks similar. Its pinout revealed that it is controlled via SPI on pins 1 (DOUT), 2 (DIN), 51 (STB) and 52 (CLK), so I added wires to these pins to spy on the protocol.

SPI Pins

Those wires were routed along with the IR "injector" to a connector at the end of the front panel board

Front Panel PCB

A the time of development, the ESP8266 had no documented way yet of using SPI slave, but with the help of me-no-dev's SPI Slave library and explanations, I finally succeeded in decoding the messages sent to the Elan chip to update the leds.

The decoded protocol is as follows:

 msg = <command> <payload>
 where command is:
   0x08: repeated every 200ms, seems to be a heartbeat. 
     payload is 0xFF 0xFF.
   0x54: update a 7-segment digit
     payload = <digit> + <value> + 0x00 + 0x00
     digit=
       80 for char 1 (left)
       40 for char 2
       C0 for char 3
       20 for char 4 (right)
     value=
       0bABCDEFGH with each bit corresponding 
       to one segment, respectively 
       <top><top_right><bottom_right><bottom><bottom_left><top_left><middle><dot>
   0xA4: update attributes1 
     payload = <value1> <value2>
     value1 = 
       0x70 for high display intensity ? (on)
       0x60 for low display intensity ? (standby)
     value2 =
       0x00 until now
   0x04: update attributes2
     payload = <value1> <value2>
     value1 =
       0x10 for power led red
       0x08 for power led green
       0x00 for power led off
     value2 =
       0xFF until now
 Notes:
   - the messages 54 and A4 always happen in bursts 
     and the bursts are repeated every 3 seconds at least
     (when not in standby), in the following order :
     <digit 1> <digit 2> <digit 3> <digit 4> <digit 1> <digit 2> <digit 3> <digit 4> <attributes1>
   - the messages 04 happen only in response to a key
     press. They are not repeated regularly.

Side-note about using an ESP8266 as SPI slave.

GPIO15 of the ESP must be low for the chip to boot normally, but it is also the "Slave Select" PIN. In SPI slave mode, we have no control on that pin which comes from the master and, more often than not, the GPIO15 pin is high upon ESP reset, preventing it from booting. So I used a little trick: a separate ESP pin (GPIO5) acts as a "SS enable", and only activates the actual SS in the setup() function, when the ESP has fully booted. The schema is as follows:

SS Enable schema

Monitoring.

The STB has a 3.5mm jack at the back that is used for maintenance and outputs serial debug information. Connecting it to the ESP's serial port was trivial altough I took care to route the signal away from the mains power area of the motherboard.

Serial routing

Here is a video of the complete hardware assembly:


Software.

The software consists of 3 parts:

1) Code running on the ESP.

That code was written using the Arduino environment and ESP8266 core (the latest stable version at the time of development was 2.3.0).

It consists of several parts:

  • The driver for the different hardware parts
  • A generic web server for serving static files stored on the internal filesystem (SPIFFS)
  • A specific URL exposing an API to emulate Remote Control keypresses. For example, calling /remote?keys=XT12^ emulates a keypress sequence of the keys eXit, Tv, 1, 2 and up arrow (^)
  • A first websocket server for streaming of front panel led updates
  • A second websocket server for streaming of serial debug logs
  • A real-time parser of the incoming logs to extract interesting information, such as the ID (DVB triplet) of the current channel

I also included the Wifimanager library, OTA, as well as a call to send the DHCP-assigned IP address to my PushBullet notification app via PushingBox (because PushBullet's API can only be accessed using HTTPS while PushingBox allows HTTP).

Issues.

I encountered several crashes during development, but most were due to the webserver, which only supports one call at a time and has poor performance (although ESP8266 core version 2.4.0 or later should be optimized in that regard). I resorted to storing the CSS and images on an external website, and inlining all the javascript inside the html pages, so that only one page is loaded at a time, and it's now quite stable.

The SPI slave was a challenge, particularly due to the lack of ESP8266 chip documentation. The ESP is incredibly powerful, but SPI slave interfacing was mostly a trial and error thing and I couldn't have made it without Me-no-dev's help. His code is now included in the ESP8266 core, but even then, advanced features rely on register configuration for which documentation is largely missing. In the end, I was lucky that the front panel protocol only used a few messages because the ESP seems (?) to only understand a protocol of the form "command-address-data" (used for memory and some LCD chips), and command can only be one of master_read / master_write / slave_read / slave_write. If you design the protocol, that's no problem, but if you are faced with a SPI transmission that doesn't follow the "command-address-data" format, I frankly doubt the ESP will be able to handle it in all cases.

I also had problems with the serial Rx buffer overflowing when the processor was too busy, but I patched the ESP8266 core with this change (which should be in core 2.4.0 or later) and it solved the issue.

2) HTML pages and javascript code.

There are:

  • Some static pages and links providing navigation between the features
  • A page presenting a picture of the remote control, with links mapped to various areas of the pictures, calling the RC API
  • A page presenting a picture of the front panel and opening a websocket connection to receive led updates. Coupled with a modified version of this Javascript 7-segment library to display changes in the UI.
  • A page presenting a scrolling log of the debug information received from the dedicated websocket
  • A page presenting the information parsed from the debug stream
  • A page with a search form which performs the search on the set-top box by navigating the on-screen keyboard

3) Tampermonkey/Greasemonkey scripts.

Those scripts are used to modify VOO's "VOOmotion" website on the fly, and to allow it to control the STB. Namely:

  • The home page was hacked to add a "setup" dialog window (e.g. to specify the IP of the Wahoocorder) and modify its VOOmotion logo to WAHOOmotion
  • In the program guide, when selecting details of a program currently on air, a "Watch on Wahoocorder" button is added to allow a direct zapping to that channel
  • In the program guide, when selecting details of a future program, a "Schedule on Wahoocorder" button is added to allow scheduling the record of that program by controlling the "manual recording" screen of the STB (including emulation of keypresses in SMS "multi-tap" style to enter program title)
  • When activating "follow zapping" in the setup dialog, the site hooks to the front panel incoming stream via websocket and pops up the details of the program currently watched on television.

Note that using a browser for both search and manual recording circumvent the tedious process of entering text using a remote with no alphabetic keys. Particularly frustrating is that those two features use a different convention (virtual on-screen keyboard for search, multi-tap for manual recordings). Forget about those as required remote keypresses will be performed for you to fill the fields ;-)

Conclusion.

This hack was made just for fun, but I was amazed at what can be performed with an outdated device by adding less than $5 in hardware.

The ESP8266 really deserves its place as the IoT king. I was stunned by the fact it could handle 3 distinct web servers while parsing two message streams without a hitch. If its documentation was more complete, no doubt it would step from IoT king to IoT emperor :-)

lundi 19 décembre 2016

A Dirt Cheap F*** Awesome Interactive Led Table



Space invaders

tl;dr

What ?

  • led coffee table
  • touch-sensitive

What's special about it ?

  • IR detection through tranclucent acrylic
  • 3 devices connected to a Raspberry Pi via a single serial port
  • Java program hacked without modifying the actual code
  • 130 EUR (140 USD)

It's finally time I document this project I completed almost one year ago.

I wanted to play with large led displays for a long time (who doesn't ?), and browsing on AliExpress, I once landed on one of those WS2811/WS2812 addressable led strips for less than 3 EUR/meter, and thought "Wow, that's dirt cheap !".
I bought one meter just to play with it, and was surprised to see how fun it was (easy to control and quite powerful), and I started to think about wiring them in a matrix shape. Most of the libraries already support custom width and height, so that's pretty cool.

A few weeks later, I came across that large led matrix by GreatScott and I thought "F*** Awesome !", and following hours of youtube suggestions, I stumbled upon this video by yohash84 and said to myself "interactivity, mmmhhh".

And so the idea was born: I want a Dirt Cheap, F*** Awesome, Interactive Led Table.


First Things First: Non-interactive Version

After a proof of concept with just my 1m long strip, cells delimited with carboard and a thin tracing paper diffuser, I was rather confident that using a strip was a simple and effective solution, and set a non-interactive table as my first goal.


Hardware

I started with IKEA's hacker-friendly Lack table. Its honeycomb cardboard structure makes it really easy to get a hollow table.

IKEA_LACK_2016-08-12_1310.png 20151023_171049.jpg 20151023_171709.jpg 20151023_172704.jpg

So first some calculations: I'll be using a strip with a density of 30 led/m, which leads to 3.33x3.33cm per matrix cell. The LACK table is 55x55cm, but the matrix cannot go from edge to edge because each corner of the table contains a bloc of particle wood (see picture above) to which the foot is attached. The frame is about 1cm and blocks are about 4x4cm each but I figured I could chop the inner corners of the blocks to maximise surface, so I settled on a matrix of 14x14 cells (or 46.67x46.67cm). The ~4 cm left around the matrix will be used to hide connections and will help the table remain sturdy. So here's the "final cut":

Final cut

OK, with these numbers, the absolute minimum bill of material is:

  • Lack table: 6 EUR
  • Led strip 7m, 30 led/m (which makes 210. I need 14x14=196): 19 EUR
  • Power Supply : 5V 20A (196 x 3 x 20mA = 12A required for the leds alone): 14 EUR
  • Controller: Arduino Nano clone: 2 EUR
  • Translucent acrylic plate: This is the hardest part to find cheap :-(. I ended up buying it locally but I it's available online for 26.5 EUR, including s/h

The rest (remains of white paint, cardboard, wires, etc.) I had laying around. Less than 70 EUR qualifies for "cheap", and if you can find or recycle a power supply or an acrylic plate, you qualify for "dirt cheap" :-) .

To create cell separations, I used thin corrugated cardboard but quickly found that cutting rectangular noches with regular spacing and depth was a very tedious task, so I piled up all the cardboard strips, stuck them between two old wooden planks and let my circular saw do the job...

20151205_185824.jpg 20151205_185925.jpg

Note that I first painted the cardboard, as well as the inside of the table and 4 small pieces of wood for the 4 inner sides in white. That way, inner reflections will increase brightness and level out the intensity of each cell. I should have taken more care though, because the veneer wood layer of the LACK table is so thin that my masking tape pulled it out at places :-( . I guess you get what you pay for.

Inside

I then pasted 14-led segments of the the self-adhesive strips to the bottom of the table, and connected the signal in series from line to line (in a zig-zag fashion), while the power for each segment is distributed from a central point to avoid the voltage drop across the 7-meter strip.

Pasting strip 20151205_191240.jpg

The result with the acrylic plate (protection sheet not fully removed yet) is quite satisfactory :


Software

Arduino Nano

The animation above is simply the output of the XYMatrix example which is part of the FastLed library - highly recommended !

Raspberry Pi

For more complex animations, and to handle the interactive part, small microcontrollers would probably be very limiting, so I switched to a Raspberry Pi as the main controller.

Without any doubt, the king of diy led matrix animation software is Glediator. On the plus side, it's a really neat piece of software which allows highly customised animations and supports many matrix types and configurations, and it's free :-) . It is written in Java, which is a surprising decision for a software where performance is important, but it runs relatively well on the Raspberry Pi. However, the main drawback is that it's not open source...

In that configuration, the Arduino Nano is used as a converter from the Pi's serial signal to the WS2812 protocol. That is required because the Glediator running on Linux can suffer inaccurate timings due to the non-realtime OS, the Java VM garbage collection, etc., while the WS2812 leds requires a very precisely timed data stream.

I had a Raspberry Pi 2 laying around so I used that but probably a Raspberry Pi Zero or an Orange Pi could do the job for less. Anyway, let's say 33 EUR for a RPi 3 to be futureproof.

So here are the steps I took to set up the system:

  • Install VNC on Pi (see here).
  • Free the serial port of the Pi (used for debug and as a shell by default in the distribution) using raspi-config in recent distributions - see here.
  • Install RxTx Java serial lib on Pi using "sudo apt-get install librxtx-java"
  • Unzip Glediator on the Pi and create a startup script in the "dist" folder as follows:

#!/bin/sh
CLASSPATH=/usr/share/java/RxTxcomm.jar
LD_LIBRARY_PATH=/usr/lib/jni
java -Djava.library.path=/usr/lib/jni -Dgnu.io.rxtx.SerialPorts=/dev/ttyAMA0 -jar Glediator_V2.jar

  • For the Arduino Nano, use the Arduino sketch provided on Glediator's website.
  • Start Glediator and configure matrix size (14x14) and pattern (in my case VS_BL for Vertical Snake starting at Bottom Left) "Output" in Glediator protocol in GRB with the serial port on 115200 bauds.

A note about baudrate: Most websites talk about a baud rate of 1000000 or 500000 bps, but with my Raspberry Pi 2, I could not achieve that bitrate at first and could not get anything to work until I reduced speed to 115200 bps as indicated above. However, the Nano (which has a 16MHz clock) is able to go much higher.

After more search, I discovered that the limitation to 115200 was on the Raspberry Pi side. The solution to go beyond 115200 was found on this post. So, to get 1000000 bps, I added the following line to the /boot/config.txt of the Raspberry Pi:
init_uart_clock=16000000
(and reboot of course).

Here is the result. Awesome :-) :




And Now for Something Completely Different: Interactivity!

Hardware

It seems quite clear that infrared reflection is the cheapest way to detect a presence at small distance. The video by yohash84, already mentioned, is a very good demonstration of how IR can be used. However, there are a few highlights in this project:

  • it has to work through translucent acrylic
  • use 1 IR detector per visible led (yohash84 has 1 detector per cell of 4 or 6 visible leds)
  • favour a circuit simpler than this :-) , if possible (no offense)
  • oh, and keep it dirt cheap of course

Basically, that last point made me choose the cheapest emitting ang receiving IR diodes I could find, which are these (1.92EUR/100pc) and these (2.27EUR/100pc). As I need 200 of each, that adds 8.5EUR to the bill.

Of course, at that price point, one shouldn't expect to find a datasheet, but adding a resistor in series with a reverse-polarized receiving diode (more exactly a photodiode, or according to the measures probably a phototransistor, but let's just call it receiving diode for simplicity :-)) and measuring the voltage drop gives an image of the incoming IR. The basic circtuit is thus:

Basic schema

The hardest point was to achieve reliable IR detection through translucent acrylic. My preliminary tests showed that many parameters influence the measured IR level:

  • direct illumination: the risk is that the receiving led gets blinded by direct flow from the emitting led
  • external lighting: natural (sun) or artificial (TL) lighting generate much IR, with the unfortunate drawback that it acts "against" the expected IR reflection (the IR flow is reduced by objects that cast a shadow while the reflection against these objects should increase the flow)
  • internal reflections: this is the obvious enemy when putting a plastic plate in front of the leds, and particularly as it is translucent and not transparent
  • component dispersion and building imprecisions

All these elements made accurate detection a real challenge. Here are the strategies I put in place to solve it:

  • limit direct illumination by putting small tubes around the leds. These are made from black cocktail straws (add 2 EUR for 2x40pcs of 12cm, which is enough for 196x2x2cm).
  • eliminate influence of external IR sources by using a differencial mode: measure IR illumination when IR source is off (Ioff) and when IR is source is on (Ion), and consider that reflection is Ion-Ioff. This is not really accurate because the observed voltage drop is not proportional with the illumination, but it gives satisfying results nonetheless
  • reduce the "mirror" effect of the acrylic plate by not putting emitting and receiving diodes at opposite corners, but on two adjacent corners of the bottom, and make them aim at the center of the top surface. Diffusing objects such as paper or hand should reflect much IR while mirror reflection of the emitted IR light against the top plate should fall in the opposite corner and not on the receiving led (see geometric model below). Also, a calibration phase is applied to measure the received light when nothing is on the table vs when a full white surface (paper) is on layed upon the table.
  • compensate difference in component dispersion and led positions by making the calibration independant per cell.

Here is a geometric model of one cell:

annotated_cell_model.png

As calibration must be made cell per cell, and with both low and high thresholds, using comparators and potentiometers was out of question. Calibration is performed in software at the very end (on the Raspberry Pi) and all measurements are achieved by the Arduino Nano using A/D inputs (note that it is a separate Arduino Nano from the one driving the led strip, which has very strict timing constraints. Here is how I came up with the final circuit, step by step:

Let's start with just one cell :

1 cell

OK, now as we don't have 196 A/D input, we have to multiplex several cells per input, like so:

2 cells A

But if the anodes of receiving diodes are all connected to GND, then the measure will combine all cells, so we use a GPIO per "column" instead of GND. When low, it acts as GND and "selects" one column (one cell in this case):

2 cells B 1

However, the "non-selected" columns will cause a problem. If we set their GPIOs to VCC, it will "short circuit" the A/D through the (now forward polarized) IR receiving diode. If we just leave them in high impedance (turning them to inputs), there is still a path that would "leak" through the IR emitter led. Both issues are illustrated below:

2 cells issue 1 2 cells issue 2

The solution is to add a diode in series with each IR receiver, polarized the opposite way, like so:

2 leds C

We need to be able to switch the emitting led on or off and compare the received IR intensity. To do so, as the "GND" of each cell is common to the emitting and receiving leds, the anode of the emitting led also has to be connected to a GPIO instead of VCC. When low, nothing will be lit, but when high, one emitting led (the one that has an both active "VCC" and active "GND") will turn on:

2 leds D

Now we can duplicate this structure for each row of the matrix. The simple solution using a single GPIO for all leds (at the cost of a transistor to increase the current) looks like this:

4 cells A

In that configuration, all leds of the same column having both VCC and GND active at the same time light up together, resulting in cross-cell illumination. Switching to one "VCC" GPIO per row avoids this issue by only activating one cell at a time. So the final model is the following:

4 cells B

Here is a video showing the proof-of-concept of this circuit in action:


Note that after more testing, in the final version, I slightly changed the resistor values and settled on 330 ohm for limiting resistors on the IR emitters and 38.3K (because that's the one slightly lower than 47K I had in stock) in series with the IR receivers.

Now we need to expand that to 14 rows by 14 cells, but of course, the number of A/D inputs is limited (8 per Arduino Nano) so two Nanos are used to perform measurements on all 14 lines. For the GPIOs, each Nano drives the "programmable VCC" for 7 rows. For "GNDs" however, we don't have enough I/Os for controlling each of them. The solution is to add an I/O expander controlled by the Nanos with fewer pins. I selected the MCP23017, which can be found for less than 1 EUR incuding s/h (add 5 EUR to the bill, 2 per Nano and 1 for the expander).

Now on to the real thing: Get the soldering iron and wire stripper!
To bend led legs precisely and assemble each of the 14 columns, I made a small jig reusing one of the planks that helped cut the notches in cardboard separators. One stripped wire for the "GND" is stretched between two screws and the angles and positions for the leds are drawn on the wood. One leg of each led (the correct one :-) ) is then bent and soldered to the wire. When all are soldered, the other led legs are bent up by the same angle and the diode preventing forward current through the IR receivers (see above) is soldered too:

Mounting animation One mounted row 4 lines

Then when all 14 columns are ready, two wires have to run perpendicularly for each line (one for "VCC" and one for measurements). Unfortunately these will cross the RGB led strips, so to avoid all risk of short circuit, they are only partially stripped, making a beautiful (and extremely boring to make) dotted pattern (grey and orange are for the 7 rows linked to the first Nano, brown and purple are for the second one) :

partially stripped wires soldering in progress fully soldered

... and with straws:

Cut straws Swtraws

Pretty cool, huh :-) ?

Controlling the GPIOs is easy, but there are two issues :

  • synchronization: the I/O expander for GNDs can only be driven by one I2C master, but two Arduino Nanos will be cycling the VCCs, and they have to perform the measurements in sync.
  • communication: the Raspberry Pi has a single serial port (and I'd rather have the data coming from a single source anyway). Note that to double the scan frequency, I wanted to run the two Nanos in parallel, so the two halves of the table are scanned simultaneously. The cross illumination between cells 7 rows apart is minimal.

Both issues were solved at once by multiplexing the Tx signals and having the Nanos "spy" on each other to speak in turn. One of Nanos is the "master" and sends its data first. The other Nano is the "slave", it listens to the master and then sends its data in turn. During that phase, the master listens to the slave, and when it's done sending, the next row is processed. When all rows have been processed, the loop restarts. The multiplexing of the Tx is done using a two diodes and a pull-up to 3.3V (remember the RPi logic levels are not 5V tolerant), creating an "or" gate, as follows:

Nano logic

For simplicity, both Arduino Nanos are identical both in software and hardware, except that pin 2 selects the behaviour (master or slave) depending if it's high or low. Each of them has 4 connectors :

  • a 3-pin header to the Raspberry Pi (GND, 5V, Tx) - any can be used. The Tx just requires a pull-up to 3.3V on the RPi side
  • a 4-pin header to the I/O expander (GND, 5V, SDA, SCL) - any can be used
  • a 4-pin header to the each other (GND, 5V, Rx, Tx). The cable crosses Rx and Tx
  • a 2x7-pin header to the led matrix (7x "VCC" and 7x measure)

Here are the schematic and a few pictures (with the Nano removed first, and then fully integrated):

Nano schema

Nano top Nano bottoms Nano integrated

The I/O expander is on its own perfboard, but it's really straightforward, with its 4-pin connector and a 2x7-pin header to the 14 "GND" lines. Here's what it looks like:

IO Expander IO Expander connected

As you can see, I used flat cables with 2x7 pin connectors to distribute signals for rows and columns. Add 4 EURs for those.

So here is the fully cabled IR matrix:


And here is the general schema:

Block Schema

Software

Arduino Nanos

Each Nano has an outer loop that counts to 14 and sets one of the "GND GPIO" low in turn, while the other 13 are high. Inside this loop, the nano basically performs 2x7 measurements in a loop which goes:

  • measure row <n> (ambiant level)
  • set "VCC GPIO" high on row <n>
  • measure row <n> again (reflection level)
  • set "VCC GPIO" low on row <n>

Then

  • it compute the delta (reflection - ambiant)
  • if the Nano is master, it sends a "start of message" byte, then the column number (0-13), then the 7 deltas, then listens and waits until it gets a "end of message" emitted by the slave
  • if the Nano is slave, it listens until it gets a "start of message", counts the 7 deltas emitted the master, then sends its own 7 delta plus a "end of message" byte

Then the Nano starts again with GND active on the next column.

One interesting thing to note is that the only sync between the two Nanos is the serial message. Even though one of them actually drives the I/O expander, the fact that they run the same code guarantees a high precision in timings as they run in parallel. The only time when code differs (the sending phase), I added a delay (measured by experience) on the slave to accomodate the time it takes the master to receive and detect the "end of message" byte.

Fun fact: one might wonder how we can be sure the Nanos are kept in sync, particularly regarding column numbers. Indeed, in case there is a temporary loss of connection, or one of the Nanos restarts, or after any condition happens that requires a resync, we have no guarantee that the outer loop will be at the same column number in both Nanos. Surprisingly, that doesn't really matter as long as the master is the one controlling the I/O expander. Indeed the only thing that matters is that the number sent by the master in the message is the number of the actual column powered by the expander. As the slave does not write the column number in the message, if it doesn't drive the expander either, the column number the slave "thinks" it is on does not matter : it just performs 2 measurements per row and returns their difference.

One more thing regarding the Arduino Nano: the Library to control the I/O expander was found in this thread.

Here is the sketch I used.

Raspberry Pi

In Glediator, a Generator is the primary class that renders contents. Generators can then be combined together to create Scenes, which can then in turn be combined to create PlayLists.

The basic idea is to create a Generator that takes the serial input from the Nanos, calibrates their values, and then renders them as a greyscale level on a matrix. After that, one can for example combine that matrix with a Rainbow Generator with a "multiply" operation, so that output is dark where no obstacle is "seen" by the IR, and rainbow-colored where an obstacle is detected.

Unfortunately, Glediator is not open source as I said, and has seen no evolution in the last two years. However, The great advantage of Java for hackers is that it is easy to change the behaviour of a Java program by overriding classes. So I quicky analyzed Glediator's classes to see what could be done, and I came up with a way to add new "generators" (much like extensions are added to browsers) by just giving replacement classes higher priority than the original ones in the classpath, and leaving the original Glediator distribution untouched.

I tried contacting the authors - SolderLed - several times to propose them to include those changes but got no reply. So I guess contributions are not welcome, or they just leave the project as abandonware :-(. In that case I hope SolderLed won't mind if I share the changes here.

For those interested, here is the code I sent them as a proof-of-concept. It contains the original Glediator v2.0.3 + a patch that overrides 4 Glediator classes to recognize "pluggable" generators + two new sample Generators: the first is a dummy "Uniform" color generator which is similar to the "Black" generator but with any color, and the second one is a "Game of Life", just because... you know.

Here's a demo with those two :


Next step: The Glediator generator reflecting exactly the values received from the IR matrix as a greyscale value (ok, there is mirroring mistake which was fixed later on). As you can see, it worked pretty well first time, but note that this is the ideal case, without white cardboard separators and translucent acrylic. Keep an eye on the screen at the right:


Note how the different cells have different values when nothing is in front of the leds. This is partly due to component dispersion, but also to the imprécisions in my build regarding led orientation. If they emitter and receiver are more or less "in front" of each other, the quantity of received IR can greatly vary, hence the calibration phase. The difference is made even less visible once the separators, and more importantly the translucent acrilyc plate are in place, so we need to amplify that difference, and that is the goal of the calibration phase too.

That calibration is thus done by determining low and high thresholds for each cell, and performing a linear interpolation in between, as follows:

Calibration logic

So the generator is first put in calibration mode, where it records the extreme "high" and "low" values observed for each cell, and stores them in two arrays. When exiting calibration mode, all incoming values will be "scaled" according to those extreme values. To check how it behaves, I developed the following UI:


With this approach, results are much better, even with the full separators and top plate:


However, this greyscale version doesn't give good results when combined with other effects using a "multiply" operation, because calibration is made with white paper laying flat on the table, which produces much higher reflection than average skin waved at the surface, resulting in a very dim image. As a first measure, a "threshold percentage" was added to convert this greyscale to binary ("black or white") values. Of course, while this percentage is a general setting, the actual value of the threshold varies from cell to cell according to the calibration. For example, a 30% threshold would be computed as follows on two cells:

Calibration with threshold

This is now much better:


As a last improvement, I replaced the unique threshold by a narrow linear interpolation from a low threshold to a high threshold. For example between 20 and 40%:

Calibration with 2 thresholds

And the results are really cool :-) :



Final bill

  • Non-interactive version (see above): 70 EUR
  • Raspberry Pi 3: 33 EUR
  • IR leds: 8.5 EUR
  • Straws: 2 EUR
  • 2 more Arduino Nano clones: 4 EUR
  • MCP23017 I/O expander: 1 EUR
  • Flat cables with 2x7 pin connectors: 4 EUR
  • Cables, soldering, resistors, perfboard, etc.: say 5-10 EUR

Total: 130 EUR (140 USD), or even less if you can recycle a power supply, a Raspberry Pi or a translucent acrylic plate.

For all the stuff there is in that table, I really think that's dirt cheap.

Final demo

pixel matrix + interactivity = ?

Tetris of course.


I say "Fucking awesome !"

Thanks for reading :-)