• High-Level Automated Hardware Debugging

    Wednesday, April 23, 2014

    4/23/2014— Tim Ryan

    The CC3000 is the on-board component that enables the Tessel to connect to WiFi. This $12 chip communicates over SPI, has a built-in TCP/IP stack, and comes with a host library that enables using it indistinguishably with the BSD sockets API.

    For example: Tessel, while not running Linux, can easily drop in a library like c-ares (used to look up the IP address correlated with a domain name) because we’re using the same API used to make TCP/UDP queries on the desktop.

    A chip this capable is naturally complicated, and so is Tessel’s firmware. Debugging can occasionally be much more difficult than throwing open a GDB console and stepping through code. Is the state of the CC3000 chip known by the code? Was an incorrect command sent most recently, or a thousand commands ago? Is this TCP command originating through the JavaScript VM or directly from third-party code?

    Instead of debugging all of these subsystems simultaneously, it’s more straightforward to just analyze a) what commands the Tessel sends and b) what commands the CC3000 sends back. Using these exposed pins on the Tessel board and a logic analyzer, we can record the SPI communication emitted by any Tessel code and dissect exactly what went on.

    Tessel exposes breakout SPI pins for the CC3000.

    We have several of the Saleae Logic line of logic analyzers around our office. The Logic and its larger brother the Logic 16 can record high-speed signals, for example the 4MHz SPI communication to the CC3000, as well as I2C, UART, or USB signals, then display them in a zoomable graph.

    The startup SPI sequence of the CC3000.

    The interface for the Logic analyzer is nice for being able to visualize erroneous communications, but we aren’t much closer to understanding what is happening on the signal level. In particular:

    1. The Logic analyzer has helpful visual tools to see what bytes are actually being sent over the protocol, but not what groups of bytes mean.
    2. We have to specify an upper bound for the number of samples we want to take, rather than continuously logging.
    3. The options to export data have to be configured each time.
    4. We can’t use the command line.

    Luckily, the open source world of reverse engineering is alive and well!

    Sigrok

    Sigrok is an open-source library with a collection of drivers for various open-source projects. In our case, Sigrok supports a family of logic analyzers using the FX2 chip, including the Logic 8. Using brew install sigrok or any version 0.4.0 or newer from your repository, we can communicate with our analyzer directly from the command line.

    sh sigrok-cli \ --driver fx2lafw \ --continuous \ --config samplerate=8M \ --output-format vcd \ --output-file /tmp/sigrok.vcd \ --probes 0=cs,1=miso,2=irq,3=mosi,4=sck,5=sw_en

    We’re using the fx2 driver, with a sample rate of 8MHz (twice the speed our CC3000 runs at), outputting in vcd format to /tmp/sigrok.vcd, with probes 0-5 (named according to how they are named in the first diagram).

    The command will begin logging data from the Logic analyzer in perpetuity using the --continuous option, as long as the USB drivers can keep up*.

    Now that we’re logging to /tmp/sigrok.vcd, we can throw together a few Node modules to utilize it:

    • tail-stream creates a stream of a file that is actively being written, outputting more data whenever the file size grows.
    • vcd parses a textual value-change dump file, turning it into a series of “sample” events.
    • cc3000-log is our own script for interpreting the signals of the CC3000, interpreting them as SPI and generating a human-readable list of communication:

    The interpreted output of the CC3000 communication. Support for interpreting the payload bytes on the right is partially implemented.

    Instead of adding extensive debug facilities to many locations in our code (slowing it down and impacting the operation of the CPU), we can now be a silent observer: tapping the communication of the Tessel and the CC3000 and viewing in real-time how the IC is being controlled. This technique doesn’t just apply to the CC3000, but for any module or IC that the Tessel communicates with!

    TLDR: Sigrok is an open source library for communicating with commercial logic analyzers. Using it, you can make powerful, customized tools to debug ICs and signals from your command line, a welcome addition to automated testing.

    We highly recommend and use the Saleae Logic. But if you’re looking for something much cheaper (in price and quality) or throwaway devices to augment automated hardware testing, there are a variety of clones from foreign markets also.


    * 8Mhz samples are a lot; originally I was trying to set --output-format as binary or hex, which would inevitably cause the device to stop responding. It wasn’t bounded by my USB driver, it was bounded by the I/O speed of piping that many bytes to a file! The VCD format only outputs when values have changed, creating a drastically smaller log. In the future, we would bind directly to libsigrok for even faster throughput, but first hack first.

    #tim ryan #wifi #cc3000 #hardware #reverse engineering #open source #tessel #technical machine #logic #saleae logic #saleae #SPI #tcp #ip #sigrok #debugging #high level #automated debugging

February 2017

November 2016

October 2016

September 2016

August 2016

July 2016

June 2016

April 2016

March 2016

February 2016

November 2015

September 2015

August 2015

July 2015

June 2015

May 2015

March 2015

February 2015

January 2015

December 2014

November 2014

October 2014

September 2014

August 2014

July 2014

June 2014

May 2014

April 2014

March 2014

February 2014

January 2014

December 2013

November 2013

October 2013

September 2013

August 2013

July 2013