• Potential Hardware Delay, Software Making Progress

    Monday, February 10, 2014

    2/102014— Updates

    It’s been snowing like crazy here in Boston, but we’re warm, indoors, and hacking away on Tessel and its modules.

    Last Friday we had one of the roughest days of our company’s relatively short existence. We discovered a bug in our hardware design just days before we were going to give our assembly house the go-ahead to start assembling all 1450 Tessel PCBs that were manufactured. The bug caused Tessel to hang each time it was plugged into USB, necessitating a push of the reset button to be able to use it.

    A Bug on the Hardware

    The tough part for us as engineers, was that we had known about this bug since July, but had diagnosed it as a minor “software issue” during several independent investigations. The root cause is a race condition between the bootup of the NXP chip (the main microcontroller) and the external flash chip. Our hardware debuggers attach to the reset pin of the Tessel, so we were changing the behavior during debugging.

    The microcontroller powers up around 100 to 200 microseconds before the flash chip, which contains the firmware. The NXP boot code (in read-only memory) checks for the firmware in flash, doesn’t find it, then only checks again 60 seconds later. We’re not the only crowdfunding team to hit NXP’s bug, and it’s not something we’re comfortable shipping out to our users.

    There is not an insurmountable problem, it’s just a matter of reconciling the consequences. Here are our options:

    Solutions and Consequences

    1. NXP has documented that they fixed the “60 second bug” on their recently manufactured Cortex M4 series chips. We’re currently trying to get in touch with someone at NXP who knows if it was fixed for the M3 series as well. If this turns out to be true, we’ll just need to make sure we have those newer chips, then we can still use the same PCB. This is the most ideal situation as our shipping schedule will likely stay the same. That being said, the current chips we have were all manufactured in the last weeks of 2013, so it is unlikely there would be newer chips with this bug fixed.

    2. We could add a reset controller to our main board, which will allow us to delay the propagation of the reset signal to the microcontroller, thereby eliminating the race condition. This will require another board revision, but allows for minimal changes to Tessel’s functionality and has the silver lining of allowing us to make a handful of incremental improvements to Tessel including making USB more robust and picking a better voltage regulator. This our most likely solution and will push our shipping schedule back to late March or early April.

    We’re still learning how to create a sustainable hardware company, and this issue was a real eye opener in terms of much more time and energy we need to dedicate towards testing prior to getting products ready for shipping. We have a lot of pressure on us to ship to you, our backers, as close to the date we expected as possible, but quality is more important to us than schedule.

    While we honestly can’t think of anything we could have done differently to avoid this specific bug (besides fix every existing software and hardware issue), we’re taking several steps to ensure the quality of our hardware and software in the future. Specifically, we’ve set up a continuous integration rig to automatically run any new firmware on every version of Tessel prior to consolidating into the rest of our code base. We’ll soon be setting up continuous integration rigs for all of our modules as well.


    But we do also have good news! We’ve been making progress on several items recently:

    • We’re able to write JS programs to flash so that they will run as soon as Tessel is powered up.
    • We’ve fixed over 50 firmware bugs reported by our beta backers.
    • We’ve been interviewing folks for various positions within the company (want to be an intern or a full-time employee? We’re looking for engineers and designers.).
    • We’ve finished the Infrared Module hardware, driver, firmware, and testbench.
    • We’ve done a major overhaul of the Audio Module. Here’s the latest prototype:

    We now support headphones and line-out and, more importantly, the module can now record audio through an onboard microphone or a line-in jack.

    We’ll keep you updated over email as things progress.


    Kelsey, Jia, Eric, Tim, Kevin, and Jon

    #Tessel #Update #Technical Machine #NXP #bug #startup #setback #progress #hardware #hardware is hard

August 2018

January 2018

July 2017

February 2017

November 2016

October 2016

September 2016

August 2016

July 2016

June 2016

April 2016

March 2016

February 2016

November 2015

September 2015

August 2015

July 2015

June 2015

May 2015

March 2015

February 2015

January 2015

December 2014

November 2014

October 2014

September 2014

August 2014

July 2014

June 2014

May 2014

April 2014

March 2014

February 2014

January 2014

December 2013

November 2013

October 2013

September 2013

August 2013

July 2013