June 24, 2019

DonkeyCar Self-Driving Challenge -- Reflection

When we were presented with the self-driving car challenge, we did not exactly understand the work ahead of us. We did have – as most people do – a vague conception of what would happen. We knew we would build a modified RC racecar, placing a small computer, camera, and extra battery onboard, and “teach” the car to drive on a race course. If a sufficiently sophisticated device is indistinguishable from magic, we were at that time in the unlucky position of magicians untrained in a trick.

Immediately, I recall two behaviors emerging. Those confident in a particular task set directly to work. And those without understanding of a task took an evangelist stance – “of course AI operates like I’d expect.” I certainly felt a participant in the second set of behaviors as I knew little by way of hardware yet knew no less than anyone else by way of machine learning. The team members who did have skill in hardware set directly to into building the physical racecar. They did an expert job. Only an hour into the competition, both racecars were ready for driving – with a human at the controller.

The next steps show us our shortcomings. The software supplied for the onboard computer took some time to acquire, verify, and then flash onto the car’s memory. Then the instructions were somewhat confusing, with a few exceptions and edge-cases written onto the online documentation. Other pertinent notes were missing. Crucially, the core piece of software – which would be the mechanism for allowing our cars to “learn” how to drive – proved flawed. Both teams would spend an extra 2-3 calendar weeks indentifying and testing a version of the software that would enable us to “train” the vehicles.

I think during this middle period, we humans learned the most. We learned the difference between reinforcement learning ( where a robot would attempt a task taking trial-and-error to discover and reward successful behaviors ) and supervised learning ( where a robot would attempt to reproduce data shown to it and use the closest matching behavior to guide itself in real life). None of the participants had much more than a cursory level of understanding before this competition; after these weeks we started to teach ourselves and each other well enough to the difference to focus our efforts in training out vehicles.

Another interesting thing happened during this phase. We started sharing knowledge between teams and individuals despite being engaged in a competition. On some level, this shouldn’t be surprising at all. Athletes train, practice, and support each other with the goal of getting better collectively. Sportsmanship, in its pure form, is a non-zero sum game. I was most happy seeing this sportsmanship on display during our competition than with any technical achievements we made.

Chris Grahn found the correct version of software that would work on our cars. I found a away to use the Sagemaker service to train our supervised learning models in an automated fashion. We traded our findings, ultimately taking them in different directions, and enabled both teams to move forward.

The competition was not without setbacks. On the red team – my team – a training accident required rebuilding the front wheels and frame of our vehicle. Again, our hardware experts jumped in to perform a speedy, successful repair. On the blue team, the vehicle burnt out a servo ( as a non-hardware person, I don’t exactly know what a “servo” is). Unfortunately, this occurred too close to the final event to perform a repair. The damage to the servo made the self-driving algorithm useless, and the team ended up forfieting the competition.

I want to close out this reflection by reiterating the shout-out message I gave to red team. These are the the contributions our individual team members gave through the entire project:

  • Scott Evtuch was game day MVP in my mind; he trained our winning models at the last-hour, using techniques he and others developed throughout the competition
  • Will Hausman and Jay Jani drove some killer training sessions; these runs gave us confidence and data to teach ourselves how to work the car
  • Chris Haun led the initial assembly as well as an emergency disassembly and repair of our front wheels; obviously, the race could not have proceeded without this very quick work
  • Merim Abdyldaeva installed our Rasbian operating system, performing a reinstall later on when we realized I messed things up.
  • Aga Kielbus and Merim stood up our first Sagemaker instances; these notebooks gave us all early access from to try out the software
  • Alex Brooke installed the donkeycar software… a few times and helped troubleshoot some of the gotchas of the library
  • Wayne Geils, George Nelson, and the rest of the team for coming out to support each other from day 1, build the vehicle, and cheer it on
  • Jeff Mauricio took great footage of the work and the race; he’ll be producing a marketing video showcasing our work
  • last but not least, Chris Grahn, Blue Team MVP, for sharing a working fork of the donkeycar software; if it weren’t for some bad luck, Blue Team may have won, and if it weren’t for sportsmanship, we may have lost anyways!

Content by © Jared Davis 2019-2020

Powered by Hugo & Kiss.