Last August I found out about Formula Pi Kickstarter campaign by @pi_borg: Self driving robot cars programmed in a Raspberry Pi (Pi Zero). I liked the idea so I backed the project and asked friends to join. @hombrenieve1983 was the most enthusiast: “I’m in!”.

Our team was called RasPerras del Infierno, as a reference to our programming contests team. Each team could draw the top part of the robot and send it back to identify each robot in the races. Here’s our team logo:

In Formula Pi the robots use the camera information to drive around the track. You get a base code that you can modify as you like to develop your own strategy for the races. It can be from as easy as “follow the center lane” to as complex as you want. There’s a simulator to test the code and there were a few testing rounds before the real races started. All races are streamed live in twitch, and they are fun to watch even when your team is not running that day.

Our idea was to use machine learning to drive the car. We are writing here about the full process during the competition, but we found it harder than we expected to get a machine learning version running in the Pi Zero.

To train the machine learning we needed to generate many images and tell what we would want the car to do in each case. To do that we integrated the race code with the wiimote (cwiid) and played in the simulator: This way we stored the simulated camera images and the position of the wiimote as desired driving direction. It was even fun!

When we had enough images we started training a scikit-learn neural network. We started our tests with a fully connected neural network with one hidden layer. We did some changes to the wiimote driving code to train it to recover when it went out of the ideal path in the track. We run our code and the simulator in our laptop and it looked promising 🙂 We sent that for the first tests but some issues with the library, not compiled for the Raspberry Pi Zero made our robot stay in the starting line 🙁

In the first race we had the library properly installed and so the neural network was going to drive the robot. This time the robot moved but our robot kept crashing against the walls, and in the recover movements it managed to do one full lap… in reverse! At least there was one other robot that did 5 laps in reverse :p

From the logs and images we got, our theory is that the Pi Zero took too long for processing each image with our code. This means it applied the driving directions to late and during too long. If it decided to turn, then turning for a full second meant driving into the walls.

For the next races we changed back to a simpler racing code: Follow the lane one right from the center.

In race 2 we did better (it’s easy to do it better than -1 laps 😉 This time we managed to get nearly 10 laps for a second place in the heat, but that only gave us 1 point. First point though, yeah!

We kept testing and training neural networks, but we couldn’t get some of them to run in a Raspberry (after the issues in the test rounds we decided to test everything in a Raspberry Pi). Some libraries didn’t even run in the raspberry, and some other tests were too slow to properly drive the robot. Running some tests at lower robot speed, it looked really promising, but when driving full speed as you want to do in a race we didn’t find a working solution that didn’t crash into the walls.

We also tried convolutional neural networks. With neupy and a convolutional neural network that would fit in the Pi Zero it got many images right, but some others were giving wrong values and the robot would crash in the simulator.

In Race 3 we did even better! 16 laps and just a couple of meters away of house robot (which was surprisingly stopped). Our robot even passed the house robot a few seconds after the 15 minutes finished. 14 more points for RasPerras del Infierno 🙂

Race 4 was even better: First win for RasPerras del Infierno!! 22 laps even though we were stuck with other robot for a couple of minutes.

Race 5 start was very good but we crashed on a robot that went backwards for some time. Then there was a good competition and two other robots went over the 23 laps limit. A third place in the race, 11 points and one of the 10 places into the semifinal! (good luck the house robots are not going into the semifinal).

We are looking forward to the final rounds of this season! Who will win? What position will RasPerras del Infierno end?

Sign us up for the next season!