Interview With [Damien George], Creator of the Micro Python Project

[Damien George] just created Micro Python (Kickstarter alert!), a lean and fast implementation of the Python scripting language that is optimized to run on a microcontroller.

It includes a complete parser, compiler, virtual machine, runtime system, garbage collector and was written from scratch. Micro Python currently supports 32-bit ARM processors like the STM32F405 (168MHz Cortex-M4, 1MB flash, 192KB ram) shown in the picture above and will be open source once the already successful campaign finishes. Running your python program is as simple as copying your file to the platform (detected as a mass storage device) and rebooting it. The official micro python board includes a micro SD card slot, 4 LEDs, a switch, a real-time clock, an accelerometer and has plenty of I/O pins to interface many peripherals. A nice video can be found on the campaign page and an interview with the project creator is embedded after the break.


[Mathieu] Hi Damien, thanks for taking the time to answer our questions. After only a few days, your Kickstarter campaign was already successful. With 17 days to go, we look forward to seeing how many people will receive your boards once the campaign finishes.

[Damien] Thanks Mathieu/Hack-a-day for the opportunity to talk about Micro Python. I’m really excited about this project and it’s awesome to see such a positive response in the first few days of the campaign. I’ve tried to combine functionality with minimalism in the Micro Python board and I’m really looking forward to sending them out to all my backers.

[Mathieu] Our feeling at HaD is that many people might underestimate the work you put in to rewrite the Python scripting language from scratch. How many hours did it take you? Can you describe your process?

[Damien] Yes, it took some time, and lots of programming tricks, to get Micro Python to fit on a microcontroller. Let me say though that it’s not 100% Python: it has exactly the same grammar as Python 3.x, and can emit the same byte code, but the libraries (which make Python what it is) are not all there, yet. Using Micro Python definitely feels the same as using Python (it has REPL running on the board), and I’m trying to get as many libraries working as possible.

I’ve been working on Micro Python for just over six months, in my spare time. Probably clocked up over 700 hours by now. It’s been a lot of fun, selecting components, designing the PCB, soldering parts by hand (mostly with a hot-air gun — they are so awesome!), and writing the Python implementation itself.
The Micro Python software was initially running just on my PC, and I started from the beginning with a simple hand-written lexical analyser.
The next part, the parser, was tricky to write because I wanted it to be as small as possible (in code) and use as little RAM as I could get away with. I ended up encoding the grammar in a static data structure (using some C preprocessor magic) and writing a single 320-line non-recursive function to interpret this table on the fly and parse the Python source. The output of the parser is a parse tree, which is fed into the compiler.
For the compiler, I wanted to be able to emit native machine code, so that Python would run as fast as possible on the micro.
I needed to learn how to encode Thumb machine instructions for this bit, and then turn the stack-machine style code that comes out of the compiler into register-machine style, to take full advantage of the RISC CPU. I also implemented a byte code emitter and virtual machine, so that Micro Python can easily run on other hardware.

Eventually Micro Python was running on my PC, and then I ported it across to the microcontroller. This went surprisingly well and I had it running in a couple of days. Initially I did not have a memory manager, so it would quickly run out of memory on the micro, but then I implemented a simple mark-sweep garbage collector and that works really well.

[Mathieu] Did you write the complete bootloader related code yourself? Did you use any library and if so which ones?

[Damien] The bootloader is built in to the micro that I use: it’s DFU (device firmware upgrade) which means you just hold a few pins at certain logic levels and reset the chip, and then you can flash it over a USB connection. This also makes it easy for users to upgrade the Micro Python image without any special hardware, and means that you can never brick the device.
For the peripherals on the board (like GPIO pins and SD card controller) I used the standard STM libraries, and interfaced them to Micro Python. I also used the awesome FatFs library by [ChaN] to provide filesystem functionality.

[Mathieu] Did you already know everything that you needed to do when starting this project? Did you learn many things along the way?

[Damien] I can’t even begin to tell you how much I learned doing this project. I have in the past written a few programming languages, including ones that compile to machine code (x86), and I’ve had experience with many different microcontrollers. But putting everything together, from the hardware right up to the high-level software, exposes you to a whole new set of problems that you’ve got to learn to solve. For example, learning how to write USB device descriptors, learning how to layout a compact PCB, and even learning new ways of doing things in C!

[Mathieu] Could you point our readers to a few good reads on how to create a lexer, parser, compiler, garbage collector and virtual machine like you did?

[Damien] My knowledge came from so many different places: books, university courses, the internet, and just mucking about. I think implementing a language is one of the most satisfying programming tasks you could ever do, and it doesn’t have to be difficult. If you want to read a book, go for “Compilers: Principles, Techniques, and Tools” by [Aho], [Lam] and [Sethi], also known as the “Dragon Book”. If you just want to start hacking, set yourself the task of implementing a ”stack machine”, as that will give you the basic understanding of most of the pieces.

[Mathieu] A few people on IRC told us that you were accustomed to lengthy projects, could you tell us a bit more about other things you’ve done?

[Damien] My day job is as a theoretical physicist, and in this field you have projects like the LHC (Large Hadron Collider) that take more than 30 years. Of course, my personal physics projects don’t last this long, but it can still take years to go from the initial idea to finally writing up the research paper. My longest hardware project was undoubtedly building my CNC machine (which features at the end of the Kickstarter video). For this I designed my own stepper motor control boards, wrote firmware for an Atmel micro to do the high-level control (not G-code, but similar), designed and built the actual machine (that took over a year…) and wrote all the PC-side software. I even wrote a 3D modelling program to design the machine. Probably overkill.

I also spent the last year and a half building, with a friend, the website paperscape.org, which maps the scientific landscape of research papers. The longest part of this project was implementing (in Python) a LaTeX/TeX parser which could automatically extract bibliographic information to work out how papers are connected.

I think Micro Python has been the most fun and most challenging project that I have worked on so far. And there’s still heaps more to do. One thing I like about this project is the feedback I’m getting from people, and all their suggestion on how to improve Micro Python. I look forward to building up a community around this and having Micro Python running on all kinds of hardware, and doing all kinds of amazing things.