Google wants to let robots write their own code

Google has released an new open source benchmark that it says will allow robots to perform tasks by writing their own code in response to human-written instructions. 

The company has launched a new website (opens in new tab) to reveal “Code as Policies” (CAP), whereby prompts written in plain English can be interpreted into language-model generated programs (LMPs) written in Python code.

CAP is the successor to PaLM-SayCan (opens in new tab), a project that similarly allowed a physical helper robot to be directed through commands in plain English. CAP promises to allow for more complex tasks to be completed with increased accuracy, in part due to allowing machines to write their own code.

Self-coding robots

In a blog post (opens in new tab) discussing CAP’s release, Google Research Intern Jacky Liang and Research Scientist Andy Zeng describe the motivation for the technology, and what it might mean for the future.

“What if when given instructions from people, robots could autonomously write their own code to interact with the world? […] Given natural language instructions, current language models are highly proficient at writing not only generic code but, as we’ve discovered, code that can control robot actions as well.”

But it might not be the time to throw away your laptop for programming just yet. In testing, Google’s researchers demonstrated simple commands with a similar structure. Test robots were able to “draw a 5 cm hexagon around the middle” [of a whiteboard], and “put the blocks in a horizontal line near the top” [of a square boundary].

In the accompanying paper (opens in new tab), titled “Code as Policies: Language Model Programs for Embodied Control”, the project team concede that CAP currently doesn’t have the ability to deal with especially abstract or complex commands or perceive descriptions of trajectory. The team’s approach also didn’t account for impossible commands being fed though CAP. 

In theory, the open source nature of Google’s “robot-centric” implementation of LMPs in Python could result in much quicker implementation of solutions to these issues. The CAP website also contains releases (opens in new tab) via Github, and an interactive demo (opens in new tab) via Google Colab to describe how robots “write” code in response to commands.