Program Synthesis Papers at ICLR 2018

Illia Polosukhin
Published in
3 min readMay 1, 2018


Example of program synthesis from natural language from our paper “Neural Program Search”

Program Synthesis is a subfield of Computer Science to automatically construct a program that satisfy given specification. Specification can be set of input/output examples (unit tests), natural language, first order logic expression or any other form that is easier to write then the expected program.

Program Synthesis has been around for awhile as a Programming Languages subfield and usually been approached with symbolic methods. In the last few years though, Machine Learning community started to apply deep learning for these problems.

I strongly believe that Program Synthesis is a great test bed for a lot of methods in deep learning (structured predictions, RL with sparse reward, reasoning, using external knowledge, meta learning, etc) and very happy to see more papers in this field at ICLR 2018. Disclaimer: NEAR has two papers in this list.

Parametrized Hierarchical Procedures for Neural Programming. Roy Fox · Richard Shin · Sanjay Krishnan · Ken Goldberg · Dawn Song · Ion Stoica. In Mon AM Posters

Towards Specification-Directed Program Repair. Richard Shin · Illia Polosukhin · Dawn Song. In Mon AM Workshops

Tree-to-tree Neural Networks for Program Translation. Xinyun Chen · Chang Liu · Dawn Song. In Mon AM Workshops

Neural-Guided Deductive Search for Real-Time Program Synthesis from Examples. Ashwin Vijayakumar · Abhishek Mohta · Alex Polozov · Dhruv Batra · Prateek Jain · Sumit Gulwani. In Mon PM Posters

Learning to Represent Programs with Graphs. Miltiadis Allamanis · Marc Brockschmidt · Mahmoud Khademi. In Tue AM Talks

Neural Sketch Learning for Conditional Program Generation. Vijayaraghavan Murali · Letao Qi · Swarat Chaudhuri · Chris Jermaine. In Tue AM Talks

Dynamic Neural Program Embeddings for Program Repair. Ke Wang · Rishabh Singh · Zhendong Su. In Tue AM Posters

Neural Program Search: Solving Programming Tasks from Description and Examples. Illia Polosukhin · Alex Skidanov. In Tue PM Workshops

Combining Symbolic Expressions and Black-box Function Evaluations in Neural Programs. Forough Arabshahi · Sameer Singh · Anima Anandkumar. In Wed AM Posters

Improving the Universality and Learnability of Neural Programmer-Interpreters with Combinator Abstraction. Da Xiao · Jo-Yu Liao · Xingyuan Yuan. In Wed AM Posters

Learning to Represent Programs with Graphs. Miltiadis Allamanis · Marc Brockschmidt · Mahmoud Khademi. In Wed PM Posters

Towards Synthesizing Complex Programs From Input-Output Examples. Xinyun Chen · Chang Liu · Dawn Song. In Wed PM Posters

Leveraging Grammar and Reinforcement Learning for Neural Program Synthesis. Rudy Bunel · Matthew Hausknecht · Jacob Devlin · Rishabh Singh · Pushmeet Kohli. In Thu AM Posters

Leveraging Constraint Logic Programming for Neural Guided Program Synthesis. Lisa Zhang · Gregory Rosenblatt · Ethan Fetaya · Renjie Liao · William Byrd · Raquel Urtasun · Richard Zemel. In Thu PM Workshops

Neural Architecture Search papers

As additional bonus, want to mention few neural architecture search, where the task is to find a neural program that best solves some machine learning problem.

Hierarchical Representations for Efficient Architecture Search. Hanxiao Liu · Karen Simonyan · Oriol Vinyals · Chrisantha Fernando · Koray Kavukcuoglu. In Mon AM Posters

Differentiable Neural Network Architecture Search. Richard Shin · Charles Packer · Dawn Song. In Mon AM Workshops

A Flexible Approach to Automated RNN Architecture Generation. Martin Schrimpf · Stephen Merity · James Bradbury · Richard Socher. In Tue PM Workshops

Searching for Activation Functions. Prajit Ramachandran · Barret Zoph · Quoc V Le. In Wed PM Workshops

Accelerating Neural Architecture Search using Performance Prediction. Bowen Baker · Otkrist Gupta · Ramesh Raskar · Nikhil Naik. In Thu AM Workshops

Faster Discovery of Neural Architectures by Searching for Paths in a Large Model. Hieu Pham · Melody Y. Guan · Barret Zoph · Quoc V Le · Jeff Dean. In Thu AM Workshops

SMASH: One-Shot Model Architecture Search through HyperNetworks. Andrew Brock · Theo Lim · James Ritchie · Nick Weston. In Thu PM Posters

Simple and efficient architecture search for Convolutional Neural Networks. Thomas Elsken · Jan Metzen · Frank Hutter. In Thu PM Workshops

Hope this list is useful to guide your search!

This is a list of paper that I’ve noticed, which may not be complete. Please let me know if I missed something.



Illia Polosukhin

Co-Founder @ NEAR Protocol. Simple. Secure. Scalable. I'm tweeting as @ilblackdragon.