This syllabus is an active document. Please be aware that course schedule will change throughout the semester; you should review the schedule weekly.

Class information

Room: Science Center 252 aka "the robot lab"
Section 1: T, TH 9:55–11:10am
Section 2: T, TH 11:20–12:35pm
Lab A: W 1:15–2:45
Lab B: W 3:00–4:30

Professor: Lisa Meeden
Office: Science Center 243
Phone: 328-8565

Introduction

This seminar will explore the topic of adaptive robotics with a focus on developmental robotics, a newly emerging paradigm of research. The goal of developmental robotics is to create intelligent robots by allowing them to go through a developmental process, rather than being directly programmed to solve a particular task. By endowing a robot with an appropriate initial control architecture and adaptive mechanisms, it can learn through interactions with the world, developing self-organized knowledge about itself and its environment. We will be studying the following sorts of questions: What should be innate in the robot? What adaptive mechanisms are needed? What motivates the robot to act?

Goals for the course

• Learn how to think like a computer science researcher.
• Analyze and critically discuss technical, research papers both in writing and in class.
• Understand the fundamental questions in a research field and be able to analyze different approaches to answering these questions.
• Formulate and evaluate a research question in a technical field by completing a substantial project.
• Relate your project to prior research via a review of related literature.
• Orally present a clear and accessible summary of your project.
• Write a coherent, complete paper describing and evaluating your project.

Grading

• Class Participation and Paper Summaries: 20%
  Each week we will discuss papers in a seminar style. This is not a lecture-bassed course; we as a collective group generate the course content, and you need to be present in class to contribute. Reading the papers and writing paper summaries prior to the first class each week is essential to preparing for a lively and informed discussion.

• Labs: 15%
  During lab we will focus on hands-on demonstrations. These demonstrations will give you a chance to learn how to control both simulated and physical robots, as well as give you the chance to implement and experiment with the models we are reading about in the primary literature. Some lab time will be designated for making progress on the midterm and final projects.

• Midterm Project: 20%
  The goal of this project is to give you a preview, on a smaller scale, of what will be expected on the final project.
• Final Project: 45%
  You will be asked to design a project related to adaptive robotics. You are strongly encouraged to work with a partner.
• Proposal 10%
  
• Checkpoint demonstration 5%
  
• Presentation 10%
  
• Paper 20%
  

Paper Summaries

Read all of the papers for the week prior to class on Tuesday. Bring a typed summary of the papers with you to class on Tuesday. You will turn this in at the end of class on Tuesday. I will write comments on it and return it to you at the start of class on Thursday.

For each paper include the following:

• A one paragraph overview that describes its main points.
• At least three questions or issues that we can discuss during class. These might include questions of clarification, such as:
  • What is the meaning of some specific terminology?
  • How does a particular algorithm work? Why is a certain step of an algorithm necessary?
  • Clarify what is being shown in a particular figure or table in the paper.
  But should also include bigger picture issues, such as:
  • In what ways does the mechanism described seem like a good tool for doing adaptive robotics? What are its advantages/distadvantages?
  • Do the paper's conclusions logically follow from the data provided?
  • How do the paper's mechanisms and conclusions relate to previous papers that we have read?
  • What follow up experiments should be done and why?

---

Schedule

WEEK	DAY	ANNOUNCEMENTS	TOPIC & READING	LAB
1	Jan 21		Introduction and Neural computation Chapters 1-2 from Computational Developmental Psychology, Thomas Schultz (1995).	pyrobot overview Lab 1: Neural network brains for robots
	Jan 23
	Jan 28		Evolutionary robotics Trends in evolutionary robotics, Chapter 9 in Soft computing for intelligent robotics systems, Lisa Meeden and Deepak Kumar (1998). Competitive coevolution through evolutionary complexification Journal of Artificial Intelligence Research, Vol. 21, Kenneth Stanley (2004).	Lab 2: Evolving neural network brains for robots
	Jan 30	Drop/Add ends (Jan 31)
	Feb 04		Novelty search Abandoning objectives: Evolution through the search for novelty alone Evolutionary Computation, Vol. 19:2, Joel Lehman and Kenneth Stanley (2011). Evolving a diversity of creatures through novelty search and local competition In the Proceedings of the Genetic and Evolutionary Computation Conference, Joel Lehman and Kenneth Stanley (2011).	Lab 3: Novelty search
	Feb 06
	Feb 11		Developmental robotics Developmental Robotics: From Babies to Robots, by Cangelosi and Schlesinger (In press), Chapters 1-2	Lab 4: Scribbler robot
	Feb 13
	Feb 18		Intrinsic motivation Intrinsic motivation systems for autonomous mental development IEEE Transactions on Evolutionary Computation, vol. 11.2, Peirre-Yves Oudeyer, Frederic Kaplan, Verena Hafner (2007). Exploring IAC	Midterm project
	Feb 20
	Feb 25		Unsupervised learning A growing neural gas learns topologies Advances in Neural Information Processing Systems 7, Bernd Fritzke (1995). Category-based intrinsic motivation, Proceedings of the Ninth International Conference on Epigenetic Robotics, Rachel Lee, Ryan Walker, Lisa Meeden, and James Marshall (2009). CBIM Slides from EpiRob09 Talk	CheckPoint Presentations on midterm project
	Feb 27
	Mar 04		Imitation learning Hierarchical attentive multiple models for execution and recognition of actions, by Yiannis Demiris and Bassam Khadhouri, Robotics and Autonomous Systems, Vol. 54, (2006).	Finish midterm project
	Mar 06	Midterm project due at 3pm (Mar 07)
	Mar 11	Spring Break
	Mar 13
	Mar 18		Reinforcement learning Reinforcement learning: An introduction by R. Sutton and A. Barto, MIT Press (1998), read Chapter 1 through section 1.5, or pages 1-13 in this pdf Horde: A scalable real-time architecture for learning knowledge from unsupervised sensorimotor interaction, by R. Sutton, J. Modayil, M. Delp, T. Degris, P. Pilarski, and A. White, in the Proceedings of the Tenth International Conference on Autonomous Agents and Multi-Agent Systems (2011).	Lab 5: The Horde
	Mar 20
	Mar 25		Developmental encodings Compositional pattern producing networks: A novel abstraction of development, by Kenneth O. Stanley, in Genetic programming and evolvable machines, Vol. 8, (2007). Unshackling evolution: Evolving soft robots with multiple materials and a powerful generative encoding by N. Cheney, R. MacCurdy, J. Clune, and H. Lipson, in Proceedings of GECCO, (2013).	Lab 6: Compositional pattern producing networks
	Mar 27	Last day to declare CR/NC or W (Mar 28)
	Apr 01		Papers selected by students Evolving neural network agents in the NERO video game, by Lee, Yosinski, Glette, Lipson, and Clune, in EvoApplications, (2013). Evolving gaits for physical robots with the HyperNEAT generative encoding: The benefits of simulation, by Stanley, Bryant, and Miikkulainen, in Symposium on Computational Intelligence and Games, (2005).	Final project
	Apr 03
	Apr 08		Papers selected by students BigDog, the Rough-Terrain Quaduped Robot, by Marc Raibert, Kevin Blankespoor, Gabriel Nelson, and Rob Playter, Boston Dynamics Developmental Perception of the Self and Action, by Ryo Saegusa, Giorgio Metta, Giulio Sandini and Lorenzo Natale, in IEEE Transactions on Neural Networks and Learning Systems, Vol 25:1 (2014).	Work on final project
	Apr 10
	Apr 15		Checkpoint presentations Schedule Evaluation form
	Apr 17
	Apr 22		Papers selected by students Exploring the Role of the Tail in Bipedal Hopping through Computational Evolution, by Jared Moore, Anne Gutmann, Craig McGowan, Philip McKinley, in the proceedings of the European Conference on Artificial Life (2013). Deep learning videos: Motivation, Example,More examples
	Apr 24
	Apr 29		Presentations Schedule Evaluation form	Presentations
	May 01	Final project due at 3pm (May 09)

Final papers

• Alex Simms and Ravenna Thielstrom "Bees?" A large scale, cooperative simulation weighing altruism and selfishness
• Amy Jin and Murphy Austin Comparative evolutions of swarm communications
• Andy Ye and Lucas Chen SoundBreeder with MultiNEAT
• Awjin Ahn and Caleb Cochrane Using NEAT to stabilize an inverted pendulum
• Ben Xie and Tyler Zon Using artificial neural networks to model Siegler's balancing task
• Chris Magnano, Kevin Roberts, and Will Schneider NERL: Neural network Emulation of Reinforcement Learners
• Cole Harbeck and KJ Bredder Exploring mutli-objective fitness functions and compositions of different fitness functions in rtNEAT
• Cynthia Ma Sexual selection: Furthering the "evolution" metaphor
• Dan Hu and Omar Chowdhury Algorithmic currency trading using NEAT-based evolutionary computation
• Daniel Redelmeier and Uriel Mandujano Evolving robots to play dodgeball
• Davis Ancona and Jake Weiner Developing frogger player intelligence using NEAT and a score-driven fitness function
• Ethan Bogdan Exploring advanced techniques for training an artificial neural network on a complex, multimodal game scenario
• Jackie Kay and Kate Derosier Simulating Mirror Neurons
• Joe Boninger and Melissa O'Connor Predation as a motivation for schooling behavior with simulated fish
• Kevin Terusaki and Vince Stigliani Emotion detection using deep belief networks
• Luis Ramirez Using artificial neural networks to play ping pong
• Mario Sanchez and Teo Gelles Comparing adaptivity of ants using NEAT and rtNEAT
• Peng Zhao and Richard Liang Applying and comparing evolutionary algorithms for robot tanks