SKILLS4ROBOTS (2015-2020; ERC Starting Grant)

The goal of SKILLS4ROBOTS is to develop an autonomous skill learning system that enables humanoid robots to acquire and improve a rich set of motor skills. This robot skill learning system will allow scaling of motor abilities up to fully anthropomorphic robots while overcoming the current limitations of skill learning systems to only few degrees of freedom. To achieve this goal, it will decompose complex motor skills into simpler elemental movements - called movement primitives - that serve as building blocks for the higher-level movement strategy and the resulting architecture will be able to address arbitrary, highly complex tasks -- up to robot table tennis for a humanoid robot. Learned primitives will be superimposed, sequenced and blended. For example, a game of robot table tennis can be represented using different stroke movement primitives, such as a forehand stroke, a backhand stroke or a smash, as well as locomotion primitives for foot placement for maintaining balance by shifting the center of mass of the robot. The resulting decomposition into building blocks is not only inherent to many motor tasks but also highly scalable and will be exploited by our learning system. Four recent breakthroughs in our research will make this project possible due to successes on the representation of the parametric probabilistic representations of the elementary movements, on probabilistic imitation learning, on relative entropy policy search-based reinforcement learning and on the modular organization of the representation. These breakthroughs will allow create a general, autonomous skill learning system that can learn many different skills in the exact same framework without changing a single line of programmed code.

Team Leader: Jan Peters
Contacts: Jan Peters, Boris Belousov, Rudolf Lioutikov, Hany Abdulsamad

SKILLS4ROBOTS Journal Papers

  •     Bib
    Tanneberg, D.; Ploeger, K.; Rueckert, E.; Peters, J. (2021). SKID RAW: Skill Discovery from Raw Trajectories, IEEE Robotics and Automation Letters (RA-L).
  •     Bib
    Koert, D.; Trick, S.; Ewerton, M.; Lutter, M.; Peters, J. (2020). Incremental Learning of an Open-Ended Collaborative Skill Library, International Journal of Humanoid Robotics (IJHR), 17, 1.
  •       Bib
    Koert, D.; Kircher, M.; Salikutluk, V.; D'Eramo, C.; Peters, J. (2020). Multi-Channel Interactive Reinforcement Learning for Sequential Tasks, Frontiers in Robotics and AI Human-Robot Interaction.
  •       Bib
    Tanneberg, D.; Rueckert, E.; Peters, J. (2020). Evolutionary Training and Abstraction Yields Algorithmic Generalization of Neural Computers, Nature Machine Intelligence, 2, 12, pp.753-763.
  •       Bib
    Tanneberg, D.; Peters, J.; Rueckert, E. (2019). Intrinsic Motivation and Mental Replay enable Efficient Online Adaptation in Stochastic Recurrent Networks, Neural Networks, 109, pp.67-80.
  •       Bib
    Belousov, B.; Peters, J. (2019). Entropic Regularization of Markov Decision Processes, Entropy, 21, 7, MDPI.
  •     Bib
    Pajarinen, J.; Thai, H.L.; Akrour, R.; Peters, J.; Neumann, G. (2019). Compatible natural gradient policy search, Machine Learning (MLJ), 108, 8, pp.1443--1466, Springer.
  •     Bib
    Koert, D.; Pajarinen, J.; Schotschneider, A.; Trick, S., Rothkopf, C.; Peters, J. (2019). Learning Intention Aware Online Adaptation of Movement Primitives, IEEE Robotics and Automation Letters (RA-L), with presentation at the IEEE International Conference on Intelligent Robots and Systems (IROS).
  •     Bib
    Ewerton, M.; Rother, D.; Weimar, J.; Kollegger, G.; Wiemeyer, J.; Peters, J.; Maeda, G. (2018). Assisting Movement Training and Execution with Visual and Haptic Feedback, Frontiers in Neurorobotics.

SKILLS4ROBOTS Conference and Workshop Papers

  •     Bib
    Wibranek, B.; Liu, Y.; Funk, N.; Belousov, B.; Peters, J.; Tessmann, O. (2021). Reinforcement Learning for Sequential Assembly of SL-Blocks: Self-Interlocking Combinatorial Design Based on Machine Learning, Proceedings of the 39th eCAADe Conference.
  •     Bib
    Eilers, C.; Eschmann, J.; Menzenbach, R.; Belousov, B.; Muratore, F.; Peters, J. (2020). Underactuated Waypoint Trajectory Optimization for Light Painting Photography, Proceedings of the IEEE International Conference on Robotics and Automation (ICRA).
  •     Bib
    Keller, L.; Tanneberg, D.; Stark, S.; Peters, J. (2020). Model-Based Quality-Diversity Search for Efficient Robot Learning, Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).
  •     Bib
    Lauri, M.; Pajarinen, J.; Peters, J. (2019). Information gathering in decentralized POMDPs by policy graph improvement, Proceedings of the International Conference on Autonomous Agents and Multiagent Systems (AAMAS).
  •     Bib
    Wibranek, B.; Belousov, B.; Sadybakasov, A.; Tessmann, O. (2019). Interactive Assemblies: Man-Machine Collaboration through Building Components for As-Built Digital Models, Computer-Aided Architectural Design Futures (CAAD Futures).
  •     Bib
    Akrour, R.; Pajarinen, J.; Neumann, G.; Peters, J. (2019). Projections for Approximate Policy Iteration Algorithms, Proceedings of the International Conference on Machine Learning (ICML).
  •     Bib
    Belousov, B.; Abdulsamad, H.; Schultheis, M.; Peters, J. (2019). Belief Space Model Predictive Control for Approximately Optimal System Identification, 4th Multidisciplinary Conference on Reinforcement Learning and Decision Making (RLDM).
  •     Bib
    Nass, D.; Belousov, B.; Peters, J. (2019). Entropic Risk Measure in Policy Search, Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).
  •     Bib
    Trick, S.; Koert, D.; Peters, J.; Rothkopf, C. (2019). Multimodal Uncertainty Reduction for Intention Recognition in Human-Robot Interaction, Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).
  •     Bib
    Belousov, B.; Sadybakasov, A.; Wibranek, B.; Veiga, F.; Tessmann, O.; Peters, J. (2019). Building a Library of Tactile Skills Based on FingerVision, Proceedings of the 2019 IEEE-RAS 19th International Conference on Humanoid Robots (Humanoids).
  •     Bib
    Schultheis, M.; Belousov, B.; Abdulsamad, H.; Peters, J. (2019). Receding Horizon Curiosity, Proceedings of the 3rd Conference on Robot Learning (CoRL).
  •     Bib
    Wibranek, B.; Belousov, B.; Sadybakasov, A.; Peters, J.; Tessmann, O. (2019). Interactive Structure: Robotic Repositioning of Vertical Elements in Man-Machine Collaborative Assembly through Vision-Based Tactile Sensing, Proceedings of the 37th eCAADe and 23rd SIGraDi Conference.
  •     Bib
    Delfosse, Q.; Stark, S.; Tanneberg, D.; Santucci, V. G.; Peters, J. (2019). Open-Ended Learning of Grasp Strategies using Intrinsically Motivated Self-Supervision, Workshop at the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).
  •     Bib
    Koert, D.; Maeda, G.; Neumann, G.; Peters, J. (2018). Learning Coupled Forward-Inverse Models with Combined Prediction Errors, Proceedings of the International Conference on Robotics and Automation (ICRA).
  •     Bib
    Koert, D.; Trick, S.; Ewerton, M.; Lutter, M.; Peters, J. (2018). Online Learning of an Open-Ended Skill Library for Collaborative Tasks, Proceedings of the International Conference on Humanoid Robots (HUMANOIDS).
  •     Bib
    Hoelscher, J.; Koert, D.; Peters, J.; Pajarinen, J. (2018). Utilizing Human Feedback in POMDP Execution and Specification, Proceedings of the International Conference on Humanoid Robots (HUMANOIDS).
  •     Bib
    Belousov, B.; Neumann, G.; Rothkopf, C.A.; Peters, J. (2017). Catching Heuristics Are Optimal Control Policies, Proceedings of the Karniel Thirteenth Computational Motor Control Workshop.
  •     Bib
    Tanneberg, D.; Peters, J.; Rueckert, E. (2017). Online Learning with Stochastic Recurrent Neural Networks using Intrinsic Motivation Signals, Proceedings of the Conference on Robot Learning (CoRL).
  •     Bib
    Rueckert, E.; Nakatenus, M.; Tosatto, S.; Peters, J. (2017). Learning Inverse Dynamics Models in O(n) time with LSTM networks, Proceedings of the International Conference on Humanoid Robots (HUMANOIDS).
  •     Bib
    Tanneberg, D.; Peters, J.; Rueckert, E. (2017). Efficient Online Adaptation with Stochastic Recurrent Neural Networks, Proceedings of the International Conference on Humanoid Robots (HUMANOIDS).
  •     Bib
    Stark, S.; Peters, J.; Rueckert, E. (2017). A Comparison of Distance Measures for Learning Nonparametric Motor Skill Libraries, Proceedings of the International Conference on Humanoid Robots (HUMANOIDS).
  •     Bib
    Thiem, S.; Stark, S.; Tanneberg, D.; Peters, J.; Rueckert, E. (2017). Simulation of the underactuated Sake Robotics Gripper in V-REP, Workshop at the International Conference on Humanoid Robots (HUMANOIDS).
  •     Bib
    Belousov, B.; Neumann, G.; Rothkopf, C.; Peters, J. (2016). Catching Heuristics Are Optimal Control Policies, Advances in Neural Information Processing Systems (NIPS / NeurIPS).
  •     Bib
    Koert, D.; Maeda, G.J.; Lioutikov, R.; Neumann, G.; Peters, J. (2016). Demonstration Based Trajectory Optimization for Generalizable Robot Motions, Proceedings of the International Conference on Humanoid Robots (HUMANOIDS).