SKILLS4ROBOTS Research
SKILLS4ROBOTS (2015-2020; ERC Starting Grant)
The goal of SKILLS4ROBOTS is to develop an autonomous skill learning system that enables humanoid robots to acquire and improve a rich set of motor skills. This robot skill learning system will allow scaling 
of motor abilities up to fully anthropomorphic robots while overcoming the current limitations of skill learning systems to only few degrees of freedom. To achieve this goal, it will decompose complex motor skills into simpler elemental movements - called movement primitives - that serve as building blocks for the higher-level movement strategy and the resulting architecture will be able to address arbitrary, highly complex tasks -- up to robot table tennis for a humanoid robot. Learned primitives will be superimposed, sequenced and blended. For example, a game of robot table tennis can be represented using different stroke movement primitives, such as a forehand stroke, a backhand stroke or a smash, as well as locomotion primitives for foot placement for maintaining balance by shifting the center of mass of the robot.
The resulting decomposition into building blocks is not only inherent to many motor tasks but also highly scalable and will be exploited by our learning system. Four recent breakthroughs in our research will make this project possible due to successes on the representation of the parametric probabilistic representations of the elementary movements, on probabilistic imitation learning, on relative entropy policy search-based reinforcement learning and on the modular organization of the representation.
These breakthroughs will allow create a general, autonomous skill learning system that can learn many different skills in the exact same framework without changing a single line of programmed code.
Team Leader: Jan Peters
Contacts: Jan Peters, Boris Belousov, Rudolf Lioutikov, Hany Abdulsamad
SKILLS4ROBOTS Journal Papers
Tanneberg, D.; Ploeger, K.; Rueckert, E.; Peters, J. (2021). SKID RAW: Skill Discovery from Raw Trajectories, IEEE Robotics and Automation Letters (RA-L).
Download Article BibTeX Reference
Koert, D.; Trick, S.; Ewerton, M.; Lutter, M.; Peters, J. (2020). Incremental Learning of an Open-Ended Collaborative Skill Library, International Journal of Humanoid Robotics (IJHR), 17, 1.
BibTeX Reference
Koert, D.; Kircher, M.; Salikutluk, V.; D'Eramo, C.; Peters, J. (2020). Multi-Channel Interactive Reinforcement Learning for Sequential Tasks, Frontiers in Robotics and AI Human-Robot Interaction.
Download Article BibTeX Reference
Tanneberg, D.; Rueckert, E.; Peters, J. (2020). Evolutionary Training and Abstraction Yields Algorithmic Generalization of Neural Computers, Nature Machine Intelligence, 2, 12, pp.753-763.
Download Article BibTeX Reference
Tanneberg, D.; Peters, J.; Rueckert, E. (2019). Intrinsic Motivation and Mental Replay enable Efficient Online Adaptation in Stochastic Recurrent Networks, Neural Networks, 109, pp.67-80.
Download Article BibTeX Reference
Belousov, B.; Peters, J. (2019). Entropic Regularization of Markov Decision Processes, Entropy, 21, 7, MDPI.
Download Article BibTeX Reference
Pajarinen, J.; Thai, H.L.; Akrour, R.; Peters, J.; Neumann, G. (2019). Compatible natural gradient policy search, Machine Learning (MLJ), 108, 8, pp.1443--1466, Springer.
Download Article BibTeX Reference
Koert, D.; Pajarinen, J.; Schotschneider, A.; Trick, S., Rothkopf, C.; Peters, J. (2019). Learning Intention Aware Online Adaptation of Movement Primitives, IEEE Robotics and Automation Letters (RA-L), with presentation at the IEEE International Conference on Intelligent Robots and Systems (IROS).
Download Article BibTeX Reference
Ewerton, M.; Rother, D.; Weimar, J.; Kollegger, G.; Wiemeyer, J.; Peters, J.; Maeda, G. (2018). Assisting Movement Training and Execution with Visual and Haptic Feedback, Frontiers in Neurorobotics.
Download Article BibTeX Reference
SKILLS4ROBOTS Conference and Workshop Papers
Wibranek, B.; Liu, Y.; Funk, N.; Belousov, B.; Peters, J.; Tessmann, O. (2021). Reinforcement Learning for Sequential Assembly of SL-Blocks: Self-Interlocking Combinatorial Design Based on Machine Learning, Proceedings of the 39th eCAADe Conference.
Download Article BibTeX Reference
Eilers, C.; Eschmann, J.; Menzenbach, R.; Belousov, B.; Muratore, F.; Peters, J. (2020). Underactuated Waypoint Trajectory Optimization for Light Painting Photography, Proceedings of the IEEE International Conference on Robotics and Automation (ICRA).
Download Article BibTeX Reference
Keller, L.; Tanneberg, D.; Stark, S.; Peters, J. (2020). Model-Based Quality-Diversity Search for Efficient Robot Learning, Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).
Download Article BibTeX Reference
Lauri, M.; Pajarinen, J.; Peters, J. (2019). Information gathering in decentralized POMDPs by policy graph improvement, Proceedings of the International Conference on Autonomous Agents and Multiagent Systems (AAMAS).
Download Article BibTeX Reference
Wibranek, B.; Belousov, B.; Sadybakasov, A.; Tessmann, O. (2019). Interactive Assemblies: Man-Machine Collaboration through Building Components for As-Built Digital Models, Computer-Aided Architectural Design Futures (CAAD Futures).
Download Article BibTeX Reference
Akrour, R.; Pajarinen, J.; Neumann, G.; Peters, J. (2019). Projections for Approximate Policy Iteration Algorithms, Proceedings of the International Conference on Machine Learning (ICML).
Download Article BibTeX Reference
Belousov, B.; Abdulsamad, H.; Schultheis, M.; Peters, J. (2019). Belief Space Model Predictive Control for Approximately Optimal System Identification, 4th Multidisciplinary Conference on Reinforcement Learning and Decision Making (RLDM).
Download Article BibTeX Reference
Nass, D.; Belousov, B.; Peters, J. (2019). Entropic Risk Measure in Policy Search, Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).
Download Article BibTeX Reference
Trick, S.; Koert, D.; Peters, J.; Rothkopf, C. (2019). Multimodal Uncertainty Reduction for Intention Recognition in Human-Robot Interaction, Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).
Download Article BibTeX Reference
Belousov, B.; Sadybakasov, A.; Wibranek, B.; Veiga, F.; Tessmann, O.; Peters, J. (2019). Building a Library of Tactile Skills Based on FingerVision, Proceedings of the 2019 IEEE-RAS 19th International Conference on Humanoid Robots (Humanoids).
Download Article BibTeX Reference
Schultheis, M.; Belousov, B.; Abdulsamad, H.; Peters, J. (2019). Receding Horizon Curiosity, Proceedings of the 3rd Conference on Robot Learning (CoRL).
Download Article BibTeX Reference
Wibranek, B.; Belousov, B.; Sadybakasov, A.; Peters, J.; Tessmann, O. (2019). Interactive Structure: Robotic Repositioning of Vertical Elements in Man-Machine Collaborative Assembly through Vision-Based Tactile Sensing, Proceedings of the 37th eCAADe and 23rd SIGraDi Conference.
Download Article BibTeX Reference
Delfosse, Q.; Stark, S.; Tanneberg, D.; Santucci, V. G.; Peters, J. (2019). Open-Ended Learning of Grasp Strategies using Intrinsically Motivated Self-Supervision, Workshop at the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).
Download Article BibTeX Reference
Koert, D.; Maeda, G.; Neumann, G.; Peters, J. (2018). Learning Coupled Forward-Inverse Models with Combined Prediction Errors, Proceedings of the International Conference on Robotics and Automation (ICRA).
Download Article BibTeX Reference
Koert, D.; Trick, S.; Ewerton, M.; Lutter, M.; Peters, J. (2018). Online Learning of an Open-Ended Skill Library for Collaborative Tasks, Proceedings of the International Conference on Humanoid Robots (HUMANOIDS).
Download Article BibTeX Reference
Hoelscher, J.; Koert, D.; Peters, J.; Pajarinen, J. (2018). Utilizing Human Feedback in POMDP Execution and Specification, Proceedings of the International Conference on Humanoid Robots (HUMANOIDS).
Download Article BibTeX Reference
Belousov, B.; Neumann, G.; Rothkopf, C.A.; Peters, J. (2017). Catching Heuristics Are Optimal Control Policies, Proceedings of the Karniel Thirteenth Computational Motor Control Workshop.
Download Article BibTeX Reference
Tanneberg, D.; Peters, J.; Rueckert, E. (2017). Online Learning with Stochastic Recurrent Neural Networks using Intrinsic Motivation Signals, Proceedings of the Conference on Robot Learning (CoRL).
Download Article BibTeX Reference
Rueckert, E.; Nakatenus, M.; Tosatto, S.; Peters, J. (2017). Learning Inverse Dynamics Models in O(n) time with LSTM networks, Proceedings of the International Conference on Humanoid Robots (HUMANOIDS).
Download Article BibTeX Reference
Tanneberg, D.; Peters, J.; Rueckert, E. (2017). Efficient Online Adaptation with Stochastic Recurrent Neural Networks, Proceedings of the International Conference on Humanoid Robots (HUMANOIDS).
Download Article BibTeX Reference
Stark, S.; Peters, J.; Rueckert, E. (2017). A Comparison of Distance Measures for Learning Nonparametric Motor Skill Libraries, Proceedings of the International Conference on Humanoid Robots (HUMANOIDS).
Download Article BibTeX Reference
Thiem, S.; Stark, S.; Tanneberg, D.; Peters, J.; Rueckert, E. (2017). Simulation of the underactuated Sake Robotics Gripper in V-REP, Workshop at the International Conference on Humanoid Robots (HUMANOIDS).
Download Article BibTeX Reference
Belousov, B.; Neumann, G.; Rothkopf, C.; Peters, J. (2016). Catching Heuristics Are Optimal Control Policies, Advances in Neural Information Processing Systems (NIPS / NeurIPS).
Download Article BibTeX Reference
Koert, D.; Maeda, G.J.; Lioutikov, R.; Neumann, G.; Peters, J. (2016). Demonstration Based Trajectory Optimization for Generalizable Robot Motions, Proceedings of the International Conference on Humanoid Robots (HUMANOIDS).
Download Article BibTeX Reference
