On-Robot Learning 

While agents can acquire complex skills by learning in a simulated environment, the sim-to-real gap is often too large to overcome using just domain randomization techniques. Indeed, it is often the case in contact-rich environments, or when learning fast and dynamic motion skills. In these settings, we need to deploy our learning algorithm directly on the real robot. This opens many crucial issues, such as learning efficiency, complying with safety and task constraints, and transferring learned skills to different tasks. In the following, we present some of the research tackling these topics. Our long-term objective is to deploy learning algorithms in the real world, allowing robots to constantly improve their performance while performing the target task.
Safe Robot Learning 

Fast Kinodynamic Planning on the Constraint Manifold with Deep Neural Networks. In recent years, learning-based solutions have become alternatives to classical approaches, but they still lack comprehensive handling of complex constraints, such as planning on a lower-dimensional manifold of the task space while considering the robot's dynamics. To solve these issues, we introduce a novel learning-to-plan framework that exploits the concept of constraint manifold, allowing the handling of kinodynamic constraints. Our approach generates plans satisfying an arbitrary set of constraints and computes them in a short constant time. This allows the robot to plan and replan reactively, making our approach suitable for dynamic environments. We validate our approach on two simulated tasks and in a demanding real-world scenario, where we use a Kuka LBR Iiwa 14 robotic arm to perform the hitting movement in robotic Air Hockey.
- Kicki, P.; Liu, P.; Tateo, D.; Bou Ammar, H.; Walas, K.; Skrzypczynski, P.; Peters, J. (2024). Fast Kinodynamic Planning on the Constraint Manifold with Deep Neural Networks, IEEE Transactions on Robotics (T-Ro), and Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 40, pp.277-297.

Robot Reinforcement Learning on the Constraint Manifold. Reinforcement learning in robotics is extremely challenging due to many practical issues, including safety, mechanical constraints, and wear and tear. Typically, these issues are not considered in the machine learning literature. One crucial problem in applying reinforcement learning in the real world is Safe Exploration, which requires physical and safety constraints satisfaction throughout the learning process. To explore in such a safety-critical environment, leveraging known information such as robot models and constraints is beneficial to provide more robust safety guarantees. Exploiting this knowledge, we propose a novel method to learn robotics tasks in simulation efficiently while satisfying the constraints during the learning process.
- Liu, P.; Tateo, D.; Bou-Ammar, H.; Peters, J. (2021). Robot Reinforcement Learning on the Constraint Manifold, Proceedings of the Conference on Robot Learning (CoRL).
- Liu, P.; Zhang, K.; Tateo, D.; Jauhri, S.; Hu, Z.; Peters, J. Chalvatzaki, G. (2023). Safe Reinforcement Learning of Dynamic High-Dimensional Robotic Tasks: Navigation, Manipulation, Interaction, 2023 IEEE International Conference on Robotics and Automation (ICRA), IEEE.
- Liu, P.; Zhang, K.; Tateo, D.; Jauhri, S.; Peters, J.; Chalvatzaki, G.; (2022). Regularized Deep Signed Distance Fields for Reactive Motion Generation, 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).
Active Exploration 
![]() ![]() |
Active Exploration for Robotic Manipulation. Robotic manipulation stands as a largely unsolved problem despite significant advances in robotics and machine learning in the last decades. One of the central challenges of manipulation is partial observability, as the agent usually does not know all physical properties of the environment and the objects it is manipulating in advance. A recently emerging theory that deals with partial observability in an explicit manner is Active Inference. It does so by driving the agent to act in a way that is not only goal-directed but also informative about the environment. In this work, we apply Active Inference to a hard-to-explore simulated robotic manipulation tasks, in which the agent has to balance a ball into a target zone. Since the reward of this task is sparse, in order to explore this environment, the agent has to learn to balance the ball without any extrinsic feedback, purely driven by its own curiosity. We show that the information-seeking behavior induced by Active Inference allows the agent to explore these challenging, sparse environments systematically. Finally, we conclude that using an information-seeking objective is beneficial in sparse environments and allows the agent to solve tasks in which methods that do not exhibit directed exploration fail. |
- Schneider, T.; Belousov, B.; Chalvatzaki, G.; Romeres, D.; Jha, D.K.; Peters, J. (2022). Active Exploration for Robotic Manipulation, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).