Carlo D'Eramo

I am now a Professor of Reinforcement Learning and Computational Decision-Making at University of Würzburg. I will keep the position of group leader of the LiteRL group at hessian.AI until 2025.

Carlo D'Eramo is an independent research group leader of the LiteRL group. Previously, Carlo has been a postdoctoral researcher at the Intelligent Autonomous Systems group from April 2019 to October 2022, after receiving his Ph.D. in Information Technology from Politecnico di Milano (Milan, Italy) in February 2019.

During several years of research, Carlo gained extensive experience in RL and provided key methodological advances in several related topics. He has made important and well-recognized contributions to uncertainty quantification and exploitation in RL, multi-task and curriculum RL, skill decomposition, residual learning, and planning. Moreover, he is the developer of MushroomRL, a widely accepted RL library for simplifying the implementation of RL experiments. The work of Carlo has been broadly published in top ML and Robotics conferences, e.g., ICML, NeurIPS, AAAI, ICLR, ICRA, RSS, and journals, e.g., JMLR, Frontiers in Robotics and AI.

He is currently conducting research revolving around the problem of how agents can efficiently acquire expert skills that account for the complexity of the real world. To answer this question, he is investigating lightweight methods to obtain adaptive autonomous agents, focusing on several RL topics including multi-task, curriculum, adversarial, options, and multi-agent RL.

Prior to his Ph.D. thesis, in 2015 he was awarded a double MSc in Computer Engineering at Politecnico di Milano and University of Illinois at Chicago (UIC), and in 2011 a BSc in Computer Engineering at Politecnico di Milano.

Key references

Dam, T.; D'Eramo, C.; Peters, J.; Pajarinen, J. (submitted). A Unified Perspective on Value Backup and Exploration in Monte-Carlo Tree Search, Submitted to the Journal of Machine Learning Research (JMLR).   Download Article [PDF]   BibTeX Reference [BibTex]

Klink, P.; D`Eramo, C.; Peters, J.; Pajarinen, J. (submitted). On the Benefit of Optimal Transport for Curriculum Reinforcement Learning, IEEE Transaction on Pattern Analysis and Machine Intelligence (PAMI).   BibTeX Reference [BibTex]

Klink, P.; D`Eramo, C.; Peters, J.; Pajarinen, J. (submitted). On the Benefit of Optimal Transport for Curriculum Reinforcement Learning, Submitted to the IEEE Transaction on Pattern Analysis and Machine Intelligence (PAMI).   BibTeX Reference [BibTex]

Urain, J.; Li, A.; Liu, P.; D'Eramo, C.; Peters, J. (in press). Composable energy policies for reactive motion generation and reinforcement learning, International Journal of Robotics Research (IJRR).   BibTeX Reference [BibTex]

Parisi, S.; Tateo, D.; Hensel, M.; D'Eramo, C.; Peters, J.; Pajarinen, J. (2022). Long-Term Visitation Value for Deep Exploration in Sparse Reward Reinforcement Learning, Algorithms, 15, 3, pp.81.   Download Article [PDF]   BibTeX Reference [BibTex]

Klink, P.; D`Eramo, C.; Peters, J.; Pajarinen, J. (2022). Boosted Curriculum Reinforcement Learning, International Conference on Learning Representations (ICLR).   Download Article [PDF]   BibTeX Reference [BibTex]

D`Eramo, C.; Chalvatzaki, G. (2022). Prioritized Sampling with Intrinsic Motivation in Multi-Task Reinforcement Learning, International Joint Conference on Neural Networks (IJCNN).   BibTeX Reference [BibTex]

Klink, P.; Yang, H.; D'Eramo, C.; Peters, J.; Pajarinen, J. (2022). Curriculum Reinforcement Learning via Constrained Optimal Transport, International Conference on Machine Learning (ICML).   Download Article [PDF]   BibTeX Reference [BibTex]

Klink, P.; Abdulsamad, H.; Belousov, B.; D'Eramo, C.; Peters, J.; Pajarinen, J. (2021). A Probabilistic Interpretation of Self-Paced Learning with Applications to Reinforcement Learning, Journal of Machine Learning Research (JMLR).   Download Article [PDF]   BibTeX Reference [BibTex]

Morgan, A.; Nandha, D.; Chalvatzaki, G.; D'Eramo, C.; Dollar, A.; Peters, J. (2021). Model Predictive Actor-Critic: Accelerating Robot Skill Acquisition with Deep Reinforcement Learning, Proceedings of the IEEE International Conference on Robotics and Automation (ICRA).   BibTeX Reference [BibTex]

Dam, T.; D'Eramo, C.; Peters, J.; Pajarinen J. (2021). Convex Regularization in Monte-Carlo Tree Search, Proceedings of the International Conference on Machine Learning (ICML).   Download Article [PDF]   BibTeX Reference [BibTex]

Urain, J.; Li, A.; Liu, P.; D'Eramo, C.; Peters, J. (2021). Composable Energy Policies for Reactive Motion Generation and Reinforcement Learning, Robotics: Science and Systems (RSS).   Download Article [PDF]   BibTeX Reference [BibTex]

D`Eramo, C.; Tateo, D.; Bonarini, A.; Restelli, M.; Peters, J. (2021). MushroomRL: Simplifying Reinforcement Learning Research, Journal of Machine Learning Research (JMLR).   BibTeX Reference [BibTex]

D`Eramo, C.; Davide, T; Bonarini, A.; Restelli, M.; Peters, J. (2021). MushroomRL: Simplifying Reinforcement Learning Research, Journal of Machine Learning Research (JMLR), 22, 131, pp.1-5.   Download Article [PDF]   BibTeX Reference [BibTex]

D`Eramo, C.; Cini, A.; Nuara, A.; Pirotta, M.; Alippi, C.; Peters, J.; Restelli, M. (2021). Gaussian Approximation for Bias Reduction in Q-Learning, Journal of Machine Learning Research (JMLR).   BibTeX Reference [BibTex]

Dam, T.; Klink, P.; D'Eramo, C.; Peters, J.; Pajarinen, J. (2020). Generalized Mean Estimation in Monte-Carlo Tree Search, Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI).   Download Article [PDF]   BibTeX Reference [BibTex]

D`Eramo, C.; Tateo, D.; Bonarini, A.; Restelli, M.; Peters, J. (2020). Sharing Knowledge in Multi-Task Deep Reinforcement Learning, International Conference in Learning Representations (ICLR).   Download Article [PDF]   BibTeX Reference [BibTex]

Koert, D.; Kircher, M.; Salikutluk, V.; D'Eramo, C.; Peters, J. (2020). Multi-Channel Interactive Reinforcement Learning for Sequential Tasks, Frontiers in Robotics and AI Human-Robot Interaction.   Download Article [PDF]   BibTeX Reference [BibTex]

Klink, P.; D'Eramo, C.; Peters, J.; Pajarinen, J. (2020). Self-Paced Deep Reinforcement Learning, Advances in Neural Information Processing Systems (NIPS / NeurIPS).   Download Article [PDF]   BibTeX Reference [BibTex]

Tosatto, S.; D'Eramo, C.; Pajarinen, J.; Restelli, M.; Peters, J. (2019). Exploration Driven By an Optimistic Bellman Equation, Proceedings of the International Joint Conference on Neural Networks (IJCNN).   Download Article [PDF]   BibTeX Reference [BibTex]

D`Eramo, C.; Cini, A.; Restelli, M. (2019). Exploiting Action-Value Uncertainty to Drive Exploration in Reinforcement Learning, IJCNN.   BibTeX Reference [BibTex]

Tosatto, S.; D'Eramo, C.; Pajarinen, J.; Restelli, M.; Peters, J. (2018). Technical Report: Exploration Driven by an Optimistic Bellman Equation.   Download Article [PDF]   BibTeX Reference [BibTex]

Tosatto, S.; D'Eramo, C.; Pirotta, M.; Restelli, M. (2017). Boosted Fitted Q-Iteration, Polytechnic University of Milan.   Download Article [PDF]   BibTeX Reference [BibTex]

Tosatto, S.; Pirotta, M.; D'Eramo, C; Restelli, M. (2017). Boosted Fitted Q-Iteration, Proceedings of the International Conference of Machine Learning (ICML).   Download Article [PDF]   BibTeX Reference [BibTex]

D`Eramo, C.; Nuara, A.; Pirotta, M.; Restelli, M. (2017). Estimating the Maximum Expected Value in Continuous Reinforcement Learning Problems, AAAI.   Download Article [PDF]   BibTeX Reference [BibTex]

Tateo, D.; D'Eramo, C.; Nuara, A.; Bonarini, A.; Restelli, M. (2017). Exploiting structure and uncertainty of Bellman updates in Markov decision processes, 2017 IEEE Symposium Series on Computational Intelligence (SSCI).   Download Article [PDF]   BibTeX Reference [BibTex]

D`Eramo, C.; Nuara, A.; Restelli, M. (2016). Estimating the Maximum Expected Value through Gaussian Approximation, ICML.   Download Article [PDF]   BibTeX Reference [BibTex]