Carlo D Eramo

I am now a Professor of Reinforcement Learning and Computational Decision-Making at University of Würzburg. I will keep the position of group leader of the LiteRL group at hessian.AI until 2025.

Research Interests

Reinforcement Learning, Decision-Making, Multi-task / Curriculum Reinforcement Learning, Multi-Agent Reinforcement Learning, Deep Reinforcement Learning

Affiliations

1. University of Würzburg, Reinforcement Learning and Computational Decision-Making
2. TU Darmstadt, Intelligent Autonomous Systems, Computer Science Department
3. Hessian Centre for Artificial Intelligence

Contact

carlo.deramo@tu-darmstadt.de
Room E323, Building S2|02, TU Darmstadt, FB-Informatik, FG-IAS, Hochschulstr. 10, 64289 Darmstadt
+49-6151-16-25376

Carlo D'Eramo is an independent research group leader of the LiteRL group. Previously, Carlo has been a postdoctoral researcher at the Intelligent Autonomous Systems group from April 2019 to October 2022, after receiving his Ph.D. in Information Technology from Politecnico di Milano (Milan, Italy) in February 2019.

During several years of research, Carlo gained extensive experience in RL and provided key methodological advances in several related topics. He has made important and well-recognized contributions to uncertainty quantification and exploitation in RL, multi-task and curriculum RL, skill decomposition, residual learning, and planning. Moreover, he is the developer of MushroomRL, a widely accepted RL library for simplifying the implementation of RL experiments. The work of Carlo has been broadly published in top ML and Robotics conferences, e.g., ICML, NeurIPS, AAAI, ICLR, ICRA, RSS, and journals, e.g., JMLR, Frontiers in Robotics and AI.

He is currently conducting research revolving around the problem of how agents can efficiently acquire expert skills that account for the complexity of the real world. To answer this question, he is investigating lightweight methods to obtain adaptive autonomous agents, focusing on several RL topics including multi-task, curriculum, adversarial, options, and multi-agent RL.

Prior to his Ph.D. thesis, in 2015 he was awarded a double MSc in Computer Engineering at Politecnico di Milano and University of Illinois at Chicago (UIC), and in 2011 a BSc in Computer Engineering at Politecnico di Milano.

Publications

  •     Bib
    Dam, T.; D'Eramo, C.; Peters, J.; Pajarinen, J. (submitted). A Unified Perspective on Value Backup and Exploration in Monte-Carlo Tree Search, Submitted to the Journal of Artificial Intelligence Research (JAIR).
  •       Bib
    Klink, P.; D'Eramo, C.; Peters, J.; Pajarinen, J. (submitted). On the Benefit of Optimal Transport for Curriculum Reinforcement Learning, Submitted to the IEEE Transaction on Pattern Analysis and Machine Intelligence (T-PAMI).
  •     Bib
    Urain, J.; Li, A.; Liu, P.; D'Eramo, C.; Peters, J. (2023). Composable energy policies for reactive motion generation and reinforcement learning, International Journal of Robotics Research (IJRR).
  •     Bib
    Vincent, T.; Belousov, B.; D'Eramo, C.; Peters, J. (2023). Iterated Deep Q-Network: Efficient Learning of Bellman Iterations for Deep Reinforcement Learning, European Workshop on Reinforcement Learning (EWRL).
  •     Bib
    Vincent, T.; Maria Metelli, A.; Peters, J.; Restelli, M.; D'Eramo, C. (2023). Parameterized projected Bellman operator, ICML Workshop on New Frontiers in Learning, Control, and Dynamical Systems.
  •   Bib
    Mittenbuehler, M.; Hendawy, A.; D'Eramo, C.; Chalvatzaki, G. (2023). Parameter-efficient Tuning of Pretrained Visual-Language Models in Multitask Robot Learning, CoRL 2023 Workshop on Learning Effective Abstractions for Planning (LEAP).
  •   Bib
    Metternich, H.; Hendawy, A.; Klink, P.; Peters, J.; D'Eramo, C. (2023). Using Proto-Value Functions for Curriculum Generation in Goal-Conditioned RL, NeurIPS 2023 Workshop on Goal-Conditioned Reinforcement Learning.
  •     Bib
    Parisi, S.; Tateo, D.; Hensel, M.; D'Eramo, C.; Peters, J.; Pajarinen, J. (2022). Long-Term Visitation Value for Deep Exploration in Sparse-Reward Reinforcement Learning, Algorithms, 15, 3, pp.81.
  •     Bib
    Klink, P.; D`Eramo, C.; Peters, J.; Pajarinen, J. (2022). Boosted Curriculum Reinforcement Learning, International Conference on Learning Representations (ICLR).
  •   Bib
    D`Eramo, C.; Chalvatzaki, G. (2022). Prioritized Sampling with Intrinsic Motivation in Multi-Task Reinforcement Learning, International Joint Conference on Neural Networks (IJCNN).
  •     Bib
    Klink, P.; Yang, H.; D'Eramo, C.; Peters, J.; Pajarinen, J. (2022). Curriculum Reinforcement Learning via Constrained Optimal Transport, International Conference on Machine Learning (ICML).
  •     Bib
    Klink, P.; Abdulsamad, H.; Belousov, B.; D'Eramo, C.; Peters, J.; Pajarinen, J. (2021). A Probabilistic Interpretation of Self-Paced Learning with Applications to Reinforcement Learning, Journal of Machine Learning Research (JMLR).
  •     Bib
    Morgan, A.; Nandha, D.; Chalvatzaki, G.; D'Eramo, C.; Dollar, A.; Peters, J. (2021). Model Predictive Actor-Critic: Accelerating Robot Skill Acquisition with Deep Reinforcement Learning, Proceedings of the IEEE International Conference on Robotics and Automation (ICRA).
  •     Bib
    Dam, T.; D'Eramo, C.; Peters, J.; Pajarinen J. (2021). Convex Regularization in Monte-Carlo Tree Search, Proceedings of the International Conference on Machine Learning (ICML).
  •     Bib
    Urain, J.; Li, A.; Liu, P.; D'Eramo, C.; Peters, J. (2021). Composable Energy Policies for Reactive Motion Generation and Reinforcement Learning, Robotics: Science and Systems (RSS).
  •     Bib
    D`Eramo, C.; Tateo, D; Bonarini, A.; Restelli, M.; Peters, J. (2021). MushroomRL: Simplifying Reinforcement Learning Research, Journal of Machine Learning Research (JMLR), 22, 131, pp.1-5.
  •     Bib
    D`Eramo, C.; Cini, A.; Nuara, A.; Pirotta, M.; Alippi, C.; Peters, J.; Restelli, M. (2021). Gaussian Approximation for Bias Reduction in Q-Learning, Journal of Machine Learning Research (JMLR).
  •       Bib
    Dam, T.; Klink, P.; D'Eramo, C.; Peters, J.; Pajarinen, J. (2020). Generalized Mean Estimation in Monte-Carlo Tree Search, Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI).
  •     Bib
    D`Eramo, C.; Tateo, D.; Bonarini, A.; Restelli, M.; Peters, J. (2020). Sharing Knowledge in Multi-Task Deep Reinforcement Learning, International Conference in Learning Representations (ICLR).
  •       Bib
    Koert, D.; Kircher, M.; Salikutluk, V.; D'Eramo, C.; Peters, J. (2020). Multi-Channel Interactive Reinforcement Learning for Sequential Tasks, Frontiers in Robotics and AI Human-Robot Interaction.
  •     Bib
    Klink, P.; D'Eramo, C.; Peters, J.; Pajarinen, J. (2020). Self-Paced Deep Reinforcement Learning, Advances in Neural Information Processing Systems (NIPS / NeurIPS).
  •     Bib
    Tosatto, S.; D'Eramo, C.; Pajarinen, J.; Restelli, M.; Peters, J. (2019). Exploration Driven By an Optimistic Bellman Equation, Proceedings of the International Joint Conference on Neural Networks (IJCNN).
  •   Bib
    D`Eramo, C.; Cini, A.; Restelli, M. (2019). Exploiting Action-Value Uncertainty to Drive Exploration in Reinforcement Learning, IJCNN.
  •       Bib
    Tosatto, S.; D'Eramo, C.; Pajarinen, J.; Restelli, M.; Peters, J. (2018). Technical Report: Exploration Driven by an Optimistic Bellman Equation.
  •     Bib
    Tosatto, S.; D'Eramo, C.; Pirotta, M.; Restelli, M. (2017). Boosted Fitted Q-Iteration, Polytechnic University of Milan.
  •     Bib
    D`Eramo, C.; Nuara, A.; Pirotta, M.; Restelli, M. (2017). Estimating the Maximum Expected Value in Continuous Reinforcement Learning Problems, AAAI.
  •       Bib
    Tateo, D.; D'Eramo, C.; Nuara, A.; Bonarini, A.; Restelli, M. (2017). Exploiting structure and uncertainty of Bellman updates in Markov decision processes, 2017 IEEE Symposium Series on Computational Intelligence (SSCI).
  •     Bib
    D`Eramo, C.; Nuara, A.; Restelli, M. (2016). Estimating the Maximum Expected Value through Gaussian Approximation, ICML.