Carlo D Eramo

I am now a Professor of Reinforcement Learning and Computational Decision-Making at University of Würzburg. I will keep the position of group leader of the LiteRL group at hessian.AI until 2025.

Research Interests

Reinforcement Learning, Decision-Making, Multi-task / Curriculum Reinforcement Learning, Multi-Agent Reinforcement Learning, Deep Reinforcement Learning

Affiliations

1. University of Würzburg, Reinforcement Learning and Computational Decision-Making
2. TU Darmstadt, Intelligent Autonomous Systems, Computer Science Department
3. Hessian Centre for Artificial Intelligence

Contact

carlo.deramo@tu-darmstadt.de
Room E323, Building S2|02, TU Darmstadt, FB-Informatik, FG-IAS, Hochschulstr. 10, 64289 Darmstadt
+49-6151-16-25376

Carlo D'Eramo is an independent research group leader of the LiteRL group. Previously, Carlo has been a postdoctoral researcher at the Intelligent Autonomous Systems group from April 2019 to October 2022, after receiving his Ph.D. in Information Technology from Politecnico di Milano (Milan, Italy) in February 2019.

During several years of research, Carlo gained extensive experience in RL and provided key methodological advances in several related topics. He has made important and well-recognized contributions to uncertainty quantification and exploitation in RL, multi-task and curriculum RL, skill decomposition, residual learning, and planning. Moreover, he is the developer of MushroomRL, a widely accepted RL library for simplifying the implementation of RL experiments. The work of Carlo has been broadly published in top ML and Robotics conferences, e.g., ICML, NeurIPS, AAAI, ICLR, ICRA, RSS, and journals, e.g., JMLR, Frontiers in Robotics and AI.

He is currently conducting research revolving around the problem of how agents can efficiently acquire expert skills that account for the complexity of the real world. To answer this question, he is investigating lightweight methods to obtain adaptive autonomous agents, focusing on several RL topics including multi-task, curriculum, adversarial, options, and multi-agent RL.

Prior to his Ph.D. thesis, in 2015 he was awarded a double MSc in Computer Engineering at Politecnico di Milano and University of Illinois at Chicago (UIC), and in 2011 a BSc in Computer Engineering at Politecnico di Milano.

Publications

  •     Bib
    Watson, J.; Song, C.; Weeger, O.; Gruner, T.; Le, A.T.; Hansel, K.; Headway, A.; Arenz, O.; Trojak, W.; Cranmer, M.; D’Eramo, C.; Bülow, F.; Goyal, T.; Peters, J.; Hoffman, M.W.; (submitted). Machine Learning with Physics Knowledge for Prediction: A Survey, Transactions on Machine Learning Research (TMLR).
  •     Bib
    Klink, P.; D'Eramo, C.; Peters, J.; Pajarinen, J. (in press). On the Benefit of Optimal Transport for Curriculum Reinforcement Learning, IEEE Transaction on Pattern Analysis and Machine Intelligence (PAMI).
  •     Bib
    Vincent, T.; Palenicek, D.; Belousov, B.; Peters, J.; D'Eramo, C. (2025). Iterated Q-Network: Beyond One-Step Bellman Updates in Deep Reinforcement Learning, Transactions on Machine Learning Research (TMLR).
  •     Bib
    Vincent, T.; Wahren, F.; Peters, J.; Belousov, B.; D'Eramo, C. (2025). Adaptive Q-Network: On-the-fly Target Selection for Deep Reinforcement Learning, International Conference on Learning Representations (ICLR).
  •   Bib
    Vincent, T.; Faust, T.; Tripathi, Y.; Peters, J.; D'Eramo, C. (2025). Eau De Q-Network: Adaptive Distillation of Neural Networks in Deep Reinforcement Learning, Conference on Reinforcement Learning and Decision Making (RLDM).
  •     Bib
    Dam, T.; D'Eramo, C.; Peters, J.; Pajarinen, J. (2024). A Unified Perspective on Value Backup and Exploration in Monte-Carlo Tree Search, Journal of Artificial Intelligence Research (JAIR), 81, pp.511-577.
  •     Bib
    Vincent, T.; Metelli, A.; Belousov, B.; Peters, J.; Restelli, M.; D'Eramo, C. (2024). Parameterized Projected Bellman Operator, Proceedings of the National Conference on Artificial Intelligence (AAAI).
  •     Bib
    Tiboni, G.; Klink, P.; Peters, J.; Tommasi, T.; D'Eramo, C.; Chalvatzaki, G. (2024). Domain Randomization via Entropy Maximization, International Conference on Learning Representations (ICLR).
  •       Bib
    Hendawy, A.; Peters, J.; D'Eramo, C. (2024). Multi-Task Reinforcement Learning with Mixture of Orthogonal Experts, International Conference on Learning Representations (ICLR).
  •       Bib
    Reddi, A.; Toelle, M.; Peters, J.; Chalvatzaki, G.; D'Eramo, C. (2024). Robust Adversarial Reinforcement Learning via Bounded Rationality Curricula, International Conference on Learning Representations (ICLR), Spotlight.
  •     Bib
    Vincent, T.; Wahren, F.; Peters, J.; Belousov, B.; D'Eramo, C.; (2024). Adaptive Q-Network: On-the-fly Target Selection for Deep Reinforcement Learning, European Workshop on Reinforcement Learning (EWRL).
  •     Bib
    Vincent, T.; Wahren, F.; Peters, J.; Belousov, B.; D'Eramo, C.; (2024). Adaptive Q-Network: On-the-fly Target Selection for Deep Reinforcement Learning, ICML Workshop on Automated Reinforcement Learning.
  •     Bib
    Holgado-Alvarez, J.H.; Reddi, A.; D'Eramo, C. (2024). Dynamic Obstacle Avoidance with Bounded Rationality Adversarial Reinforcement Learning, CoRL 2024 Locolearn Workshop.
  •     Bib
    Urain, J.; Li, A.; Liu, P.; D'Eramo, C.; Peters, J. (2023). Composable energy policies for reactive motion generation and reinforcement learning, International Journal of Robotics Research (IJRR).
  •     Bib
    Vincent, T.; Belousov, B.; D'Eramo, C.; Peters, J. (2023). Iterated Deep Q-Network: Efficient Learning of Bellman Iterations for Deep Reinforcement Learning, European Workshop on Reinforcement Learning (EWRL).
  •     Bib
    Vincent, T.; Metelli, A.; Peters, J.; Restelli, M.; D'Eramo, C. (2023). Parameterized projected Bellman operator, ICML Workshop on New Frontiers in Learning, Control, and Dynamical Systems.
  •   Bib
    Mittenbuehler, M.; Hendawy, A.; D'Eramo, C.; Chalvatzaki, G. (2023). Parameter-efficient Tuning of Pretrained Visual-Language Models in Multitask Robot Learning, CoRL 2023 Workshop on Learning Effective Abstractions for Planning (LEAP).
  •   Bib
    Metternich, H.; Hendawy, A.; Klink, P.; Peters, J.; D'Eramo, C. (2023). Using Proto-Value Functions for Curriculum Generation in Goal-Conditioned RL, NeurIPS 2023 Workshop on Goal-Conditioned Reinforcement Learning.
  •     Bib
    Parisi, S.; Tateo, D.; Hensel, M.; D'Eramo, C.; Peters, J.; Pajarinen, J. (2022). Long-Term Visitation Value for Deep Exploration in Sparse-Reward Reinforcement Learning, Algorithms, 15, 3, pp.81.
  •     Bib
    Klink, P.; D`Eramo, C.; Peters, J.; Pajarinen, J. (2022). Boosted Curriculum Reinforcement Learning, International Conference on Learning Representations (ICLR).
  •   Bib
    D`Eramo, C.; Chalvatzaki, G. (2022). Prioritized Sampling with Intrinsic Motivation in Multi-Task Reinforcement Learning, International Joint Conference on Neural Networks (IJCNN).
  •     Bib
    Klink, P.; Yang, H.; D'Eramo, C.; Peters, J.; Pajarinen, J. (2022). Curriculum Reinforcement Learning via Constrained Optimal Transport, International Conference on Machine Learning (ICML).
  •     Bib
    Klink, P.; Abdulsamad, H.; Belousov, B.; D'Eramo, C.; Peters, J.; Pajarinen, J. (2021). A Probabilistic Interpretation of Self-Paced Learning with Applications to Reinforcement Learning, Journal of Machine Learning Research (JMLR).
  •     Bib
    Morgan, A.; Nandha, D.; Chalvatzaki, G.; D'Eramo, C.; Dollar, A.; Peters, J. (2021). Model Predictive Actor-Critic: Accelerating Robot Skill Acquisition with Deep Reinforcement Learning, Proceedings of the IEEE International Conference on Robotics and Automation (ICRA).
  •     Bib
    Dam, T.; D'Eramo, C.; Peters, J.; Pajarinen J. (2021). Convex Regularization in Monte-Carlo Tree Search, Proceedings of the International Conference on Machine Learning (ICML).
  •     Bib
    Urain, J.; Li, A.; Liu, P.; D'Eramo, C.; Peters, J. (2021). Composable Energy Policies for Reactive Motion Generation and Reinforcement Learning, Robotics: Science and Systems (RSS).
  •     Bib
    D`Eramo, C.; Tateo, D; Bonarini, A.; Restelli, M.; Peters, J. (2021). MushroomRL: Simplifying Reinforcement Learning Research, Journal of Machine Learning Research (JMLR), 22, 131, pp.1-5.
  •     Bib
    D`Eramo, C.; Cini, A.; Nuara, A.; Pirotta, M.; Alippi, C.; Peters, J.; Restelli, M. (2021). Gaussian Approximation for Bias Reduction in Q-Learning, Journal of Machine Learning Research (JMLR).
  •       Bib
    Dam, T.; Klink, P.; D'Eramo, C.; Peters, J.; Pajarinen, J. (2020). Generalized Mean Estimation in Monte-Carlo Tree Search, Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI).
  •     Bib
    D`Eramo, C.; Tateo, D.; Bonarini, A.; Restelli, M.; Peters, J. (2020). Sharing Knowledge in Multi-Task Deep Reinforcement Learning, International Conference in Learning Representations (ICLR).