Daniel Palenicek

Research Interests

  • Reinforcement Learning
  • Model-based Reinforcement Learning
  • Machine Learning

Affiliations

  1. TU Darmstadt, Intelligent Autonomous Systems, Computer Science Department
  2. Hessian Centre for Artificial Intelligence

Contact Information

daniel.palenicek@tu-darmstadt.de
Room E304, Building S2|02, TU Darmstadt, FB-Informatik, FG-IAS, Hochschulstr. 10, 64289 Darmstadt
+49-6151-16-25387


Reviewing

NeurIPS, ICLR, CoRL, RLC (Senior Reviewer), IROS, and various ML & Robotics workshops.

Teaching Assistant

LectureYears
CE and RoboticsSS 2022, WS 2022/23
Statistical Machine LearningSS 2023, WS 2023/24
Probabilistic Methods in CSWS 2024/25
Robot Learning Integrated Project 1 & 2SS 2025

Daniel joined the Intelligent Autonomous System lab on October 1st, 2021 as a Ph.D. student. He is part of the 3AI project with Hessian.AI. In his research, Daniel focuses on increasing sample efficiency of model-based reinforcement learning algorithms by studying the impact which model-errors have on the learning.

Before starting his Ph.D., Daniel completed his Bachelor's degree and Master's degree in Wirtschaftsinformatik at the Technische Universität Darmstadt. He wrote his Master's thesis entitled "Dyna-Style Model-Based Reinforcement Learning with Value Expansion" in the Computer Science Department under the supervision of Michael Lutter and Jan Peters. During his studies, Daniel further did two research internships, at the Bosch Center for AI and at Huawei Noah’s Ark Lab London.

Publications

  •     Bib
    Palenicek, D.; Lutter, M.; Carvalho, J.; Dennert, D.; Ahmad, F.; Peters, J. (submitted). Diminishing Return of Value Expansion Methods, Submitted to the IEEE Transaction on Pattern Analysis and Machine Intelligence (PAMI).
  •     Bib
    Palenicek, D.; Vogt, F.; Watson, J.; Peters, J. (submitted). Scaling Off-Policy Reinforcement Learning with Batch and Weight Normalization, Under review.
  •     Bib
    Vincent, T.; Palenicek, D.; Belousov, B.; Peters, J.; D'Eramo, C. (2025). Iterated Q-Network: Beyond One-Step Bellman Updates in Deep Reinforcement Learning, Transactions on Machine Learning Research (TMLR).
  •     Bib
    Toelle, M.; Gruner, T.; Palenicek, D.; Schneider, T. Guenster, J.; Watson, J.; Tateo, D.; Liu, P.; Peters, J. (2025). Towards Safe Robot Foundation Models using Inductive Biases, SafeVLM Workshop @ IEEE International Conference on Robotics and Automation (ICRA), [Spotlight].
  •   Bib
    Palenicek, D.; Vogt, F.; Peters, J. (2025). Scaling CrossQ with Weight Normalization, Multi-disciplinary Conference on Reinforcement Learning and Decision Making (RLDM).
  •     Bib
    Lenz, J.; Pfenning, I.; Gruner, T.; Palenicek, D.; Schneider, T.; Peters, J. (2025). Exploring the Role of Vision and Touch in Reinforcement Learning for Dexterous Insertion Tasks, Multi-disciplinary Conference on Reinforcement Learning and Decision Making (RLDM).
  •   Bib
    Celik, O.; Li, Z.; Blessing, D.; Li, G.; Palenicek, D.; Peters, J.; Chalvatzaki, G.; Neumann, G. (2025). Diffusion-Based Maximum Entropy Reinforcement Learning, 7th Robot Learning Workshop: Towards Robots with Human-Level Abilities.
  •     Bib
    Scherer, C. F.; Tölle, M.; Gruner, T.; Palenicek, D.; Schneider, T.; Schramowski, P.; Belousov, B.; Peters, J. (2025). AllmAN: A German Vision-Language-Action Model, German Robotics Conference (GRC).
  •       Bib
    Toelle, M.; Gruner, T.; Palenicek, D.; Guenster, J.; Liu, P.; Watson, J.; Tateo, D.; Peters, J. (2025). Towards Safe Robot Foundation Models, German Robotics Conference (GRC).
  •       Bib
    Bohlinger, N.; Kinzel, J.; Palenicek, D.; Antczak, L.; Peters, J. (2025). Gait in Eight: Efficient On-Robot Learning for Omnidirectional Quadruped Locomotion, International Conference on Intelligent Robots and Systems (IROS).
  •     Bib
    Celik, O.; Li, Z.; Blessing, D.; Li, G.; Palenicek, D.; Peters, J.; Chalvatzaki, G.; Neumann, G. (2025). DIME: Diffusion-Based Maximum Entropy Reinforcement Learning, International Conference on Machine Learning (ICML).
  •   Bib
    Celik, O.; Li, Z.; Blessing, D.; Li, G.; Palenicek, D.; Peters, J.; Chalvatzaki, G.; Neumann, G. (2025). DIME: Diffusion-Based Maximum Entropy Reinforcement Learning, EXAIT Workshop at International Conference on Machine Learning (ICML).
  •       Bib
    Bhatt, A.; Palenicek, D.; Belousov, B.; Argus, M.; Amiranashvili, A.; Brox, T.; Peters, J. (2024). CrossQ: Batch Normalization in Deep Reinforcement Learning for Greater Sample Efficiency and Simplicity, International Conference on Learning Representations (ICLR), [Spotlight].
  •     Bib
    Palenicek, D.; Gruner, T.; Schneider, T.; Böhm, A.; Lenz, J.; Pfenning, I. and Krämer, E.; Peters, J. (2024). Learning Tactile Insertion in the Real World, IEEE ICRA 2024 Workshop on Robot Embodiment through Visuo-Tactile Perception.
  •     Bib
    Palenicek, D.; Gruner, T.; Schneider, T.; Böhm, A.; Lenz, J.; Pfenning, I. and Krämer, E.; Peters, J. (2024). Learning Tactile Insertion in the Real World, 40th Anniversary of the IEEE International Conference on Robotics and Automation (ICRA@40).
  •     Bib
    Bhatt, A.; Palenicek, D.; Belousov, B.; Argus, M.; Amiranashvili, A.; Brox, T.; Peters, J. (2024). CrossQ: Batch Normalization in Deep Reinforcement Learning for Greater Sample Efficiency and Simplicity, European Workshop on Reinforcement Learning (EWRL).
  •     Bib
    Lenz, J.; Gruner, T.; Palenicek, D.; Schneider, T.; Pfenning, I.; Peters J. (2024). Analysing the Interplay of Vision and Touch for Dexterous Insertion Tasks, CoRL 2024 Workshop on Learning Robot Fine and Dexterous Manipulation: Perception and Control.
  •     Bib
    Palenicek, D.; Lutter, M.; Carvalho, J.; Peters, J. (2023). Diminishing Return of Value Expansion Methods in Model-Based Reinforcement Learning, International Conference on Learning Representations (ICLR).
  •     Bib
    Gruner, T.; Belousov, B.; Muratore, F.; Palenicek, D.; Peters, J. (2023). Pseudo-Likelihood Inference, Advances in Neural Information Processing Systems (NIPS / NeurIPS).
  •     Bib
    Cowen-Rivers, A.I.; Palenicek, D.; Moens, V.; Abdullah, M.A.; Sootla, A.; Wang, J.; Bou-Ammar, H. (2022). SAMBA: safe model-based & active reinforcement learning, Machine Learning.
  •     Bib
    Palenicek, D.; Lutter, M., Peters, J. (2022). Revisiting Model-based Value Expansion, Multi-disciplinary Conference on Reinforcement Learning and Decision Making (RLDM).
  •     Bib
    Palenicek, D. (2021). A Survey on Constraining Policy Updates Using the KL Divergence, Reinforcement Learning Algorithms: Analysis and Applications, pp.49-57.
  •       Bib
    Zhou, M.; ...; Palenicek, D; ... (2020). SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Driving, Conference on Robot Learning (CoRL), Best System Paper Award.

Talks and Interviews

Supervised Theses and Projects

Thesis/ProjectTopicStudent(s)Together with
M.Sc. ThesisInvestigating bottlenecks of CrossQ's sample efficiencyVogt F. 
M.Sc. ThesisOn-robot Deep Reinforcement Learning for Quadruped LocomotionKinzel J.Nico Bohlinger
B.Sc. ThesisDiminishing Return of Value Expansion Methods in Offline Model-Based Reinforcement LearningDennert D. 
B.Sc. ThesisDiminishing Return of Value Expansion Methods in Discrete Model-Based Reinforcement LearningAhmad F. 
RL:IP.SS24Training Large Scale Robot Transformer ModelsScherer C.Tim Schneider & Theo Gruner & Maximilian Tölle
RL:IP.SS24XXX: eXploring X-Embodiment with RT-XJacobs T.Tim Schneider & Theo Gruner & Maximilian Tölle
RL:IP.SS24Unveiling the Unseen: Tactile Perception and Reinforcement Learning in the Real WorldBöhm A., Krämer E.Tim Schneider & Theo Gruner
RL:IP.WS23/24XXX: eXploring X-Embodiment with RT-XDennert D., Scherer C., Ahmad F.Tim Schneider & Theo Gruner & Maximilian Tölle
RL:IP.WS23/24XXX: eXploring X-Embodiment with RT-XJacobs T.Tim Schneider & Theo Gruner & Maximilian Tölle
RL:IP.WS23/24Unveiling the Unseen: Tactile Perception and Reinforcement Learning in the Real WorldBöhm A., Pfenning I., Lenz J.Tim Schneider & Theo Gruner
RL:IP.SS23Latent Tactile Representations for Model-based RLKrämer E.Tim Schneider & Theo Gruner