Boris Belousov

Boris Belousov has graduated but remains adjunct to IAS while being a Senior Researcher and Deputy Head at the German Research Center for AI (DFKI), Research Department: Systems AI for Robot Learning (SAIROL). He holds a MSc degree in Electrical Engineering from FAU Erlangen-Nürnberg with a major in Communications and Multimedia Engineering and a BSc degree in Applied Mathematics and Physics from Moscow Institute of Physics and Technology with a specialization in Electrical Engineering and Cybernetics.

The research of Boris Belousov aims towards building an intelligent and autonomous general-purpose robot — an embodied agent. There are many technical challenges that need to be overcome to realize this vision. Driven by the observation that learning is crucial to enabling advanced robotic capabilities, Boris and his team are working on learning-based approaches and developing tools and algorithms to facilitate efficient robot learning. Thus, SAIROL is building simulation environments for robots (e.g., tactile simulation TacEx for manipulation), sample-efficient reinforcement learning (RL) algorithms (e.g., CrossQ, iDQN, AdaQN), controllers based on imitation learning (IL) and optimization (e.g., Diffusion Policy-based and MPC-based controllers for humanoid locomotion), and working towards integrating vision-language models (VLMs) for driving intelligent robot behavior (e.g., by developing generalizable task representations for robotic manipulation).

Publications

Reinforcement Learning

1. •     Bib
     Vincent, T.; Wahren, F.; Peters, J.; Belousov, B.; D'Eramo, C.; (2024). Adaptive Q-Network: On-the-fly Target Selection for Deep Reinforcement Learning, European Workshop on Reinforcement Learning (EWRL).
2. •       Bib
     Bhatt, A.; Palenicek, D.; Belousov, B.; Argus, M.; Amiranashvili, A.; Brox, T.; Peters, J. (2024). CrossQ: Batch Normalization in Deep Reinforcement Learning for Greater Sample Efficiency and Simplicity, International Conference on Learning Representations (ICLR), Spotlight.
3. •     Bib
     Vincent, T.; Metelli, A.; Belousov, B.; Peters, J.; Restelli, M.; D'Eramo, C. (2024). Parameterized Projected Bellman Operator, Proceedings of the National Conference on Artificial Intelligence (AAAI).
4. •     Bib
     Faust, T.L.; Maraqten, H.; Aghadavoodi, E.; Belousov, B.; Peters, J. (2024). Velocity-History-Based Soft Actor-Critic: Tackling IROS'24 Competition AI Olympics with RealAIGym, IROS'24 Competition AI Olympics with RealAIGym.
5. •     Bib
     Wiebe, F.; Turcato, N.; Dalla Libera, A.; Zhang, C.; Vincent, T.; Vyas, S.; Giacomuzzo, G.; Carli, R.; Romeres, D.; Sathuluri, A.; Zimmermann, M.; Belousov, B.; Peters, J.; Kirchner, F.; Kumar, S. (2024). Reinforcement Learning for Athletic Intelligence: Lessons from the 1st “AI Olympics with RealAIGym” Competition, The 33rd International Joint Conference on Artificial Intelligence.
6. •     Bib
     Vincent, T.; Belousov, B.; D'Eramo, C.; Peters, J. (2023). Iterated Deep Q-Network: Efficient Learning of Bellman Iterations for Deep Reinforcement Learning, European Workshop on Reinforcement Learning (EWRL).
7. •       Bib
     Lutter, M.; Belousov, B.; Mannor, S.; Fox, D.; Garg, A.; Peters, J. (2023). Continuous-Time Fitted Value Iteration for Robust Policies, IEEE Transaction on Pattern Analysis and Machine Intelligence (PAMI).
8. •     Bib
     Belousov, B.; Abdulsamad H.; Klink, P.; Parisi, S.; Peters, J. (2021). Reinforcement Learning Algorithms: Analysis and Applications, Studies in Computational Intelligence, Springer International Publishing.

Humanoid Control

1. •       Bib
     Kaidanov, O.; Al-Hafez, F.; Süvari, Y.; Belousov, B.; Peters, J. (2024). The Role of Domain Randomization in Training Diffusion Policies for Whole-Body Humanoid Control, CoRL 2024 Workshop on Whole-body Control and Bimanual Manipulation: Applications in Humanoids and Beyond.
2. •     Bib
     Meser, M.; Bhatt, A.; Belousov, B.; Peters, J. (2024). MuJoCo MPC for Humanoid Control: Evaluation on HumanoidBench, 40th Anniversary of the IEEE International Conference on Robotics and Automation (ICRA@40).
3. •     Bib
     Galljamov, R.; Zhao, G.; Belousov, B.; Seyfarth, A.; Peters, J. (2022). Improving Sample Efficiency of Example-Guided Deep Reinforcement Learning for Bipedal Walking, 2022 IEEE-RAS 21st International Conference on Humanoid Robots (Humanoids).

Bayesian Inference

1. •     Bib
     Watson, J.; Hahner, B.; Belousov, B.; Peters, J. (2024). Tractable Bayesian Dynamics Priors from Differentiable Physics for Learning and Control, 40th Anniversary of the IEEE International Conference on Robotics and Automation (ICRA@40).
2. •     Bib
     Gruner, T.; Belousov, B.; Muratore, F.; Palenicek, D.; Peters, J. (2023). Pseudo-Likelihood Inference, Advances in Neural Information Processing Systems (NIPS / NeurIPS).
3. •     Bib
     Muratore, F.; Gruner, T.; Wiese, F.; Belousov, B.; Gienger, M.; Peters, J. (2021). Neural Posterior Domain Randomization, Conference on Robot Learning (CoRL).

Foundation Models

1. •     Bib
     Toelle, M.; Belousov, B.; Peters, J. (2023). A Unifying Perspective on Language-Based Task Representations for Robot Control, CoRL Workshop on Language and Robot Learning: Language as Grounding.
2. •       Bib
     Siebenborn, M.; Belousov, B.; Huang, J.; Peters, J. (2022). How Crucial is Transformer in Decision Transformer?, Foundation Models for Decision Making Workshop at Neural Information Processing Systems.

Tactile Sensing

1. •     Bib
     Nguyen, D.H.; Schneider, T.; Duret, G.; Kshirsagar, A.; Belousov, B.; Peters, J. (2025). TacEx: GelSight Tactile Simulation in Isaac Sim – Combining Soft-Body and Visuotactile Simulators, German Robotics Conference (GRC).
   •       Bib
     Nguyen, D.H.; Schneider, T.; Duret, G.; Kshirsagar, A.; Belousov, B.; Peters, J. (2024). TacEx: GelSight Tactile Simulation in Isaac Sim – Combining Soft-Body and Visuotactile Simulators, CoRL 2024 Workshop on Learning Robot Fine and Dexterous Manipulation: Perception and Control.
2. •     Bib
     Kshirsagar, A.; Heller, F.; Gomez Andreu, M.; Belousov, B.; Schneider, T.; Lin, L. P. Y.; Doerschner, K.; Drewing, K.; Peters, J. (2024). Hardness Similarity Detection Using Vision-Based Tactile Sensors, 40th Anniversary of the IEEE International Conference on Robotics and Automation (ICRA@40).
3. •       Bib
     Helmut, E.; Dziarski, L.; Funk, N.; Belousov, B.; Peters, J. (2024). Learning Force Distribution Estimation for the GelSight Mini Optical Tactile Sensor Based on Finite Element Analysis, 2nd NeurIPS Workshop on Touch Processing: From Data to Knowledge.
4. •     Bib
     Boehm, A.; Schneider, T.; Belousov, B.; Kshirsagar, A.; Lin, L.; Doerschner, K.; Drewing, K.; Rothkopf, C.A.; Peters, J. (2024). What Matters for Active Texture Recognition With Vision-Based Tactile Sensors, Proceedings of the IEEE International Conference on Robotics and Automation (ICRA).
5. •       Bib
     Funk, N.; Mueller, P.-O.; Belousov, B.; Savchenko, A.; Findeisen, R.; Peters, J. (2023). High-Resolution Pixelwise Contact Area and Normal Force Estimation for the GelSight Mini Visuotactile Sensor Using Neural Networks, Embracing Contacts-Workshop at ICRA 2023.
6. •     Bib
     Zhu, Y.; Nazirjonov, S.; Jiang, B.; Colan, J.; Aoyama, T.; Hasegawa, Y.; Belousov, B.; Hansel, K.; Peters, J. (2023). Visual Tactile Sensor Based Force Estimation for Position-Force Teleoperation, IEEE International Conference on Cyborg and Bionic Systems (CBS), pp.49-52.
7. •     Bib
     Boehm, A.; Schneider, T.; Belousov, B.; Kshirsagar, A.; Lin, L.; Doerschner, K.; Drewing, K.; Rothkopf, C.A.; Peters, J. (2023). Tactile Active Texture Recognition With Vision-Based Tactile Sensors, NeurIPS Workshop on Touch Processing: a new Sensing Modality for AI.
8. •     Bib
     Belousov, B.; Sadybakasov, A.; Wibranek, B.; Veiga, F.; Tessmann, O.; Peters, J. (2019). Building a Library of Tactile Skills Based on FingerVision, Proceedings of the 2019 IEEE-RAS 19th International Conference on Humanoid Robots (Humanoids).

Robotic Architectural Assembly

1. •       Bib
     Liu, Y.; Belousov, B.; Schneider, T.; Harsono, K.; Cheng, T.W.; Shih, S.G.; Tessmann, O.; Peters, J. (2024). Advancing Sustainable Construction: Discrete Modular Systems & Robotic Assembly, Sustainability, 16, pp.6678, MDPI.
2. •     Bib
     Liu, Y.; Belousov, B.; Funk, N.; Chalvatzaki, G.; Peters, J.; Tessman, O. (2023). Auto(mated)nomous Assembly, International Conference on Trends on Construction in the Post-Digital Era, pp.167-181, Springer, Cham.
3. •       Bib
     Belousov, B.; Wibranek, B.; Schneider, J.; Schneider, T.; Chalvatzaki, G.; Peters, J.; Tessmann, O. (2022). Robotic Architectural Assembly with Tactile Skills: Simulation and Optimization, Automation in Construction, 133, pp.104006.
4. •       Bib
     Funk, N.; Chalvatzaki, G.; Belousov, B.; Peters, J. (2021). Learn2Assemble with Structured Representations and Search for Robotic Architectural Construction, Conference on Robot Learning (CoRL).
5. •     Bib
     Wibranek, B.; Liu, Y.; Funk, N.; Belousov, B.; Peters, J.; Tessmann, O. (2021). Reinforcement Learning for Sequential Assembly of SL-Blocks: Self-Interlocking Combinatorial Design Based on Machine Learning, Proceedings of the 39th eCAADe Conference.
6. •     Bib
     Wibranek, B.; Belousov, B.; Sadybakasov, A.; Peters, J.; Tessmann, O. (2019). Interactive Structure: Robotic Repositioning of Vertical Elements in Man-Machine Collaborative Assembly through Vision-Based Tactile Sensing, Proceedings of the 37th eCAADe and 23rd SIGraDi Conference.
7. •     Bib
     Wibranek, B.; Belousov, B.; Sadybakasov, A.; Tessmann, O. (2019). Interactive Assemblies: Man-Machine Collaboration through Building Components for As-Built Digital Models, Computer-Aided Architectural Design Futures (CAAD Futures).

Active Exploration

1. •       Bib
     Schneider, T.; Belousov, B.; Chalvatzaki, G.; Romeres, D.; Jha, D.K.; Peters, J. (2022). Active Exploration for Robotic Manipulation, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).
2. •       Bib
     Schneider, T.; Belousov, B.; Abdulsamad, H.; Peters, J. (2022). Active Inference for Robotic Manipulation, 5th Multidisciplinary Conference on Reinforcement Learning and Decision Making (RLDM).
3. •     Bib
     Belousov, B.; Abdulsamad, H.; Schultheis, M.; Peters, J. (2019). Belief Space Model Predictive Control for Approximately Optimal System Identification, 4th Multidisciplinary Conference on Reinforcement Learning and Decision Making (RLDM).
4. •     Bib
     Schultheis, M.; Belousov, B.; Abdulsamad, H.; Peters, J. (2019). Receding Horizon Curiosity, Proceedings of the 3rd Conference on Robot Learning (CoRL).

Self-Paced Curriculum Learning

1. •     Bib
     Klink, P.; Abdulsamad, H.; Belousov, B.; D'Eramo, C.; Peters, J.; Pajarinen, J. (2021). A Probabilistic Interpretation of Self-Paced Learning with Applications to Reinforcement Learning, Journal of Machine Learning Research (JMLR).
2. •     Bib
     Klink, P.; Abdulsamad, H.; Belousov, B.; Peters, J. (2019). Self-Paced Contextual Reinforcement Learning, Proceedings of the 3rd Conference on Robot Learning (CoRL).

Risk-Sensitive and Distributionally-Robust Control

1. •     Bib
     Abdulsamad, H.; Dorau, T.; Belousov, B.; Zhu, J.-J; Peters, J. (2021). Distributionally Robust Trajectory Optimization Under Uncertain Dynamics via Relative Entropy Trust-Regions, arXiv.
2. •     Bib
     Nass, D.; Belousov, B.; Peters, J. (2019). Entropic Risk Measure in Policy Search, Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

MaxEnt RL

1. •       Bib
     Belousov, B.; Peters, J. (2019). Entropic Regularization of Markov Decision Processes, Entropy, 21, 7, MDPI.
2. •     Bib
     Belousov, B.; Peters, J. (2018). Mean Squared Advantage Minimization as a Consequence of Entropic Policy Improvement Regularization, European Workshops on Reinforcement Learning (EWRL).
3. •     Bib
     Belousov, B.; Peters, J. (2017). f-Divergence Constrained Policy Improvement, arXiv.

Stochastic Optimal Control

1. •     Bib
     Eilers, C.; Eschmann, J.; Menzenbach, R.; Belousov, B.; Muratore, F.; Peters, J. (2020). Underactuated Waypoint Trajectory Optimization for Light Painting Photography, Proceedings of the IEEE International Conference on Robotics and Automation (ICRA).
2. •   Bib
     Lutter, M.; Clever, D.; Belousov, B.; Listmann, K.; Peters, J. (2020). Evaluating the Robustness of HJB Optimal Feedback Control, International Symposium on Robotics.
3. •     Bib
     Lutter, M.; Belousov, B.; Listmann, K.; Clever, D.; Peters, J. (2019). HJB Optimal Feedback Control with Deep Differential Value Functions and Action Constraints, Conference on Robot Learning (CoRL).
4. •     Bib
     Belousov, B.; Neumann, G.; Rothkopf, C.; Peters, J. (2016). Catching Heuristics Are Optimal Control Policies, Advances in Neural Information Processing Systems (NIPS / NeurIPS).