Boris Belousov

Boris Belousov has graduated but remains adjunct to IAS while being a Senior Researcher and Deputy Head at the German Research Center for AI (DFKI), Research Department: Systems AI for Robot Learning (SAIROL). He holds a MSc degree in Electrical Engineering from FAU Erlangen-Nürnberg with a major in Communications and Multimedia Engineering and a BSc degree in Applied Mathematics and Physics from Moscow Institute of Physics and Technology with a specialization in Electrical Engineering and Cybernetics.

Supervision

Boris Belousov has supervised numerous theses and projects. See Supervised Theses for details.

Teaching

Reinforcement Learning WS'18
Statistical Machine Learning SS'18
Robot Learning IP WS'17

Reviewing

JMLR, NeurIPS, ICML, AAAI, ICLR, CORL, ICRA, IROS, AURO, RA-L, TR-O, R:SS

The research of Boris Belousov aims towards building an intelligent and autonomous general-purpose robot — an embodied agent. There are many technical challenges that need to be overcome to realize this vision. Driven by the observation that learning is crucial to enabling advanced robotic capabilities, Boris and his team are working on learning-based approaches and developing tools and algorithms to facilitate efficient robot learning. Thus, SAIROL is building simulation environments for robots (e.g., tactile simulation for precise manipulation), sample-efficient reinforcement learning (RL) algorithms (e.g., CrossQ, iDQN, and AdaQN), controllers based on imitation learning (IL) and optimization (e.g., AMP- and MPC-based controllers for humanoid locomotion), and working towards integrating vision-language models (VLMs) for driving intelligent robot behavior (e.g., by developing generalizable task representations for robotic manipulation).

Publications

Reinforcement Learning

    •     Bib
      Vincent, T.; Wahren, F.; Peters, J.; Belousov, B.; D'Eramo, C.; (2024). Adaptive Q-Network: On-the-fly Target Selection for Deep Reinforcement Learning, European Workshop on Reinforcement Learning (EWRL).
    •     Bib
      Bhatt, A.; Palenicek, D.; Belousov, B.; Argus, M.; Amiranashvili, A.; Brox, T.; Peters, J. (2024). CrossQ: Batch Normalization in Deep Reinforcement Learning for Greater Sample Efficiency and Simplicity, International Conference on Learning Representations (ICLR), Spotlight.
    •     Bib
      Vincent, T.; Metelli, A.; Belousov, B.; Peters, J.; Restelli, M.; D'Eramo, C. (2024). Parameterized Projected Bellman Operator, Proceedings of the National Conference on Artificial Intelligence (AAAI).
    •     Bib
      Wiebe, F.; Turcato, N.; Dalla Libera, A.; Zhang, C.; Vincent, T.; Vyas, S.; Giacomuzzo, G.; Carli, R.; Romeres, D.; Sathuluri, A.; Zimmermann, M.; Belousov, B.; Peters, J.; Kirchner, F.; Kumar, S. (2024). Reinforcement Learning for Athletic Intelligence: Lessons from the 1st “AI Olympics with RealAIGym” Competition, The 33rd International Joint Conference on Artificial Intelligence.
    •     Bib
      Vincent, T.; Belousov, B.; D'Eramo, C.; Peters, J. (2023). Iterated Deep Q-Network: Efficient Learning of Bellman Iterations for Deep Reinforcement Learning, European Workshop on Reinforcement Learning (EWRL).
    •       Bib
      Lutter, M.; Belousov, B.; Mannor, S.; Fox, D.; Garg, A.; Peters, J. (2023). Continuous-Time Fitted Value Iteration for Robust Policies, IEEE Transaction on Pattern Analysis and Machine Intelligence (PAMI).
    •     Bib
      Belousov, B.; Abdulsamad H.; Klink, P.; Parisi, S.; Peters, J. (2021). Reinforcement Learning Algorithms: Analysis and Applications, Studies in Computational Intelligence, Springer International Publishing.

Humanoid Control

    •       Bib
      Kaidanov, O.; Al-Hafez, F.; Süvari, Y.; Belousov, B.; Peters, J. (2024). The Role of Domain Randomization in Training Diffusion Policies for Whole-Body Humanoid Control, CoRL 2024 Workshop on Whole-body Control and Bimanual Manipulation: Applications in Humanoids and Beyond.
    •     Bib
      Meser, M.; Bhatt, A.; Belousov, B.; Peters, J. (2024). MuJoCo MPC for Humanoid Control: Evaluation on HumanoidBench, 40th Anniversary of the IEEE International Conference on Robotics and Automation (ICRA@40).
    •     Bib
      Galljamov, R.; Zhao, G.; Belousov, B.; Seyfarth, A.; Peters, J. (2022). Improving Sample Efficiency of Example-Guided Deep Reinforcement Learning for Bipedal Walking, 2022 IEEE-RAS 21st International Conference on Humanoid Robots (Humanoids).

Bayesian Inference

    •     Bib
      Watson, J.; Hahner, B.; Belousov, B.; Peters, J. (2024). Tractable Bayesian Dynamics Priors from Differentiable Physics for Learning and Control, 40th Anniversary of the IEEE International Conference on Robotics and Automation (ICRA@40).
    •     Bib
      Gruner, T.; Belousov, B.; Muratore, F.; Palenicek, D.; Peters, J. (2023). Pseudo-Likelihood Inference, Advances in Neural Information Processing Systems (NIPS / NeurIPS).
    •     Bib
      Muratore, F.; Gruner, T.; Wiese, F.; Belousov, B.; Gienger, M.; Peters, J. (2021). Neural Posterior Domain Randomization, Conference on Robot Learning (CoRL).

Foundation Models

    •     Bib
      Toelle, M.; Belousov, B.; Peters, J. (2023). A Unifying Perspective on Language-Based Task Representations for Robot Control, CoRL Workshop on Language and Robot Learning: Language as Grounding.
    •       Bib
      Siebenborn, M.; Belousov, B.; Huang, J.; Peters, J. (2022). How Crucial is Transformer in Decision Transformer?, Foundation Models for Decision Making Workshop at Neural Information Processing Systems.

Tactile Sensing

    •     Bib
      Nguyen, D.H.; Schneider, T.; Duret, G.; Kshirsagar, A.; Belousov, B.; Peters, J. (2024). TacEx: GelSight Tactile Simulation in Isaac Sim – Combining Soft-Body and Visuotactile Simulators, CoRL 2024 Workshop on Learning Robot Fine and Dexterous Manipulation: Perception and Control.
    •     Bib
      Kshirsagar, A.; Heller, F.; Gomez Andreu, M.; Belousov, B.; Schneider, T.; Lin, L. P. Y.; Doerschner, K.; Drewing, K.; Peters, J. (2024). Hardness Similarity Detection Using Vision-Based Tactile Sensors, 40th Anniversary of the IEEE International Conference on Robotics and Automation (ICRA@40).
    •     Bib
      Helmut, E.; Dziarski, L.; Funk, N.; Belousov, B.; Peters, J. (2024). Learning Force Distribution Estimation for the GelSight Mini Optical Tactile Sensor Based on Finite Element Analysis, 2nd NeurIPS Workshop on Touch Processing: From Data to Knowledge.
    •     Bib
      Boehm, A.; Schneider, T.; Belousov, B.; Kshirsagar, A.; Lin, L.; Doerschner, K.; Drewing, K.; Rothkopf, C.A.; Peters, J. (2024). What Matters for Active Texture Recognition With Vision-Based Tactile Sensors, Proceedings of the IEEE International Conference on Robotics and Automation (ICRA).
    •       Bib
      Funk, N.; Mueller, P.-O.; Belousov, B.; Savchenko, A.; Findeisen, R.; Peters, J. (2023). High-Resolution Pixelwise Contact Area and Normal Force Estimation for the GelSight Mini Visuotactile Sensor Using Neural Networks, Embracing Contacts-Workshop at ICRA 2023.
    •     Bib
      Zhu, Y.; Nazirjonov, S.; Jiang, B.; Colan, J.; Aoyama, T.; Hasegawa, Y.; Belousov, B.; Hansel, K.; Peters, J. (2023). Visual Tactile Sensor Based Force Estimation for Position-Force Teleoperation, IEEE International Conference on Cyborg and Bionic Systems (CBS), pp.49-52.
    •     Bib
      Boehm, A.; Schneider, T.; Belousov, B.; Kshirsagar, A.; Lin, L.; Doerschner, K.; Drewing, K.; Rothkopf, C.A.; Peters, J. (2023). Tactile Active Texture Recognition With Vision-Based Tactile Sensors, NeurIPS Workshop on Touch Processing: a new Sensing Modality for AI.
    •     Bib
      Belousov, B.; Sadybakasov, A.; Wibranek, B.; Veiga, F.; Tessmann, O.; Peters, J. (2019). Building a Library of Tactile Skills Based on FingerVision, Proceedings of the 2019 IEEE-RAS 19th International Conference on Humanoid Robots (Humanoids).

Robotic Architectural Assembly

    •       Bib
      Liu, Y.; Belousov, B.; Schneider, T.; Harsono, K.; Cheng, T.W.; Shih, S.G.; Tessmann, O.; Peters, J. (2024). Advancing Sustainable Construction: Discrete Modular Systems & Robotic Assembly, Sustainability, 16, pp.6678, MDPI.
    •     Bib
      Liu, Y.; Belousov, B.; Funk, N.; Chalvatzaki, G.; Peters, J.; Tessman, O. (2023). Auto(mated)nomous Assembly, International Conference on Trends on Construction in the Post-Digital Era, pp.167-181, Springer, Cham.
    •       Bib
      Belousov, B.; Wibranek, B.; Schneider, J.; Schneider, T.; Chalvatzaki, G.; Peters, J.; Tessmann, O. (2022). Robotic Architectural Assembly with Tactile Skills: Simulation and Optimization, Automation in Construction, 133, pp.104006.
    •       Bib
      Funk, N.; Chalvatzaki, G.; Belousov, B.; Peters, J. (2021). Learn2Assemble with Structured Representations and Search for Robotic Architectural Construction, Conference on Robot Learning (CoRL).
    •     Bib
      Wibranek, B.; Liu, Y.; Funk, N.; Belousov, B.; Peters, J.; Tessmann, O. (2021). Reinforcement Learning for Sequential Assembly of SL-Blocks: Self-Interlocking Combinatorial Design Based on Machine Learning, Proceedings of the 39th eCAADe Conference.
    •     Bib
      Wibranek, B.; Belousov, B.; Sadybakasov, A.; Peters, J.; Tessmann, O. (2019). Interactive Structure: Robotic Repositioning of Vertical Elements in Man-Machine Collaborative Assembly through Vision-Based Tactile Sensing, Proceedings of the 37th eCAADe and 23rd SIGraDi Conference.
    •     Bib
      Wibranek, B.; Belousov, B.; Sadybakasov, A.; Tessmann, O. (2019). Interactive Assemblies: Man-Machine Collaboration through Building Components for As-Built Digital Models, Computer-Aided Architectural Design Futures (CAAD Futures).

Active Exploration

    •       Bib
      Schneider, T.; Belousov, B.; Chalvatzaki, G.; Romeres, D.; Jha, D.K.; Peters, J. (2022). Active Exploration for Robotic Manipulation, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).
    •       Bib
      Schneider, T.; Belousov, B.; Abdulsamad, H.; Peters, J. (2022). Active Inference for Robotic Manipulation, 5th Multidisciplinary Conference on Reinforcement Learning and Decision Making (RLDM).
    •     Bib
      Belousov, B.; Abdulsamad, H.; Schultheis, M.; Peters, J. (2019). Belief Space Model Predictive Control for Approximately Optimal System Identification, 4th Multidisciplinary Conference on Reinforcement Learning and Decision Making (RLDM).
    •     Bib
      Schultheis, M.; Belousov, B.; Abdulsamad, H.; Peters, J. (2019). Receding Horizon Curiosity, Proceedings of the 3rd Conference on Robot Learning (CoRL).

Self-Paced Curriculum Learning

    •     Bib
      Klink, P.; Abdulsamad, H.; Belousov, B.; D'Eramo, C.; Peters, J.; Pajarinen, J. (2021). A Probabilistic Interpretation of Self-Paced Learning with Applications to Reinforcement Learning, Journal of Machine Learning Research (JMLR).
    •     Bib
      Klink, P.; Abdulsamad, H.; Belousov, B.; Peters, J. (2019). Self-Paced Contextual Reinforcement Learning, Proceedings of the 3rd Conference on Robot Learning (CoRL).

Risk-Sensitive and Distributionally-Robust Control

    •     Bib
      Abdulsamad, H.; Dorau, T.; Belousov, B.; Zhu, J.-J; Peters, J. (2021). Distributionally Robust Trajectory Optimization Under Uncertain Dynamics via Relative Entropy Trust-Regions, arXiv.
    •     Bib
      Nass, D.; Belousov, B.; Peters, J. (2019). Entropic Risk Measure in Policy Search, Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

MaxEnt RL

    •       Bib
      Belousov, B.; Peters, J. (2019). Entropic Regularization of Markov Decision Processes, Entropy, 21, 7, MDPI.
    •     Bib
      Belousov, B.; Peters, J. (2018). Mean Squared Advantage Minimization as a Consequence of Entropic Policy Improvement Regularization, European Workshops on Reinforcement Learning (EWRL).
    •     Bib
      Belousov, B.; Peters, J. (2017). f-Divergence Constrained Policy Improvement, arXiv.

Stochastic Optimal Control

    •     Bib
      Eilers, C.; Eschmann, J.; Menzenbach, R.; Belousov, B.; Muratore, F.; Peters, J. (2020). Underactuated Waypoint Trajectory Optimization for Light Painting Photography, Proceedings of the IEEE International Conference on Robotics and Automation (ICRA).
    •   Bib
      Lutter, M.; Clever, D.; Belousov, B.; Listmann, K.; Peters, J. (2020). Evaluating the Robustness of HJB Optimal Feedback Control, International Symposium on Robotics.
    •     Bib
      Lutter, M.; Belousov, B.; Listmann, K.; Clever, D.; Peters, J. (2019). HJB Optimal Feedback Control with Deep Differential Value Functions and Action Constraints, Conference on Robot Learning (CoRL).
    •     Bib
      Belousov, B.; Neumann, G.; Rothkopf, C.; Peters, J. (2016). Catching Heuristics Are Optimal Control Policies, Advances in Neural Information Processing Systems (NIPS / NeurIPS).