Learning Models for Control

Bringing anthropomorphic robots into human daily life requires backdrivable robots with compliant control in order to ensure the safe interaction with human beings. In contrast, traditional industrial robots employ high control gains which results in an inherent stiffness and, thus, are ill-suited for this aim. To achieve accurate but compliant tracking, it is essential to predict the torques required for the current movement accurately [2,4]. It is well-known that for sufficiently complex robots (e.g., humanoids, service robots), the standard rigid body dynamics (RBD) models no longer describe the dynamics properly, and data-driven approximate methods become a promising alternative.

(:youtube OBTpkqTrV0k :) Using modern machine learning techniques has a multitude of advantages ranging from higher precision torque prediction to adaptation to altered dynamics with online learning. Nevertheless, online learning of robot dynamics poses a tremendous technical challenge as the learning method has to deal with an endless stream of data while learning needs to take place in real-time at sampling rates of approximately 100Hz. While modern machine learning approaches such as Gaussian process regression and support vector regression, yield significantly higher accuracy than traditional RBD models, their computational requirements can become prohibitively costly as they grow with number of data points. Thus, it is infeasible to simply use off-the-shelf regression techniques and the development of domain-appropriate versions of these methods is essential [3,5].

One possibility for reducing the computational cost is the partitioning of the data such that only the regionally interesting data is included in a local regression and, subsequently, combining these local predictions into a joint prediction. As a result, one can reach a significantly higher learning and prediction speed while having a comparable accuracy. The resulting method employs Gaussian process regression for learning local models and uses a weighted combination of the local models for the global prediction [1,3]. Due to the reduced computational cost, this approach can be made to work on a real Barrett WAM arm where it was able to improve the tracking performance while learning online. It can be shown that it out-performs RBD models and, due to the online improvement, also the global regression techniques.

Another possibility is to employ sparsification methods in combination with an incremental model learning approach [5]. The main idea is to find a sparse representation of the data - called dictionary - which can be used for model learning. Based on a kernel independence measure, a criterion is developed allowing an efficient online selection (and optionally removing) of data points for the dictionary. In combination with an incremental learning approach, such as incremental Gaussian process regression or incremental support vector regression, a model approximation method can be obtained which is applicable in real-time online learning.

Contact persons: Duy Nguyen-Tuong, Jan Peters
Collaborators: Stefan Schaal (USC), Matthias Seeger (Saarbruecken University), Bernhard Schölkopf


    •     Bib
      Nguyen Tuong, D.; Seeger, M.; Peters, J. (2009). Local Gaussian Process Regression for Real Time Online Model Learning and Control, Advances in Neural Information Processing Systems 22 (NIPS/NeurIPS), Cambridge, MA: MIT Press.
    •     Bib
      Peters, J.; Seeger, M. (2008). Computed Torque Control with Nonparametric Regressions Techniques, American Control Conference.
    •     Bib
      Nguyen Tuong, D.; Seeger, M.; Peters, J. (2009). Model Learning with Local Gaussian Process Regression, Advanced Robotics, 23, 15, pp.2015-2034.
    •     Bib
      Nguyen Tuong, D.; Peters, J. (2010). Using Model Knowledge for Learning Inverse Dynamics, IEEE International Conference on Robotics and Automation.