We will develop and study the scientific principles of Semi-Autonomous Human-Robot Collaboration. This new robotics paradigm will result in a class of robotic systems that have the following principles: they are proactive, able to be programmed and commanded by instruction, capable of skill self-assessment and have an explicit model of the team behaviour. These principles will allow a revolution in the way factories work by changing the usual way we program and interact with robots: Instruction replaces Programming. Due to frequent occurrence of new situations, it is not possible to program each new task with abstract programming languages. Instead, intuitive programing methods must be devised. Our instruction framework will generalize learning from demonstration, active learning, mutual adaptation and human guidance to allow laymen to command a robot to work on collaborative tasks. Knowledge Transfer enables Fast Task-Switching. Although new tasks will be continually added to the required skills of the robots, most will have significant similarities. If such invariant properties are exploited, we can substantially improve the efficiency of task switching - significantly beyond what either instruction only or standard generalization properties of the learning algorithms can provide. Hence, we must devise methods that at any state know what they know as well as what knowledge needs to be acquired. Semi-Autonomy replaces Teleoperation or Full-Autonomy. New products will require a level of dexterity that is only possible in the collaboration between robots and people. In this setting, it is essential that the robot is capable to ask for help when its skills are not sufficient. Also, for a fluid interaction it will need to interpret the users' needs and anticipate their requests, either to act according to the operators' plans or to avoid unsafe situations, as well as for smoother action generation.

Expected Results

Current robots in real-world industrial applications are either pre-programmed or tele-operated (with only few exceptions), lacking any autonomy at all. At the other extreme, the field of artificial intelligence has so far been unable to endow robots with full autonomy, and the prospect of fully-autonomous robots is uncertain at best. We challenge the current thinking in industry and academia on both the future of robotics and artificial intelligence as wells as on the nature of the long-awaited robotic killer application. We claim that a novel alternative lies between these extremes. Semi-autonomous robots go beyond tele-operated robots as they are trained to do their jobs without step-by-step guidance. Instead of requiring an operator, these robots operate proactively and blend their operation with that of their human coworkers. The key idea is the combination of the precision, force and speed of robots with the dexterity, reasoning and intelligence of humans. Rather than operating independently, the robot becomes a semi-autonomous part of a mixed human/robot assembly team, within which it incrementally learns to fulfil its role based on intuitive human instruction. We introduce instruction as the combination of demonstrations (both correct behaviour and counterexamples), guidance, self-adaptation, and active querying in an interactive process. Human and robotic coworkers can switch roles, and the robot can predict and adapt to the human co-worker both at a low control level and the higher level of understanding preferences and limitations of its collaborator. While fully autonomous robots are very far from deployment, such semi-autonomous systems are within reach in the foreseeable future, if the new technologies and scientific principles proposed here are developed in a coordinated way. This project will introduce many novel aspects, all of them necessary requirements for a truly effective 3rdHand robot. Learning from multi-agent demonstrations to extract basic relevant features, motor primitives and hierarchical structure of the collaborative task. Learning from teacher instruction in an online interactive session in order to facilitate this learning challenge and to online correct and improve on the learnt model in a proactive fashion. This system will learn an explicit model of the team behaviour, enabling it to optimally work in a task where different human collaborators might have different preferences, limitations and/or skills. This calls for Behaviour modelling of user capabilities and preferences} to ensure the precise Behaviour prediction that will ensure that accurate complementarity actions are executed. Active Task Transfer and Task Transfer from Instruction will ensure that knowledge is reused and re-instructing is reduced even in situations of frequent changes of tasks either by the composition and/or adaptation of previously acquired skills or by performing queries to the user. It will also ensure that robots model their own uncertainty to avoid unsafe situations and/or tedious confirmation requests to users. This high-level modelling of the task knowledge will call for new task representations that besides ensuring an efficient reproduction of the task from the demonstration with the subsequent optimization for the physical system are able to: work in collaborative environments, are universal task representations (including hierarchical, sequential and concurrent tasks) and have sound models of the uncertainty.

Expected Impact

Our project is directly aimed at developing a set of new scientific contributions to trigger a revolution on how robots collaborate with humans and how manufacturing of goods is performed. The overall expected impact will be on a combination of the following axis. 3rdHand will:

  • help improve the competitiveness of Europe in the manufacturing sector by introducing a new approach for the manufacturing industry.
  • develop a new scientific and technological robot with new functionalities, precision and safety and test it in a realistic human-robot collaboration scenario.
  • contribute with foundational algorithms on semi-autonomy, learning from instruction and human-robot collaboration
  • contribute with high impact scientific publications and training programs.
  • disseminate our results on special scientific events, open-source software or industrial patents.

Main Innovations:

In the recent years robotic systems have been growing at a fast pace in either consumer makers, i.e. personal vacuum cleaners, or autonomous surveillance systems, e.g. drones, while for industrial robots the speed of growing has been slower. In most of these cases the interaction with humans is much reduced or very simplified. This project will create the principle of semi-autonomous robots that works as an extension of the human body. These machines allow a more intuitive way of interaction with the human always on the loop, allowing the robot to show some autonomy but with always the power to override the command decided by the machine. The main innovations will be:

  • A new form to represent complex, hierarchical and concurrent, tasks.
  • Methods to program robot from instruction combining: learning from demonstration, feedback, guidance and kinesthetic teaching.
  • Efficient adaption of task descriptions.
  • New forms of instruction for robots and machines.
  • Semi-Autonomous methods for task transfer.
  • Intuitive human-robot collaboration through mutual adaptation and role selection.
  • Human intention understanding.
zum Seitenanfang