Follow me

Tuesday, September 25, 2012

A Systematic Approach to Predict Performance of Human-Automation System


IEEE Transactions on Systems, Man, and Cybernetics – Part C: Applications and Reviews, Vol.37, NO.4, July 2007
Summary
                  One of the key issues in human-robot interaction is which tasks is better to be performed by humans, by robots or by a combination of the two and with which level of cooperation. The paper discusses an approach for predicting system performance resulting from humans and robots preforming repetitive tasks in a collaborative  manner. Two are the main factors affecting performance: the approximation of the decline in performance associated with the constant mental/resource load required to complete a task and the quantification of how well an agent achieves a certain task. It is known that automation systems are highly discouraged in presence of unexpected events or uncertainty, where cooperation with human can improve the overall performance of the system. The author introduces HumAns (Human Automantion system performance), that evaluates the effects of workload on human performance, and estimates performance derived from task allocation. The procedure to be followed consists of 4 steps: 1) decompose the scenario in primitive major functional tasks; 2) estimate the performance of both human and robot; 3) calculate the performance score based on satisfaction and effects on both human and robot; 4) compute a composite task score to enable tradeoff studies in order to allocate tasks between humans and robots.
The method used for scenario decomposition is the task diagram interview sequence (part of the applied cognitive task, ACTA, technique), so that the scenario is broken into 3 to 6 functional primitives, then the cognitive skill/mental demand are derived (through a classification which creates 3 macro-levels: perception, cognition and motor activities) and a task diagram can then give an overview. The primitives are selected independently from each other to emphasize different aspects. Performance metrics are measured on workload values and execution time components, the first is relative to decline in performance associated with mental/resource load required for task completion. The performance metrics are performed for each primitive found previously. ACT-R is the framework used for modeling human cognition in this case (it studies how it works in different scenarios, based on the assumption that come from psychology experiments). At this point for each primitive there is a time of execution and workload, so these further steps can be done: 1) calculate the execution rank for each of the three zones in which the data (execution time vs. ranking value) is divided (a logarithmic function, given at page 597); 2) calculate the Workload considering it distributed logarithmically and dependent on the Execution Rank (page 597); 3) calculating the Composite Task Score. In order to obtain the Composite Task Score the Markov Decision Process (for which each agent, either robot or human, is allowed to chose individual actions based on maximizing an optimization function for the entire system, this incorporates already both workload ranking values and execution ranking times, the algorithm is introduced briefly in the paper (page 598).
Key Concepts
Human-Robot Cooperation, Performances of Cooperation
Key Results
Two different setups have been done testing tele-operation control and fully autonomous control. HumAns is denoting how as time elapses the task score decreased, while for a fully automated robot the scores stays fairly constant, in the comparison we could say that for short elapsing times teleoperation is more convenient than full automation, which is not true in the long and repetitive case. HumAns appears to be a very important and useful method to predict workload on human and machine and the consecutive effects in the overall system, still studies on the effects on accuracy and repeatability have to be performed.

No comments: