Switching Between Collaboration Levels in a Human-Robot Target Recognition System

Itshak Tkach, Avital Bechar, Yael Edan

IEEE Transactions on Systems, man and cybernetics – Part C: Applications and Reviews. Vol. 41, No.6, November 2011

Summary

                  Human operator (HO) excel in recognition capabilities, therefore a combined system robot and human working together, may work together well in this kind of tasks. Bechar et al. previously defined four human-robot collaboration levels for target recognition tasks in unstructured environments, based on the assumption that integration of an HO in a robotic system tends to reduce complexity of the robotic system. Performances in a human-robot environment are mainly affected by: the state of human, the environmental conditions and the system parameters. According on different conditions of the working environment, it might be good to have the possibility of switching between collaboration levels, as proposed in this paper, proposing a logical controller that considers important parameters to maintain maximum performance. The human-robot environment is defined as a system composed by the HO subsystem, where a human perceived information from a display, the robot subsystem is comprising the autonomous operations that are defined through programming. The HO has the possibility of deciding how and when to intervene in the current work state in any collaboration level (which, from Bechar et al. are: the human operator detect and marks targets solely, the human operator supervises the robot, the human operator completes robot’s detections and the robot acts autonomously). The objective function is expressed through the operation cost: V_IS=V_HS+V_MS+V_FAS+V_CRS+V_TS  (where V_FAS and V_HS are penalties, V_HS is the gain from detecting targets, V_MS is the cost of missed targets, V_CRS is the benefit gained from correct rejection and V_TS is the cost of time and actions). The terms V_FAS and V_HS have negative values (they are penalties), so the idea is improving them through technology characteristics enhancements, other terms are estimated through their probabilities. The system time is computed as a superposition of all the possible time probabilities (time for the human to confirm robots hits, time for the human to detect additional targets, time for the human to correct the robot false alarms, time for the human to mark false alarms, time for the robot to process the image and to achieve hits or false alarms). The time computation appears to be important for the calculation of V_TS. The parameters used for the calculation of the objective function are divided in 4 categories: human performance parameters, robot performance parameters, task performance parameters, environmental performance parameters. The controller is designed so that it is capable of switching between collaboration levels to provide optimal collaboration level, the human-robot system appears then to be a closed loop system with logical controller, where through the score of the optimal collaboration level (OCL), compared with the current collaboration level (CCL), it is able of apply the change and provides a manipulated input to the process u(t). Assumptions to the system are considering human not influencing robots, human and robot not influencing the target, the new image sampling after target achieved, noise and signals with the same distribution and system inputs achieved prior to each image sample, so that the switching objective function is V_IS= (V_ISoptimal- V_IScurrent) + t_response x V_t + Ψ x V_p , where V_p is the penality to switch earlier than the nominal value of the switching frequency and Ψ is the deviation value from the switching frequency and V_t is the penality for the system’s response time.

Different switching algorithms are proposed (page 961-062).

Key Concepts

Team Working, Human Robot Cooperation

Key Results

Dynamic switching in 100% conditions may not increase necessary performances, global optimum may not be achieved because of influence of local optimums, but it works under different probabilities and greatly increases the system performance in most of the simulated scenarios.

Information Sharing via Projection Function for Coexistence of Robot and Human

Yujin Wakita, Shigeoki Hirai, Takashi Suehiro, Toshio Hori

Autonomous Robots, N0.10, 2001

Summary

                  The authors introduce the concept of safety based on intelligent augmentation of robotic systems. In previous studies the authors introduced the concept of tele-robotic systems (1992,1995,1996), where a robot is operated from another position with no physical contact and monitored through a television, and intelligent monitoring (1992), a system allowing conveyance of only required information through selection of data. The expansion of this last system has been the snapshot function (1995), where a laser pointer helps in teaching mode to estimate the deviation of the position, while the operator can move the robot, teaching the estimated relative deviation. A further implementation is the here proposed projection function (2001), where a robot and human jointly operate through a Digital Desk, a special environment provided with a projector perpendicular to the working table and a speaker. The aim of this research is to achieve intelligent augmentation in order to prevent and avoid undesirable contact, information sharing is a fundamental aspect in cooperative tasks between a person and a robot (Wakita, 1998). The experiment test a human and robot operating in mainly 5 states (initial, approach, grasp, release and final), the main issue is this kind of problem to be solves are: the person does not know the delivery coordinate, the person must keep holding the object until it is released, the person might be frightened by the robot movement.

The projection function consists of projecting on the table the simulated images of the moving robot, so that the human operator knows in real time the robots trajectory and understand the delivery trajectory. Force sensors in the robot’s fingers are used in order to allow the robot understand when the object has been grasped by the operator. A new teaching method also is introduced: the operator activated the teaching mode by touching the robot’s hand, then, instead of physically moving the manipulator, the projected image of the robot follows the operator’s hand to destination, the advantage is that only the model is required and no robot movement; the robot confirm through the speakers that the teaching trajectory has been saved.

The force sensors are an efficient communication method only during grasping, visual monitoring appears to be necessary for the entire delivery task.

It can be observed that humans in cooperation require visual feedback in order to understand that their motion and activity has been understood, each person expects to be observed during their action. So visual information appears to be extremely important by means of perception and it enhance safety in the system.

The digital desks comes to help once again in monitoring and indicating robots and humans in the system, in fact while operating a symbol (in the experiment it is a white rectangle) is projected on the hand of the operator when the robot has detected an action, in this way the human is aware that the robot knows about its presence.

In order to perform the experiment, a CCD camera was used for detection of human’s hand and robot position, and a video projector (SANYO LP-SG60) mounted on the ceiling in parallel with the camera.

The system as programmed, projects a white rectangle on the human’s hand when the CCD and the computer had performed the detection, while stationary hand is recognized a the delivery position.

Key Concepts

Human-Robot Interaction, Human-Robot Cooperation, Team Working

Key Results

The experiment appears to be useful prompting the importance of communication between robots and humans working together, a communication which need also visual feedback in order to ensure safety. A big part of communication is in fact performed not only by direct communication, but also by indirect feedback, showing that the message has been properly received. Future research may require adding information to the system.

Toward a Framework for a Human-Robot Interaction

Sebastian Thrun

Human-Computer Interaction, No.19, 2004

Summary

                  The field of robotics has undergone a considerable change from the time it first appeared as a complete science, robots now perform many assembly and transportation tasks, often equipped with minimal sensing and computing, slaved to perform a repetitive task. The future is more and more seeing the introduction of service robots and this is mainly thanks to reduce in costs of many technologies required and increase in autonomy capabilities.

Robotics appears to be a broad discipline and therefore definitions of this science are not unique, a general definition has been done by the author in a previous paper (Thrun, 2002) a system of robotic sensors, actuators and algorithms. The United Nations has categorized robotics in three fields: industrial robotics, professional service robotics and personal service robotics.

Industrial robotics are the earliest commercial success; an industrial robot operates manipulating its physical environment, it is computer controlled and operates in industrial settings (for example on conveyor belts).

Industrial robotics started in the 60s with the first commercial manipulator, the Unimate, later on in the 70s Nissan Corporation automated an entire assembly line with robots, starting a real “robotic revolution”, simply it can be considered that today the ration human to worker to robots is approximately 10:1 (the automotive industry is definitely the one with biggest application of robotics). However industrial robots are not intended to operate directly with humans.

Professional service robots are the younger kind of robots and are projected to assist people, perhaps in accessible environments or in tasks where speed and precision won’t definitely be met by human operators (as it is becoming more common in surgery).

Personal service robots posses today the highest expected growth rate, they are projected to assist people in domestic tasks and for recreational activities, often these robots are humanoids.

In all three of these fields two are the main drivers: cost and safety, these appear to be the challenges of robotics.

Autonomy refers to the ability the robot has to accommodate variation in the environment, it is a very important factory in human-robot interaction. Industrial robots are not considered to be highly autonomous, they often are called for repetitive tasks and therefore can be programmed, a different scenario appears to be the on of service robots where complexity of the environment brings them to be design to be very autonomous since they have to be able to predict the environment uncertainties, to detect and accommodate people and so on.

Of course there is also a cost issue, which necessitates the personal robots to be low-cost, therefore it they are the most complicated since the need high levels of autonomy and low costs. In human robot interaction extremely important become the interface mechanism, industrial robots are often limited, in fact they hard programmed and programming language and simulation softwares appear to be intermediary between the robot and the human. Service robots of course require richer interfaces and therefore distinguished are indirect and direct interaction methods. Indirect interaction consists of a person operating a robot through a command, while direct interaction consist of a robot taking decision on its on in parallel with a human.

Different technologies exist in order to achieve different method of communication, an interesting example appears to be the Robonaus (Ambrose et al., 2001), a master-slave idea demonstrating how a robot can cooperate with astronaut on a space station. Speech synthetisers and screens also appear to be interesting direct interaction methods.

Investigating humanoids and appearance, together with social aspect of service robots are also important aspect which researched are today investigating for the future of robotics.

Key Concepts

Human Robot Interaction, Human Robot Cooperation

An Empirical Analysis of Team Coordination Behaviors and Action Planning with Application to Human-Robot Teaming

Julie Shah, Cynthia Breazeal

Human Factors, The Journal of the Human Factors and Ergonomics Society, April 2010

Summary

                  Robots working in team with humans are increasing and the field of application is also covering stressed, highly uncertain, ambiguous and time pressured environments, therefore studies on team-working and human-robot cooperation are of deep interest. The approach in this paper is studying human-human interaction in order to apply the discovered rules in the design of a human-robot cooperation environment.

Studies on implicit and explicit communication affecting team working have been done, in fact it is demonstrated that implicit communication, including non-verbal cues improve team working in terms of efficiency. It also has been studied that team under pressure, uncertainty and complicated conditions can perform in the same way, if not even better, than teams not facing this kind of conditions.

There are already a certain number of HRI researches investigating robot capturing humans expression, gesture, there are systems capable also of processing human spoken orders and Fong et al. (2006) provided also the Human-Robot Interaction Operating System, which accomplished cooperation through a central task manager capable of decompose goals into high-level task assigns tasks either to a robot or a human.

Examples of implicit communication, which are used for improving team performances, are the use of periodic situation assessments, preplanning and dynamically redistributing workload among the team.

In implicit coordination the use of Shared Mental Models (SMM) is the main strategy working in the background, for example “cross-trained” team member share responsibilities and aspects which are capable of making the overall team more performing (Volpe, 1996), people tend through SMM to incorporate resources and capabilities of other team members into their own action planning. Stout et al. (1996) has identified 9 methods for enhancing SMM: 1) creating an open environment, 2) setting goals and awareness, 3) exchanging preferences and expectations, 4)clarifying roles and information to be created, 5) clarifying sequencing and timing, 6) discussing handling of unexpected events, 7) discussing how high workload affects performance, 8)pre-preparing information and 9) self-correcting. Stoud et al. (1999) and Orasanu (1990) found out that the most effective team tend to generate more type of planning behaviours.

The authors of the paper discuss the importance of “switching cost”, being an explanation for the benefits of implicit communication, meaning that the immediate response of a team member (caused by explicit communication) would cause degrade in team’s performance since a responding to the command may imply a waste of time in changing activity and this tends to be magnified in complex environments, provoking lack of flexibility and therefore efficiency.

Three hypothesis are taken: 0) team exhibit increased use of implicit coordination behaviour as time pressure increases, and coordination behaviour is positively correlated with improved team performance outcomes, 1) explicit communication will provoke immediate response, 2) explicit communication has higher specificity.

The experiment involve 30 couples, half of them working in a competition and time pressured environment; the task was regarding building 4 structures with toy bricks and, although the users could perform the 4 task simultaneously, not all the pieces for doing so where given, without letting them know.

Key Concepts

Human-Robot Cooperation, Team Working

Key Results

Hypothesis all were proven to be true. This study appears to be useful for robotics designers and suggests that robots should use explicit cues for an action that required immediate response, while efficient coordination should be promoted through implicit cues. If these principles are followed, communication will be more natural.