Human-robot cooperation control for installing heavy construction materials

Seung Yeol Lee, Kye Young Lee, Sang Heon Lee

Autonomous Robots, N0.22, 2007

Summary

                  Automation System and Robotic Construction is an issue which has came to discussion among researched due to the importance of safety, productivity, quality and work environment. The first robot applied in construction field was in 1983, followed by other examples of robots using pneumatic actuators and servo motors in hybrid solutions or in master-slave solutions. With the new trend of tall and dangerous buildings, advances in new material have be done, but still the process remains mainly assigned to manpower. An automation system (ASCI: Automation System for Curtain Wall Installation) has been developed for mechanized construction to enable simple and precise installations, specially in regard of safety improvement. Automation in construction faces some characteristics which make it much different from manufacturing, in fact rarely processes are highly repetitive, therefore the requirements for this system are: the robot must follow the operator in various works, a robot must share the work space with the operator, there must be coordination between the operator’s and the robot’s force and an intuitive operation method that reflect dexterity of an operator should be performed. Human robot cooperation has been an interest for many different fields of application, in the 60’s the department of defence enhanced the capability of carrying heavy material with “Suit of Armor” and similar ideas where carried later on.  The paper introduces a robot control method for the installation of heavy construction material in cooperation with a human operator, basically the system allows the operator to handle heavy material by exerting operational force with a certain power assist ratio. Target dynamics model the interactions among human, robot and environment, an impendance controller which considers each environment contacting is performed considering two cases: constrained and unconstrained, which differ in the presence or not of environment contact. In unconstrained condition the operator handle heavy materials on an obstacle free area, the force is measured by a force (torque) sensor, F_h(T) is the force generated by the interaction between the operator and a heavy material, M_pt(M_ot) and B_pt(B_ot) are respectively the n x n positive definite diagonal inertia and damping matrices. The desired dynamics, given input F_f,(T_h) is obtained by impedance, by means of λ_p and λ_o, respectively the power assist ration of the position and orientation. The Stiffness Matrices is K is to not be considered having the property of a spring.  Constrained condition differed in the presence in the impedance of F_e(T_e), the experimental force (torque) and in this case, if contact occurs the generalized active impedance is considered. High stability is achieved s through damping, where too much damping may decrease mobility of the system so M_pt(M_ot) and B_pt(B_ot) are adjusted according to the requirements of the operator and appropriate values are found through simulation. Motion control is made stiffness so as enhance disturbance disturbance rejection, so a reference frame and a desired frame are introduced in the previous models. Experiment have been carried on requiring the operator to perform a circle applying force on the robot’s gripper.

Key Concepts

Human Robot Interaction, Human Robot Cooperation, Impedance Control, Control Systems

Key Results

The results show that and increase in M_pt(M_ot) stability decreases (although the with the increase of M_pt an object can move to a long-distance place with small operational force), while the opposite is for B_pt(B_ot), making it necessary to make a trade-off.

Also it is shown that a robot operation without the inner motion control (which balances the system) shows inferior desired position following performance in an unconstrained condition.

The force required by an operator gets smaller with higher values of the power ratio, but this won’t cause hang in F_e (force that reflect in the contacting condition.

Information Sharing via Projection Function for Coexistence of Robot and Human

Yujin Wakita, Shigeoki Hirai, Takashi Suehiro, Toshio Hori

Autonomous Robots, N0.10, 2001

Summary

                  The authors introduce the concept of safety based on intelligent augmentation of robotic systems. In previous studies the authors introduced the concept of tele-robotic systems (1992,1995,1996), where a robot is operated from another position with no physical contact and monitored through a television, and intelligent monitoring (1992), a system allowing conveyance of only required information through selection of data. The expansion of this last system has been the snapshot function (1995), where a laser pointer helps in teaching mode to estimate the deviation of the position, while the operator can move the robot, teaching the estimated relative deviation. A further implementation is the here proposed projection function (2001), where a robot and human jointly operate through a Digital Desk, a special environment provided with a projector perpendicular to the working table and a speaker. The aim of this research is to achieve intelligent augmentation in order to prevent and avoid undesirable contact, information sharing is a fundamental aspect in cooperative tasks between a person and a robot (Wakita, 1998). The experiment test a human and robot operating in mainly 5 states (initial, approach, grasp, release and final), the main issue is this kind of problem to be solves are: the person does not know the delivery coordinate, the person must keep holding the object until it is released, the person might be frightened by the robot movement.

The projection function consists of projecting on the table the simulated images of the moving robot, so that the human operator knows in real time the robots trajectory and understand the delivery trajectory. Force sensors in the robot’s fingers are used in order to allow the robot understand when the object has been grasped by the operator. A new teaching method also is introduced: the operator activated the teaching mode by touching the robot’s hand, then, instead of physically moving the manipulator, the projected image of the robot follows the operator’s hand to destination, the advantage is that only the model is required and no robot movement; the robot confirm through the speakers that the teaching trajectory has been saved.

The force sensors are an efficient communication method only during grasping, visual monitoring appears to be necessary for the entire delivery task.

It can be observed that humans in cooperation require visual feedback in order to understand that their motion and activity has been understood, each person expects to be observed during their action. So visual information appears to be extremely important by means of perception and it enhance safety in the system.

The digital desks comes to help once again in monitoring and indicating robots and humans in the system, in fact while operating a symbol (in the experiment it is a white rectangle) is projected on the hand of the operator when the robot has detected an action, in this way the human is aware that the robot knows about its presence.

In order to perform the experiment, a CCD camera was used for detection of human’s hand and robot position, and a video projector (SANYO LP-SG60) mounted on the ceiling in parallel with the camera.

The system as programmed, projects a white rectangle on the human’s hand when the CCD and the computer had performed the detection, while stationary hand is recognized a the delivery position.

Key Concepts

Human-Robot Interaction, Human-Robot Cooperation, Team Working

Key Results

The experiment appears to be useful prompting the importance of communication between robots and humans working together, a communication which need also visual feedback in order to ensure safety. A big part of communication is in fact performed not only by direct communication, but also by indirect feedback, showing that the message has been properly received. Future research may require adding information to the system.

Toward a Framework for a Human-Robot Interaction

Sebastian Thrun

Human-Computer Interaction, No.19, 2004

Summary

                  The field of robotics has undergone a considerable change from the time it first appeared as a complete science, robots now perform many assembly and transportation tasks, often equipped with minimal sensing and computing, slaved to perform a repetitive task. The future is more and more seeing the introduction of service robots and this is mainly thanks to reduce in costs of many technologies required and increase in autonomy capabilities.

Robotics appears to be a broad discipline and therefore definitions of this science are not unique, a general definition has been done by the author in a previous paper (Thrun, 2002) a system of robotic sensors, actuators and algorithms. The United Nations has categorized robotics in three fields: industrial robotics, professional service robotics and personal service robotics.

Industrial robotics are the earliest commercial success; an industrial robot operates manipulating its physical environment, it is computer controlled and operates in industrial settings (for example on conveyor belts).

Industrial robotics started in the 60s with the first commercial manipulator, the Unimate, later on in the 70s Nissan Corporation automated an entire assembly line with robots, starting a real “robotic revolution”, simply it can be considered that today the ration human to worker to robots is approximately 10:1 (the automotive industry is definitely the one with biggest application of robotics). However industrial robots are not intended to operate directly with humans.

Professional service robots are the younger kind of robots and are projected to assist people, perhaps in accessible environments or in tasks where speed and precision won’t definitely be met by human operators (as it is becoming more common in surgery).

Personal service robots posses today the highest expected growth rate, they are projected to assist people in domestic tasks and for recreational activities, often these robots are humanoids.

In all three of these fields two are the main drivers: cost and safety, these appear to be the challenges of robotics.

Autonomy refers to the ability the robot has to accommodate variation in the environment, it is a very important factory in human-robot interaction. Industrial robots are not considered to be highly autonomous, they often are called for repetitive tasks and therefore can be programmed, a different scenario appears to be the on of service robots where complexity of the environment brings them to be design to be very autonomous since they have to be able to predict the environment uncertainties, to detect and accommodate people and so on.

Of course there is also a cost issue, which necessitates the personal robots to be low-cost, therefore it they are the most complicated since the need high levels of autonomy and low costs. In human robot interaction extremely important become the interface mechanism, industrial robots are often limited, in fact they hard programmed and programming language and simulation softwares appear to be intermediary between the robot and the human. Service robots of course require richer interfaces and therefore distinguished are indirect and direct interaction methods. Indirect interaction consists of a person operating a robot through a command, while direct interaction consist of a robot taking decision on its on in parallel with a human.

Different technologies exist in order to achieve different method of communication, an interesting example appears to be the Robonaus (Ambrose et al., 2001), a master-slave idea demonstrating how a robot can cooperate with astronaut on a space station. Speech synthetisers and screens also appear to be interesting direct interaction methods.

Investigating humanoids and appearance, together with social aspect of service robots are also important aspect which researched are today investigating for the future of robotics.

Key Concepts

Human Robot Interaction, Human Robot Cooperation

An Empirical Analysis of Team Coordination Behaviors and Action Planning with Application to Human-Robot Teaming

Julie Shah, Cynthia Breazeal

Human Factors, The Journal of the Human Factors and Ergonomics Society, April 2010

Summary

                  Robots working in team with humans are increasing and the field of application is also covering stressed, highly uncertain, ambiguous and time pressured environments, therefore studies on team-working and human-robot cooperation are of deep interest. The approach in this paper is studying human-human interaction in order to apply the discovered rules in the design of a human-robot cooperation environment.

Studies on implicit and explicit communication affecting team working have been done, in fact it is demonstrated that implicit communication, including non-verbal cues improve team working in terms of efficiency. It also has been studied that team under pressure, uncertainty and complicated conditions can perform in the same way, if not even better, than teams not facing this kind of conditions.

There are already a certain number of HRI researches investigating robot capturing humans expression, gesture, there are systems capable also of processing human spoken orders and Fong et al. (2006) provided also the Human-Robot Interaction Operating System, which accomplished cooperation through a central task manager capable of decompose goals into high-level task assigns tasks either to a robot or a human.

Examples of implicit communication, which are used for improving team performances, are the use of periodic situation assessments, preplanning and dynamically redistributing workload among the team.

In implicit coordination the use of Shared Mental Models (SMM) is the main strategy working in the background, for example “cross-trained” team member share responsibilities and aspects which are capable of making the overall team more performing (Volpe, 1996), people tend through SMM to incorporate resources and capabilities of other team members into their own action planning. Stout et al. (1996) has identified 9 methods for enhancing SMM: 1) creating an open environment, 2) setting goals and awareness, 3) exchanging preferences and expectations, 4)clarifying roles and information to be created, 5) clarifying sequencing and timing, 6) discussing handling of unexpected events, 7) discussing how high workload affects performance, 8)pre-preparing information and 9) self-correcting. Stoud et al. (1999) and Orasanu (1990) found out that the most effective team tend to generate more type of planning behaviours.

The authors of the paper discuss the importance of “switching cost”, being an explanation for the benefits of implicit communication, meaning that the immediate response of a team member (caused by explicit communication) would cause degrade in team’s performance since a responding to the command may imply a waste of time in changing activity and this tends to be magnified in complex environments, provoking lack of flexibility and therefore efficiency.

Three hypothesis are taken: 0) team exhibit increased use of implicit coordination behaviour as time pressure increases, and coordination behaviour is positively correlated with improved team performance outcomes, 1) explicit communication will provoke immediate response, 2) explicit communication has higher specificity.

The experiment involve 30 couples, half of them working in a competition and time pressured environment; the task was regarding building 4 structures with toy bricks and, although the users could perform the 4 task simultaneously, not all the pieces for doing so where given, without letting them know.

Key Concepts

Human-Robot Cooperation, Team Working

Key Results

Hypothesis all were proven to be true. This study appears to be useful for robotics designers and suggests that robots should use explicit cues for an action that required immediate response, while efficient coordination should be promoted through implicit cues. If these principles are followed, communication will be more natural.