Control Scheme of Object Manipulation Based on Tactile ... · presented. Object manipulation...

6
Abstract— Object manipulation is one of the important tasks in robotics. Tactile sensor system is essential as a sensory device to support robot control system. This paper present a precision control scheme of multi-fingered humanoid robot arm based on tactile sensing information to perform object manipulation tasks. With the aim to enhance the ability to recognize and manipulate object in humanoid robot, we developed a novel optical three-axis tactile sensor system mounted on fingertips of the humanoid robot fingers. This tactile sensor applies an optical waveguide transduction method, and capable of acquiring normal and shearing force. Trajectory generation based on kinematical solutions at the arm and fingers, together with control system structure and sensing principle of the tactile sensor system are presented. Object manipulation experiments are conducted using hard and soft objects. Experimental results revealed that the proposed control scheme enable the finger system to recognize low force interactions based on tactile sensing information to grasp the object surface and manipulate it without causing damage to the object and the sensor elements. Index Terms—Tactile sensing, optical waveguide, object manipulation, multi-fingered arm, humanoid robot. I. INTRODUCTION The sense of touch or tactile sensing is the process of determining physical properties and events through contact with objects in the world. A tactile sensor system is essential as a sensory device to support the robot control system [1-3], particularly in object manipulation tasks. This tactile sensor is capable of sensing normal force, shearing force, and slippage, thus offering exciting possibilities for application in the field of robotics for determining object shape, texture, hardness, etc. However, tactile sensor is an especially appropriate sensing device that has too often been neglected in favor of vision based approaches. To date, while a great deal of research has been applied to the development of sensors especially visual and auditory sensors, comparatively little progress has been made with regards to sensors translating the sense of touch [3]. A part of this study was supported by fiscal 2006 Grant-in-Aid for Scientific Research in Exploratory Research from the Japan Ministry of Education, Culture, Sports, Science and Technology (Grant no. 18656079). Hanafiah Yussof and Masahiro Ohka are with the Graduate School of Information Science, Nagoya University, Furo-cho Chikusa-ku Nagoya 464-8601 Japan (phone: +81-52-789-4251; fax: +81-52-789-4800; e-mail: Hanafiah: [email protected] , Ohka: [email protected] ) Hirofumi suzuki and Nobuyuki Morisawa are with the Graduate School of Engineering, Nagoya University, Nagoya, Japan (e-mail: [email protected] , [email protected] ) Jumpei Takata is with the Olympus Corporation, Japan. Research on tactile sensor is basically motivated by tactile sensing system of human skin. It is well known that human’s tactile sense is very accurate and sensitive. Eventually, the most important distinction between the sensor types is between static and dynamics, in the sense that dynamics do not react to constant pressure. The dynamics sensors are of special importance for actively checking surface texture and properties, such as roughness, flatness, etc. Meanwhile, static sensors are more on improvement of imperfect grips. In human, the skin structure provides mechanism to sense static and dynamics pressure simultaneously with extremely high accuracy. On the other hand, most tactile sensors developed nowadays are capable of detecting both static and dynamic pressure. Although accuracy and consistency are still remaining as main problems, the tactile sensing characteristic offers exciting possibilities for application in the field of robotics, especially for application in robotic finger or gripper system to perform object manipulation tasks. Basically, in order to effectively perform the object manipulation tasks, the robotic systems required at least two types of tactile information: contact sense and slippage. Therefore, the tactile sensor system must be able to measure force in direction of three axes. The contact sense is normally defined by measuring normal force, by means of static tactile sensing. Meanwhile, the slippage is defined by measuring shearing force, by means of dynamic tactile sensing. In this research, we developed and analyze the performance of a novel optical three-axis tactile sensor system mounted on robotic fingers of a humanoid robot arm to conduct object manipulation tasks. Fig. 1 shows structure of the multi-fingered humanoid robot arm used in this research. It consists of two robotic fingers and the tactile sensors are mounted on each fingertip. In this report, at first we present the development of the optical three-axis tactile sensor. Secondly, we explain the structure and kinematical formulations of the multi-fingered Control Scheme of Object Manipulation Based on Tactile Sensing in Humanoid Robot Arm Hanafiah Yussof, Masahiro Ohka, Hirofumi Suzuki, Nobuyuki Morisawa and Jumpei Takata Fig. 1. Multi-fingered humanoid robot arm mounted with optical three-axis tactile sensors at fingertips. Proceedings of the World Congress on Engineering and Computer Science 2007 WCECS 2007, October 24-26, 2007, San Francisco, USA ISBN:978-988-98671-6-4 WCECS 2007

Transcript of Control Scheme of Object Manipulation Based on Tactile ... · presented. Object manipulation...

Page 1: Control Scheme of Object Manipulation Based on Tactile ... · presented. Object manipulation experiments are conducted using hard and soft objects. Experimental results revealed that

Abstract— Object manipulation is one of the important tasks in

robotics. Tactile sensor system is essential as a sensory device to support robot control system. This paper present a precision control scheme of multi-fingered humanoid robot arm based on tactile sensing information to perform object manipulation tasks. With the aim to enhance the ability to recognize and manipulate object in humanoid robot, we developed a novel optical three-axis tactile sensor system mounted on fingertips of the humanoid robot fingers. This tactile sensor applies an optical waveguide transduction method, and capable of acquiring normal and shearing force. Trajectory generation based on kinematical solutions at the arm and fingers, together with control system structure and sensing principle of the tactile sensor system are presented. Object manipulation experiments are conducted using hard and soft objects. Experimental results revealed that the proposed control scheme enable the finger system to recognize low force interactions based on tactile sensing information to grasp the object surface and manipulate it without causing damage to the object and the sensor elements.

Index Terms—Tactile sensing, optical waveguide, object manipulation, multi-fingered arm, humanoid robot.

I. INTRODUCTION The sense of touch or tactile sensing is the process of

determining physical properties and events through contact with objects in the world. A tactile sensor system is essential as a sensory device to support the robot control system [1-3], particularly in object manipulation tasks. This tactile sensor is capable of sensing normal force, shearing force, and slippage, thus offering exciting possibilities for application in the field of robotics for determining object shape, texture, hardness, etc. However, tactile sensor is an especially appropriate sensing device that has too often been neglected in favor of vision based approaches. To date, while a great deal of research has been applied to the development of sensors especially visual and auditory sensors, comparatively little progress has been made with regards to sensors translating the sense of touch [3].

A part of this study was supported by fiscal 2006 Grant-in-Aid for Scientific

Research in Exploratory Research from the Japan Ministry of Education, Culture, Sports, Science and Technology (Grant no. 18656079).

Hanafiah Yussof and Masahiro Ohka are with the Graduate School of Information Science, Nagoya University, Furo-cho Chikusa-ku Nagoya 464-8601 Japan (phone: +81-52-789-4251; fax: +81-52-789-4800; e-mail: Hanafiah: [email protected], Ohka: [email protected])

Hirofumi suzuki and Nobuyuki Morisawa are with the Graduate School of Engineering, Nagoya University, Nagoya, Japan (e-mail: [email protected], [email protected])

Jumpei Takata is with the Olympus Corporation, Japan.

Research on tactile sensor is basically motivated by tactile sensing system of human skin. It is well known that human’s tactile sense is very accurate and sensitive. Eventually, the most important distinction between the sensor types is between static and dynamics, in the sense that dynamics do not react to constant pressure. The dynamics sensors are of special importance for actively checking surface texture and properties, such as roughness, flatness, etc. Meanwhile, static sensors are more on improvement of imperfect grips. In human, the skin structure provides mechanism to sense static and dynamics pressure simultaneously with extremely high accuracy. On the other hand, most tactile sensors developed nowadays are capable of detecting both static and dynamic pressure. Although accuracy and consistency are still remaining as main problems, the tactile sensing characteristic offers exciting possibilities for application in the field of robotics, especially for application in robotic finger or gripper system to perform object manipulation tasks. Basically, in order to effectively perform the object manipulation tasks, the robotic systems required at least two types of tactile information: contact sense and slippage. Therefore, the tactile sensor system must be able to measure force in direction of three axes. The contact sense is normally defined by measuring normal force, by means of static tactile sensing. Meanwhile, the slippage is defined by measuring shearing force, by means of dynamic tactile sensing.

In this research, we developed and analyze the performance of a novel optical three-axis tactile sensor system mounted on robotic fingers of a humanoid robot arm to conduct object manipulation tasks. Fig. 1 shows structure of the multi-fingered humanoid robot arm used in this research. It consists of two robotic fingers and the tactile sensors are mounted on each fingertip. In this report, at first we present the development of the optical three-axis tactile sensor. Secondly, we explain the structure and kinematical formulations of the multi-fingered

Control Scheme of Object Manipulation Based on Tactile Sensing in Humanoid Robot Arm

Hanafiah Yussof, Masahiro Ohka, Hirofumi Suzuki, Nobuyuki Morisawa and Jumpei Takata

Fig. 1. Multi-fingered humanoid robot arm mounted with optical three-axis tactile sensors at fingertips.

Proceedings of the World Congress on Engineering and Computer Science 2007WCECS 2007, October 24-26, 2007, San Francisco, USA

ISBN:978-988-98671-6-4 WCECS 2007

Page 2: Control Scheme of Object Manipulation Based on Tactile ... · presented. Object manipulation experiments are conducted using hard and soft objects. Experimental results revealed that

humanoid robot arm, in which the tactile sensor is mounted on fingertips of each finger. This multi-fingered arm system is developed for experimental and evaluations of the tactile sensor system towards future application in real humanoid robot. Next, we explain control algorithm of the robotic fingers based on the tactile sensing information. Finally, we present experimental results of object manipulation tasks with soft and hard objects.

II. STATE-OF-THE-ART SURVEY OF TACTILE SENSING IN ROBOT MANIPULATION

Robot manipulation is fundamentally relies on contact interaction between the robot and the world [4]. As blind people convincingly demonstrate, tactile sensing alone can support extremely sophisticated manipulation. Unfortunately, many traditional tactile sensing technologies do not fit the requirements of robot manipulation in human environments due to lack of sensitivity, dynamic range and material strength.

Recent research in robot manipulation has been focusing in development of new tactile sensor that take advantage of advances in materials, microelectromechanical systems (MEMS), and semiconductor technology. For instance, a research team at MIT have developed sensor with a protruding shape that allows them to easily make contact with the world from many directions in a similar way to the ridges of a human fingerprint or the hair on human skin. By measuring the deformation of the compliant dome, the sensors can estimate the magnitude and direction of applied forces with great sensitivity. This tactile sensor has been applied in a compliant hand of humanoid robot Obrero [5]. Another interesting example is the human skin-like combined sensor which offers extremely high-sensitivity with forces as low as 5mN can be detected. This sensor is suitable for determining dynamic tactile information in object exploration and has been tested with two-jar gripper [6].

Obviously, tactile sensor offers exciting possibilities for application in the field of robotics for determining object shape, texture, hardness, etc. Humanoid robotics is one of the application areas in robotics that have generated the most interest presently in regards to tactile sensor applications. This is because humanoid robots are the type of robot that practically suitable to coexist with human because of its anthropomorphism, human friendly design and applicability of locomotion. In fact, to effectively work in built-for-human environments, humanoid robot requires sensing device that can measure physical properties and recognize shapes of the given object through contact interaction, thus the robot can generate suitable trajectories to handle and manipulate the object. To date, many great efforts by robotic researchers have been reported related with development of tactile sensor systems that are compatible for use in humanoid robot platform [2][7][8]. These efforts are aimed towards the successful introduction of this type of robot into human environments.

However, there are serious issues in object manipulation that based on tactile sensing. The first issue is that the robot must use low force interactions to explore the object surface without unduly altering the physical property of the object or causing

damage. The second issue is that the robot will rarely be able to control the exact angle at which its tactile sensor makes contact with the object. The third issue is related with mechanical structure and material strength, whereby the tactile sensor elements which normally made from soft and elastic materials make it not robust enough to handle strong impact and pressure during object manipulation tasks. Consequently, besides the development of novel tactile sensor system that capable of precisely measuring normal and shearing forces, the development of precision control algorithm based on the tactile sensing information in the robot control system is inevitably important. Moreover, it is desirable for the robots to feature reliable object manipulation ability, and at the same time ensure the safety of tactile sensors and the manipulated objects.

III. OPTICAL THREE-AXIS TACTILE SENSOR Tactile sensor is a device that can measure a given property

of an object or contact event through physical contact between the sensor and the object. To date, several basic sensing principles are commonly in use in tactile sensor, such as capacitive sensor, piezoelectrical sensor, inductive sensor, optoelectrical sensor and piezoresistive sensor [3][5]. In this research, with the aim of establishing object manipulation ability in real humanoid robot, we have developed an optical three-axis tactile sensor capable of acquiring normal and shearing force to mount on fingertips of humanoid robot arm. This tactile sensor is using optical waveguide transduction method, applying image processing techniques. This type of sensing principle is comparatively provides better sensing accuracy to detect contact phenomena from acquisition of three axial directions of forces, thus normal force and shearing force can be measured simultaneously [9]. The proposed three-axis tactile sensor has high potential compared to ordinal tactile sensor for fitting to a dextrose robotic arm.

The optical three-axis tactile sensor developed in this research is designed in a hemispherical dome shape consist of an array sensing elements. This shape is to mimic human fingertips structure for easy compliance with various shapes of objects. The hardware novelty consists of an acrylic hemispherical dome, an array of 41 peaces of sensing elements made from silicon rubber, a light source, an optical fibre-scope, and a CCD camera, as shown in Fig. 2. The silicone rubber sensing element that also indicated in this figure comprises one columnar feeler and eight conical feelers. The eight conical feelers remain in contact with the acrylic surface while the tip of the columnar feeler touches an object. The sensing elements are arranged on the hemispherical acrylic dome in a concentric configuration with 41 sub-regions as shown in Fig. 3. The hardware system structure of this tactile sensor is shown in Fig. 4. Referring to Fig. 2, the light emitted from the light source is directed towards the edge of the hemispherical acrylic dome through optical fibres. When an object contacts the columnar feelers, resulting in contact pressure, the feelers collapse. At the points where the conical feelers collapse, light is diffusely reflected out of the reverse surface of the acrylic surface because the rubber has a higher reflective index.

Proceedings of the World Congress on Engineering and Computer Science 2007WCECS 2007, October 24-26, 2007, San Francisco, USA

ISBN:978-988-98671-6-4 WCECS 2007

Page 3: Control Scheme of Object Manipulation Based on Tactile ... · presented. Object manipulation experiments are conducted using hard and soft objects. Experimental results revealed that

The contact phenomena consisting of bright spots caused by

the feelers collapse are observed as image data, which are retrieved by the optical fibre-scope connected to the CCD camera and are transmitted to the computer. Fig. 5 shows the image data acquired by the CCD camera. The dividing procedure, digital filtering, integrated gray-scale value and centroid displacement are controlled on the PC using auto analysis programme applying the image analysis software Cosmos32. In this situation, the normal force of Fx, Fy and Fz values are calculated using the integrated gray-scale value G, while shearing force is based on the horizontal centroid displacement. The displacement of the gray-scale distribution u is defined in (1), where i and j are orthogonal base vectors of the x- and y-axes of a Cartesian coordinate, respectively.

u = uxi + uyj (1) This equation is according to calibration experiments,

whereby material functions are identified with piecewise approximate curves such as bi-linear and curves [7]. Consequently, each force component is defined in (2).

Fx = f(ux), Fy = f(uy), Fz = g(G) (2)

IV. MULTI-FINGERED HUMANOID ROBOT ARM

A. Humanoid Robot Arm The humanoid robot arm developed in this research consists

of 3-dofs: two dof (pitch and roll) at the shoulder and one dof (roll) at the elbow. The arm structure and control system was designed based on the earlier developed 21-dof humanoid robot Bonten-Maru II. However, some refinement and new modules have been added to comply with the robotic finger and tactile sensor system. Each joint is driven by a DC servomotor with a rotary encoder and a harmonic drive-reduction system, and is controlled by a PC with the Fedora Core Linux OS.

In the arm, upper link l1 is connecting shoulder joints with elbow joint, while lower link l2 connecting elbow joint with two robotic fingers. Trajectory generation of the arm is generated by determination of forward and inverse kinematics solutions. To describe translation and rotational relationship between adjacent joint links, we employ a matrix method proposed by Denavit-Hartenberg [10], which systematically establishes a coordinate system for each link of an articulated chain. Fig. 6 displays a model of the robot arm describing the configurations and orientation of each joint coordinates, and five sets of joint-coordinates frames. Consequently, corresponding link parameters of the arm can be defined as shown in Table 1. From the Denavit-Hartenberg convention, definitions of the homogeneous transform matrix of the link parameters can be described as in (3). This equation is initially used to obtain forward kinematics for the arm.

0h T= 0

1 T 12 T 2

3 T 3h T

⎥⎥⎥⎥

⎢⎢⎢⎢

+−−++−

=

1000)clcl(cssccc

slsl0cs)clcl(scsscs

2322111231231

232212323

2322111231231

(3)

Consequently, the end-effector’s orientation Rarm with respect to the reference coordinate and the position of the end-effector Parm in regard to global axes Px, Py and Pz are shown in (4).

0h Rarm

⎥⎥⎥

⎢⎢⎢

−=

1231231

2323

1231231

ssccc0cscsscs

, 0Ph

⎥⎥⎥

⎢⎢⎢

+−++

=)(

)(

232211

23221

232211

clclcslslclcls

arm (4)

Here, si, ci, sij and cij are respective abbreviations of sinθi, cosθi, sin(θi+θj) and cos(θi+θj) where (i, j=1,2,3).

Fig. 2. Structure of hemispherical optical three-axis tactile sensor and sensing element.

X

Y

Fig. 3. Arrangement of sensing elements on fingertip.

Fig. 4: Layout of optical three-axis tactile sensor hardware system structure.

Auto image sampling module

Auto image analysis module

Fig. 5. CCD camera-captured image of contact phenomenon in the optical three-axis tactile sensor.

Proceedings of the World Congress on Engineering and Computer Science 2007WCECS 2007, October 24-26, 2007, San Francisco, USA

ISBN:978-988-98671-6-4 WCECS 2007

Page 4: Control Scheme of Object Manipulation Based on Tactile ... · presented. Object manipulation experiments are conducted using hard and soft objects. Experimental results revealed that

As understood from (3) and (4), a forward kinematics

equation can be used to compute the Cartesian coordinates of the robot arm when the joint angles are known. However, in real-time applications it is more practical to provide the end-effector’s position and orientation data to the robot’s control system than to define each joint angle that involved complicated calculations. Therefore, inverse kinematics solutions are more favorable. To define joint angles θ1arm, θ2arm, θ3arm in an inverse kinematics problem, at first each position element in (4) is multiplied and added to each other according to (5), which can also be arranged as in (6). Thus, θ3arm is defined in (7).

3212

22

12

z2

y2

x cll2llPPP ++=++ armarmarm (5)

Cll2

llPPPc

21

22

21

2z

2y

2x

3 =+−++

=)(armarmarm (6)

⎟⎠⎞⎜

⎝⎛ −±= CC1 2

3 ,atan2armθ (7)

Consequently, θ3arm is used to define θ2arm, as shown in (8) ~ (12), where newly polar coordinates are defined in Eq. (10). Finally, θ1arm can be defined as in (13).

3223211 slkcllk −=+= , (8)

2122y2221xz skckpskckp −=+= , (9)

),(atan2 21 kk=φ (10)

( )yxz

yxz2

pp

rp

rp

,atan2

,atan2arm

=

⎟⎟⎠

⎞⎜⎜⎝

⎛=+θφ

(11)

( ) ( )21yxz2 kkpp ,atan2,atan2arm −=θ (12)

( )zx

xz

z

xz

x1

pppp

pp

,atan2

,atan2arm

=

⎟⎟⎠

⎞⎜⎜⎝

⎛=θ

(13)

B. Robotic Fingers The robotic finger system comprises of two articulated

fingers as shown in Fig. 7. Each finger has 3-dof with micro-actuators (YR-KA01-A000, Yasukawa) is used in each joint. The micro-actuator consists of a micro AC servomotor, a harmonic gear (reduction ratio: 1/80, maximum torque: 0.7 Nm) and a digital encoder. The optical three-axis tactile sensor is mounted on each fingertip. Hardware system structure of the robotic finger is shown in Fig. 8. Each joint connects to a PC via a motor driver and motor control board. The PC is installed with the Windows OS, and a Visual C++ compiler. In this research, the integrated robotic fingers and tactile sensor system was designed to comply with the humanoid robot arm in term of mechanical design and control structure.

Trajectory generation of the fingers is defined by kinematical solutions derived by the same convention as the humanoid robot arm. According to the Denavit-Hartenberg notation, a model of the finger consist of configurations and orientation of each joint coordinates and five sets of joint-coordinates frames is shown in Fig. 7. The actuator angular velocity is derived by

Ox1

z1

y1

x2y2

z2

x3

y3

z3

x4

y4

z4

x5

y5

z5

O1

O2

O3

O4O5

Fig. 7. Robotic fingers mounted with optical three-axis tactile sensor and configuration of joint coordinate frames at the finger.

Bus BridgeCSI-PCI04

Driver ControllerPCI-7414V

Driver ControllerPCI-7414V

AC Servo DriverJZRCH-ZKD01

AC Servo DriverJZRCH-ZKD01

AC Servo DriverJZRCH-ZKD01

Windows XPVisual C++ 6.0

Fig. 8. Layout of robotic fingers system structure.

Fig. 6. Configuration of joint coordinates of the robot arm and appearance of the humanoid robot Bonyten-Maru II.

oz

ox1z

1x

2z

2x

3x

hx

3z

hz

oz

ox1z

1x

2z

2x

3x

hx

3z

hz

Table 1. Link parameters at humanoid robot arm.

Link θiarm d α l 0 θ1arm-90º 0 90 0 1 θ2arm 0 -90 0 2 θ3arm 0 0 l1 3 0 0 0 l2

Proceedings of the World Congress on Engineering and Computer Science 2007WCECS 2007, October 24-26, 2007, San Francisco, USA

ISBN:978-988-98671-6-4 WCECS 2007

Page 5: Control Scheme of Object Manipulation Based on Tactile ... · presented. Object manipulation experiments are conducted using hard and soft objects. Experimental results revealed that

kinematics-based resolved motion rate control which commonly known as an algorithm for solving path-tracking problem in robotic control.

At this point, since joints angle of the finger are defined by kinematics solution as θ=[θ1,θ2,θ3]T, and put the fingertip moving velocity in global coordinate space as Tzyxr ],,[ &&&& = , the joint rotation velocity at the finger is defined as following:

rJ && 1−= )(θθ . (14) Here, inverse Jacobian matrix was employed to solve joint angle velocity which consequently satisfies the specified velocity vector r& of the fingertip in global coordinate plane. Initially, the Jacobian matrix is defined in (15).

⎥⎥⎥

⎢⎢⎢

++++−++++−++++−

=

432432432332331323423233

422422422322332323423223

412412412312311323423213

lRlRlRcRsRlclcllRlRlRlRcRsRlclcllRlRlRlRcRsRlclcllR

J

)()()()()()(

)(

θθθθθθθθθθθθ

θ

(15) Meanwhile, rotational transformation from the local frame of fingertip where tactile sensor is attached, to the frame of the workspace (refer Fig. 7) is calculated as follows:

⎥⎥⎥

⎢⎢⎢

−−−

=

⎥⎥⎥

⎢⎢⎢

−⎥⎥⎥

⎢⎢⎢

⎡=

313233

212223

111213

333231

232221

1312110s

RRRRRRRRR

9009001090090

RRRRRRRRR

Roo

oo

cossin

sincos

. (16)

A direction cosine is obtained to estimate the slippage direction of the grasped object during object handling. Referring to definition of sensor element coordinate position at hemispherical shape tactile sensor as shown in Fig. 9, the direction cosine of k-th sensing element in the local frame of the tactile sensor (αk, βk, γk) is define in (17).

⎥⎥⎥

⎢⎢⎢

⎡=

⎥⎥⎥

⎢⎢⎢

k

kk

kk

k

k

k

θφθφθ

γβα

cossinsincossin

. (17)

Hence, the direction cosine in the frame of workspace (αGk, βGk, γGk) is calculated as follows:

⎥⎥⎥

⎢⎢⎢

⎥⎥⎥

⎢⎢⎢

−−−

=⎥⎥⎥

⎢⎢⎢

k

kk

kk

313233

212223

111213

Gk

Gk

Gk

RRRRRRRRR

θφθφθ

γβα

cossinsincossin

. (18)

C. Control System Structure Fig. 10 shows control system structure of the multi-fingered

humanoid robot arm with robotic fingers and optical three-axis tactile sensor. This system comprises of three main controllers: arm controller, finger controller and tactile sensor controller. Each of these controllers is connected to each other using TCP/IP protocols via internet. The arm controller consists of two main modules: robot controller and motion instructor. Shared memory is used to connect these two modules. Meanwhile control system architecture for the robot finger controller which is based on tactile sensing is shown in Fig. 11.

Fig. 9. Coordination of sensing position in hemispherical dome tactile sensor

Fig. 10. Control system structure of multi-fingered humanoid robot arm with optical three-axis tactile sensor.

Fig. 11. System conception diagram of finger controller

This controller comprises of three modules: connection module, thinking routines and finger control module. It is connected with tactile sensor controller by the connection module using TCP/IP protocols. In addition, to obtain low force interactions of the fingers during exploring object surface without causing damage, rotation velocity at each joint is define precisely based on joint angle obtained in kinematics calculations, whereby force-position controls were performed.

In the tactile sensor controller, based on image data captured by CCD camera, an image processing board controlled the dividing procedure, digital filtering, calculation of integrated gray-scale value and centroid displacement. Since the image was warps due to projection from a hemispherical surface as shown in Fig. 5, software Cosmos32 installed in the computer modifies the warped image data and calculates G, ux and uy to obtain the three-axis force applied to the tip of the sensing element using (2). These control schemes enable the finger controller to perform force-position control to adjust grasp pressure of the two fingers on the given object.

Proceedings of the World Congress on Engineering and Computer Science 2007WCECS 2007, October 24-26, 2007, San Francisco, USA

ISBN:978-988-98671-6-4 WCECS 2007

Page 6: Control Scheme of Object Manipulation Based on Tactile ... · presented. Object manipulation experiments are conducted using hard and soft objects. Experimental results revealed that

V. EXPERIMENT AND RESULTS The ability to sense hardness and/or softness will be

particularly important in future applications to humanoid robot. Therefore, we have conducted this set of experiment in order to recognize and manipulate hard and soft object. We used a cubic shape wood block represents a hard object, and a cubic shape thin paper box represents a soft object. Since the parameters of the human hand and fingers that are involved in sensing the hardness and/or softness of an object have not been fully researched, we conduct calibration test on the tactile sensor system to grasp objects with different hardness. This is to obtain basic data for estimation of optimum grasping. From the test results we estimate the force parameters as shown in Table 2, which also indicate parameters of the finger control system.

In this experiment, at first the two fingers grasp the object to define optimum gripping pressure. At this moment, the grasp pressure in controlled by parameters of normal force thresholds. Then both fingers lift up the object to z-axis direction while maintaining the optimum grasp pressure. During this motion, both normal pressure and slippage are concerned. Therefore the finger controller utilized parameters of normal force and centroid change thresholds. Here, when shearing force exceeds the centroid change threshold, the finger’s velocity for reinforcing the grasping pressure is calculated using (19), whereby vector velocity of the finger v+∆v is defined by finger control module in the finger controller.

⎥⎥⎥

⎢⎢⎢

⎡=

Gk

Gk

Gk

pvvγβα

∆ (19)

Fig. 12 shows photographs of the robot arm performing object manipulation with wood block and paper box. In both experiments, the fingers managed to grasp the objects within optimum grasp pressure and lift it to upwards direction. An example of shearing force detected at x-axis and its relation with the fingertip movement at z-axis in experiment with paper box is shown in Fig. 13. The proposed control scheme managed to recognize low force interactions to grasp the object surface within optimum grasp pressure without causing damage to the object and the sensor elements. In addition, the formulations applied in this system enable precise control of the fingertips from determination of joint rotation angles and velocity.

VI. CONCLUSION We proposed a new control scheme of object manipulation

tasks based on tactile sensing in a multi-fingered humanoid robot arm towards application in real humanoid robot. We developed a novel optical three-axis tactile sensor system capable of acquiring normal and shearing force. To evaluate performance of the proposed system, we conduct object manipulation experiment with hard and soft object. Experimental results revealed that the proposed system managed to recognize low force interactions to grasp and manipulate different hardness of objects. It is anticipated that by using this system technology will bring forward the evolution of humanoid robots working effectively in real life.

Table 2. Parameters in experiments with hard and soft object. Category Parameter

Interval for sampling Sensor 100ms Finger 25ms Threshold of normal force F1 0.5N F2 1.8N Threshold of centroid change dr 0.004mm Velocity of repush vp 2mm/s Increment of normal force ∆F 0.08N Progress time ∆t 0.1s

Fig. 12. Experiments of object manipulation with wood block and paper box.

-0.05

-0.04

-0.03

-0.02

-0.01

0

0.01

0.02

0.03

0.04

0.05

0 10000 20000 30000 40000 50000 60000

Time, t [msec]

Cen

troid

cha

nge

at x

-axi

s d

x G m

m

-5.0

0.0

5.0

10.0

15.0

20.0

25.0

Fing

ertip

Pos

ition

, z[

mm

]

#00#01#03#05#07#09#11#13#15#17#19#21#23

Z-Pos.

-0.05

-0.04

-0.03

-0.02

-0.01

0

0.01

0.02

0.03

0.04

0.05

0 10000 20000 30000 40000 50000 60000

Time, t [msec]

Cen

troid

cha

nge

at x

-axi

s d

x G m

m

-5.0

0.0

5.0

10.0

15.0

20.0

25.0

Fing

ertip

Pos

ition

, z[

mm

]

#00#01#03#05#07#09#11#13#15#17#19#21#23

Z-Pos.

Fig. 13. Relation between amount of x-directional centroid change and z-directional fingertip position at right finger for experiment with paper box.

REFERENCES [1] S. Omata, Y. Murayama and C.E. Constantinou, “Real time robotic tactile

sensor system for determination of the physical properties of biomaterials”, J. Sensors and Actuators A, 112(2-3), 2004, pp 278-285.

[2] O. Kerpa, K. Weiss and H. Worn, “Development of a flexible tactile sensor system for a humanoid robot”, Proceeding of IROS2003, vol. 1, Oct. 2003, Las Vegas, USA, pp. 1-6.

[3] M.H. Lee and H.R. Nicholls, “Tactile sensing for mechatronics: a state of the art survey”, Journal Mechatronics, 9(1), 1999, pp 1-31.

[4] A. Edsinger and C. Kemp, “Manipulation in human environments”, Proc. IEEE/RSJ Int. Conf. on Humanoid Robotics (Humanoids06), 2006, Italy.

[5] L. Natale and E. Torres-Jara, “A sensitive approach to grasping”, Proc. 6th International Conference on Epigenetic Robotics, Sept. 2006, France.

[6] P. A. Schmidt, E. Mael, and R. P. Wurtz, “A sensor for dynamic tactile information with applications in human-robot interaction and object exploration”, J. Robotics & Autonomous Systems, vol. 54, issue 12, Dec. 2006, pp. 1005-1014.

[7] Y. Hanafiah, M. Ohka, H. Kobayashi, J. Takata, M. Yamano and Y. Nasu “Contribution to the development of contact interaction-based humanoid robot navigation system: Application of an optical three-axis tactile sensor”, Proceeding of 3rd ICARA06, Dec. 2006, New Zealand, pp. 63-68.

[8] M. Ohka, H. Kobayashi and Y. Mitsuya, “Sensing precision of an optical three-axis tactile sensor for a robotic finger”, Proceeding of 15th RO-MAN06, Sept. 2006, United Kingdom, pp. 220-225.

[9] M. Ohka, Y. Mitsuya, Y. Matsunaga, and S. Takeuchi, “Sensing characteristics of an optical three-axis tactile sensor under combined loading”, Robotica, vol. 22, 2004, pp. 213-221.

[10] J. Denavit and S. Hartenberg, “A Kinematic Notation for Lower-pair Mechanisms Based upon Matrices”, Journal of Applied Mechanics, 1955, vol. 77, pp. 215-221.

Proceedings of the World Congress on Engineering and Computer Science 2007WCECS 2007, October 24-26, 2007, San Francisco, USA

ISBN:978-988-98671-6-4 WCECS 2007