Robotization of Orchid Protocorm Transplanting in Tissue Culture

Tsuguo OKAMOTO

Department of Agricultural Engineering, The University of Tokyo (Bunkyo, Tokyo, 113 Japan)

Abstract

Nowadays, orchid plants are propagated by mericlone culture. When the seed propagation method of this culture is used, a protocorm transplant is necessary during the growth stage. This study on robotization of the transplant for orchid protocorms was conducted to prevent biological contamination and to reduce labor costs. The image processing algorithm developed in the study was useful for recognizing protocorms about 0.5 mm in diameter, interspersed on the culture medium. A small, light gripper driven by a shape memory alloy actuator was constructed and was able to handle a very small object. The intelligent robotic system with machine vision and micro-handling mechanism that was developed could perform protocorm transfers automatically, with a higher accuracy and efficiency than humans.

Discipline: Agricultural machinery/BiotechnologyAdditional key words: automation, end-effector, machine vision, micropropagation

JARQ 30, 213 - 220 (1996)

Introduction

Mass production of orchid plants is commonly achieved by tissue culture. However , to promote the germination and growth of orchid seeds, the seed propagation method has become popular. The seed of the orchid plant is exalbuminous and the germi­nation percentage is very low. The plant also grows very slowly. Using the seed propagation method, an orchid seed planted in agar growth medium gener­ates a protocorm. If this protocorm is transferred to a fresh culture medium at the proper time during its growth stage, the germination percentage increases, and growth is improved.

The seed propagation technique is one of the sterile culture methods. About I to 3 weeks after planting an orchid seed, a slender ovule, about 0.5 to 1.0 mm in length appears and enlarges. The ovule tears open its seed coat and swells with a sphere at its center. The protocorm then becomes light green and, when transplanted to fresh growth medium, it germinates and takes root. A rooted plant that has grown to 2 to 5 cm, should be transferred to a pot. When the protocorm is cultured in a liquid rotary system it grows into an aggregation of protocorms.

If the aggregation becomes too large, it should be divided into several pieces and propagated again in the liquid rotary system.

The size of the orchid protoeorm is very small, about 0.5 to 1.0 mm in diameter. For the transfer, a technician must pick it up from the exhausted cul­ture medium and gently place it into the fresh cul­ture medium in another container. These operations which must be delicately and subtly handled under sterile conditions to avoid biological contamination are difficult and tedious even for a trained worker. Most of the biological contamination is caused by various microorganisms transmitted by workers' hands. The best way to prevent contamination is to remove the main contamination source, i.e. manual operation. The authors earlier developed a robotic transfer system 1- 5> for plant tissues such as callus at the subculture stage. This system which was found to be effective did not enable to handle an object as small as an orchid protocorm. In this study, an intelligent robotic system was developed for efficiently transferring orchid protocorms, preventing biologi­cal contamination, and reducing labor costs. The robot employed an image processing and micro­handling technique to transfer minute and delicate

objects such as protocorms.

214

Materials and methods

Protocorms used in lhe study were those of 8/etiJ/a stria/a Reichenbach Fil., with a s1andard size (Plate I). Since the process of growth is al.so abo·ut the same, the robo1 system that was developed would be used to transfer most orchid plants. Fig. I shows an outline of the system. The robot consisted of a manipulat0r control, an end-effector control, an image processor and a computer comrol.

JARQ 30(4) 1996

I) Manip11la1or The manipulator used, a SCARA type robot

(SONY SRX-4CH), is depicted in Plate 2. The length of the first arm was 350 mm and of the second arm 250 mm. The synthetic maximum velocity of the 2 arms was 5.2 m .s- 1

• Maximum velocity of the Z axis was 300 mm.s- i, and maximum rotating speed of the R axis was I 080° s- 1

• The positional accuracy of the X, Y and Z axes was ± 0.05 mm. The rotational accuracy of the R axis was ± 0.05°.

Plate I. Protocorms of 8/etilla slriala Reichenbach Fil.

I __ ~in.!!Jlo~be'l;h,

I I I I I I I I L Protocorm

I At= Bl=i

Petri dishos --------

I I I I

Image data processor

SMAdriving EJ controller 0 ••

D Manipulator controller

Personal computer

Fig. I. Schematic diagram of a robotic system f'or transplaming or orchid protocorms in seed propagation

T. Okamoto: Rol,01iuttio11 of Orchid Proloform Tm11spl<1111i11g i11 Tiss11£' C111!11re 215

Plate 2. Robotic transplanting of orchid protocorms in mericlone culture

2) Machine vision sys1em A monochromatic TV camera was used for imag­

ing. With its image sensor of the metal oxide semi­conductor type, it took 2-dimensional image data of the protocorms dotted on the agar growth medi­um. The composite video signal output from the camera was changed from an analog signal to a digi­tal signal by a 4-bit A-D converter. The image data of a field were stored in the field memory. Video image data in a single field consisted of 64,000 sam­pling points in an area of 320 x 200 pixels. The data stored in the field memory were loaded onto the memory of the computer in response to an order. lt took about 0.3 s to transfer 32 Kbyte of data. The TV camera was installed with its back toward a gripper in the R axis of t he manipulator as shown in Fig. I. The camera enabled to observe a given work area. A magnifying ring for the camera was auached to the lens system. To obtain the resolu­tion necessary to recognize a protocorm, the foca l length of the TV camera was adjusted to 15 mm (lengt h) and 24 mm (width) on the surface or the agar culture medium. As a result, the size of a pixel in the picture was equivalent to 75 x 75 tt 2 on the work surface.

Plate 3. Gripper for prolQcorm transplanting

3) So/I-handling gripper Various grippers have been studied for soft

handling of plant tissues 1- 6>, rnicrocutting6l and plantlets 7>. Here, a gripper was developed to handle a very small object. The structure of the gripper that has a pair of finger-l ike tweezers is shown in Plate 3. II is operated by a shape memory a lloy (SMA) actuator. ASMA wire, 0.15 mm in diameter and 270 mm in length, is stretched tightly so that it passes the A, C, E, F, D and B points shown in Fig. 2. An electric current applied between termi­nal A and terminal B heats the SMA wire. As the temperature of the SMA rises , the length of the wi re decreases. The result is that point C and point D are pulled apart, causing the fingers to open. The gripper ls DOrmally in a closed position.

The transformation temperature of SMA is con­trolled by heating with an input electric current using the pulse width modulation (PWM) method and by cooling in the atmosphere. The pulse frequency of the current is 5 kHz and the height of the current pulse, supplied by a constant current regulator, is 400 mA. The SMA material used was a Ni-Ti a lloy that had a resistivity o f about 42 n.m-1

• The phase transformation temperatures of SMA, As, Af,

216

R-axis

Gripper

/ 147

(Unit ; rrtn)

Fig. 2. Finger pair driven by shape memory alloy actuator for handling protocorms

Ms and Mf were 90, 120, 90 and 70°C, respectively. The shape memory recovery ratio of the alloy is generally equal to or less than 30Jo of the full length. To magnify the displacement of the fingers, the SMA wire was tightened by winding as shown in Fig. 2.

The SMA actuator, without mechanical parts, a l­lows the gripper mechanism to be smaller and lighter than in other actuators, and the movement is smooth and quiet. The gripper fingers were made of a phos­phorus bronze plate, 0.4 mm thick. A fingertip made of stainless steel 2 mm wide and 0.5 mm thick was attached to each finger. Only after the gripping motion had begun, was the normally closed pair of fingers opened by the SMA actuator. To grip an object, the applied current to ihe SMA was shut off and the pair of fingers closed by the bending force of the finger plates. The force required to grip a protocorm was set at about 0. I N. The bend­ing force of the finger plate acts as a bias force that extends the SMA material to its original length when not active. SMA under the shape memory non-recovery condition cannot generate power. That is, the SMA can generate a shrinking force itself just by opening the fingers.

4) Computer The computer (NEC PC-98RL) used here had a

32-bil CPU with a clock speed of 20 MHz. T he

JARQ 30(4) 1996

manipulator was controlled through a serial inter­face RS232C cable that connected the manipulator controller to the computer. The communication speed was 9,600 bps, and the control program was written by N88BASIC. The programs of image processing and SMA driving that would have required more time to process were written in assembly language.

Image processing algorithm

A TV camera first rook a picture of the proto­corms dotted on the agar culture medium in a petri dish. The procedure of recognizing protocorms is shown in the now chart in Fig. 3. Processing the main part of the procedure was performed by the following algorithm.

1) Geuing an image of protocorms The comrast and the brightness of the TV camera

were initially adjusted. By illumination with fluores­cent light from above, specular reflectance on the agar growth medium was minimized. P rotocorms pictured were growing on semitransparent agar growth medium and their color, when ready for transfer, was light green. A petri dish with protocorms was pUL on top of a black sponge. The background in­tensity was then lowered so that the contrast in t he image obtained was sufficient for the computer to

( IMAGE DATA PROCESSING ) l

INITIALIZATION Lighting, Brightness, Contrast,

Threshold for binarization I

I GET PROTOCORM IMAGE THROUGH TV CAMERA I I

iFIL TERING WITH LAPLACIAN FILTER FOR SHARP OUTLINE I l

I NOISE ELIMINATION THROUGH SPACE FILTER I I

GET IMAGE BINARIZED ON THRESHOLD I

LABELING OF CONNECTED COMPONENT I

CALCULATION Centroid, Area, Vertical and horizontal Ferers diameter

l

I SELECT PROTOCORM OBJECTS TO BE TRANSPLANTED I I

(RETURN)

Fig. 3. Image processing procedure

T. Okamoro: Robo1iwtio11 of Orchid Prorocorm Tro11spl11111i11g i11 'f'issue C11il11re 217

Plate 4. Print of an original picture image of orchid prococorrns

recognize individual protocorms. Input image data were sent to the computer

ml!mory. The data were displayed as a monochrome picture with a gray scale of 16 intensity levels. It was also possible to display a 16-pseudo-color im­age. Plate 4 is an example of an original image. A pixel with an intensity of O displayed a black dot while a pixel with an intensi ty over 1 displayed a white dot. Some protocorms were in close contact and overlapped seed coats. The picture also shows noise due to specular reflection on the agar growth medium.

2) Image filters Various spatial filters were used to enhance the

image and extract characteristics efficiently. Picture noise caused by irregular reflectio ns on the medium surface occurred in the original image. As the im­age of the seed coat surrounded the globular proto­corm image, the edge of the latter was indistinct. To remove noise and make the picture edge clear, a low pass filter and a Laplacian filter were used.

To make the edge clear, t he result of the Lapla­cian filtering process was deducted from the original image data. T he intensity inclination in the edge then became steep. This computational process is described by the equation,

+ X1.J+ I - 4X1.; )

= 5X;,j - X ;- l,j - X ;+ I.J - Xi,; - I - X 1,J+ I

where X1.1 is the original intensity of pixel (i, j), and Y;,; is the value of pixel (i, j) after the filtering process. The expression in parentheses refers to the Laplacian process.

The low pass filter for smoothing operates the processing shown in the next equation.

y . . 1,/ = ~[111· . .J::·1

m • i - 1

11 -D j + I

I; X,,,, ,, + 3X1.1 ,, : j - 1

3) Binarization It was necessary to identify individual protocorms

that were close together. To locate the centroid of a protocorm, the image data were analyzed. Before binarizat.ion, processing of the original image was performed using a Laplacian filter to' clar ify it. After noise was removed by the low pass filter,

IJ•f\\ry utogo,

' . '

. '

• • . . . • . • • • • • ~ .... . .. • ·,

Fig. 4 . Binary image of figured components based on the image in Fig. 5

218

a binary image was obtained using a global threshol.d that was experimentally determined.

Since the shape of a protocorm is globular, the intensity of the top part is greater than that of other parts. The center of a protocorm could thus be identified by discriminating the high intensity pari.. Fig. 4 is a binary image based on the original image in Plate 4.

4) labeling fn a binary image, the value of each pixel is 0

or t. An area where every pixel has the same value and connects is designated as a 'connected compo­nent'. There were many connected components seg­mented in the central part of a protocorm that had a value of 1 in a binary image. To recognize each individual component, a unique label must be given to each. A labeling process assigned different names to t hem. Various algorithms can be used for the labeling. In this study, pixel data were observed from the top left to the botiom right on the image. Four connected pixels of a pixel were examined and a label was given to the pixels in the areas with a value of 1. After a ll the pixels were examined, the identity of the label was determined. The same labels were given to every pixel of the connected component.

5) Feature calculation Centroid, bounding rectangle and area were cal­

culated as the features of the connected components. The detected centroid positions are indicated by the + marks in Fig. 5. To calculate these parameters most efficiently, a rectangle that bounded the con­nected component was drawn. Only the small area inside the rectangle was used when the features wer,e calculated.

'

Fig. 5. Detection of centers of protocorm image with labeling

JARQ 30(4) 1996

( START )

I CALIBRATION AND COOROINATE MATCHING Machine vision. Manipulator. Finger.pair I

I MANIPULATOR MOVES TO STANDING POSITION I I

I MOVE CAMERA TO A POINT FOR GETTING PROTOCORM IMAGE l

!GET IMAGE OF PROTOCORMS ON SURFACE OF CULTURE MEDIUM I

I IMAGE OATA PROCESSING AND ANALYSIS

Recogniling prolocorm particles Measuring center or <lbjects

I SELECT TARGETS TO BE TRANSPLANTED

I OPEN GRIPPER

I I

I

I GUIDE flNGER·PAJR TO A PROTOCORM I '

I CLOSE GRIPPER ANO PICK UP OBJECT I l

I TRANSFER PROTOCORM TO A TRANSPLANTING P0St110N I

I PUT FINGER TIPS INTO MEDIUM ANO OPEN TIPS I

I PLACE PROTOCORM ON SURFACE OF CULTURE MEDIUM I

J

ALL OF SELECTED TARGETS WERE

TRANSPLANTED Tves

ENO TRANSPLANTING

I ves ( STOP )

no

no

Pig. 6. Plow chan of robotic transplanting of orchid protocorms

Robotizalion of protocorm transplant

I) Transplant procedure A protocorm should be transferred to fresh agar

growth medium within 2 lO 3 weeks after the seed is planted. The robot picked up protocorms from exhausted culture medium, transferred them individu­ally to a petri dish containing a fresh culture medi­um, and placed them in a pin grid array at regular in tervals of 7 mm. The robotic transfer procedure is shown in Fig. 6.

2) Coordinate matching between image and manipulator

First, the manipulator moved to the machine origin of the manipulator base coordinate system and then moved to the task origin. T his position was also the original position of the manipulator. The origin of the image coordinate system was in the center of the image field. The optical axis of the TV camera passed through this point. The optical axis of the camera lens and the center line of the

T. Okamoto: Robotiza1io11 of Orchid Prowcorm Tra11spl11111ing i11 Tissue Culture 219

gripper were parallel to the rotational axis, R, but there was a space between them. Therefore, the grip­per and image coordinate systems were related to the task coordinate system geometrically.

3) Selection of object After protocorms were recognized in the image,

their spatial positions were determined. Since orchid plant seeds are too small and light to be planted at regular intervals on the surface of agar growth medium, the generated protocorms were planted close together randomly and even formed a cluster in places. As a result, it was difficult to pick up just I protocorm with the gripper. The robot sometimes gripped a couple of protocorms. Even a trained

Fig. 7. Selected objects 10 be transferred

worker may occasionally trans fer more than I pro­tocorm at a time.

A work area for the gripper had to be provided with enough space for the pair of fingers to operate. The area selected was reCLangular as shown in Fig. 7. Even if more than I protocorm was present in that area, the protocorm in the center marked by a large +, as shown in Fig. 4, was selected as the only object to be transferred. Starting from the top left and moving to the bouom right of an image, work areas were identified. An object in the center of one area was present only in that particular area, and not in any other area.

Fig. 8 shows the motion of the pair of fingers transferring a protocorm. First, a shape memory alloy actuator made the fingers open only by I mm by applying an elect-ric current. The open fingers were led to a position right over the protocorm and lowered by the manipulator as shown in Fig. 8(a). The fingers were dipped into the agar growth medi­um to a depth of I mm. Next, the electric current running through the shape memory alloy element was shut off so that it was free from the shape memory state. Then, the spring force of the finger plates caused the pair of fingers to close around the proto­corm. Since the protocorms must be gently and care­fully handled, the gripping force of the fingers was adjusted to about 0.1 N so that it would not damage the protocorms.

~~1!~~~'. approach .- dip into - > close .... pick ~

(a) FINGER-PAIR MOTION AT GRIPPING

I I I I loPview

{b) FINGER-PAIR MOTION AT PLACING

Fig. 8. Finger pair operat ion for transplanting of orchid protocorms

220

The gripper was then led to a pre-programmed position on the petri dish holding the fresh culture medium. The fingers immersed the protocorm into the medium to a depth of 1 nun, then ope11ed (2 mm wide) as shown in Fig. 8(b). To ensure that the object was released into the culture medium, the manipulator shifted the fingers 2 mm to the right of the area where the protocorm was released.

4) Pe,f ormance Although the time of one image processing de­

pended on the number of protocorms transferred, it took less than 4 s from obtaining the image to determining the centroid of objects. The selected protocorm was then transferred to a culture medium in a pin grid array at 7 mm intervals. About 8 s were required for each transfer. After all selected protocorms in the image frame had been transferred, the TV camera then gave on image of an adjacent area of the culture surface of the petri dish and the trans fer procedure was repeated.

ln contrast, it takes 5 to JO s for a trained work­er to transfer a protocorm. In a manual operalion, much time is required for preparation because of the need to sterilize and disinfect instruments. Also, it is difficult for a worker to place a protocorm on the medium at regular intervals by hand and to con­tinue such a tedious task for many hours. The ro­botic system developed in this study was more efficient than a technician performing the work manually.

Conclusion

The transfer of orchid protocorms is still per­formed manually by technicians skilled in biological routines, and is very costly. Most of the biological contamination is caused by various microorganisms transmitted by workers' hands. The best way to pre­vent contamination is to remove the main contami­nation source, manual operation. The current study on the robotic transfer of these orchid protocorms was performed to prevent biological contamination

JARQ 30(4) 1996

and to reduce labor cost. The image processing algorithm developed in this

srndy was useful in recognizing protocorms about 0.5 mm in diameter. A small , light gripper driven by a shape memory alloy actuator was constructed and enabled 10 handle a very small object successful­ly. An intelligem robotic system wi1h machine vi­sion and micro-handling mechanism was developed. The robot picked up protocorms from exhausted culture medium, transferred 1hem individually to a petri dish containing fresh medium, and placed them in a pin grid array at regular intervals of 7 mm. It took about 8 s to transfer each protocorm. The efficiency was higher than in the case of manual operation.

References

I) Okamolo, T. & Kitani, 0. (1989): Smdies on robot­ics for biotechnological operations. l. On callus han­dling robot. J. Jpn. Soc. Agric. Mach., 51 (5), 37-45 [In Japanese with English summary].

2) Okamoto, T. & Kitani, 0. (1990): Studies on robotics for bio1cchnological operations. IL I111ellige111 robot for transplanting of calluses in subculture. J. Jpn. Soc. Agric. Mach., 52(5), 79-85 !I n Japanese with English summary).

3) Okamoto, T ., Kitani, 0. & Torii, T. (1991): Smdies on robotics for biotechnological operations. II l. Fuz­zy control of robot finger set driven by shape memory alloy actuators. J. Jpn. Soc. Agric. Mach., 53 (5), 85-91 [In Japanese with English summary].

4) Okamoto, T., Kitani, O. & Torii, T. (1992): Tissue proliferation robot in plam tissue culture. ASAE paper No. 923516.

5) Okamoto, T., Kitani, 0. & Torii, T. (1993): Robotic transplanting of orchid protocorm in mcriclone cul­rnre. ASAE paper No. 933090.

6) Simonton, W. (1991): Robotic end effector for han­dling greenhouse pla111 material. Trans. ASAE, 34(6) , 2615 - 2621.

7) Ting, K. C., Giacomelli , G. A., Shen, S. J. & Kabala, W. P. (1990) : Robot workccll for transplanting of seedlings. II. End effector development. Trans. ASAE, 33(3), !013- 1018.

( Received for publication, November 6, 1995)