February 15, 1991 / Vol. 16, No.4 / OPTICS LETTERS 247

Space-time-sharing optical neural network

Francis T. S. Yu, Xiangyang Yang, and Taiwei Lu

Department of Electrical and Computer Engineering, The Pennsylvania State University, University Park, Pennsylvania 16802

Received July 2, 1990; accepted November 14, 1990

A space-time-sharing optical neural network for implementing a large-scale operation is presented. If the intercon-nection weight matrix is partitioned into an array of submatrices, a large space-bandwidth pattern can be processedwith a smaller neural network. We show that the processing time increases as a square function of the space-bandwidth product of the pattern. To illustrate the space-time-sharing operation, experimental and simulatedresults are provided.

The massive interconnection and parallel processingcapabilities of optics offer advantages for optical com-puting. In recent years, much effort has been appliedfor the optical implementation of neural networks.1 -10To provide the neural network with learning ability,spatial light modulators (SLM's), such as a video mon-itor7' 8 and a liquid-crystal television9' 10 (LCTV), havebeen used as the programmable devices. In a fullyinterconnected neural network, every neuron at theinput plane is connected to all the neurons at theoutput plane. For example, 1000 neurons would re-quire a million interconnections. Thus an extremelyhigh-resolution SLM is required for the massive inter-connection. However, the resolution of the currentlyavailable SLM's is rather limited. This poses an ob-stacle in developing an optical neural network forlarge-scale operation. In this Letter we use a space-time sharing technique to alleviate this constraint.

For an N X N neuron network, the interative equa-tion can be described by2

N N

Vlk(n7 + l) = fT,,ijUij(n) ,i=l j=l

(1)

where n denotes the nth iteration, f(.) represents anonlinear operator, Vlk and Uij represent the state ofthe lkth and ijth neurons, respectively, and Tlkij is theconnection strength from the lkth to the ijth neuron.[Tlkij] is known as the interconnection weight matrix(IWM) and can be partitioned into an array of N X Nsubmatrices. 2' 8

An optical neural network (Fig. 1) is used in ourdiscussion, in which the IWM and the input patternare displayed onto LCTV1 and LCTV2, respectively.Each lens in the lenslet array images a specific subma-trix onto the input LCTV2 to establish the properinterconnections. The lenslet array is imaged by theimaging lens onto the output plane. The overall in-tensity transmitted by each lens in the lenslet array ispicked up by the charge-coupled-device (CCD) detec-tor as the output value. We let the resolution LCTV1be limited by R X R pixels, and the lenslet array isequal to L X L neurons. If the size of the IWM islarger than the resolution of LCTV1, i.e., N2 > R, the

IWM cannot be fully displayed onto LCTV1. Wediscuss in the following cases how a small neural net-work may accommodate a large-size IWM:

For N2 > R and LN < R, we let D = int(N/L), whereint(-) is the integer function. The IWM can be parti-tioned into D X D sub-IWM's, and each sub-IWMconsists of L X L submatrices of N X N size, as shownin Fig. 2. To complete the iterative operation of Eq.(1), D X D sequential operations of the sub-IWM's arerequired. Thus a smaller neural network can handle alarger space-bandwidth product (SBP) input patternthrough sequential operation of the sub-IWM's.

For N2 > R and LN > R, we let D = int(N/L) and d =int(LN/R). In this case the submatrices within thesub-IWM's are further divided into d X d smallersubmatrices, and each submatrix is (Nid) X (Nid) insize, as shown in Fig. 3. The iterative equation can bewritten as

Vlk(n + 1)

(d-ld-1F (p+1)(N/d)(q+1)(N/d)

p=Oq= i=pd+l j=qd+l(2)

The input pattern is also partitioned into d X d sub-matrices, and each submatrix is (N/d) X (Nid) in size.Thus by sequentially displaying each of the IWM sub-matrices with respect to the input submatrices ontoLCTV1 and LCTV2, respectively, one can process anextremely large SBP pattern with a small neural net-

LightSource

-I ,co-

11 \

L CTU 1GI Lenslet

Diffuser Rrray Imagingnrce CTD2 system r

Z~~~~~ne ac [S!9

Fig. 1. Schematic diagram of the optical neural network.

0146-9592/91/040247-03$5.00/00 © 1991 Optical Society of America

TlkijUij(n) .

248 OPTICS LETTERS / Vol. 16, No.4 / February 15, 1991

... ... ... ... ... ...

DM 11 IVZ4 12

interconnection weight, with which the resolution ofthe LCTV1 is essentially reduced to 120 X 80 ele-ments.

By referring to N = 12, L = 6, and R = 80 and notingthat N2 = 144 > R and LN = 72 < R, we have D =int(N/L) = 2. The IWM can be divided into a 2 X 2

l11 1 lo ..11

LI .. 1 ° ...21

2M22

Fig. 2.2).

Partition of the IWM into D X D sub-IWM's (D =

IWMpq

(b) (c) (d)

Fig. 4. Experimental results of processing 12 X 12 elementpatterns on a 6 X 6 optical neural network. (a) Four refer-ence patterns stored in the IWM, (b) the partial input pat-terns, (c) one of the four 6 X 6 suboutput arrays, (d) thecomposed output pattern.

(a)

0 ... i0 0 1 1 ... El E . l l..E

Me6 ... 0 0 .. El . El...ED(b)

Fig. 3. Partition of a sub-IWM into d X d submatrices (d =2). (a) The pqth sub-IWM, (b) d X d smaller submatrices.

work. It is trivial that the price we paid is prolongingthe processing speed by D2 X d2 times.

Generally, the processing time increases as thesquare function of the SBP, of the input pattern, asgiven by

T2 = [(N2 X N 2)/(Nl X Nl)] 2T, = (N21NJ)4T1,

(N2 > N 1 > = L), (3)

where T2 and T1 are the processing times for the inputpatterns with N2 X N2 and N1 X N 1 resolution ele-ments, respectively. For instance, if the resolutionelements of the input pattern increase four times ineach dimension, i.e., N2 = 4N1 , the processing timewould be 44 = 256 times longer.

For experimental demonstrations we show that pat-terns with 12 X 12 resolution elements can be pro-cessed using the 6 X 6 neuron network shown in Fig. 1.A 240 X 480 element Hitachi color LCTV (LCTV1) isused for displaying the IWM. Since each color pixel iscomposed of red, green, and blue elements, the resolu-tion is actually reduced to 240 X 160 pixels. In theexperiment, however, we used 2 X 2 pixels for each

(a)

(b)

(C)

(d)

Fig. 5. Computer-simulated results of processing 24 X 24element patterns on a 6 X 6 optical neural network. (a) Fourreference patterns stored in the IWM, (b) the partial inputpattern, (c) four of sixteen 6 X 6 suboutput arrays, (d) thecomposed output pattern.

(a)

February 15,1991 / Vol. 16, No.4 / OPTICS LETTERS 249

sub-IWM array and displayed one by one ontoLCTV1. If an input pattern is displayed on LCTV2,the signals collected by the CCD camera can be thresh-olded and then composed to provide an output pat-tern, which has a SBP four times larger than that ofthe optical neural network.

One of the experimental results is shown in Fig. 4.The training set is shown in Fig. 4(a), and each of thesepatterns is limited by a 12 X 12 pixel matrix. Figure4(b) shows a part of the second image with 12 X 5pixels blocked as the input pattern. The sub-IWM'sare then sequentially displayed, one by one, ontoLCTV1. Different parts of the output pattern areobtained, and one of the output parts is shown in Fig.4(c). The final output pattern recalled by the opticalneural network is given in Fig. 4(d).

To demonstrate the larger-scale operation further, a24 X 24 neuron IWM is used. Since N = 24, L = 6, R =80, N2 = 576 > R, and LN = 96 > R, we take D int(N/L)= 4 and d = int(LN/R) = 2. Thus the IWM is parti-tioned into 4 X 4 sub-IWM's, and each sub-IWM isdivided into 2 X 2 smaller submatrices, as illustratedin Fig. 3. In this case the input pattern is also dividedinto 2 X 2 submatrices, and each submatrix is 12 X 12in size. It is apparent that by sequentially displayingthe submatrices of IWM and the input submatricesonto LCTV1 and LCTV2, respectively, a 24 X 24 out-put pattern can be obtained.

Figure 5(a) shows four 24 X 24 pixel images as thetraining set. A partial pattern of the third image,shown in Fig. 5(b), is used as the input pattern, whichis divided into 2 X 2 matrices of a 12 X 12 size duringthe processing. Four of sixteen 6 X 6 output parts areshown in Fig. 5(c). In going through all the partitionsof the sub-IWM's, we compose an output pattern ofthe image, as shown in Fig. 5(d). The whole processtakes D2 X d2 = 64 operations of the optical neuralnetwork.

In conclusion, we have used a space-time-sharingtechnique for large-scale neuron operation. We haveshown that to achieve a large SBP of the system, addi-tional expenditure of processing time is needed. Theamount of processing time increases as the squarefunction of the SBP of the input pattern. For experi-mental demonstration, we have implemented the pro-cessing of 12 X 12 element patterns on a 6 X 6 neuronnetwork. The processing of 24 X 24 element patternson a 6 X 6 neuron network has been further demon-strated by computer simulation.

We acknowledge the support of the U.S. Army Mis-sile Command through the U.S. Army Research Officeunder contract DAAL03-87-0147.

References

1. N. H. Farhat, D. Psaltis, A. Prata, and E. Paek, Appl.Opt. 24, 1469 (1985).

2. N. H. Farhat, and D. Psaltis, in Optical Signal Process-ing, J. L. Horner, ed. (Academic, New York, 1987), p.129.

3. B. Macukow and H. H. Arsenault, Appl. Opt. 26, 924(1987).

4. N. Farhat, Appl. Opt. 26, 5093 (1987).5. R. A. Athale and C. W. Stirk, Opt. Eng. 28, 447 (1989).6. D. Psaltis, D. Brady, X.-G. Gu, and K. Hsu, in Optical

Processing and Computing, H. H. Arsenault, ed. (Aca-demic, San Diego, Calif., 1989), p. 251.

7. S. Wu, T. Lu, X. Xu, and F. T. S. Yu, Microwave Opt.Technol. Lett. 2, 252 (1989).

8. T. Lu, S. Wu, X. Xu, and F. T. S. Yu, Appl. Opt. 28,4908(1989).

9. F. T. S. Yu, T. Lu, X. Yang, and D. A. Gregory, Opt. Lett.15, 863 (1990).

10. X. Yang, T. Lu, and F. T. S. Yu, Appl. Opt. 29, 5223(1990).