1

EE630 PROJECT 3D OVERVIEW OF PHASE MEASURING PROFILOMETRY STRUCTURED LIGHT ILLUMINATION by LGH 11-7-2014 Updated 11-10-2014: We introduce you to a Structured Light Illumination (SLI) method known as Phase Measuring Profilometry (PMP). In this method we analyze each pixel of the captured pattern sequence as a temporal signal. Each signal is exactly one cycle long. From the phase shift we can determine the depth of the surface at that pixel. Our coordinate system is comprised of the phase dimension (vertical axis) and the orthogonal dimension (horizontal axis). We are using a dual-frequency version of PMP where we first project patterns with a long spatial period or what we refer to as the baseline kc=1 pattern set. The baseline set of patterns yields a non-ambiguous phase result which tends to be noisy and sometimes distorted. To reduce the noise and distortion we project a second set of patterns which are higher frequency along the phase dimension. The second set yields a wrapped phased result which is lower in noise but ambiguous in depth. So the first baseline phase is used to unwrap the second wrapped phase result yielding both a precise surface measure and non-ambiguous depth. The unwrapped phase images are converted to 3-D coordinates via perspective transformations found from a calibration data set. 1. INPUT TARGET DATA Given a sequence of 24 bit BMP files, we generate an inphase and quadrature phase image pattern.

Figure 1: Last captured image in sequence.

2

Fig. 1 shows the mannequin “Fred” with sinusoidal pattern projected at an angle from the camera’s optical axis. Figs 2 and 3 show the complete 32 pattern sequence for the baseline and second frequency sequences, respectively.

Figure 2: Baseline frequency kc=1 image sequence.

Notice the spatial movement of the pattern across the FOV. For the baseline image sequence, the spatial period is greater than the vertical distance of the FOV. The second frequency set is shown in Fig. 2. The frequency and number of patterns are set up in MATLAB as: clear all; Nf = 2; % number of different frequencies Nk(1)=1; % kc0=1 (k=0) first of Nf frequences Np(1)=32; % number pattern shifts for first frequency Nk(2)=16; % kc1=16 (k=1) second of Nf frequences Np(2)=32; % number pattern shifts for second frequency

3

Figure 3: Second frequency (kc=16)

File names, sizes, initialization of phase, albedo and indicator images are given by Pathname='Data3D' % path or folder name in reference to your default path Filename='pattern' % name of files not including the index or suffix Filesuffix='bmp' % suffix or image type index=0; % first index is 0 Fullname=sprintf('%s%c%s%d%c%s',Pathname,'\',Filename,index,'.',Filesuffix); A_bmp=double(imread(Fullname)); % load pattern#.bmp [My, Nx, Pz] =size(A_bmp); My Nx Pz Phase0=zeros(My,Nx); Phase1=zeros(My,Nx); AlbedoC=zeros(My,Nx); AlbedoI=zeros(My,Nx); For both frequencies, the number of patterns is 32. We will form a phase image for both frequencies, albedo image as the average of all patterns and an indicator image based only on the second frequency. In addition to this, the second phase image will be unwrapped during the input process. The entire code is as follows: index=0; % first index is 0

4

for k=1:Nf; % frequency group Iq=zeros(My,Nx); Ii=zeros(My,Nx); for p=1:Np(k); Fullname=sprintf('%s%c%s%d%c%s',Pathname,'\',Filename,index,'.',Filesuffix); A_bmp=double(imread(Fullname)); % load pattern#.bmp Ar=A_bmp(:,:,1); %% if k==1 figure(2); else figure(3); end; subplot(8,4,p,'align'); imagesc(Ar); colormap gray axis image; axis off %title('Captured Pattern Sequence'); % average in albedo AlbedoC=AlbedoC+Ar; % determin the sin and cos phase shift coefficients sincoef=sin(2*pi*(p-1)/Np(k)); coscoef=cos(2*pi*(p-1)/Np(k)); % average in weighted quadrature and inphase images Iq=Iq+Ar.*sincoef; Ii=Ii+Ar.*coscoef; % increment to next pattern index=index+1; end; % loop over p figure(2); print -dtiff 'B_SubplotsOfBaselineFreq.tif';%% figure(3); print -dtiff 'C_SubplotsOfSecondFreq.tif';%% if Np(k)>1 & k==Nf Iq=Iq/Np(k); Ii=Ii/Np(k); end; % process first set of patterns into phase image if k==1 Phase0=atan2(Iq,Ii)+pi; end; AlbedoI=sqrt(Ii.*Ii+Iq.*Iq); % calculate the peak to peak response of patterns % process second set of patterns into phase image if k==2 Phase1=atan2(Iq,Ii)+pi; %Phase unwrapping %Phase cycles Cycle0=floor((Nk(2)*Phase0-Phase1)/(2*pi)); % find cycle remainder between unwrapped phase1 and Phase0 remainder=((Nk(2)*Phase0-Phase1)-Cycle0*2*pi)/(2*pi); % find where remainder (phase distance) is greater than +1/2 cycle, Iplus=find(remainder>0.5); Cycle0(Iplus)=Cycle0(Iplus)+1; % find where remainder (phase distance) is less than -1/2 cycle,

5

Iminus=find(remainder<-0.5); Cycle0(Iminus)=Cycle0(Iminus)-1; % put unwrapping Cycle0 back to radians and add original Phase 1 Phase00=(Cycle0*2*pi)/Nk(2); Phase00=Phase00+(Phase1/Nk(2)); end end; % loop over k 2. GENERATE PHASE, ALBEDO AND INDICATOR IMAGES In this section we will describe the mathematics behind the code presented in section 1. Let the kth frequency, nth shifted pattern image captured by the camera is represented as Ik,n(x,y) where x and y are the column and row pixel locations, respectively. The indices are k = 0, (Nf -1) and n = 0, 1, … (N-1) where Nf =2 and N=32. The two frequencies are kc0=1 and kc1=16. 2.1 Albedo Image Technically the albedo refers to the reflectance of the target surface independent of local gradient. We will use the term more loosely by simply representing the average reflectance across all pattern sequences. To find the albedo image C.bmp, average the pattern images together and scale between 0 and 255;

fN

k

N

nnk

fC yxI

NNyxI

0

1

0, ,

1, (2.1)

The code for this, given in the first section is: % average in albedo AlbedoC=AlbedoC+Ar; where “Ar” is the input image intensity. The result is shown in Fig. 4.

Figure 4: Average image intensity or "Albedo."

6

2.2 In-phase and Quadrature Image Information Along with image intensity we determine what is basically the “inphase” component and the “quadrature” component of the temporal signal at each pixel. The inphase term is

1

0,, /2cos,,

N

nnkkIn NnyxIyxI (2.2)

and the quadrature component image is

1

0,, /2sin,,

N

nnkkQuad NnyxIyxI (2.3)

The matlab code for the inphase and quadrature images is % determin the sin and cos phase shift coefficients sincoef=sin(2*pi*(p-1)/Np(k)); coscoef=cos(2*pi*(p-1)/Np(k)); % average in weighted quadrature and inphase images Iq=Iq+Ar.*sincoef; Ii=Ii+Ar.*coscoef; 2.3 Indicator and Phase images Given the in-phase and quadrature images, we can generate the indicator image and the various phase images. The indicator matrix I, in I.bmp, is based on the modulation amplitude [1] of the patterns such that:

2,2

, ,,, yxIyxIyxI kQuadkInI (2.4)

where the k value selects the highest frequency pattern set. The matlab code is: if Np(k)>1 & k==Nf Iq=Iq/Np(k); Ii=Ii/Np(k); end; AlbedoI=sqrt(Ii.*Ii+Iq.*Iq); % calculate the peak to peak response of An example is shown in Fig. 5.

7

Figure 5: Indicator image.

The phase image for the baseline k=0 and the high frequency k=1 are determined by

yxI

yxIyx

kIn

kQuadk ,

,arctan,

,

, (2)

For implementation, you should use atan2(y,x) instead of atan(y/x) because atan() only yields values for 2 quadrants where as atan2() yields angles spanning all 4 quadrants. The is added to make the phase go from 0 to 2 rather than – to . The baseline phase is shown in Fig. 6 (left) and the high frequency phase is shown in Fig. 6 (right).

Figure 6: (left) Phase k=0 and (right) phase k=1.

If we take a cross section of both phase images in Fig. 6, we plot together, shown in Fig. 7, we can see that the k=0 phase is non-ambiguous where as the k=1 is wrapped. Both vary from 0 to 2.

8

Figure 7:(blue) Phase k=0 cross section and (green) phase k=1 cross section.

In preparation to unwrap k=1, we normalize it by dividing by its pattern spatial frequency of kc = 16 and offsetting first wrap section.

Figure 8: (blue) Phase k=0, (green) phase k=1 divided by its projected spatial frequency of kc1=16 cycles per FOV and (red) the unwrapped phase.

As seen in Fig. 8, the phase gradients of the (k=0) kc0=1 and (k=1) kc1=16 are about the same when normalized. We need to use k=0 phase to estimate how many integer multiples of 2 to align the k=1 phase with the k=0 phase. Once that is done, the k=0 phase is not included in the final phase estimate, thus removing its noise contribution. The first step [2] is to obtain the integer number of phase unwrapping wavelengths (unwrapping cycles) that are at or below the desired value such that 2/,,1kfloor, 10c yxyxyx (3)

In units of cycles, we find the difference between the baseline phase and the unwrapping cycles such that

9

2/,2,,1k, 10c yxyxyxyxr (4)

A final adjustment of +/- 1 unwrapping cycle is done by evaluating the remainder such that

5.0,for1,

5.0,for1,,

yxryx

yxryxyxu

(5)

The final phase image is then the unwrapping cycles plus the higher frequency phase such that 1/,,2, 1 cu kyxyxyx (6)

Thus, the k=1 phase is unwrapped and shown in Fig. 8.

Figure 9: Unwrapped phase image.

The matlab code for the unwrapping is: if k==1 Phase0=atan2(Iq,Ii)+pi; end; AlbedoI=sqrt(Ii.*Ii+Iq.*Iq); % calculate the peak to peak response of patterns % process second set of patterns into phase image if k==2 Phase1=atan2(Iq,Ii)+pi; %Phase unwrapping %Phase cycles Cycle0=floor((Nk(2)*Phase0-Phase1)/(2*pi)); % find cycle remainder between unwrapped phase1 and Phase0 remainder=((Nk(2)*Phase0-Phase1)-Cycle0*2*pi)/(2*pi); % find where remainder (phase distance) is greater than +1/2 cycle, Iplus=find(remainder>0.5); Cycle0(Iplus)=Cycle0(Iplus)+1; % find where remainder (phase distance) is less than -1/2 cycle, Iminus=find(remainder<-0.5); Cycle0(Iminus)=Cycle0(Iminus)-1; % put unwrapping Cycle0 back to radians and add original Phase 1 Phase00=(Cycle0*2*pi)/Nk(2); Phase00=Phase00+(Phase1/Nk(2));

10

end 3. CONVERTING PHASE TO 3-DIMENIONSIONAL COORDINATES To map the C, I and phase matrices to C, I, and X, Y, Z coordinates we start with the calibration data stored in the G file to determine the M matrix of coefficients (we will provide this for you). A linear triangulation paper will be provided with the assignment for explanation. Include M values in your report. The world to camera coordinate transforms for a pin hole lens are

12,11,10,9

4,3,2,1, mzmymxm

mzmymxmx

nwnwnw

nwnwnwnc

(7)

and

12,11,10,9

8,7,6,5, mzmymxm

mzmymxmy

nwnwnw

nwnwnwnc

(8)

where n=1, 2, …. N known points. Because there is an unknown coefficient with every term in the above transformations, there are an infinite number of solutions. However, if we make

112 m then the transformation is linear at the world origin so we fix the common coefficient value of m12 to be unity such that

112 m (9) A coefficient vector, not including the m12 coefficient is defined as 11x1 column vector such that

Tmmmmmmmmmmmm 1110987654321 (10)

Given a linear equation as

mAp (11)

where

zyc

zxc

p

pp (12)

is a 2Nx1 vector constructed from the augmentation of two Nx1 vectors given by

TNcnczxcxmxmp ,12,12 ... (13)

and

TNcnczycymymp ,12,12 ... (14)

The factor matrix A is 2Nx11 and constructed by the augmentation of two matrices as

11

yc

xc

A

AA (15)

where

NwNcNwNcNwNcNwNwNw

wcwcwcwww

xc

zxyxxxzyx

zxyxxxzyx

A

,,,,,,,,,

1,1,1,1,1,1,1,1,1,

00001

...........

00001

(16)

and

NwNcNwNcNwNcNwNwNw

wcwcwcwww

yc

zyyyxyzyx

zyyyxyzyx

A

,,,,,,,,,

1,1,1,1,1,1,1,1,1,

10000

...........

10000

(17)

The Least Squares Solution for the coefficient vector is

pAAAm TT 1 (18)

Use the M matrix of coefficients to reconstruct the 3-D surface (we provide this MATLAB code as well). While the solution to the coefficient values is a linear solution, the pin hole model perspective correction is in no way linear because of the denominators dependency on world coordinates. So to determine the world coordinates from the camera and projector coordinates is not exactly a linear solution. That is, the solution is a matrix vector operation, it is not a linear combination of the input camera and projector coordinates. By algebraically manipulating the world to camera coordinate transforms and world to projector coordinates to group together the world coordinates as

nwncnwncnwncnc zxmmyxmmxxmmmmx ,,113,,102,,91412, (19)

and

nwncnwncnwncnc zymmyymmxymmmmy ,,117,,106,,95812, (20)

and the projector phase is reconfigured as

nwnpPAPAnwnpPAPAnwnpPAPAPAPAnp zymmyymmxymmmmy ,,117,,106,,95812, (21)

Let the left side of Eqs. (19) thru (21) be combined into the matrix

12

812,

812,

412,

PAPAnp

nc

nc

n

mmy

mmy

mmx

D (22)

and combine the coefficients from the right side as

npPAPAnpPAPAnpPAPA

ncncnc

ncncnc

n

ymmymmymm

ymmymmymm

xmmxmmxmm

C

,117,106,95

,117,106,95

,113,102,91

(23)

The world coordinate vector is defined as

TnwnwnwW

n zyxP ,,, (24)

The linear algebraic solution for the world coordinate is then

nnW

n DCP 1 (25)

Save the C,I, X,Y,Z data to mat5 format and then view in mat5 viewer. Use print screen to capture image for this report.

Figure 10: GL3DView program displaying mat5 of Fred.

The code for creating the MAT 5 data is as follows:

13

%Albedo0=Albedo0/index; ImageI=uint8(AlbedoI*2); ImageC=uint8(AlbedoC/index); % plot out data figure(4); imagesc(ImageC); colormap gray; axis image; title('Image C (average intensity)'); print -dtiff 'D_AverageIntensity.tif';%% figure(5); imagesc(ImageI); colormap gray; axis image; title('Image I (p-p intensity)'); print -dtiff 'E_QualityImage.tif';%% figure(6); imagesc(Phase0); colormap gray; axis image; title('kc=1 Phase Image'); print -dtiff 'F_BaselinePhase.tif';%% figure(7); imagesc(Phase1); colormap gray; axis image; title('Wrapped Phase Image'); print -dtiff 'G_WrappedPhase.tif';%% figure(8); imagesc(Phase00); colormap gray; title('Unwrapped Phase Image'); print -dtiff 'H_UnWrappedPhase.tif';%% % P0(1:My)=Phase0(1:My,Nx/2); P1(1:My)=Phase1(1:My,Nx/2); P00(1:My)=Phase00(1:My,Nx/2); m=1:My; figure(9); plot(m,P0,m,P1); title('kc=1 and kc=16 phase cross section'); print -dtiff 'I_PhaseCrossSection.tif'; figure(10); P1A=P0(1)/Nk(1)+P1/Nk(2); plot(m,P0/Nk(1),m,P1A,m,P00); title('Multi-frequency phase vs. kc=1'); legend('baseline phase','f=16 phase','multi-f phase'); print -dtiff 'J_SuperimposedCrossSections.tif'; %% data image file OUTPUTS

14

B_bmp(:,:,1)=ImageC; B_bmp(:,:,2)=ImageC; B_bmp(:,:,3)=ImageC; bmpimageC=B_bmp; B_bmp(:,:,1)=ImageI; B_bmp(:,:,2)=ImageI; B_bmp(:,:,3)=ImageI; bmpimageI=B_bmp; imwrite(bmpimageC,'Albedo.bmp','BMP'); % Albedo8=uint8(Phase0*255/(2*pi)); max(max(Albedo8)) min(min(Albedo8)) B_bmp(:,:,1)=Albedo8; B_bmp(:,:,2)=Albedo8; B_bmp(:,:,3)=Albedo8; imwrite(B_bmp,'Phase0.bmp','BMP'); % Albedo8=uint8(Phase1*255/(2*pi)); max(max(Albedo8)) min(min(Albedo8)) B_bmp(:,:,1)=Albedo8; B_bmp(:,:,2)=Albedo8; B_bmp(:,:,3)=Albedo8; imwrite(B_bmp,'Phase1.bmp','BMP'); % Albedo8=uint8(Phase00*255/(2*pi)); max(max(Albedo8)) min(min(Albedo8)) B_bmp(:,:,1)=Albedo8; B_bmp(:,:,2)=Albedo8; B_bmp(:,:,3)=Albedo8; imwrite(B_bmp,'PhaseYP.bmp','BMP'); result=CPleastsqmethod(); % out=getWorldCoordsB(ImageI,Phase00,32); xw=out(:,:,1); yw=out(:,:,2); zw=out(:,:,3); mat5file='fred'; result = mat5write(mat5file,bmpimageC,bmpimageI,xw,yw,zw); result = mat52fileF(mat5file,bmpimageC,ImageI,xw,yw,zw); REFERENCES

1. P.S. Huang, Q. Hu, F. Jin, F.P. Chiang, “Color-encoded digital fringe projection technique for high-speed three-dimensional surface contouring,” Optical Engineering 38(6), pp 1065-1071, (June 1999)

2. Jielin Li, L.G. Hassebrook and Chun Guan, “Optimized Two-Frequency Phase-Measuring-Profilometry Light-Sensor Temporal-Noise Sensitivity,” JOSA A, 20(1), 106-115, (2003).

15

3. Kai Liu, Yongchang Wang, D.L. Lau, Q. Hao and L.G. Hassebrook, “Gamma model and its analysis for phase measuring profilometry,” JOSA A, In press, (2011).

APPENDIX: MATLAB READ IN MAT5 (mat5read.m) % Veeraganesh Yalla % template script to read the % MAT5 files % Date: Jan 30 2004 function [xw,yw,zw,imageI,imageC] = mat5read(matfile); % open the world coordinate % files % the x,y,z are 1-D arrays % need to be reshaped based on % the dimensions of I and C images fnamex = strcat(matfile,'X.byt');fnamey = strcat(matfile,'Y.byt'); fnamez = strcat(matfile,'Z.byt');fnameC = strcat(matfile,'C.bmp'); fnameI = strcat(matfile,'I.bmp'); % fpx = fopen(fnamex,'rb');x = fread(fpx,'float');fclose(fpx); % fpy = fopen(fnamey,'rb');y = fread(fpy,'float');fclose(fpy); % fpz = fopen(fnamez,'rb');z = fread(fpz,'float');fclose(fpz); %open the I and C images imageC = imread(fnameC);imageI = imread(fnameI); %get the dimensions [my,nx,pz] = size(imageI); %reshape the 1D world coordinate vectors xw = reshape(x,nx,my)'; yw = reshape(y,nx,my)'; zw = reshape(z,nx,my)'; READ IN CALIBRATION FILE *G.byt (read_calib_data.m) % read the calibration data % function needs the input calibration grid data file '.byt' % Npoints number of calibration points function [coods] = read_calib_data(infname,Npoints) % fp = fopen(infname,'rb'); calib_data = fread(fp,'float') fclose(fp); %read the coordinate information [m,n] = size(calib_data); coods = calib_data(2:m); coods = reshape(coods,7,Npoints)'; % ASCII FORMAT OF calgridG.byt Note that the data below is for a grid that has “Npoint” = 28 points. The “npoint” indicates the last point set being edited and carries no information in itself. Each line of data or point set has 7

16

floating point values. Going from left to right, they are Xw, Yw, Zw, Xc, Yc, Xp, Yp where “w” indicates a world coordinate, “c” indicates a camera coordinate and “p” indicates a projector coordinate. Notice that Xp’s are all 0 because this is not used in the reconstruction of the 3-D data for most applications. Npoint=28 npoint=0 0.000000 0.000000 0.000000 937.007996 1490.270020 0.000000 4.174830 0.000000 15.875000 0.000000 935.841003 1260.900024 0.000000 3.619083 0.000000 31.750000 0.000000 934.781982 1027.890015 0.000000 3.059905 0.000000 47.625000 0.000000 933.473999 796.344971 0.000000 2.512760 15.875000 0.000000 12.700000 1178.099976 1512.079956 0.000000 3.981830 15.875000 15.875000 12.700000 1176.050049 1278.420044 0.000000 3.413457 15.875000 31.750000 12.700000 1178.459961 1040.170044 0.000000 2.843287 15.875000 47.625000 12.700000 1176.650024 800.879028 0.000000 2.280516 31.750000 0.000000 0.000000 1397.069946 1484.609985 0.000000 4.176435 31.750000 15.875000 0.000000 1397.150024 1257.400024 0.000000 3.619735 31.750000 31.750000 0.000000 1396.910034 1025.890015 0.000000 3.062361 31.750000 47.625000 0.000000 1396.810059 795.119995 0.000000 2.511117 47.625000 0.000000 12.700000 1647.489990 1504.859985 0.000000 3.982054 47.625000 15.875000 12.700000 1648.849976 1271.560059 0.000000 3.409640 47.625000 31.750000 12.700000 1648.719971 1036.400024 0.000000 2.840146 47.625000 47.625000 12.700000 1649.569946 800.890991 0.000000 2.282020 63.500000 0.000000 0.000000 1848.689941 1479.130005 0.000000 4.177021 63.500000 15.875000 0.000000 1850.829956 1252.540039 0.000000 3.619135 63.500000 31.750000 0.000000 1851.040039 1023.830017 0.000000 3.064629 63.500000 47.625000 0.000000 1851.250000 795.603027 0.000000 2.514975 79.375000 0.000000 12.700000 2106.350098 1500.050049 0.000000 3.987183 79.375000 15.875000 12.700000 2110.250000 1265.739990 0.000000 3.405886 79.375000 31.750000 12.700000 2111.489990 1032.670044 0.000000 2.839072 79.375000 47.625000 12.700000 2113.550049 799.932007 0.000000 2.285129 95.250000 0.000000 0.000000 2292.300049 1472.890015 0.000000 4.178957 95.250000 15.875000 0.000000 2294.800049 1248.540039 0.000000 3.622159 95.250000 31.750000 0.000000 2297.090088 1021.719971 0.000000 3.066538 95.250000 47.625000 0.000000 2300.159912 796.306030 0.000000 2.518003 WRITE IN MAT5 (mat5write.m) function [result] = mat5write(matfile,xw,yw,zw,imageI,imageC); % open the world coordinate files fnamex = strcat(matfile,'X.byt');fnamey = strcat(matfile,'Y.byt'); fnamez = strcat(matfile,'Z.byt');fnameC = strcat(matfile,'C.bmp'); fnameI = strcat(matfile,'I.bmp'); %get the dimensions [my,nx,pz] = size(imageI); %reshape the 1D world coordinate vectors x = reshape(xw',1,my*nx)';y = reshape(yw',1,nx*my)';z = reshape(zw',1,nx*my)'; %xw fpx = fopen(fnamex,'wb');fwrite(fpx,x,'float32');fclose(fpx); %yw fpy = fopen(fnamey,'wb');fwrite(fpy,y,'float32');fclose(fpy); %zw fpz = fopen(fnamez,'wb');fwrite(fpz,z,'float32');fclose(fpz); %C

17

imwrite(imageC,fnameC,'bmp');imwrite(imageI,fnameI,'bmp'); % result = 1; LEAST SQUARES METHOD FOR M MATRIX (result=CPleastsqmethod()) %% Generates the perspective matrix M parameters given the calibration data function out=CPleastsqmethod(); % to read data from G.byt file % infname='CalgridG.byt'; fid=fopen(infname); G=fread(fid,'float'); fclose(fid); %no of elements% %n=G(1); %A=reshape(G(2:141),7,20)'; A=read_calib_data(infname,18); %reading the world coordinates% xw=A(:,1); yw=A(:,2); zw=A(:,3); %reading the camera coordinates% xc=A(:,4) yc=A(:,5) %reading the projector coordinates% xp=A(:,6); yp=A(:,7) %%%calculate transformation matrix using linear triangulation%%% %%%perspective matrix for camera%%%% N=18; for i=1:N Ac(2*i-1,:)=[xw(i) yw(i) zw(i) 1 0 0 0 0 -xc(i)*xw(i) -xc(i)*yw(i) -xc(i)*zw(i)]; Ac(2*i,:)=[0 0 0 0 xw(i) yw(i) zw(i) 1 -yc(i)*xw(i) -yc(i)*yw(i) -yc(i)*zw(i)]; Pc(2*i-1,1)=xc(i); Pc(2*i,1)=yc(i); end mc=(inv(Ac'*Ac))*Ac'*Pc; mc(12)=1; mwc=(reshape(mc,4,3))'; %%%perspective matrix for projector%%%% for i=1:N Ayp(i,:)=[xw(i) yw(i) zw(i) 1 -yp(i)*xw(i) -yp(i)*yw(i) -yp(i)*zw(i)]; Pp(i,1)=yp(i); end mp=(inv(Ayp'*Ayp))*Ayp'*Pp; mp(8)=1; mwp=(reshape(mp,4,2))';

18

%%%%%reconstruct the 3D world co-ordinates%%%%% M=18; for i=1:M c(1,1)=mwc(1,1)-mwc(3,1)*xc(i); c(1,2)=mwc(1,2)-mwc(3,2)*xc(i); c(1,3)=mwc(1,3)-mwc(3,3)*xc(i); c(2,1)=mwc(2,1)-mwc(3,1)*yc(i); c(2,2)=mwc(2,2)-mwc(3,2)*yc(i); c(2,3)=mwc(2,3)-mwc(3,3)*yc(i); c(3,1)=mwp(1,1)-mwp(2,1)*yp(i); c(3,2)=mwp(1,2)-mwp(2,2)*yp(i); c(3,3)=mwp(1,3)-mwp(2,3)*yp(i); d(1,1)=xc(i)-mwc(1,4); d(2,1)=yc(i)-mwc(2,4); d(3,1)=yp(i)-mwp(1,4); W(:,i)=inv(c)*d; end new_xw(:,1)=W(1,:); new_yw(:,1)=W(2,:); new_zw(:,1)=W(3,:); pdisplay=zeros(18,3); olddisplay=pdisplay; pdisplay(:,1)=new_xw(:,1); pdisplay(:,2)=new_yw(:,1); pdisplay(:,3)=new_zw(:,1); olddisplay(:,1)=xw; olddisplay(:,2)=yw; olddisplay(:,3)=zw; pdisplay olddisplay errval(:,1)= xw-new_xw(:,1); errval(:,2)= yw-new_yw(:,1); errval(:,3)= zw-new_zw(:,1) % erval = sqrt(errval(:,1).^2+errval(:,2).^2+errval(:,3).^2) meanerval=mean(erval) %write the projector parameters fp = fopen('proj.txt','w'); fprintf(fp,'%f ',mwp(1,1));fprintf(fp,'%f ',mwp(1,2));fprintf(fp,'%f ',mwp(1,3));fprintf(fp,'%f\n',mwp(1,4)); fprintf(fp,'%f ',mwp(2,1));fprintf(fp,'%f ',mwp(2,2));fprintf(fp,'%f ',mwp(2,3));fprintf(fp,'%f\n',mwp(2,4)); %fprintf(fp,'%f ',mwp(3,1));fprintf(fp,'%f ',mwp(3,2));fprintf(fp,'%f ',mwp(3,3));fprintf(fp,'%f\n',mwp(3,4)); fclose(fp); %write the camera parameters

19

fp = fopen('cam.txt','w'); fprintf(fp,'%f ',mwc(1,1));fprintf(fp,'%f ',mwc(1,2));fprintf(fp,'%f ',mwc(1,3));fprintf(fp,'%f\n',mwc(1,4)); fprintf(fp,'%f ',mwc(2,1));fprintf(fp,'%f ',mwc(2,2));fprintf(fp,'%f ',mwc(2,3));fprintf(fp,'%f\n',mwc(2,4)); fprintf(fp,'%f ',mwc(3,1));fprintf(fp,'%f ',mwc(3,2));fprintf(fp,'%f ',mwc(3,3));fprintf(fp,'%f\n',mwc(3,4)); fclose(fp); mwc mwp out=1; % 3D FROM M AND PHASE (out=getWorldCoordsB(ImageI,Phase00,32)) function out=getWorldCoordsB(I,yp,ithresh) % reads in cam.txt and proj.txt and processes yp, out is XYZ of a mat5 cam=textread('cam.txt','%f'); cam=reshape(cam,[4,3]); cam=cam'; proj=textread('proj.txt','%f'); proj=reshape(proj,[4,2]); proj=proj'; camMatrix=cam; projMatrix=proj; [M N]=size(yp); d=double(camMatrix); p=double(projMatrix); yp=double(yp); X=zeros(M,N); Y=X; Z=X; mwc=d; mwp=p; for n=1:N for m=1:M if I(m,n)>=ithresh xc=double(n); yc=double(m); c(1,1)=mwc(1,1)-mwc(3,1)*xc; c(1,2)=mwc(1,2)-mwc(3,2)*xc; c(1,3)=mwc(1,3)-mwc(3,3)*xc; c(2,1)=mwc(2,1)-mwc(3,1)*yc; c(2,2)=mwc(2,2)-mwc(3,2)*yc; c(2,3)=mwc(2,3)-mwc(3,3)*yc;

20

c(3,1)=mwp(1,1)-mwp(2,1)*yp(m,n); c(3,2)=mwp(1,2)-mwp(2,2)*yp(m,n); c(3,3)=mwp(1,3)-mwp(2,3)*yp(m,n); d(1,1)=xc-mwc(1,4); d(2,1)=yc-mwc(2,4); d(3,1)=yp(m,n)-mwp(1,4); P=inv(c)*d; X(m,n)=P(1); Y(m,n)=P(2); Z(m,n)=P(3); else X(m,n)=0; Y(m,n)=0; Z(m,n)=0; end % if floor(Y(m,n))>22 && floor(Y(m,n))<28 && floor(X(m,n))>52 && floor(X(m,n))<60 % testingthing=[X(m,n), Y(m,n), Z(m,n)] % end % if n>1000 && n<1100 && m>200 && m<300 && X(m,n)~=0 % testingthing=[X(m,n), Y(m,n), Z(m,n), yp(m,n)] % end end end out=zeros(M,N,3); out(:,:,1)=X(:,:); out(:,:,2)=Y(:,:); out(:,:,3)=Z(:,:);