CSE 559A: Computer Visionayan/courses/cse559a/PDFs/lec4.pdf · CSE 559A: Computer Vision Fall 2019:...

CSE 559A: Computer Vision

Fall 2020: T-R: 11:30-12:50pm @ Wrighton 300 / Zoom

Instructor: Ayan Chakrabarti ([email protected]).Course Staff: Adith Boloor, Patrick Williams

Sep 24, 2020

http://www.cse.wustl.edu/~ayan/courses/cse559a/

1

ADMINISTRIVIAADMINISTRIVIAREMINDER: Office Hours start this Friday. Zoom link available through Canvas.

Received your problem set 0 solutions. Will be giving individual feedback Soon.Meanwhile, a few points:

Make sure your write-up is named solution.pdf, and not something else.Make sure you git add the solution.pdf file before commit & push. We need the PDF, not just the .tex.Don’t rename the .py files. Your submitted code should be added in to the original prob?.py files.Be a little careful about googling for answers to specific questions. Problem set 0 was pretty basic, so bothwere the same. But in problem set 1, don’t google: how to implement histogram equalization!

Stack overflow type web-sites can especially get problematic sometimes.Do not use OpenCV! You have to implement everything using numpy / scipy. The point is to know howcomputer vision algorithms work and are implemented.

2

LAST TIMELAST TIMETalked about point-wise operations.Began talking about efficient implementations in numpy (with parallel elementwise operations, reshapes,and broadcasting).

3

POINTWISE OPERATIONS IN NUMPYPOINTWISE OPERATIONS IN NUMPYImages themselves are arrays not matrices. (In numpy, * does element-wise multiply, np.matmul doesmatrix multiplication).But for such linear operations, we can form matrices by stacking all pixel locations, in some pre-determinedorder, as rows. Represent as:

matrix: color images

vector: grayscale images.

X

(HW) × 3

(HW) × 1

Y[n] = C X[n] ⇒ Y = ?

4

POINTWISE OPERATIONS IN NUMPYPOINTWISE OPERATIONS IN NUMPYImages themselves are arrays not matrices. (In numpy, * does element-wise multiply, np.matmul doesmatrix multiplication).But for such linear operations, we can form matrices by stacking all pixel locations, in some pre-determinedorder, as rows. Represent as:

matrix: color images

vector: grayscale images.

X

(HW) × 3

(HW) × 1

Y[n] = C X[n] ⇒ Y = X CT

# Begin with X as (H,W,3) array

Xflt = np.reshape(X,(-1,3)) # Flatten X to a (H*W, 3) matrix

Yflt = np.matmul(Xflt,C.T) # Post-multiply by C

Y = np.reshape(Yflt,X.shape) # Turn Y back to an image array

5

POINTWISE OPERATIONS IN NUMPYPOINTWISE OPERATIONS IN NUMPYOne last trick: what if your pointwise function is a lookup table (for grayscale images).Assume that you are trying to map all intensities in an array X, which is of type integer (uint8, etc.), and youhave a lookup table as a vector H, where you want to map an intensity f to H[f].So what you want to do is: Y[i,j] = H[X[i,j]]You can do this simply as Y = H[X]. Y will be the same shape as X (and not H).

6

CONVOLUTIONCONVOLUTION

7


8


9


10


11


12


13


14


15


Double summation over the support / size of the kernel

We assume is scalar vaued.

If is scalar, so is .

If is a color image, each channel convolved with independently.

To go from m to n channels in a “conv layer”: is matrix valued, and is a matrix-vector product.

Notation: Y = X ∗ k

Y[n] = k[ ] X[n − ]∑n′

n′

n′

Y[ , ] = k[ , ] X[( − ), ( − )]nx ny ∑n′x

∑n′y

n′x n′y nx n′x ny n′y

k

k[n] ∈ ℝ

X[n] Y[n]

X k

k[n] ∈ ℝn×mk[ ] X[n − ]n

′n

′

16


17


18


19


20


21


22


23


24


25


26


27


28


29


30


31

CONVOLUTION: PROPERTIESCONVOLUTION: PROPERTIESLet , , and denote full, valid, and same convolution (with zero padding for full and same)

Linear / Distributive: For scalars ;

If , then:

If and , ( , same size):


X k∗full

X k∗val

X k∗same

α, β

Y = X ∗ k X ∗ (αk) = (αX) ∗ k = αY

= X ∗Y1 k1 = X ∗Y2 k2 k1 k2 X ∗ (α + β ) = α + βk1 k2 Y1 Y2

= ∗ kY1 X1 = ∗ kY2 X2 X1 X2 (α + β ) ∗ k = α + βX1 X2 Y1 Y2

X ∗ (α + β ) [n] = (α [ ] + β [ ])X[n − ]k1 k2 ∑n′

k1 n′

k2 n′

n′

= α [ ]X[n − ] + β [ ]X[n − ]∑n′

k1 n′

n′

∑n′

k2 n′

n′

32



If , then:



Associative

X k∗full

X k∗val

X k∗same

α, β

Y = X ∗ k X ∗ (αk) = (αX) ∗ k = αY



(X ) = X ( )∗fullk1 ∗

fullk2 ∗

fullk1 ∗full

k2

(X ) = X ( )∗valk1 ∗

valk2 ∗

valk1 ∗full

k2

(X ) ≠ X ( )∗samek1 ∗

samek2 ∗

samek1 ∗full

k2

(X ) [n] = [ ](X ∗ )[n − ] = [ ] [ ]X[n − − ]∗fullk1 ∗

fullk2 ∑

n′

k2 n′

k1 n′

∑n′

k2 n′

∑n″

k1 n″

n′

n″

= ( [ ] [( + ) − ])X[n − ( + )]∑+n′ n″∑n′

k2 n′k1 n

′n″

n′

n′

n″

33



If , then:



Associative

Commutative:

X k∗full

X k∗val

X k∗same

α, β

Y = X ∗ k X ∗ (αk) = (αX) ∗ k = αY



(X ) = X ( )∗fullk1 ∗

fullk2 ∗

fullk1 ∗full

k2

(X ) = X ( )∗valk1 ∗

valk2 ∗

valk1 ∗full

k2

(X ) ≠ X ( )∗samek1 ∗

samek2 ∗

samek1 ∗full

k2

=k1 ∗fullk2 k2 ∗full

k1

(X ) = (X )∗fullk1 ∗

fullk2 ∗

fullk2 ∗

fullk1

(X ) = (X )∗valk1 ∗

valk2 ∗

valk2 ∗

valk1

(X ) ≠ (X )∗samek1 ∗

samek2 ∗

samek2 ∗

samek1


35


36


37


38


39


40


41


42


43


44


45


46


47


48


49


50


51


52


53


54


55


56


57


58


59


60

APPLICATION: EDGE DETECTIONAPPLICATION: EDGE DETECTION

61


62


63


64


65


66


67


68


69


70


71


72


73


74


75


76


77


78


79


80


81

CSE 559A: Computer Visionayan/courses/cse559a/PDFs/lec4.pdf · CSE 559A: Computer Vision Fall 2019:...

Documents

Transcript of CSE 559A: Computer Visionayan/courses/cse559a/PDFs/lec4.pdf · CSE 559A: Computer Vision Fall 2019:...