Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a...
-
Upload
giles-boyd -
Category
Documents
-
view
213 -
download
0
Transcript of Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a...
![Page 1: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/1.jpg)
Prelude• A pattern of activation in a NN is a vector• A set of connection weights between units is
a matrix• Vectors and matrices have well-understood
mathematical and geometric properties• Very useful for understanding the properties
of NNs
![Page 2: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/2.jpg)
Operations on Vectors and Matrices
![Page 3: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/3.jpg)
Outline1) The Players: Scalars, Vectors and Matrices
2) Vectors, matrices and neural nets
3) Geometric Analysis of Vectors
4) Multiplying Vectors by Scalars
5) Multiplying Vectors by Vectors
a) The inner product (produces a scalar)
b) The outer product (produces a matrix)
6) Multiplying Vectors by Matrices
7) Multiplying Matrices by Matrices
![Page 4: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/4.jpg)
Scalars, Vectors and Matrices1) Scalar: A single number (integer or real)
2) Vector: An ordered list of scalars
[ 1 2 3 4 5 ] [ 0.4 1.2 0.07 8.4 12.3 ] [ 12 10 ] [ 2 ]
![Page 5: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/5.jpg)
Scalars, Vectors and Matrices1) Scalar: A single number (integer or real)
2) Vector: An ordered list of scalars
[ 1 2 3 4 5 ] [ 0.4 1.2 0.07 8.4 12.3 ] [ 12 10 ] [ 2 ]
[ 12 10 ] ≠ [ 10 12 ]
![Page 6: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/6.jpg)
Scalars, Vectors and Matrices1) Scalar: A single number (integer or real)
2) Vector: An ordered list of scalars
[ 1 2 3 4 5 ] [ 0.4 1.2 0.07 8.4 12.3 ] [ 12 10 ] [ 2 ]
Row vectors
![Page 7: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/7.jpg)
Scalars, Vectors and Matrices1) Scalar: A single number (integer or real)
2) Vector: An ordered list of scalars
[ 1 2 3 4 5 ] [ 0.4 1.2 0.07 8.4 12.3 ] [ 12 10 ] [ 2 ]
Row vectors
Column Vectors12345
1.5
0.3
6.2
12.0
17.1
![Page 8: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/8.jpg)
Scalars, Vectors and Matrices1) Scalar: A single number (integer or real)
2) Vector: An ordered list of scalars
3) Matrix: An ordered list of vectors:
1 2 6 1 7 8
2 5 9 0 0 3
3 1 5 7 6 3
2 7 9 3 3 1
![Page 9: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/9.jpg)
Scalars, Vectors and Matrices1) Scalar: A single number (integer or real)
2) Vector: An ordered list of scalars
3) Matrix: An ordered list of vectors:
1 2 6 1 7 8
2 5 9 0 0 3
3 1 5 7 6 3
2 7 9 3 3 1
Row vectors
![Page 10: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/10.jpg)
Scalars, Vectors and Matrices1) Scalar: A single number (integer or real)
2) Vector: An ordered list of scalars
3) Matrix: An ordered list of vectors:
1 2 6 1 7 8
2 5 9 0 0 3
3 1 5 7 6 3
2 7 9 3 3 1
Column vectors
![Page 11: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/11.jpg)
Scalars, Vectors and Matrices1) Scalar: A single number (integer or real)
2) Vector: An ordered list of scalars
3) Matrix: An ordered list of vectors:
1 2 6 1 7 8
2 5 9 0 0 3
3 1 5 7 6 3
2 7 9 3 3 1
Matrices are indexed (row, column)
M =
![Page 12: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/12.jpg)
Scalars, Vectors and Matrices1) Scalar: A single number (integer or real)
2) Vector: An ordered list of scalars
3) Matrix: An ordered list of vectors:
1 2 6 1 7 8
2 5 9 0 0 3
3 1 5 7 6 3
2 7 9 3 3 1
Matrices are indexed (row, column)
M =M(1,3) = 6 (row 1, column 3)
![Page 13: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/13.jpg)
Scalars, Vectors and Matrices1) Scalar: A single number (integer or real)
2) Vector: An ordered list of scalars
3) Matrix: An ordered list of vectors:
1 2 6 1 7 8
2 5 9 0 0 3
3 1 5 7 6 3
2 7 9 3 3 1
Matrices are indexed (row, column)
M =M(1,3) = 6 (row 1, column 3)
M(3,1) = 3 (row 3, column 1)
![Page 14: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/14.jpg)
Variable Naming Conventions1) Scalars: Lowercase, italics
x, y, z…
2) Vectors: Lowercase, bold
u, v, w…
• Matrices: Uppercase, bold
M, N, O …
• Constants: Greek
, , , , …
![Page 15: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/15.jpg)
Transposing VectorsIf u is a row vector…
u = [ 1 2 3 4 5 ]
…then u’ (“u-transpose”) is a column vector
12345
… and vice-versa.
u’ =
![Page 16: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/16.jpg)
Transposing VectorsIf u is a row vector…
u = [ 1 2 3 4 5 ]
…then u’ (“u-transpose”) is a column vector
12345
… and vice-versa.
u’ =Why in the world
would I care??
You
![Page 17: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/17.jpg)
Transposing VectorsIf u is a row vector…
u = [ 1 2 3 4 5 ]
…then u’ (“u-transpose”) is a column vector
12345
… and vice-versa.
u’ = Answer: It’ll matter when we come to vector multiplication.
![Page 18: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/18.jpg)
Transposing VectorsIf u is a row vector…
u = [ 1 2 3 4 5 ]
…then u’ (“u-transpose”) is a column vector
12345
… and vice-versa.
u’ = OK.
![Page 19: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/19.jpg)
Vectors, Matrices & Neural Nets
![Page 20: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/20.jpg)
Vectors, Matrices & Neural Nets
j1 j2 j3 Input units, j
![Page 21: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/21.jpg)
Vectors, Matrices & Neural Nets
i1 i2
j1 j2 j3 Input units, j
Output units, i
![Page 22: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/22.jpg)
Vectors, Matrices & Neural Nets
i1 i2
j1 j2 j3 Input units, j
Output units, i
Connection weights, wij
w11
w12 w13w21w22
w23
![Page 23: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/23.jpg)
Vectors, Matrices & Neural Nets
i1 i2
0.2 0.9 0.5 Input units, j
Output units, i
Connection weights, wij
w11
w12 w13w21w22
w23
The activations of the input nodes can be represented as a 3-dimensional vector:
j = [ 0.2 0.9 0.5 ]
![Page 24: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/24.jpg)
Vectors, Matrices & Neural Nets
1.0 0.0
j1 j2 j3 Input units, j
Output units, i
Connection weights, wij
w11
w12 w13w21w22
w23
The activations of the output nodes can be represented as a 2-dimensional vector:
i = [ 1.0 0.0 ]
![Page 25: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/25.jpg)
Vectors, Matrices & Neural Nets
i1 i2
j1 j2 j3 Input units, j
Output units, i
Connection weights, wij
w11
w12 w13w21w22
w23
The weights leading into any output node can be represented as a 3-dimensional vector:
w1j = [ 0.1 1.0 0.2 ]
0.1
1.0 0.2
![Page 26: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/26.jpg)
Vectors, Matrices & Neural Nets
i1 i2
j1 j2 j3 Input units, j
Output units, i
Connection weights, wij
w11
w12 w13w21w22
w23
The complete set of weights can be represented as a 3 (row) X 2 (column) matrix:
0.1
1.0 0.21.0 0.1
-0.9
W =0.1 1.0 0.2 1.0 0.1 -0.9
![Page 27: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/27.jpg)
Vectors, Matrices & Neural Nets
i1 i2
j1 j2 j3 Input units, j
Output units, i
Connection weights, wij
w11
w12 w13w21w22
w23
The complete set of weights can be represented as a 2 (row) X 3 (column) matrix:
0.1
1.0 0.21.0 0.1
-0.9
W =
Why in the world would I care??
0.1 1.0 0.2 1.0 0.1 -0.9
![Page 28: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/28.jpg)
Vectors, Matrices & Neural Nets
W
Why in the world would I care??
1. Because the mathematics of vectors and matrices is well-understood.
2. Because vectors have a very useful geometric interpretation.
3. Because Matlab “thinks” in vectors and matrices.
4. Because you are going to have to learn to think in Matlab.
![Page 29: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/29.jpg)
Vectors, Matrices & Neural Nets
OK.
1. Because the mathematics of vectors and matrices is well-understood.
2. Because vectors have a very useful geometric interpretation.
3. Because Matlab “thinks” in vectors and matrices.
4. Because you are going to have to learn to think in Matlab.
![Page 30: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/30.jpg)
Geometric Analysis of VectorsDimensionality: The number of numbers in a vector
![Page 31: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/31.jpg)
Geometric Analysis of Vectors
Vector as a point Vector as arrow
x1
x2
x3
.3
.2x1
x2
x3
.5
.3
.5
.2
Dimensionality: The number of numbers in a vector
![Page 32: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/32.jpg)
Geometric Analysis of Vectors
Vector as a point Vector as arrow
x1
x2
x3
.3
.2x1
x2
x3
.5
.3
.5
.2
Dimensionality: The number of numbers in a vector
![Page 33: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/33.jpg)
Geometric Analysis of Vectors
Implications for neural networks
• Auto-associative nets• State of activation at time t is a vector (a point in a space)
• As activations change, vector moves through that space
• Will prove invaluable in understanding Hopfield nets
• Layered nets (“perceptrons”)• Input vectors activate output vectors
• Points in input space map to points in output space
• Will prove invaluable in understanding perceptrons and back-propagation learning
![Page 34: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/34.jpg)
Multiplying a Vector by a Scalar
[ 5 4 ] * 2 =
5
4
![Page 35: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/35.jpg)
Multiplying a Vector by a Scalar
[ 5 4 ] * 2 = [ 10 8 ]
Lengthens the vector but does not change its orientation
5
4
10
8
![Page 36: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/36.jpg)
Adding a Vector to a Scalar
[ 5 4 ] + 2 =
5
4
![Page 37: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/37.jpg)
Adding a Vector to a Scalar
[ 5 4 ] + 2 = NAN
Is Illegal.
5
4
![Page 38: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/38.jpg)
Adding a Vector to a Vector
[ 5 4 ] + [ 3 6 ] =
5
4
3
6
![Page 39: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/39.jpg)
Adding a Vector to a Vector
[ 5 4 ] + [ 3 6 ] = [ 8 10 ]
Forms a parallelogram.
5
4
3
6
8
10
![Page 40: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/40.jpg)
Multiplying a Vector by a Vector 1:The Inner Product (aka “Dot Product”)
If u and v are both row vectors of the same dimensionality…
u = [ 1 2 3 ]
v = [ 4 5 6 ]
![Page 41: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/41.jpg)
Multiplying a Vector by a Vector 1:The Inner Product (aka “Dot Product”)
If u and v are both row vectors of the same dimensionality…
u = [ 1 2 3 ]
v = [ 4 5 6 ]
… then the product
u · v =
![Page 42: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/42.jpg)
Multiplying a Vector by a Vector 1:The Inner Product (aka “Dot Product”)
If u and v are both row vectors of the same dimensionality…
u = [ 1 2 3 ]
v = [ 4 5 6 ]
… then the product
u · v = NAN
Is undefined.
![Page 43: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/43.jpg)
Multiplying a Vector by a Vector 1:The Inner Product (aka “Dot Product”)
If u and v are both row vectors of the same dimensionality…
u = [ 1 2 3 ]
v = [ 4 5 6 ]
… then the product
u · v = NAN
Is undefined.
Huh??Why??
That’s BS!
![Page 44: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/44.jpg)
Multiplying a Vector by a Vector 1:The Inner Product (aka “Dot Product”)
I told you you’d eventually care about transposing vectors…
?
![Page 45: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/45.jpg)
Multiplying a Vector by a Vector 1:The Inner Product (aka “Dot Product”)
• The Mantra: “Rows by Columns”• Multiply rows (or row vectors) by columns (or
column vectors)
![Page 46: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/46.jpg)
Multiplying a Vector by a Vector 1:The Inner Product (aka “Dot Product”)
• The Mantra: “Rows by Columns”• Multiply rows (or row vectors) by columns (or
column vectors)
u = [ 1 2 3 ]
v = [ 4 5 6 ]
![Page 47: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/47.jpg)
Multiplying a Vector by a Vector 1:The Inner Product (aka “Dot Product”)
• The Mantra: “Rows by Columns”• Multiply rows (or row vectors) by columns (or
column vectors)
u = [ 1 2 3 ]
v = [ 4 5 6 ] v’ =
4
5
6
![Page 48: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/48.jpg)
Multiplying a Vector by a Vector 1:The Inner Product (aka “Dot Product”)
• The Mantra: “Rows by Columns”• Multiply rows (or row vectors) by columns (or
column vectors)
u = [ 1 2 3 ]
v’ =
4
5
6
u · v’ = 32
![Page 49: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/49.jpg)
Multiplying a Vector by a Vector 1:The Inner Product (aka “Dot Product”)
• The Mantra: “Rows by Columns”• Multiply rows (or row vectors) by columns (or
column vectors)
u = [ 1 2 3 ]
v’
4
5
6
u · v’ = 32
Imagine rotating your row vector into a (pseudo) column vector…
1
2
3
![Page 50: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/50.jpg)
Multiplying a Vector by a Vector 1:The Inner Product (aka “Dot Product”)
• The Mantra: “Rows by Columns”• Multiply rows (or row vectors) by columns (or
column vectors)
u = [ 1 2 3 ]
v’
4
5
6
u · v’ = 32
Now multiply corresponding elements and add up the products…
1
2
3
4
![Page 51: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/51.jpg)
Multiplying a Vector by a Vector 1:The Inner Product (aka “Dot Product”)
• The Mantra: “Rows by Columns”• Multiply rows (or row vectors) by columns (or
column vectors)
u = [ 1 2 3 ]
v’
4
5
6
u · v’ = 32
Now multiply corresponding elements and add up the products…
1
2
3
4
10
![Page 52: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/52.jpg)
Multiplying a Vector by a Vector 1:The Inner Product (aka “Dot Product”)
• The Mantra: “Rows by Columns”• Multiply rows (or row vectors) by columns (or
column vectors)
u = [ 1 2 3 ]
v’
4
5
6
u · v’ = 32
Now multiply corresponding elements and add up the products…
1
2
3
4
10
18
![Page 53: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/53.jpg)
Multiplying a Vector by a Vector 1:The Inner Product (aka “Dot Product”)
• The Mantra: “Rows by Columns”• Multiply rows (or row vectors) by columns (or
column vectors)
u = [ 1 2 3 ]
v’
4
5
6
u · v’ = 32
Now multiply corresponding elements and add up the products…
1
2
3
4
10
1832
![Page 54: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/54.jpg)
4
5
6
4
10
1832
Multiplying a Vector by a Vector 1:The Inner Product (aka “Dot Product”)
• Inner product is commutative as long as you transpose correctly
u = [ 1 2 3 ]
v’
u · v’ = 32
v · u’ = 32
u’
1
2
3
v = [ 4 5 6 ]
v
4
5
6
4
10
1832
![Page 55: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/55.jpg)
The Inner (“Dot”) Product
• In scalar notation…v’
4
5
6
1
2
3
4
10
1832
u
i
d
iivuv'u
1
![Page 56: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/56.jpg)
The Inner (“Dot”) Product
• In scalar notation…
• Remind you of…
… the net input to a unit
i
d
iivuv'u
1
j
d
jiji awn
1
![Page 57: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/57.jpg)
The Inner (“Dot”) Product
• In scalar notation…
• Remind you of…
j
d
jijij awn
1
… the net input to a unit
• In vector notation…
awn
i
d
iivuv'u
1
![Page 58: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/58.jpg)
What Does the Dot Product “Mean”?
![Page 59: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/59.jpg)
What Does the Dot Product “Mean”?
• Consider u u’
![Page 60: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/60.jpg)
What Does the Dot Product “Mean”?
• Consider u u’
u = [ 3, 4 ]
3
4
![Page 61: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/61.jpg)
What Does the Dot Product “Mean”?
• Consider u u’
u = [ 3, 4 ]
u’
3
4
3
4
9
16
25
u
3
4
![Page 62: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/62.jpg)
What Does the Dot Product “Mean”?
• Consider u u’
u = [ 3, 4 ]
u’
3
4
3
4
9
16
25
u
3
4
u uu'
u ui2
5
![Page 63: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/63.jpg)
What Does the Dot Product “Mean”?
• Consider u u’
u = [ 3, 4 ]
u’
3
4
3
4
9
16
25
u
3
4
u uu'
u ui2
5True for vectors of any dimensionality
![Page 64: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/64.jpg)
What Does the Dot Product “Mean”?
• So:
u uu'
![Page 65: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/65.jpg)
What Does the Dot Product “Mean”?
• What about u v where u v?
![Page 66: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/66.jpg)
What Does the Dot Product “Mean”?
cos(uv ) uvu v
Well…
• What about u v where u v?
![Page 67: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/67.jpg)
What Does the Dot Product “Mean”?
cos(uv ) uvu v
Well…
• What about u v where u v?
… and cos(uv) is a length-invariant measure of the similarity of u and v
![Page 68: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/68.jpg)
What Does the Dot Product “Mean”?
cos(uv ) uvu v
• What about u v where u v?
cos(uv) is a length-invariant measure of the similarity of u and v
U = [ 1, 0 ]
V’ = [ 1, 1 ]
![Page 69: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/69.jpg)
What Does the Dot Product “Mean”?
cos(uv ) uvu v
• What about u v where u v?
cos(uv) is a length-invariant measure of the similarity of u and v
U = [ 1, 0 ]
V’ = [ 1, 1 ]
uv = 45º; cos(uv) = .707
U V = (1 * 1) + (1 * 0) = 1
![Page 70: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/70.jpg)
What Does the Dot Product “Mean”?
cos(uv ) uvu v
• What about u v where u v?
cos(uv) is a length-invariant measure of the similarity of u and v
U = [ 1, 0 ]
V’ = [ 1, 1 ] U V = (1 * 1) + (1 * 0) = 1
cos(uv ) 1
u v
uv = 45º; cos(uv) = .707
![Page 71: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/71.jpg)
What Does the Dot Product “Mean”?
cos(uv ) uvu v
• What about u v where u v?
cos(uv) is a length-invariant measure of the similarity of u and v
U = [ 1, 0 ]
V’ = [ 1, 1 ]||u|| = sqrt(1) = 1
||v|| = sqrt(2) = 1.414
cos(uv ) 1
u v
uv = 45º; cos(uv) = .707
![Page 72: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/72.jpg)
What Does the Dot Product “Mean”?
cos(uv ) uvu v
• What about u v where u v?
cos(uv) is a length-invariant measure of the similarity of u and v
U = [ 1, 0 ]
V’ = [ 1, 1 ]||u|| = sqrt(1) = 1
||v|| = sqrt(2) = 1.414
cos(uv ) 1
1*1.414.707
uv = 45º; cos(uv) = .707
![Page 73: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/73.jpg)
What Does the Dot Product “Mean”?
cos(uv ) uvu v
• What about u v where u v?
cos(uv) is a length-invariant measure of the similarity of u and v
U = [ 1, 0 ]
V’ = [ 0, 1 ]
uv = 90º; cos(uv) = 0
![Page 74: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/74.jpg)
What Does the Dot Product “Mean”?
cos(uv ) uvu v
• What about u v where u v?
cos(uv) is a length-invariant measure of the similarity of u and v
U = [ 1, 0 ]
V’ = [ 0, -1 ]
uv = 270º; cos(uv) = 0
![Page 75: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/75.jpg)
What Does the Dot Product “Mean”?
cos(uv ) uvu v
• What about u v where u v?
cos(uv) is a length-invariant measure of the similarity of u and v
U = [ 1, 0 ]V’ = [ -1,0 ]
uv = 180º; cos(uv) = -1
![Page 76: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/76.jpg)
What Does the Dot Product “Mean”?
cos(uv ) uvu v
• What about u v where u v?
cos(uv) is a length-invariant measure of the similarity of u and v
U = [ 1, 0 ]
V’ = [ 2.2,0 ]
uv = 0º; cos(uv) = 1
![Page 77: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/77.jpg)
What Does the Dot Product “Mean”?
cos(uv ) uvu v
• What about u v where u v?
cos(uv) is a length-invariant measure of the similarity of u and v
In general…
cos(uv) -1…1
True regardless of dimensionality
![Page 78: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/78.jpg)
What Does the Dot Product “Mean”?
cos(uv ) uvu v
• What about u v where u v?
cos(uv) is a length-invariant measure of the similarity of u and v
To see why, consider the cosine expressed in scalar notation…
cos(uv ) uiv i
ui2 v i
2
![Page 79: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/79.jpg)
What Does the Dot Product “Mean”?
cos(uv ) uvu v
• What about u v where u v?
cos(uv) is a length-invariant measure of the similarity of u and v
… and compare it to the equation for the correlation coefficient…
cos(uv ) uiv i
ui2 v i
2
r(u,v) (ui u )(v i v )
(ui u )2 (v i v )2
![Page 80: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/80.jpg)
What Does the Dot Product “Mean”?
cos(uv ) uvu v
• What about u v where u v?
cos(uv) is a length-invariant measure of the similarity of u and v
… and compare it to the equation for the correlation coefficient…
cos(uv ) uiv i
ui2 v i
2
r(u,v) (ui u )(v i v )
(ui u )2 (v i v )2
if u and v have means of zero, then cos(uv) = r(u,v)
![Page 81: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/81.jpg)
What Does the Dot Product “Mean”?
cos(uv ) uvu v
• What about u v where u v?
cos(uv) is a length-invariant measure of the similarity of u and v
… and compare it to the equation for the correlation coefficient…
cos(uv ) uiv i
ui2 v i
2
r(u,v) (ui u )(v i v )
(ui u )2 (v i v )2
if u and v have means of zero, then cos(uv) = r(u,v)
The cosine is a special case of the correlation coefficient!
![Page 82: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/82.jpg)
What Does the Dot Product “Mean”?
cos(uv ) uvu v
• What about u v where u v?
cos(uv) is a length-invariant measure of the similarity of u and v
… and let’s compare the cosine to the dot product…
![Page 83: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/83.jpg)
What Does the Dot Product “Mean”?
cos(uv ) uvu v
• What about u v where u v?
cos(uv) is a length-invariant measure of the similarity of u and v
… and let’s compare the cosine to the dot product…
If u and v have lengths of 1, then the dot product is equal to the cosine.
![Page 84: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/84.jpg)
What Does the Dot Product “Mean”?
cos(uv ) uvu v
• What about u v where u v?
cos(uv) is a length-invariant measure of the similarity of u and v
… and let’s compare the cosine to the dot product…
If u and v have lengths of 1, then the dot product is equal to the cosine.
The dot product is a special case of the cosine, which is a special case of the correlation coefficient, which is a measure of vector similarity!
![Page 85: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/85.jpg)
What Does the Dot Product “Mean”?
aw ijj
iji awn • The most common input rule is a dot product between unit i’s
vector of weights and the activation vector on the other end
• Such a unit is computing the (biased) similarity between what it expects (wi) and what it’s getting (a).
• It’s activation is a positive function of this similarity
![Page 86: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/86.jpg)
What Does the Dot Product “Mean”?
• The most common input rule is a dot product between unit i’s vector of weights and the activation vector on the other end
• Such a unit is computing the (biased) similarity between what it expects (wi) and what it’s getting (a).
• It’s activation is a positive function of this similarity
ai
ni
asymptotic
aw ijj
iji awn
![Page 87: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/87.jpg)
What Does the Dot Product “Mean”?
• The most common input rule is a dot product between unit i’s vector of weights and the activation vector on the other end
• Such a unit is computing the (biased) similarity between what it expects (wi) and what it’s getting (a).
• It’s activation is a positive function of this similarity
ai
ni
Step (BTU)
aw ijj
iji awn
![Page 88: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/88.jpg)
What Does the Dot Product “Mean”?
• The most common input rule is a dot product between unit i’s vector of weights and the activation vector on the other end
• Such a unit is computing the (biased) similarity between what it expects (wi) and what it’s getting (a).
• It’s activation is a positive function of this similarity
ai
ni
logistic
aw ijj
iji awn
![Page 89: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/89.jpg)
Multiplying a Vector by a Vector 2:The Outer Product
The two vectors need not have the same dimensionality.
Same Mantra: Rows by Columns.
This time, multiply a column vector by a row vector:
![Page 90: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/90.jpg)
Multiplying a Vector by a Vector 2:The Outer Product
The two vectors need not have the same dimensionality.
Same Mantra: Rows by Columns.
This time, multiply a column vector by a row vector:
u’ =1
2v = [ 4 5 6 ]
M = u’ * v
![Page 91: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/91.jpg)
Multiplying a Vector by a Vector 2:The Outer Product
The two vectors need not have the same dimensionality.
Same Mantra: Rows by Columns.
This time, multiply a column vector by a row vector:
1
2v = [ 4 5 6 ]
M = u’ * v
M =u’ =
![Page 92: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/92.jpg)
Multiplying a Vector by a Vector 2:The Outer Product
The two vectors need not have the same dimensionality.
Same Mantra: Rows by Columns.
This time, multiply a column vector by a row vector:
1
2v = [ 4 5 6 ]
M = u’ * v
M =
Row 1
u’ =
![Page 93: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/93.jpg)
Multiplying a Vector by a Vector 2:The Outer Product
The two vectors need not have the same dimensionality.
Same Mantra: Rows by Columns.
This time, multiply a column vector by a row vector:
1
2v = [ 4 5 6 ]
M = u’ * v
M =
Row 1 times column 1
u’ =
![Page 94: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/94.jpg)
Multiplying a Vector by a Vector 2:The Outer Product
The two vectors need not have the same dimensionality.
Same Mantra: Rows by Columns.
This time, multiply a column vector by a row vector:
1
2v = [ 4 5 6 ]
M = u’ * v
M =
Row 1 times column 1 goes into row 1, column 1
4 u’ =
![Page 95: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/95.jpg)
Multiplying a Vector by a Vector 2:The Outer Product
The two vectors need not have the same dimensionality.
Same Mantra: Rows by Columns.
This time, multiply a column vector by a row vector:
1
2v = [ 4 5 6 ]
M = u’ * v
M =
Row 1 times column 2 goes into row 1, column 2
4 5 u’ =
![Page 96: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/96.jpg)
Multiplying a Vector by a Vector 2:The Outer Product
The two vectors need not have the same dimensionality.
Same Mantra: Rows by Columns.
This time, multiply a column vector by a row vector:
1
2v = [ 4 5 6 ]
M = u’ * v
M =
Row 1 times column 3 goes into row 1, column 3
4 5 6 u’ =
![Page 97: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/97.jpg)
Multiplying a Vector by a Vector 2:The Outer Product
The two vectors need not have the same dimensionality.
Same Mantra: Rows by Columns.
This time, multiply a column vector by a row vector:
u =1
2v = [ 4 5 6 ]
M = u’ * v
M =
Row 2 times column 1 goes into row 2, column 1
4 5 6 8
![Page 98: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/98.jpg)
Multiplying a Vector by a Vector 2:The Outer Product
The two vectors need not have the same dimensionality.
Same Mantra: Rows by Columns.
This time, multiply a column vector by a row vector:
1
2v = [ 4 5 6 ]
M = u’ * v
M =
Row 2 times column 2 goes into row 2, column 2
4 5 6 8 10
u’ =
![Page 99: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/99.jpg)
Multiplying a Vector by a Vector 2:The Outer Product
The two vectors need not have the same dimensionality.
Same Mantra: Rows by Columns.
This time, multiply a column vector by a row vector:
1
2v = [ 4 5 6 ]
M = u’ * v
M =
Row 2 times column 3 goes into row 2, column 3
4 5 6 8 10 12
u’ =
![Page 100: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/100.jpg)
u’ =
Multiplying a Vector by a Vector 2:The Outer Product
The two vectors need not have the same dimensionality.
Same Mantra: Rows by Columns.
This time, multiply a column vector by a row vector:
12
v = [ 4 5 6 ]
M = u’ * v
= M4 5 6 8 10 12
A better way to visualize it…
![Page 101: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/101.jpg)
12
Multiplying a Vector by a Vector 2:The Outer Product
Outer product is not exactly commutative…
u’ =
v = [ 4 5 6 ]
M = u’ * v
= M4 5 6 8 10 12
M = v’ * u
u = [ 1 2 ]
456
v’ =4 85 106 12
![Page 102: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/102.jpg)
Multiplying a Vector by a Matrix• Same Mantra: Rows by Columns
![Page 103: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/103.jpg)
Rows by Columns
A row vector:
[ .2 .6 .3 .7 .9 .4 .3 ]
![Page 104: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/104.jpg)
Rows by Columns
A row vector:
[ .2 .6 .3 .7 .9 .4 .3 ]
A matrix:
.3 .4 .8 .1 .2 .3
.5 .2 0 .1 .5 .2
.1 .1 .9 .2 .5 .3
.2 .4 .1 .7 .8 .5
.9 .9 .2 .5 .3 .5
.4 .1 .2 .7 .8 .2
.1 .2 .2 .5 .7 .2
![Page 105: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/105.jpg)
Rows by Columns
A row vector:
[ .2 .6 .3 .7 .9 .4 .3 ]
A matrix:
.3 .4 .8 .1 .2 .3
.5 .2 0 .1 .5 .2
.1 .1 .9 .2 .5 .3
.2 .4 .1 .7 .8 .5
.9 .9 .2 .5 .3 .5
.4 .1 .2 .7 .8 .2
.1 .2 .2 .5 .7 .2
Multiply rows
![Page 106: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/106.jpg)
Rows by Columns
A row vector:
[ .2 .6 .3 .7 .9 .4 .3 ]
A matrix:
.3 .4 .8 .1 .2 .3
.5 .2 0 .1 .5 .2
.1 .1 .9 .2 .5 .3
.2 .4 .1 .7 .8 .5
.9 .9 .2 .5 .3 .5
.4 .1 .2 .7 .8 .2
.1 .2 .2 .5 .7 .2
Multiply rows by columns
![Page 107: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/107.jpg)
Each such multiplication is a simple dot product
A row vector:
[ .2 .6 .3 .7 .9 .4 .3 ]
A matrix:
.3 .4 .8 .1 .2 .3
.5 .2 0 .1 .5 .2
.1 .1 .9 .2 .5 .3
.2 .4 .1 .7 .8 .5
.9 .9 .2 .5 .3 .5
.4 .1 .2 .7 .8 .2
.1 .2 .2 .5 .7 .2
![Page 108: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/108.jpg)
Each such multiplication is a simple dot product
A row vector:
[ .2 .6 .3 .7 .9 .4 .3 ]
A matrix:
.3 .4 .8 .1 .2 .3
.5 .2 0 .1 .5 .2
.1 .1 .9 .2 .5 .3
.2 .4 .1 .7 .8 .5
.9 .9 .2 .5 .3 .5
.4 .1 .2 .7 .8 .2
.1 .2 .2 .5 .7 .2
Make a proxy column vector…
![Page 109: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/109.jpg)
Each such multiplication is a simple dot product
A row vector:
[ .2 .6 .3 .7 .9 .4 .3 ]
A matrix:
.3 .4 .8 .1 .2 .3
.5 .2 0 .1 .5 .2
.1 .1 .9 .2 .5 .3
.2 .4 .1 .7 .8 .5
.9 .9 .2 .5 .3 .5
.4 .1 .2 .7 .8 .2
.1 .2 .2 .5 .7 .2
.2
.6
.3
.7
.9
.4
.3
![Page 110: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/110.jpg)
Each such multiplication is a simple dot product
A row vector:
[ .2 .6 .3 .7 .9 .4 .3 ]
A matrix:
.3 .4 .8 .1 .2 .3
.5 .2 0 .1 .5 .2
.1 .1 .9 .2 .5 .3
.2 .4 .1 .7 .8 .5
.9 .9 .2 .5 .3 .5
.4 .1 .2 .7 .8 .2
.1 .2 .2 .5 .7 .2
.2
.6
.3
.7
.9
.4
.3
Now compute the dot product of the (proxy) row vector with each column of the matrix…
[ 1.5 ]
![Page 111: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/111.jpg)
Each such multiplication is a simple dot product
A row vector:
[ .2 .6 .3 .7 .9 .4 .3 ]
A matrix:
.3 .4 .8 .1 .2 .3
.5 .2 0 .1 .5 .2
.1 .1 .9 .2 .5 .3
.2 .4 .1 .7 .8 .5
.9 .9 .2 .5 .3 .5
.4 .1 .2 .7 .8 .2
.1 .2 .2 .5 .7 .2
.2
.6
.3
.7
.9
.4
.3
[ 1.5 1.4 ]
![Page 112: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/112.jpg)
Each such multiplication is a simple dot product
A row vector:
[ .2 .6 .3 .7 .9 .4 .3 ]
A matrix:
.3 .4 .8 .1 .2 .3
.5 .2 0 .1 .5 .2
.1 .1 .9 .2 .5 .3
.2 .4 .1 .7 .8 .5
.9 .9 .2 .5 .3 .5
.4 .1 .2 .7 .8 .2
.1 .2 .2 .5 .7 .2
.2
.6
.3
.7
.9
.4
.3
[ 1.5 1.4 0.8 ]
![Page 113: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/113.jpg)
Each such multiplication is a simple dot product
A row vector:
[ .2 .6 .3 .7 .9 .4 .3 ]
A matrix:
.3 .4 .8 .1 .2 .3
.5 .2 0 .1 .5 .2
.1 .1 .9 .2 .5 .3
.2 .4 .1 .7 .8 .5
.9 .9 .2 .5 .3 .5
.4 .1 .2 .7 .8 .2
.1 .2 .2 .5 .7 .2
.2
.6
.3
.7
.9
.4
.3
[ 1.5 1.4 0.8 1.5 ]
![Page 114: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/114.jpg)
Each such multiplication is a simple dot product
A row vector:
[ .2 .6 .3 .7 .9 .4 .3 ]
A matrix:
.3 .4 .8 .1 .2 .3
.5 .2 0 .1 .5 .2
.1 .1 .9 .2 .5 .3
.2 .4 .1 .7 .8 .5
.9 .9 .2 .5 .3 .5
.4 .1 .2 .7 .8 .2
.1 .2 .2 .5 .7 .2
.2
.6
.3
.7
.9
.4
.3
[ 1.5 1.4 0.8 1.5 1.9 ]
![Page 115: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/115.jpg)
Each such multiplication is a simple dot product
A row vector:
[ .2 .6 .3 .7 .9 .4 .3 ]
A matrix:
.3 .4 .8 .1 .2 .3
.5 .2 0 .1 .5 .2
.1 .1 .9 .2 .5 .3
.2 .4 .1 .7 .8 .5
.9 .9 .2 .5 .3 .5
.4 .1 .2 .7 .8 .2
.1 .2 .2 .5 .7 .2
.2
.6
.3
.7
.9
.4
.3
[ 1.5 1.4 0.8 1.5 1.9 1.2 ]
![Page 116: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/116.jpg)
A row vector:
[ .2 .6 .3 .7 .9 .4 .3 ]
A matrix:
.3 .4 .8 .1 .2 .3
.5 .2 0 .1 .5 .2
.1 .1 .9 .2 .5 .3
.2 .4 .1 .7 .8 .5
.9 .9 .2 .5 .3 .5
.4 .1 .2 .7 .8 .2
.1 .2 .2 .5 .7 .2The result is a row vector with as many columns (dimensions) as the matrix (not the vector)
[ 1.5 1.4 0.8 1.5 1.9 1.2 ]
![Page 117: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/117.jpg)
A row vector:
[ .2 .6 .3 .7 .9 .4 .3 ]
A matrix:
.3 .4 .8 .1 .2 .3
.5 .2 0 .1 .5 .2
.1 .1 .9 .2 .5 .3
.2 .4 .1 .7 .8 .5
.9 .9 .2 .5 .3 .5
.4 .1 .2 .7 .8 .2
.1 .2 .2 .5 .7 .2
[ 1.5 1.4 0.8 1.5 1.9 1.2 ]
7-dimensional vector
![Page 118: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/118.jpg)
A row vector:
[ .2 .6 .3 .7 .9 .4 .3 ]
A matrix:
.3 .4 .8 .1 .2 .3
.5 .2 0 .1 .5 .2
.1 .1 .9 .2 .5 .3
.2 .4 .1 .7 .8 .5
.9 .9 .2 .5 .3 .5
.4 .1 .2 .7 .8 .2
.1 .2 .2 .5 .7 .2
[ 1.5 1.4 0.8 1.5 1.9 1.2 ]
7-dimensional vector
6-dimensional vector
![Page 119: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/119.jpg)
A row vector:
[ .2 .6 .3 .7 .9 .4 .3 ]
A matrix:
.3 .4 .8 .1 .2 .3
.5 .2 0 .1 .5 .2
.1 .1 .9 .2 .5 .3
.2 .4 .1 .7 .8 .5
.9 .9 .2 .5 .3 .5
.4 .1 .2 .7 .8 .2
.1 .2 .2 .5 .7 .2
[ 1.5 1.4 0.8 1.5 1.9 1.2 ]
7-dimensional vector
6-dimensional vector
7 (rows) X 6 (columns) matrix
![Page 120: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/120.jpg)
A row vector:
[ .2 .6 .3 .7 .9 .4 .3 ]
A matrix:
.3 .4 .8 .1 .2 .3
.5 .2 0 .1 .5 .2
.1 .1 .9 .2 .5 .3
.2 .4 .1 .7 .8 .5
.9 .9 .2 .5 .3 .5
.4 .1 .2 .7 .8 .2
.1 .2 .2 .5 .7 .2
[ 1.5 1.4 0.8 1.5 1.9 1.2 ]
7-dimensional vector
6-dimensional vector
7 (rows) X 6 (columns) matrix
NOT Commutative!
![Page 121: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/121.jpg)
Multiplying a Matrix by a Matrix
• The Same Mantra: Rows by Columns
![Page 122: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/122.jpg)
Rows by Columns
A 2 X 3 matrix A 3 X 2 matrix
1 2 3
1 2 3
1 2
1 2
1 2
![Page 123: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/123.jpg)
Row 1 X Column 1
A 2 X 3 matrix A 3 X 2 matrix
1 2 3
1 2 3
1 2
1 2
1 2
1
2
3
(proxy)
Row 1 Column 1
![Page 124: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/124.jpg)
Row 1 X Column 1
A 2 X 3 matrix A 3 X 2 matrix
1 2 3
1 2 3
1 2
1 2
1 2
1
2
3
Row 1 Column 1
Result = 6
![Page 125: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/125.jpg)
Row 1 X Column 1
A 2 X 3 matrix A 3 X 2 matrix
1 2 3
1 2 3
1 2
1 2
1 2
1
2
3
Row 1 Column 1
Result = 6
Place the result in row 1,
![Page 126: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/126.jpg)
Row 1 X Column 1
A 2 X 3 matrix A 3 X 2 matrix
1 2 3
1 2 3
1 2
1 2
1 2
1
2
3
Row 1 Column 1
Result = 6
Place the result in row 1, column 1
![Page 127: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/127.jpg)
Row 1 X Column 1
A 2 X 3 matrix A 3 X 2 matrix
1 2 3
1 2 3
1 2
1 2
1 2
1
2
3
Row 1 Column 1
Result = 6
Place the result in row 1, column 1 of a new matrix…
6
![Page 128: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/128.jpg)
Row 1 X Column 2
A 2 X 3 matrix A 3 X 2 matrix
1 2 3
1 2 3
1 2
1 2
1 2
1
2
3
Row 1 Column 2
Result = 12
6
![Page 129: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/129.jpg)
Row 1 X Column 2
A 2 X 3 matrix A 3 X 2 matrix
1 2 3
1 2 3
1 2
1 2
1 2
1
2
3
Row 1 Column 2
Result = 12
6 12Place the result in row 1, column 2 of the new matrix…
![Page 130: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/130.jpg)
Row 2 X Column 1
A 2 X 3 matrix A 3 X 2 matrix
1 2 3
1 2 3
1
2
3
Row 2
Column 1
Result = 6
6 12
1 2
1 2
1 2
![Page 131: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/131.jpg)
Row 2 X Column 1
A 2 X 3 matrix A 3 X 2 matrix
1 2 3
1 2 3
1
2
3
Row 2
Column 1
Result = 6
6 12
6
1 2
1 2
1 2
Place the result in row 2, column 1 of the new matrix…
![Page 132: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/132.jpg)
Row 2 X Column 2
A 2 X 3 matrix A 3 X 2 matrix
1 2 3
1 2 3
1
2
3
Row 2
Column 2
6 12
6
1 2
1 2
1 2
Result = 12
![Page 133: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/133.jpg)
Row 2 X Column 2
A 2 X 3 matrix A 3 X 2 matrix
1 2 3
1 2 3
1
2
3
Row 2
Column 2
6 12
6 12
1 2
1 2
1 2
Result = 12
Place the result in row 2, column 2 of the new matrix…
![Page 134: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/134.jpg)
So…
A 2 X 3 matrix A 3 X 2 matrix
1 2 3
1 2 3
6 12
6 12
1 2
1 2
1 2*
=
A 2 X 2 matrix
![Page 135: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/135.jpg)
So…
A 2 X 3 matrix A 3 X 2 matrix
1 2 3
1 2 3
6 12
6 12
1 2
1 2
1 2*
=
A 2 X 2 matrix
The result has the same number of rows as the first matrix…
![Page 136: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/136.jpg)
So…
A 2 X 3 matrix A 3 X 2 matrix
1 2 3
1 2 3
6 12
6 12
1 2
1 2
1 2*
=
A 2 X 2 matrix
The result has the same number of rows as the first matrix…
…and the same number of columns as the second.
![Page 137: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/137.jpg)
…and…
A 2 X 3 matrix A 3 X 2 matrix
1 2 3
1 2 3
6 12
6 12
1 2
1 2
1 2*
=
A 2 X 2 matrix
…and the number of columns in the first matrix…
![Page 138: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/138.jpg)
…and…
A 2 X 3 matrix A 3 X 2 matrix
1 2 3
1 2 3
6 12
6 12
1 2
1 2
1 2*
=
A 2 X 2 matrix
…and the number of columns in the first matrix…
…must be equal to the number of rows in the second.
![Page 139: Prelude A pattern of activation in a NN is a vector A set of connection weights between units is a matrix Vectors and matrices have well-understood mathematical.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649dd45503460f94acb580/html5/thumbnails/139.jpg)
This is basic (default) matrix
multiplication.
There’s other more complicated stuff, too.
You (probably) won’t need it for this class.