NumPy Refresherwhy and how to use NumPy

28 Nov 2013 @lukmdo

Photo by Tony Delgrosso used under CC BY-NC-SA 2.0 /“Tile+Crop”

Agenda (<10min)

• Why: descriptiveness, memory, speed, data type

• How: conceptualise and use

Why?

import array import heapq from collections import deaque !__builtin__.bytearray !!

Why? (descriptive) def foo(matrix_2D, val): """ :type matrix_2D: ... """ out = [] for row in matrix_2D: for item in row: if item > val: out.append(item) return out

Why? (descriptive) def foo(matrix_2D, val): """ :type matrix_2D: ... """ return [i for r in matrix_2D for i in r if i > val]

Why? (descriptive) def foo(matrix_2D, val): """ :type matrix_2D: matrix """ return matrix_2D[matrix_2D > val]

Why? (descriptive)dtypes = [ bool, int, numpy.int8, numpy.int16, numpy.int32, numpy.int64, float, numpy.float16, numpy.float32, numpy.float64, numpy.float128, complex, ... str, unicode, object, numpy.void]

Why? (descriptive)!• constructors • selectors • shapes • types • methods • modules

Why? (descriptive)constructors = .copy(), array(), ndarray(), matrix(), empty(), empty_like(), zeros(), zeros_like(), ones(), ones_like(), fill(), arange(), linespace(), eye() !selectors = take(), where(), choose() !shapes = .shape, .reshape(), .resize(), .flatten(), .ravel(), .flat, .T, .transpose(), hstack(), vstack(), concentrate(), hsplit(), vsplit(), split() !types = .dtype, .astype(), dtype() !methods = .sum(), .min(), .max(), average(), var() ... !modules = linalg, random, polynomial, fft, ...

Why? (descriptive)

!a = np.array([20, 21, 22, 23, 24, 25], dtype=np.uint8) a = np.arange(20, 26, dtype=np.uint8) a = np.arange(20, 26) !>>> a[::2] array([20, 22, 24], dtype=uint8)

Why? (descriptive)a = np.arange(20, 26) !a[a > 21] a[where(a > 21)] take(a, a > 21) take(a, *where(a > 21)) !# small difference >>> where(a > 21) (array([2, 3, 4, 5]),) >>> a > 21 array([False, False, True, True, True, True])

Why? (descriptive)>>> a = np.arange(20, 26).reshape(2,3) array([[20, 21, 22], [23, 24, 25]]) !>>> a.mean() >>> a.mean(axis=0) >>> a.mean(axis=1)

Why? (descriptive)>>> a = np.arange(20, 26).reshape(2,3) array([[20, 21, 22], [23, 24, 25]]) !>>> a.mean() 22.5 >>> a.mean(axis=0) array([ 21.5, 22.5, 23.5]) >>> a.mean(axis=1) array([ 21., 24.])

Why? (descriptive)>>> a = np.arange(20, 26).reshape(2,3) array([[20, 21, 22], [23, 24, 25]]) !>>> a.mean() 22.5 >>> a.mean(axis=0) array([ 21.5, 22.5, 23.5]) >>> a.mean(axis=1) array([ 21., 24.]) !>>> a[ a.mean(axis=1) > 21, ] >>> a[ :, a.mean(axis=0) > 22 ]

Why? (descriptive)>>> a = np.arange(20, 26).reshape(2,3) array([[20, 21, 22], [23, 24, 25]]) !>>> a.mean() 22.5 >>> a.mean(axis=0) array([ 21.5, 22.5, 23.5]) >>> a.mean(axis=1) array([ 21., 24.]) >>> a[ a.mean(axis=1) > 21, ] array([[23, 24, 25]]) >>> a[ :, a.mean(axis=0) > 22 ] array([[21, 22], [24, 25]])

Why? (memory)>>> [ sys.getsizeof(None), sys.getsizeof(False), sys.getsizeof(0), sys.getsizeof(10**20)] [16, 24, 24, 36] !!>>> np.array([1,2,3], dtype=np.uint8).nbytes 3 >>> np.array([1,2,3], dtype=np.uint16).nbytes 6 >>> np.array([1,2,3], dtype=np.uint32).nbytes 12 >>> np.array([1,2,3], dtype=np.uint64).nbytes 24

Why? (speed)

1 < 10 (100, 1000 …)

How?• axis

• stride

• shape

• order

• out

How?a = np.arange(20, 26, dtype=np.uint8) !>>> a.strides (1,) >>> a.reshape(2, 3).strides (3, 1) >>> a.reshape(3, 2).strides (2, 1) >>> a.reshape(3, 2, order='F').strides (1, 3) >>> a.astype(np.int).reshape(3, 2).strides (16, 8)

21 22 242320 25

How?

• share

• copy / deep copy

• ipython + pylab + notebook + …

Linkswww.numpy.org

docs.scipy.org/doc/numpy/

scipy-lectures.github.io

pydata.org

scikit-learn.org

scikit-image.org

statsmodels.sourceforge.net

orange.biolab.si

pandas.pydata.org

blaze.pydata.org

www.sagemath.org

Photo by Tony Delgrosso used under CC BY-NC-SA 2.0 /“Tile+Crop”