Scientific Computing and Data Science Applications with ...978-1-4842-4246-9/1.pdf · Numerical...

Numerical PythonScientific Computing and Data

Science Applications with Numpy, SciPy and Matplotlib

Second Edition

Robert Johansson

Numerical Python: Scientific Computing and Data Science Applications with Numpy, SciPy and Matplotlib

ISBN-13 (pbk): 978-1-4842-4245-2 ISBN-13 (electronic): 978-1-4842-4246-9https://doi.org/10.1007/978-1-4842-4246-9

Library of Congress Control Number: 2018966798

Copyright © 2019 by Robert Johansson

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed.

Trademarked names, logos, and images may appear in this book. Rather than use a trademark symbol with every occurrence of a trademarked name, logo, or image we use the names, logos, and images only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark.

The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights.

While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein.

Managing Director, Apress Media LLC: Welmoed SpahrAcquisitions Editor: Todd GreenDevelopment Editor: James MarkhamCoordinating Editor: Jill Balzano

Cover designed by eStudioCalamar

Cover image designed by Freepik (www.freepik.com)

Distributed to the book trade worldwide by Springer Science+Business Media New York, 233 Spring Street, 6th Floor, New York, NY 10013. Phone 1-800-SPRINGER, fax (201) 348-4505, e-mail [email protected], or visit www.springeronline.com. Apress Media, LLC is a California LLC and the sole member (owner) is Springer Science + Business Media Finance Inc (SSBM Finance Inc). SSBM Finance Inc is a Delaware corporation.

For information on translations, please e-mail [email protected]; for reprint, paperback, or audio rights, please email [email protected].

Apress titles may be purchased in bulk for academic, corporate, or promotional use. eBook versions and licenses are also available for most titles. For more information, reference our Print and eBook Bulk Sales web page at http://www.apress.com/bulk-sales.

Any source code or other supplementary material referenced by the author in this book is available to readers on GitHub via the book's product page, located at www.apress.com/9781484242452. For more detailed information, please visit http://www.apress.com/source-code.

Printed on acid-free paper

Robert JohanssonUrayasu-shi, Chiba, Japan

https://doi.org/10.1007/978-1-4842-4246-9

To Mika and Erika.

v

About the Author ��xv

About the Technical Reviewers ��xvii

Introduction ��xxi

Table of Contents

Chapter 1: Introduction to Computing with Python �� 1

Environments for Computing with Python �� 5

Python �� 6

Interpreter �� 7

IPython Console �� 8

Input and Output Caching �� 9

Autocompletion and Object Introspection �� 11

Documentation �� 11

Interaction with the System Shell �� 12

IPython Extensions �� 13

Jupyter �� 19

The Jupyter QtConsole �� 20

The Jupyter Notebook �� 21

Jupyter Lab �� 24

Cell Types �� 25

Editing Cells �� 26

Markdown Cells �� 28

Rich Output Display �� 30

nbconvert �� 34

https://doi.org/10.1007/978-1-4842-4246-9_1

https://doi.org/10.1007/978-1-4842-4246-9_1

https://doi.org/10.1007/978-1-4842-4246-9_1#Sec1

https://doi.org/10.1007/978-1-4842-4246-9_1#Sec2

https://doi.org/10.1007/978-1-4842-4246-9_1#Sec3

https://doi.org/10.1007/978-1-4842-4246-9_1#Sec4

https://doi.org/10.1007/978-1-4842-4246-9_1#Sec5

https://doi.org/10.1007/978-1-4842-4246-9_1#Sec6

https://doi.org/10.1007/978-1-4842-4246-9_1#Sec7

https://doi.org/10.1007/978-1-4842-4246-9_1#Sec8

https://doi.org/10.1007/978-1-4842-4246-9_1#Sec9

https://doi.org/10.1007/978-1-4842-4246-9_1#Sec16

https://doi.org/10.1007/978-1-4842-4246-9_1#Sec17

https://doi.org/10.1007/978-1-4842-4246-9_1#Sec18

https://doi.org/10.1007/978-1-4842-4246-9_1#Sec19

https://doi.org/10.1007/978-1-4842-4246-9_1#Sec20

https://doi.org/10.1007/978-1-4842-4246-9_1#Sec21

https://doi.org/10.1007/978-1-4842-4246-9_1#Sec22

https://doi.org/10.1007/978-1-4842-4246-9_1#Sec23

https://doi.org/10.1007/978-1-4842-4246-9_1#Sec24

vi

Spyder: An Integrated Development Environment �� 37

Source Code Editor �� 38

Consoles in Spyder �� 40

Object Inspector �� 40

Summary�� 41

Further Reading �� 41

References �� 41

Chapter 2: Vectors, Matrices, and Multidimensional Arrays �� 43

Importing the Modules �� 44

The NumPy Array Object �� 44

Data Types �� 46

Order of Array Data in Memory �� 49

Creating Arrays �� 50

Arrays Created from Lists and Other Array-Like Objects �� 52

Arrays Filled with Constant Values �� 52

Arrays Filled with Incremental Sequences �� 54

Arrays Filled with Logarithmic Sequences �� 54

Meshgrid Arrays �� 55

Creating Uninitialized Arrays �� 56

Creating Arrays with Properties of Other Arrays �� 56

Creating Matrix Arrays �� 57

Indexing and Slicing �� 58

One-Dimensional Arrays �� 58

Multidimensional Arrays �� 60

Views �� 62

Fancy Indexing and Boolean-Valued Indexing �� 63

Reshaping and Resizing �� 66

Vectorized Expressions �� 70

Arithmetic Operations �� 72

Elementwise Functions �� 76

Table of ConTenTs

https://doi.org/10.1007/978-1-4842-4246-9_1#Sec28

https://doi.org/10.1007/978-1-4842-4246-9_1#Sec29

https://doi.org/10.1007/978-1-4842-4246-9_1#Sec30

https://doi.org/10.1007/978-1-4842-4246-9_1#Sec31

https://doi.org/10.1007/978-1-4842-4246-9_1#Sec32

https://doi.org/10.1007/978-1-4842-4246-9_1#Sec33

https://doi.org/10.1007/978-1-4842-4246-9_1#Sec34

https://doi.org/10.1007/978-1-4842-4246-9_2

https://doi.org/10.1007/978-1-4842-4246-9_2

https://doi.org/10.1007/978-1-4842-4246-9_2#Sec1

https://doi.org/10.1007/978-1-4842-4246-9_2#Sec2

https://doi.org/10.1007/978-1-4842-4246-9_2#Sec3

https://doi.org/10.1007/978-1-4842-4246-9_2#Sec5

https://doi.org/10.1007/978-1-4842-4246-9_2#Sec6

https://doi.org/10.1007/978-1-4842-4246-9_2#Sec7

https://doi.org/10.1007/978-1-4842-4246-9_2#Sec8

https://doi.org/10.1007/978-1-4842-4246-9_2#Sec9

https://doi.org/10.1007/978-1-4842-4246-9_2#Sec10

https://doi.org/10.1007/978-1-4842-4246-9_2#Sec11

https://doi.org/10.1007/978-1-4842-4246-9_2#Sec12

https://doi.org/10.1007/978-1-4842-4246-9_2#Sec13

https://doi.org/10.1007/978-1-4842-4246-9_2#Sec14

https://doi.org/10.1007/978-1-4842-4246-9_2#Sec15

https://doi.org/10.1007/978-1-4842-4246-9_2#Sec16

https://doi.org/10.1007/978-1-4842-4246-9_2#Sec17

https://doi.org/10.1007/978-1-4842-4246-9_2#Sec18

https://doi.org/10.1007/978-1-4842-4246-9_2#Sec19

https://doi.org/10.1007/978-1-4842-4246-9_2#Sec20

https://doi.org/10.1007/978-1-4842-4246-9_2#Sec21

https://doi.org/10.1007/978-1-4842-4246-9_2#Sec22

https://doi.org/10.1007/978-1-4842-4246-9_2#Sec23

vii

Aggregate Functions �� 79

Boolean Arrays and Conditional Expressions �� 82

Set Operations �� 85

Operations on Arrays �� 87

Matrix and Vector Operations �� 88

Summary�� 95



Chapter 3: Symbolic Computing �� 97

Importing SymPy �� 98

Symbols �� 99

Numbers �� 102

Expressions �� 109

Manipulating Expressions �� 110

Simplification �� 111

Expand �� 112

Factor, Collect, and Combine �� 114

Apart, Together, and Cancel �� 115

Substitutions �� 115

Numerical Evaluation �� 117

Calculus �� 118

Derivatives �� 119

Integrals �� 121

Series �� 123

Limits �� 125

Sums and Products �� 126

Equations �� 127

Linear Algebra �� 130

Table of ConTenTs

https://doi.org/10.1007/978-1-4842-4246-9_2#Sec24

https://doi.org/10.1007/978-1-4842-4246-9_2#Sec25

https://doi.org/10.1007/978-1-4842-4246-9_2#Sec26

https://doi.org/10.1007/978-1-4842-4246-9_2#Sec27

https://doi.org/10.1007/978-1-4842-4246-9_2#Sec28

https://doi.org/10.1007/978-1-4842-4246-9_2#Sec29

https://doi.org/10.1007/978-1-4842-4246-9_2#Sec30

https://doi.org/10.1007/978-1-4842-4246-9_2#Sec31

https://doi.org/10.1007/978-1-4842-4246-9_3

https://doi.org/10.1007/978-1-4842-4246-9_3

https://doi.org/10.1007/978-1-4842-4246-9_3#Sec1

https://doi.org/10.1007/978-1-4842-4246-9_3#Sec2

https://doi.org/10.1007/978-1-4842-4246-9_3#Sec3

https://doi.org/10.1007/978-1-4842-4246-9_3#Sec9

https://doi.org/10.1007/978-1-4842-4246-9_3#Sec10

https://doi.org/10.1007/978-1-4842-4246-9_3#Sec11

https://doi.org/10.1007/978-1-4842-4246-9_3#Sec12

https://doi.org/10.1007/978-1-4842-4246-9_3#Sec13

https://doi.org/10.1007/978-1-4842-4246-9_3#Sec14

https://doi.org/10.1007/978-1-4842-4246-9_3#Sec15

https://doi.org/10.1007/978-1-4842-4246-9_3#Sec16

https://doi.org/10.1007/978-1-4842-4246-9_3#Sec17

https://doi.org/10.1007/978-1-4842-4246-9_3#Sec18

https://doi.org/10.1007/978-1-4842-4246-9_3#Sec19

https://doi.org/10.1007/978-1-4842-4246-9_3#Sec20

https://doi.org/10.1007/978-1-4842-4246-9_3#Sec21

https://doi.org/10.1007/978-1-4842-4246-9_3#Sec22

https://doi.org/10.1007/978-1-4842-4246-9_3#Sec23

https://doi.org/10.1007/978-1-4842-4246-9_3#Sec24

viii

Summary�� 134


Reference �� 134

Chapter 4: Plotting and Visualization �� 135

Importing Modules �� 136

Getting Started �� 137

Interactive and Noninteractive Modes �� 141

Figure �� 143

Axes �� 145

Plot Types �� 146

Line Properties �� 147

Legends �� 152

Text Formatting and Annotations �� 153

Axis Properties �� 156

Advanced Axes Layouts �� 168

Insets �� 168

Subplots �� 170

Subplot2grid �� 172

GridSpec �� 173

Colormap Plots �� 174

3 D Plots �� 177

Summary�� 180



Chapter 5: Equation Solving �� 183


Linear Equation Systems �� 185

Square Systems �� 186

Rectangular Systems�� 192

Table of ConTenTs

https://doi.org/10.1007/978-1-4842-4246-9_3#Sec25

https://doi.org/10.1007/978-1-4842-4246-9_3#Sec26

https://doi.org/10.1007/978-1-4842-4246-9_3#Sec27

https://doi.org/10.1007/978-1-4842-4246-9_4

https://doi.org/10.1007/978-1-4842-4246-9_4

https://doi.org/10.1007/978-1-4842-4246-9_4#Sec1

https://doi.org/10.1007/978-1-4842-4246-9_4#Sec2

https://doi.org/10.1007/978-1-4842-4246-9_4#Sec3

https://doi.org/10.1007/978-1-4842-4246-9_4#Sec4

https://doi.org/10.1007/978-1-4842-4246-9_4#Sec5

https://doi.org/10.1007/978-1-4842-4246-9_4#Sec6

https://doi.org/10.1007/978-1-4842-4246-9_4#Sec7

https://doi.org/10.1007/978-1-4842-4246-9_4#Sec8

https://doi.org/10.1007/978-1-4842-4246-9_4#Sec9

https://doi.org/10.1007/978-1-4842-4246-9_4#Sec10

https://doi.org/10.1007/978-1-4842-4246-9_4#Sec17

https://doi.org/10.1007/978-1-4842-4246-9_4#Sec18

https://doi.org/10.1007/978-1-4842-4246-9_4#Sec19

https://doi.org/10.1007/978-1-4842-4246-9_4#Sec20

https://doi.org/10.1007/978-1-4842-4246-9_4#Sec21

https://doi.org/10.1007/978-1-4842-4246-9_4#Sec22

https://doi.org/10.1007/978-1-4842-4246-9_4#Sec23

https://doi.org/10.1007/978-1-4842-4246-9_4#Sec24

https://doi.org/10.1007/978-1-4842-4246-9_4#Sec25

https://doi.org/10.1007/978-1-4842-4246-9_4#Sec26

https://doi.org/10.1007/978-1-4842-4246-9_5

https://doi.org/10.1007/978-1-4842-4246-9_5

https://doi.org/10.1007/978-1-4842-4246-9_5#Sec1

https://doi.org/10.1007/978-1-4842-4246-9_5#Sec2

https://doi.org/10.1007/978-1-4842-4246-9_5#Sec3

https://doi.org/10.1007/978-1-4842-4246-9_5#Sec4

ix

Eigenvalue Problems�� 196

Nonlinear Equations �� 198

Univariate Equations �� 199

Systems of Nonlinear Equations �� 207

Summary�� 212



Chapter 6: Optimization �� 213


Classification of Optimization Problems �� 214

Univariate Optimization �� 217

Unconstrained Multivariate Optimization �� 221

Nonlinear Least Square Problems �� 230

Constrained Optimization �� 232

Linear Programming �� 238

Summary�� 241



Chapter 7: Interpolation �� 243


Interpolation �� 244

Polynomials �� 245

Polynomial Interpolation �� 249

Spline Interpolation �� 255

Multivariate Interpolation �� 258

Summary�� 265



Table of ConTenTs

https://doi.org/10.1007/978-1-4842-4246-9_5#Sec5

https://doi.org/10.1007/978-1-4842-4246-9_5#Sec6

https://doi.org/10.1007/978-1-4842-4246-9_5#Sec7

https://doi.org/10.1007/978-1-4842-4246-9_5#Sec8

https://doi.org/10.1007/978-1-4842-4246-9_5#Sec9

https://doi.org/10.1007/978-1-4842-4246-9_5#Sec10

https://doi.org/10.1007/978-1-4842-4246-9_5#Sec11

https://doi.org/10.1007/978-1-4842-4246-9_6

https://doi.org/10.1007/978-1-4842-4246-9_6

https://doi.org/10.1007/978-1-4842-4246-9_6#Sec1

https://doi.org/10.1007/978-1-4842-4246-9_6#Sec2

https://doi.org/10.1007/978-1-4842-4246-9_6#Sec3

https://doi.org/10.1007/978-1-4842-4246-9_6#Sec4

https://doi.org/10.1007/978-1-4842-4246-9_6#Sec5

https://doi.org/10.1007/978-1-4842-4246-9_6#Sec6

https://doi.org/10.1007/978-1-4842-4246-9_6#Sec7

https://doi.org/10.1007/978-1-4842-4246-9_6#Sec8

https://doi.org/10.1007/978-1-4842-4246-9_6#Sec9

https://doi.org/10.1007/978-1-4842-4246-9_6#Sec10

https://doi.org/10.1007/978-1-4842-4246-9_7

https://doi.org/10.1007/978-1-4842-4246-9_7

https://doi.org/10.1007/978-1-4842-4246-9_7#Sec1

https://doi.org/10.1007/978-1-4842-4246-9_7#Sec2

https://doi.org/10.1007/978-1-4842-4246-9_7#Sec3

https://doi.org/10.1007/978-1-4842-4246-9_7#Sec4

https://doi.org/10.1007/978-1-4842-4246-9_7#Sec5

https://doi.org/10.1007/978-1-4842-4246-9_7#Sec6

https://doi.org/10.1007/978-1-4842-4246-9_7#Sec7

https://doi.org/10.1007/978-1-4842-4246-9_7#Sec8

https://doi.org/10.1007/978-1-4842-4246-9_7#Sec9

x

Chapter 8: Integration�� 267


Numerical Integration Methods�� 269

Numerical Integration with SciPy �� 274

Tabulated Integrand �� 277

Multiple Integration �� 280

Symbolic and Arbitrary-Precision Integration �� 285

Line Integrals �� 288

Integral Transforms �� 289

Summary�� 292



Chapter 9: Ordinary Differential Equations �� 295


Ordinary Differential Equations �� 296

Symbolic Solution to ODEs �� 298

Direction Fields �� 304

Solving ODEs Using Laplace Transformations �� 309

Numerical Methods for Solving ODEs �� 313

Numerical Integration of ODEs Using SciPy �� 317

Summary�� 332



Chapter 10: Sparse Matrices and Graphs �� 335


Sparse Matrices in SciPy �� 336

Functions for Creating Sparse Matrices �� 342

Sparse Linear Algebra Functions �� 345

Table of ConTenTs

https://doi.org/10.1007/978-1-4842-4246-9_8

https://doi.org/10.1007/978-1-4842-4246-9_8

https://doi.org/10.1007/978-1-4842-4246-9_8#Sec1

https://doi.org/10.1007/978-1-4842-4246-9_8#Sec2

https://doi.org/10.1007/978-1-4842-4246-9_8#Sec3

https://doi.org/10.1007/978-1-4842-4246-9_8#Sec4

https://doi.org/10.1007/978-1-4842-4246-9_8#Sec5

https://doi.org/10.1007/978-1-4842-4246-9_8#Sec6

https://doi.org/10.1007/978-1-4842-4246-9_8#Sec7

https://doi.org/10.1007/978-1-4842-4246-9_8#Sec8

https://doi.org/10.1007/978-1-4842-4246-9_8#Sec9

https://doi.org/10.1007/978-1-4842-4246-9_8#Sec10

https://doi.org/10.1007/978-1-4842-4246-9_8#Sec11

https://doi.org/10.1007/978-1-4842-4246-9_9

https://doi.org/10.1007/978-1-4842-4246-9_9

https://doi.org/10.1007/978-1-4842-4246-9_9#Sec1

https://doi.org/10.1007/978-1-4842-4246-9_9#Sec2

https://doi.org/10.1007/978-1-4842-4246-9_9#Sec3

https://doi.org/10.1007/978-1-4842-4246-9_9#Sec4

https://doi.org/10.1007/978-1-4842-4246-9_9#Sec5

https://doi.org/10.1007/978-1-4842-4246-9_9#Sec6

https://doi.org/10.1007/978-1-4842-4246-9_9#Sec7

https://doi.org/10.1007/978-1-4842-4246-9_9#Sec8

https://doi.org/10.1007/978-1-4842-4246-9_9#Sec9

https://doi.org/10.1007/978-1-4842-4246-9_9#Sec10

https://doi.org/10.1007/978-1-4842-4246-9_10

https://doi.org/10.1007/978-1-4842-4246-9_10

https://doi.org/10.1007/978-1-4842-4246-9_10#Sec1

https://doi.org/10.1007/978-1-4842-4246-9_10#Sec2

https://doi.org/10.1007/978-1-4842-4246-9_10#Sec3

https://doi.org/10.1007/978-1-4842-4246-9_10#Sec4

xi

Linear Equation Systems �� 345

Graphs and Networks �� 352

Summary�� 360



Chapter 11: Partial Differential Equations �� 363


Partial Differential Equations �� 365

Finite-Difference Methods �� 366

Finite-Element Methods �� 373

Survey of FEM Libraries �� 377

Solving PDEs Using FEniCS �� 378

Summary�� 403



Chapter 12: Data Processing and Analysis �� 405


Introduction to Pandas �� 407

Series �� 407

DataFrame �� 410

Time Series �� 422

The Seaborn Graphics Library �� 434

Summary�� 440



Chapter 13: Statistics �� 443


Review of Statistics and Probability �� 444

Random Numbers �� 446

Table of ConTenTs

https://doi.org/10.1007/978-1-4842-4246-9_10#Sec5

https://doi.org/10.1007/978-1-4842-4246-9_10#Sec7

https://doi.org/10.1007/978-1-4842-4246-9_10#Sec8

https://doi.org/10.1007/978-1-4842-4246-9_10#Sec9

https://doi.org/10.1007/978-1-4842-4246-9_10#Sec10

https://doi.org/10.1007/978-1-4842-4246-9_11

https://doi.org/10.1007/978-1-4842-4246-9_11

https://doi.org/10.1007/978-1-4842-4246-9_11#Sec1

https://doi.org/10.1007/978-1-4842-4246-9_11#Sec2

https://doi.org/10.1007/978-1-4842-4246-9_11#Sec3

https://doi.org/10.1007/978-1-4842-4246-9_11#Sec4

https://doi.org/10.1007/978-1-4842-4246-9_11#Sec5

https://doi.org/10.1007/978-1-4842-4246-9_11#Sec6

https://doi.org/10.1007/978-1-4842-4246-9_11#Sec7

https://doi.org/10.1007/978-1-4842-4246-9_11#Sec8

https://doi.org/10.1007/978-1-4842-4246-9_11#Sec9

https://doi.org/10.1007/978-1-4842-4246-9_12

https://doi.org/10.1007/978-1-4842-4246-9_12

https://doi.org/10.1007/978-1-4842-4246-9_12#Sec1

https://doi.org/10.1007/978-1-4842-4246-9_12#Sec2

https://doi.org/10.1007/978-1-4842-4246-9_12#Sec3

https://doi.org/10.1007/978-1-4842-4246-9_12#Sec4

https://doi.org/10.1007/978-1-4842-4246-9_12#Sec5

https://doi.org/10.1007/978-1-4842-4246-9_12#Sec6

https://doi.org/10.1007/978-1-4842-4246-9_12#Sec7

https://doi.org/10.1007/978-1-4842-4246-9_12#Sec8

https://doi.org/10.1007/978-1-4842-4246-9_12#Sec9

https://doi.org/10.1007/978-1-4842-4246-9_13

https://doi.org/10.1007/978-1-4842-4246-9_13

https://doi.org/10.1007/978-1-4842-4246-9_13#Sec1

https://doi.org/10.1007/978-1-4842-4246-9_13#Sec2

https://doi.org/10.1007/978-1-4842-4246-9_13#Sec3

xii

Random Variables and Distributions �� 451

Hypothesis Testing �� 460

Nonparametric Methods �� 466

Summary�� 469



Chapter 14: Statistical Modeling �� 471


Introduction to Statistical Modeling �� 473

Defining Statistical Models with Patsy �� 474

Linear Regression �� 485

Example Datasets �� 494

Discrete Regression �� 496

Logistic Regression �� 496

Poisson Model �� 502

Time Series �� 506

Summary�� 511



Chapter 15: Machine Learning �� 513


Brief Review of Machine Learning �� 515

Regression �� 518

Classification �� 529

Clustering �� 535

Summary�� 540



Table of ConTenTs

https://doi.org/10.1007/978-1-4842-4246-9_13#Sec4

https://doi.org/10.1007/978-1-4842-4246-9_13#Sec5

https://doi.org/10.1007/978-1-4842-4246-9_13#Sec6

https://doi.org/10.1007/978-1-4842-4246-9_13#Sec7

https://doi.org/10.1007/978-1-4842-4246-9_13#Sec8

https://doi.org/10.1007/978-1-4842-4246-9_13#Sec9

https://doi.org/10.1007/978-1-4842-4246-9_14

https://doi.org/10.1007/978-1-4842-4246-9_14

https://doi.org/10.1007/978-1-4842-4246-9_14#Sec1

https://doi.org/10.1007/978-1-4842-4246-9_14#Sec2

https://doi.org/10.1007/978-1-4842-4246-9_14#Sec3

https://doi.org/10.1007/978-1-4842-4246-9_14#Sec4

https://doi.org/10.1007/978-1-4842-4246-9_14#Sec5

https://doi.org/10.1007/978-1-4842-4246-9_14#Sec6

https://doi.org/10.1007/978-1-4842-4246-9_14#Sec7

https://doi.org/10.1007/978-1-4842-4246-9_14#Sec8

https://doi.org/10.1007/978-1-4842-4246-9_14#Sec9

https://doi.org/10.1007/978-1-4842-4246-9_14#Sec10

https://doi.org/10.1007/978-1-4842-4246-9_14#Sec11

https://doi.org/10.1007/978-1-4842-4246-9_14#Sec12

https://doi.org/10.1007/978-1-4842-4246-9_15

https://doi.org/10.1007/978-1-4842-4246-9_15

https://doi.org/10.1007/978-1-4842-4246-9_15#Sec1

https://doi.org/10.1007/978-1-4842-4246-9_15#Sec2

https://doi.org/10.1007/978-1-4842-4246-9_15#Sec3

https://doi.org/10.1007/978-1-4842-4246-9_15#Sec4

https://doi.org/10.1007/978-1-4842-4246-9_15#Sec5

https://doi.org/10.1007/978-1-4842-4246-9_15#Sec6

https://doi.org/10.1007/978-1-4842-4246-9_15#Sec7

https://doi.org/10.1007/978-1-4842-4246-9_15#Sec8

xiii

Chapter 16: Bayesian Statistics �� 543


Introduction to Bayesian Statistics �� 545

Model Definition �� 548

Sampling Posterior Distributions �� 553

Linear Regression �� 558

Summary�� 571



Chapter 17: Signal Processing �� 573


Spectral Analysis �� 574

Fourier Transforms �� 575

Windowing �� 581

Spectrogram �� 585

Signal Filters �� 590

Convolution Filters �� 590

FIR and IIR Filters �� 593

Summary�� 598



Chapter 18: Data Input and Output �� 601


Comma-Separated Values �� 603

HDF5 �� 608

h5py �� 610

PyTables �� 623

Pandas HDFStore �� 629

Table of ConTenTs

https://doi.org/10.1007/978-1-4842-4246-9_16

https://doi.org/10.1007/978-1-4842-4246-9_16

https://doi.org/10.1007/978-1-4842-4246-9_16#Sec1

https://doi.org/10.1007/978-1-4842-4246-9_16#Sec2

https://doi.org/10.1007/978-1-4842-4246-9_16#Sec3

https://doi.org/10.1007/978-1-4842-4246-9_16#Sec4

https://doi.org/10.1007/978-1-4842-4246-9_16#Sec5

https://doi.org/10.1007/978-1-4842-4246-9_16#Sec6

https://doi.org/10.1007/978-1-4842-4246-9_16#Sec7

https://doi.org/10.1007/978-1-4842-4246-9_16#Sec8

https://doi.org/10.1007/978-1-4842-4246-9_17

https://doi.org/10.1007/978-1-4842-4246-9_17

https://doi.org/10.1007/978-1-4842-4246-9_17#Sec1

https://doi.org/10.1007/978-1-4842-4246-9_17#Sec2

https://doi.org/10.1007/978-1-4842-4246-9_17#Sec3

https://doi.org/10.1007/978-1-4842-4246-9_17#Sec5

https://doi.org/10.1007/978-1-4842-4246-9_17#Sec6

https://doi.org/10.1007/978-1-4842-4246-9_17#Sec7

https://doi.org/10.1007/978-1-4842-4246-9_17#Sec8

https://doi.org/10.1007/978-1-4842-4246-9_17#Sec9

https://doi.org/10.1007/978-1-4842-4246-9_17#Sec10

https://doi.org/10.1007/978-1-4842-4246-9_17#Sec11

https://doi.org/10.1007/978-1-4842-4246-9_17#Sec12

https://doi.org/10.1007/978-1-4842-4246-9_18

https://doi.org/10.1007/978-1-4842-4246-9_18

https://doi.org/10.1007/978-1-4842-4246-9_18#Sec1

https://doi.org/10.1007/978-1-4842-4246-9_18#Sec2

https://doi.org/10.1007/978-1-4842-4246-9_18#Sec3

https://doi.org/10.1007/978-1-4842-4246-9_18#Sec4

https://doi.org/10.1007/978-1-4842-4246-9_18#Sec9

https://doi.org/10.1007/978-1-4842-4246-9_18#Sec10

xiv

JSON �� 631

Serialization �� 636

Summary�� 639


Reference �� 640

Chapter 19: Code Optimization �� 641


Numba�� 644

Cython �� 652

Summary�� 664



Appendix: Installation �� 667

Miniconda and Conda �� 668

A Complete Environment�� 676

Summary�� 680


Index �� 683

Table of ConTenTs

https://doi.org/10.1007/978-1-4842-4246-9_18#Sec11

https://doi.org/10.1007/978-1-4842-4246-9_18#Sec12

https://doi.org/10.1007/978-1-4842-4246-9_18#Sec13

https://doi.org/10.1007/978-1-4842-4246-9_18#Sec14

https://doi.org/10.1007/978-1-4842-4246-9_18#Sec15

https://doi.org/10.1007/978-1-4842-4246-9_19

https://doi.org/10.1007/978-1-4842-4246-9_19

https://doi.org/10.1007/978-1-4842-4246-9_19#Sec1

https://doi.org/10.1007/978-1-4842-4246-9_19#Sec2

https://doi.org/10.1007/978-1-4842-4246-9_19#Sec3

https://doi.org/10.1007/978-1-4842-4246-9_19#Sec4

https://doi.org/10.1007/978-1-4842-4246-9_19#Sec5

https://doi.org/10.1007/978-1-4842-4246-9_19#Sec6

xv

About the Author

Robert Johansson is an experienced Python programmer

and computational scientist, with a Ph.D. in Theoretical

Physics from Chalmers University of Technology, Sweden.

He has worked with scientific computing in academia and

industry for over 10 years, and he has participated in both

open source development and proprietary research projects.

His open source contributions include work on QuTiP, a

popular Python framework for simulating the dynamics of

quantum systems; and he has also contributed to several

other popular Python libraries in the scientific computing

landscape. Robert is passionate about scientific computing

and software development and about teaching and communicating best practices for

bringing these fields together with optimal outcome: novel, reproducible, and extensible

computational results. Robert’s background includes 5 years of postdoctoral research in

theoretical and computational physics, and he is now working as a data scientist in the

IT industry.

xvii

About the Technical Reviewers

Massimo Nardone has more than 24 years of experiences

in security, web/mobile development, cloud, and IT

architecture. His true IT passions are security and Android.

He has been programming and teaching how to program

with Android, Perl, PHP, Java, VB, Python, C/C++, and

MySQL for more than 20 years.

He holds an M.Sc. degree in computing science from the

University of Salerno, Italy.

He has worked as a project manager, software engineer,

research engineer, chief security architect, information

security manager, PCI/SCADA auditor, and senior lead IT security/cloud/SCADA

architect for many years.

His technical skills include security, Android, cloud, Java, MySQL, Drupal, Cobol,

Perl, web and mobile development, MongoDB, D3, Joomla!, Couchbase, C/C++, WebGL,

Python, Pro Rails, Django CMS, Jekyll, Scratch, etc.

He worked as visiting lecturer and supervisor for exercises at the Networking

Laboratory of the Helsinki University of Technology (Aalto University). He holds four

international patents (PKI, SIP, SAML, and Proxy areas).

He currently works as chief information security officer (CISO) for Cargotec Oyj, and

he is a member of the ISACA Finland Chapter Board.

Massimo has reviewed more than 45 IT books for different publishers and has

coauthored Pro JPA 2 in Java EE 8 (Apress, 2018), Beginning EJB in Java EE 8

(Apress, 2018), and Pro Android Games (Apress, 2015).

xviii

Chinmaya Patnayak is an embedded software developer at

NVIDIA and is skilled in C++, CUDA, deep learning, Linux,

and file systems. He has been a speaker and instructor for

deep learning at various major technology events across

India. Chinmaya holds an M.Sc. degree in physics and

B.E. in electrical and electronics engineering from BITS

Pilani. He has previously worked with Defence Research

and Development Organization (DRDO) on encryption

algorithms for video streams. His current interest lies in

neural networks for image segmentation and applications in biomedical research and

self-driving cars. Find more about him at http://chinmayapatnayak.github.io.

Michael Thomas has worked in software development

for more than 20 years as an individual contributor, team

lead, program manager, and vice president of engineering.

Michael has more than 10 years of experience working with

mobile devices. His current focus is in the medical sector,

using mobile devices to accelerate information transfer

between patients and health-care providers.

David Stansby is a Ph.D. student at Imperial College

London and an active Python developer. He is on the core

development team of Matplotlib, Python’s most popular

plotting library, and the creator of HelioPy, a Python package

for space science data analysis.

abouT The TeChniCal RevieweRs

http://chinmayapatnayak.github.io/

xix

Jason Whitehorn is an experienced entrepreneur and

software developer and has helped many oil and gas

companies automate and enhance their oilfield solutions

through field data capture, SCADA, and machine learning.

Jason obtained his B.SC. in computer science from Arkansas

State University, but he traces his passion for development

back many years before then, having first taught himself

to program BASIC on his family’s computer while still in

middle school.

When he’s not mentoring and helping his team at work, writing, or pursuing one of

his many side projects, Jason enjoys spending time with his wife and four children and

living in the Tulsa, Oklahoma region. More information about Jason can be found on his

web site: https://jason.whitehorn.us.

abouT The TeChniCal RevieweRs

https://jason.whitehorn.us/

xxi

Introduction

Scientific and numerical computing is a booming field in research, engineering, and

analytics. The revolution in the computer industry over the last several decades has

provided new and powerful tools for computational practitioners. This has enabled

computational undertakings of previously unprecedented scale and complexity. Entire

fields and industries have sprung up as a result. This development is still ongoing, and

it is creating new opportunities as hardware, software, and algorithms keep improving.

Ultimately the enabling technology for this movement is the powerful computing

hardware that has been developed in recent decades. However, for a computational

practitioner, the software environment used for computational work is as important as, if

not more important than, the hardware on which the computations are carried out. This

book is about one popular and fast-growing environment for numerical computing: the

Python programming language and its vibrant ecosystem of libraries and extensions for

computational work.

Computing is an interdisciplinary activity that requires experience and expertise

in both theoretical and practical subjects: a firm understanding of mathematics and

scientific thinking is a fundamental requirement for effective computational work.

Equally important is solid training in computer programming and computer science.

The role of this book is to bridge these two subjects by introducing how scientific

computing can be done using the Python programming language and the computing

environment that has appeared around this language. In this book the reader is assumed

to have some previous training in mathematics and numerical methods and basic

knowledge about Python programming. The focus of the book is to give a practical

introduction to computational problem-solving with Python. Brief introductions to the

theory of the covered topics are given in each chapter, to introduce notation and remind

readers of the basic methods and algorithms. However, this book is not a self-consistent

treatment of numerical methods. To assist readers that are not previously familiar with

some of the topics of this book, references for further reading are given at the end of each

chapter. Likewise, readers without experience in Python programming will probably find

it useful to read this book together with a book that focuses on the Python programming

language itself.

xxii

How This Book Is OrganizedThe first chapter in this book introduces general principles for scientific computing

and the main development environments that are available for work with computing in

Python: the focus is on IPython and its interactive Python prompt, the excellent Jupyter

Notebook application, and the Spyder IDE.

In Chapter 2, an introduction to the NumPy library is given, and here we also

discuss more generally array-based computing and its virtues. In Chapter 3, we turn our

attention to symbolic computing – which in many respects complements array-based

computing – using the SymPy library. In Chapter 4, we cover plotting and visualization

using the Matplotlib library. Together, Chapters 2 to 4 provide the basic computational

tools that will be used for domain-specific problems throughout the rest of the book:

numerics, symbolics, and visualization.

In Chapter 5, the topic of study is equation solving, which we explore with both

numerical and symbolic methods, using the SciPy and SymPy libraries. In Chapter 6, we

explore optimization, which is a natural extension of equation solving. Here we mainly

work with the SciPy library and briefly with the cvxopt library. Chapter 7 deals with

interpolation, which is another basic mathematical method with many applications of

its own, and important roles in higher-level algorithms and methods. In Chapter 8, we

cover numerical and symbolic integration. Chapters 5 to 8 cover core computational

techniques that are pervasive in all types of computational work. Most of the methods

from these chapters are found in the SciPy library.

In Chapter 9, we proceed to cover ordinary differential equations. Chapter 10 is

a detour into sparse matrices and graph methods, which helps prepare the field for

the following chapter. In Chapter 11, we discuss partial differential equations, which

conceptually are closely related to ordinary differential equations, but require a different

set of techniques that necessitates the introduction of sparse matrices, the topic of

Chapter 10.

Starting with Chapter 12, we make a change of direction and begin exploring data

analysis and statistics. In Chapter 12, we introduce the Pandas library and its excellent

data analysis framework. In Chapter 13, we cover basic statistical analysis and methods

from the SciPy stats package. In Chapter 14, we move on to statistical modeling,

using the statsmodels library. In Chapter 15, the theme of statistics and data analysis

is continued with a discussion of machine learning, using the scikit-learn library. In

Chapter 16, we wrap up the statistics-related chapters with a discussion of Bayesian

statistics and the PyMC library. Together, Chapters 12 to 16 provide an introduction to

inTRoduCTion

https://doi.org/10.1007/978-1-4842-4246-9_2

https://doi.org/10.1007/978-1-4842-4246-9_3

https://doi.org/10.1007/978-1-4842-4246-9_4

https://doi.org/10.1007/978-1-4842-4246-9_2

https://doi.org/10.1007/978-1-4842-4246-9_4

https://doi.org/10.1007/978-1-4842-4246-9_5

https://doi.org/10.1007/978-1-4842-4246-9_6

https://doi.org/10.1007/978-1-4842-4246-9_7

https://doi.org/10.1007/978-1-4842-4246-9_8

https://doi.org/10.1007/978-1-4842-4246-9_5

https://doi.org/10.1007/978-1-4842-4246-9_8

https://doi.org/10.1007/978-1-4842-4246-9_9

https://doi.org/10.1007/978-1-4842-4246-9_10

https://doi.org/10.1007/978-1-4842-4246-9_11

https://doi.org/10.1007/978-1-4842-4246-9_10

https://doi.org/10.1007/978-1-4842-4246-9_12

https://doi.org/10.1007/978-1-4842-4246-9_12

https://doi.org/10.1007/978-1-4842-4246-9_13

https://doi.org/10.1007/978-1-4842-4246-9_14

https://doi.org/10.1007/978-1-4842-4246-9_15

https://doi.org/10.1007/978-1-4842-4246-9_16

https://doi.org/10.1007/978-1-4842-4246-9_12

https://doi.org/10.1007/978-1-4842-4246-9_16

xxiii

the broad field of statistics and data analytics: a field that has been developing rapidly

within and outside of the scientific Python community in recent years.

In Chapter 17, we briefly return to a core subject in scientific computing: signal

processing. In Chapter 18, we discuss data input and output, and several methods for

reading and writing numerical data to files, which is a basic topic that is required for

most types of computational work. In Chapter 19, the final regular chapter in this book,

two methods for speeding up Python code are introduced, using the Numba and Cython

libraries.

The Appendix covers the installation of the software used in this book. To install

the required software (mostly Python libraries), we use the conda package manager.

Conda can also be used to create virtual and isolated Python environments, which is an

important topic for creating stable and reproducible computational environments. The

Appendix also discusses how to work with such environments using the conda package

manager.

Source Code ListingsEach chapter in this book has an accompanying Jupyter Notebook that contains the

chapter’s source code listings. These notebooks, and the data files required to run them,

can be downloaded by clicking the Download Source Code button located at

www.apress.com/9781484242452.

inTRoduCTion

https://doi.org/10.1007/978-1-4842-4246-9_17

https://doi.org/10.1007/978-1-4842-4246-9_18

https://doi.org/10.1007/978-1-4842-4246-9_19

http://www.apress.com/9781484242452

Scientific Computing and Data Science Applications with ...978-1-4842-4246-9/1.pdf · Numerical...

Documents

Transcript of Scientific Computing and Data Science Applications with ...978-1-4842-4246-9/1.pdf · Numerical...