IPython: moderntoolsforinteractive&web-enabled ......(Incomplete)CastofCharacters...
Transcript of IPython: moderntoolsforinteractive&web-enabled ......(Incomplete)CastofCharacters...
![Page 1: IPython: moderntoolsforinteractive&web-enabled ......(Incomplete)CastofCharacters BrianGranger-Physics,CalStateSanLuisObispo MinRagan-Kelley-NuclearEngineering,UCBerkeley MatthiasBussonnier-Physics,InstitutCurie,Paris](https://reader033.fdocuments.in/reader033/viewer/2022053119/609f7072b140510856612775/html5/thumbnails/1.jpg)
SciPy IPython
IPython: modern tools for interactive & web-enabledscientific computing
Fernando Pérezhttp://fperez.org, @fperez_org
Helen Wills Neuroscience Institute, UC Berkeley
NERSC User Day, LBL, Berkeley,Feb 13, 2013
![Page 2: IPython: moderntoolsforinteractive&web-enabled ......(Incomplete)CastofCharacters BrianGranger-Physics,CalStateSanLuisObispo MinRagan-Kelley-NuclearEngineering,UCBerkeley MatthiasBussonnier-Physics,InstitutCurie,Paris](https://reader033.fdocuments.in/reader033/viewer/2022053119/609f7072b140510856612775/html5/thumbnails/2.jpg)
SciPy IPython
Outline
1 Scientific Python
2 IPython: Interactive Python
FP (UC Berkeley) IPython 2/13/13 2 / 21
![Page 3: IPython: moderntoolsforinteractive&web-enabled ......(Incomplete)CastofCharacters BrianGranger-Physics,CalStateSanLuisObispo MinRagan-Kelley-NuclearEngineering,UCBerkeley MatthiasBussonnier-Physics,InstitutCurie,Paris](https://reader033.fdocuments.in/reader033/viewer/2022053119/609f7072b140510856612775/html5/thumbnails/3.jpg)
Beyond (Floating Point) Number Crunching
Hardwarefloating point
Arbitrary precisionintegers
Rationals
Interval arithmetic
Symbolic manipulation
FORTRAN
Extended precisionfloating point
Text processing
Databases
Graphical userinterfaces
Web interfacesHardwarecontrol
Multi-languageintegration
Data formats: HDF5, XML, ...
![Page 4: IPython: moderntoolsforinteractive&web-enabled ......(Incomplete)CastofCharacters BrianGranger-Physics,CalStateSanLuisObispo MinRagan-Kelley-NuclearEngineering,UCBerkeley MatthiasBussonnier-Physics,InstitutCurie,Paris](https://reader033.fdocuments.in/reader033/viewer/2022053119/609f7072b140510856612775/html5/thumbnails/4.jpg)
Scientific Python: a Rich Ecosystem
IPython
NetworkX
![Page 5: IPython: moderntoolsforinteractive&web-enabled ......(Incomplete)CastofCharacters BrianGranger-Physics,CalStateSanLuisObispo MinRagan-Kelley-NuclearEngineering,UCBerkeley MatthiasBussonnier-Physics,InstitutCurie,Paris](https://reader033.fdocuments.in/reader033/viewer/2022053119/609f7072b140510856612775/html5/thumbnails/5.jpg)
SciPy IPython
Outline
1 Scientific Python
2 IPython: Interactive Python
FP (UC Berkeley) IPython 2/13/13 5 / 21
![Page 6: IPython: moderntoolsforinteractive&web-enabled ......(Incomplete)CastofCharacters BrianGranger-Physics,CalStateSanLuisObispo MinRagan-Kelley-NuclearEngineering,UCBerkeley MatthiasBussonnier-Physics,InstitutCurie,Paris](https://reader033.fdocuments.in/reader033/viewer/2022053119/609f7072b140510856612775/html5/thumbnails/6.jpg)
SciPy IPython
The Lifecycle of a Scientific Idea (schematically)
1 Individual exploratory work2 Collaborative development3 Production work (HPC, cloud, parallel)4 Publication (with reproducible results!)5 Education6 Goto 1.
The Problem with most toolsBarriers and discontinuities in workflow in between all the steps
FP (UC Berkeley) IPython 2/13/13 6 / 21
![Page 7: IPython: moderntoolsforinteractive&web-enabled ......(Incomplete)CastofCharacters BrianGranger-Physics,CalStateSanLuisObispo MinRagan-Kelley-NuclearEngineering,UCBerkeley MatthiasBussonnier-Physics,InstitutCurie,Paris](https://reader033.fdocuments.in/reader033/viewer/2022053119/609f7072b140510856612775/html5/thumbnails/7.jpg)
SciPy IPython
The Lifecycle of a Scientific Idea (schematically)
1 Individual exploratory work2 Collaborative development3 Production work (HPC, cloud, parallel)4 Publication (with reproducible results!)5 Education6 Goto 1.
The Problem with most toolsBarriers and discontinuities in workflow in between all the steps
FP (UC Berkeley) IPython 2/13/13 6 / 21
![Page 8: IPython: moderntoolsforinteractive&web-enabled ......(Incomplete)CastofCharacters BrianGranger-Physics,CalStateSanLuisObispo MinRagan-Kelley-NuclearEngineering,UCBerkeley MatthiasBussonnier-Physics,InstitutCurie,Paris](https://reader033.fdocuments.in/reader033/viewer/2022053119/609f7072b140510856612775/html5/thumbnails/8.jpg)
IPython’s goal:Fluid transitions in all these steps
![Page 9: IPython: moderntoolsforinteractive&web-enabled ......(Incomplete)CastofCharacters BrianGranger-Physics,CalStateSanLuisObispo MinRagan-Kelley-NuclearEngineering,UCBerkeley MatthiasBussonnier-Physics,InstitutCurie,Paris](https://reader033.fdocuments.in/reader033/viewer/2022053119/609f7072b140510856612775/html5/thumbnails/9.jpg)
Demo
![Page 10: IPython: moderntoolsforinteractive&web-enabled ......(Incomplete)CastofCharacters BrianGranger-Physics,CalStateSanLuisObispo MinRagan-Kelley-NuclearEngineering,UCBerkeley MatthiasBussonnier-Physics,InstitutCurie,Paris](https://reader033.fdocuments.in/reader033/viewer/2022053119/609f7072b140510856612775/html5/thumbnails/10.jpg)
SciPy IPython
Pillar #1: An architecture for interactive computing
FP (UC Berkeley) IPython 2/13/13 9 / 21
![Page 11: IPython: moderntoolsforinteractive&web-enabled ......(Incomplete)CastofCharacters BrianGranger-Physics,CalStateSanLuisObispo MinRagan-Kelley-NuclearEngineering,UCBerkeley MatthiasBussonnier-Physics,InstitutCurie,Paris](https://reader033.fdocuments.in/reader033/viewer/2022053119/609f7072b140510856612775/html5/thumbnails/11.jpg)
SciPy IPython
Pillar #2: the Notebook Format
JSON but version control-friendlyEasy for machine processing, fixable by hand if need be.Lots of hooks for metadataNot Python-specific (R and Ruby notebooks exist, Julia planned)Produce Markdown, reST, LATEX, HTML, etc...
An open format for sharing, publishing andarchiving executable computational work
FP (UC Berkeley) IPython 2/13/13 10 / 21
![Page 12: IPython: moderntoolsforinteractive&web-enabled ......(Incomplete)CastofCharacters BrianGranger-Physics,CalStateSanLuisObispo MinRagan-Kelley-NuclearEngineering,UCBerkeley MatthiasBussonnier-Physics,InstitutCurie,Paris](https://reader033.fdocuments.in/reader033/viewer/2022053119/609f7072b140510856612775/html5/thumbnails/12.jpg)
Documented protocols and formats:a growing ecosystem around IPython
![Page 13: IPython: moderntoolsforinteractive&web-enabled ......(Incomplete)CastofCharacters BrianGranger-Physics,CalStateSanLuisObispo MinRagan-Kelley-NuclearEngineering,UCBerkeley MatthiasBussonnier-Physics,InstitutCurie,Paris](https://reader033.fdocuments.in/reader033/viewer/2022053119/609f7072b140510856612775/html5/thumbnails/13.jpg)
Microsoft Visual Studio 2010 integrated consoleDino Viehland and Shahrokh Mortazavi (Microsoft); http://pytools.codeplex.com
![Page 14: IPython: moderntoolsforinteractive&web-enabled ......(Incomplete)CastofCharacters BrianGranger-Physics,CalStateSanLuisObispo MinRagan-Kelley-NuclearEngineering,UCBerkeley MatthiasBussonnier-Physics,InstitutCurie,Paris](https://reader033.fdocuments.in/reader033/viewer/2022053119/609f7072b140510856612775/html5/thumbnails/14.jpg)
A vim client to control an IPython kernel/consolePaul Ivanov (Berkeley), https://github.com/ivanov/vim-ipython
![Page 15: IPython: moderntoolsforinteractive&web-enabled ......(Incomplete)CastofCharacters BrianGranger-Physics,CalStateSanLuisObispo MinRagan-Kelley-NuclearEngineering,UCBerkeley MatthiasBussonnier-Physics,InstitutCurie,Paris](https://reader033.fdocuments.in/reader033/viewer/2022053119/609f7072b140510856612775/html5/thumbnails/15.jpg)
Notebooks on Windows Azure CloudShahrokh Mortazavi (Microsoft), B.G., F.P.: http://bit.ly/JQeojD.
![Page 16: IPython: moderntoolsforinteractive&web-enabled ......(Incomplete)CastofCharacters BrianGranger-Physics,CalStateSanLuisObispo MinRagan-Kelley-NuclearEngineering,UCBerkeley MatthiasBussonnier-Physics,InstitutCurie,Paris](https://reader033.fdocuments.in/reader033/viewer/2022053119/609f7072b140510856612775/html5/thumbnails/16.jpg)
Star Cluster: IPython parallel+Notebook on Amazon EC2Justin Riley (MIT): http://web.mit.edu/star/cluster
![Page 17: IPython: moderntoolsforinteractive&web-enabled ......(Incomplete)CastofCharacters BrianGranger-Physics,CalStateSanLuisObispo MinRagan-Kelley-NuclearEngineering,UCBerkeley MatthiasBussonnier-Physics,InstitutCurie,Paris](https://reader033.fdocuments.in/reader033/viewer/2022053119/609f7072b140510856612775/html5/thumbnails/17.jpg)
One-click single notebook on Amazon EC2Carl Smith (UK): https://notebookcloud.appspot.com.
![Page 18: IPython: moderntoolsforinteractive&web-enabled ......(Incomplete)CastofCharacters BrianGranger-Physics,CalStateSanLuisObispo MinRagan-Kelley-NuclearEngineering,UCBerkeley MatthiasBussonnier-Physics,InstitutCurie,Paris](https://reader033.fdocuments.in/reader033/viewer/2022053119/609f7072b140510856612775/html5/thumbnails/18.jpg)
Other projects using IPython
ScientificSoftware CarpentryEPD: Enthought Python Distribution.Continuum: Anaconda, Wakari.Sage: open source mathematics.PyRAF: Space Telescope Science InstituteCASA: Nat. Radio Astronomy ObservatoryGanga: CERNPyMAD: neutron spectrom., Laue LangevinSardana: European Synchrotron RadiationASCEND: eng. modeling (Carnegie Mellon).JModelica: dynamical systems.DASH: Denver Aerosol Sources and Health.Trilinos: Sandia National Lab.DoD: baseline configuration.Mayavi: 3d visualization, Enthought.NiPype: computational pipelines, MIT.
...
Web/OtherVisual Studio 2010: MS.Django.Turbo Gears.Pylons web frameworkZope and Plone CMS.Axon Shell, BBCKamaelia.Schevo database.Pitz: distributedtask/bug tracking.iVR (interactive VirtualReality).Movable Python(portable Pythonenvironment)....
![Page 19: IPython: moderntoolsforinteractive&web-enabled ......(Incomplete)CastofCharacters BrianGranger-Physics,CalStateSanLuisObispo MinRagan-Kelley-NuclearEngineering,UCBerkeley MatthiasBussonnier-Physics,InstitutCurie,Paris](https://reader033.fdocuments.in/reader033/viewer/2022053119/609f7072b140510856612775/html5/thumbnails/19.jpg)
(Incomplete) Cast of CharactersBrian Granger - Physics, Cal State San Luis ObispoMin Ragan-Kelley - Nuclear Engineering, UC BerkeleyMatthias Bussonnier - Physics, Institut Curie, ParisBrad Froehle - Mathematics, UC BerkeleyPaul Ivanov - Neuroscience, UC Berkeley.Robert Kern - EnthoughtThomas Kluyver - Biology, U. SheffieldJonathan March- EnthoughtEvan Patterson - Physics, Caltech/EnthoughtJörgen Stenarson - Elect. Engineering, Sweden.John Hunter - TradeLink Securities, Chicago.Stefan van der Walt - Applied Math, U. StellenboschPrabhu Ramachandran - Aerospace Engineering, IIT Bombay.Satra Ghosh- MIT NeuroscienceGaël Varoquaux - Neurospin (Orsay, France)Ville Vainio - CS, Tampere University of Technology, FinlandBarry Wark - Neuroscience, U. Washington.Ondrej Certik - Physics, U Nevada RenoDarren Dale - CornellJustin Riley - MITMark Voorhies - UC San FranciscoNicholas Rougier - INRIA Nancy Grand EstThomas Spura - Fedora projectMany more! (~150 commit authors)
![Page 20: IPython: moderntoolsforinteractive&web-enabled ......(Incomplete)CastofCharacters BrianGranger-Physics,CalStateSanLuisObispo MinRagan-Kelley-NuclearEngineering,UCBerkeley MatthiasBussonnier-Physics,InstitutCurie,Paris](https://reader033.fdocuments.in/reader033/viewer/2022053119/609f7072b140510856612775/html5/thumbnails/20.jpg)
SupportThank you!
Enthought, Austin, TX: Lots!Microsoft: WinHPC support, Visual Studio integration, Azure(thanks to Shahrokh Mortazavi).DoD/DRC Inc: funding through Sept. 2012 (thanks to JoseUnpingco and Chris Keees).NIH: via NiPy grantNSF: via Sage compmath grantGoogle: summer of code 2005, 2010.Tech-X Corp., Boulder, CO: Parallel/notebook (previous versions)
![Page 21: IPython: moderntoolsforinteractive&web-enabled ......(Incomplete)CastofCharacters BrianGranger-Physics,CalStateSanLuisObispo MinRagan-Kelley-NuclearEngineering,UCBerkeley MatthiasBussonnier-Physics,InstitutCurie,Paris](https://reader033.fdocuments.in/reader033/viewer/2022053119/609f7072b140510856612775/html5/thumbnails/21.jpg)
Recent Stable Funding
2-year funding, core team (7 people)
Stanford: reproducibility & data sharing(neuroimaging, other data-intensive cases)
![Page 22: IPython: moderntoolsforinteractive&web-enabled ......(Incomplete)CastofCharacters BrianGranger-Physics,CalStateSanLuisObispo MinRagan-Kelley-NuclearEngineering,UCBerkeley MatthiasBussonnier-Physics,InstitutCurie,Paris](https://reader033.fdocuments.in/reader033/viewer/2022053119/609f7072b140510856612775/html5/thumbnails/22.jpg)
NumFOCUS: http://numfocus.orgFoundation to promote Open Source Scientific Computing
Support the development of multiple projects.Community-created and driven.A neutral ground for industry, academia and government to supportscientific open source.501(c)3 - donations are tax-exempt in the USA