Python and pandas, oh my:a rich ecosystem of open source

tools for petrophysical and geological assessment

Dan HallauThe Discovery Group, Inc.Denver, CO

Disclaimer / Notes

• I am not a computer scientist!

• Just an enthusiast who has learned the python programming language in his spare time and now uses it every day.

11/3/2016 www.discovery-group.com 2

Outline

• What is python?• What can you do with python in

geoscience/petrophysics?

11/3/2016 3www.discovery-group.com

Python

• Interpreted programming language

11/3/2016 www.discovery-group.com 4

Python

• Interpreted programming language

11/3/2016 www.discovery-group.com 5

Python

• Interpreted programming language

11/3/2016 www.discovery-group.com 6

• “Batteries included”– Lots of

excellent tools in the standard library

11/3/2016 www.discovery-group.com 7

Python cont’d

• “Batteries included”– Lots of

excellent tools in the standard library

11/3/2016 www.discovery-group.com 8

Python cont’d

• “Batteries included”– Lots of

excellent tools in the standard library

11/3/2016 www.discovery-group.com 9

Python cont’d

• “Batteries included”– Lots of

excellent tools in the standard library

11/3/2016 www.discovery-group.com 10

Python cont’d

• Open source– You are free to

study the source code, modify it, distribute it

11/3/2016 www.discovery-group.com 11

Python cont’d

• Cross-platform

11/3/2016 www.discovery-group.com 12

Python cont’d

11/3/2016 www.discovery-group.com 13

Python cont’d

• Also an excellent ecosystem!• Dead simple to install external

libraries$ pip install pandas

$ python>>> import pandas>>>

11/3/2016 www.discovery-group.com 14

Python cont’d

• Also an excellent ecosystem!• Dead simple to install external

libraries$ pip install pandas

$ python>>> import pandas>>>

https://xkcd.com/353/https://xkcd.com/353/https://xkcd.com/353/

Example project and workflow

• Lance Formation– Stack of undifferentiated fluvial sand

bodies• Notoriously difficult to correlate

• Wanted to compare sandstone and reservoir quality off- vs. on-structure

• I won’t be showing results from the study so much as I’ll be showing the python tools we used

11/3/2016 www.discovery-group.com 15

Tools/libraries used in this example workflow

11/3/2016 www.discovery-group.com 16

pandas.pydata.org

numpy.org

github.com/kinverarity1/lasio

jupyter.org

matplotlib.org

Jupyter notebook

11/3/2016 www.discovery-group.com 17

Jupyter notebook

11/3/2016 www.discovery-group.com 18

Jupyter notebook

11/3/2016 www.discovery-group.com 19

Workflow

• Pick tops, export LAS data• Read all LAS files, concatenate into a

single dataframe• Compute basic petrophysics• Create zones• Run statistics on zones, groupby

various selectors• Create visualizations11/3/2016 www.discovery-group.com 20

Petra

Python

Loading LAS files

11/3/2016 www.discovery-group.com 21

Create zones

www.discovery-group.com 22

Design petrophysical functions

11/3/2016 www.discovery-group.com 23

functions

11/3/2016 www.discovery-group.com 24

describe

11/3/2016 www.discovery-group.com 25

groupbys

11/3/2016 www.discovery-group.com 26

groupbys cont’d

11/3/2016 www.discovery-group.com 27

Plotting

11/3/2016 www.discovery-group.com 28

More plots

11/3/2016 www.discovery-group.com 29

More plots

11/3/2016 www.discovery-group.com 30

Ternary plots

11/3/2016 www.discovery-group.com 31

github.com/marcharper/python-ternary

• You don’t need an expensive specialized software package to get excellent, publication-quality results

Ternary plot example

11/3/2016 www.discovery-group.com 32

Machine learning• Domain of computer science / statistics

where computers “learn” using large datasets

• e.g. exploratory data analysis through hierarchical clustering. Plot data with respect to various selectors, cluster the data based on similarities.

• Could then create a classifier. If log data is being used:

11/3/2016 www.discovery-group.com 33

Machine learning

11/3/2016 www.discovery-group.com 34

scikit-learn.org/stable/

caffe.berkeleyvision.org/

tensorflow.org/versions/r0.10/api_docs/python/index.html

Other python uses

• Endless possibilities– Automation–Web-scraping– File system manipulations• Multi-rename files• Create archives

– Sending emails, text messages

11/3/2016 www.discovery-group.com 35