How do Python Programmers Use Python? - PyCon Sweden › 2014 › assets › slides ›...

How do Python Programmers Use Python?Python Dynamicity & Other Ideas

Beatrice Åkerblom

Department of Computer and Systems SciencesStockholm University [email protected]

Saturday, May 31, 14

mailto:[email protected]:[email protected]

2

Statically typed languagesDynamically typed languages

Lawful evil

Lawful good

Chaotic good

Chaotic evil

Dyn

amic

pro

pone

nt’s

vi

ewSta

tic

prop

onen

t’s

view


“Historically” in Language Research

• Type inference (Smalltalk, Various Python projects, Diamondback Ruby)• Gradual typing (e.g. Siek, Taha)• Soft typing (e.g. Fagan)• Pluggable types (e.g. Bracha)

• Generally tries to make dynamic languages more “controllable” and predictable, that is static

• Assumptions are made about how programs are developed

3


Approaches used before

Selected examples:• “Usually, no further properties are defined

after the initialization and the type of the properties rarely changes.” -- Peter Thiemann

• “Giving people a dynamically-typed language does not mean that they write dynamically-typed programs” -- John Aycock

• “Yet while the presence of such abundant dynamism makes traditional static optimization impossible, in most programs, there is surprisingly little dynamism present.” -- Michael Salib

4


Approaches used before

Selected examples:• “Usually, no further properties are defined

after the initialization and the type of the properties rarely changes.” -- Peter Thiemann

• “Giving people a dynamically-typed language does not mean that they write dynamically-typed programs” -- John Aycock

• “Yet while the presence of such abundant dynamism makes traditional static optimization impossible, in most programs, there is surprisingly little dynamism present.” -- Michael Salib

5

True?We don’t know


When/Where, How & Why (if at all) is the dynamic power of dynamic

languages used inreal applications?

6


What’s Dynamic?

7

• Dynamic features - use of introspection, reflection, dynamic code evaluation

• Duck typing - polymorphism without need for inheritance or declared interfaces

• Dynamic objects - how dynamic are class and object structures





8

What’s Dynamic?

what is the program?do our objects reflect the class definitions?how dynamic are variable accesses, etc?how common is dynamic code generation?


9

do variables change type?will different paths lead to different types? how polymorphic are method calls?can common supertypes be found?

What’s Dynamic?





10

how stable is the OO (objects, classes, inheritance structures) of Python programs?do we find interface-like structures in Python programs?

What’s Dynamic?





Why is this important?

We’ll be able to:

• know how much of a “typical” Python program could (or could not) be annotated with types

• know how well Python source code does represent the running program

• know to what extent we need to support dynamic behaviour e.g. when building tools or new language constructs for Python

• emphasize the focus on how Python is used when designing new constructs

11


• Programs (Quantitative)– Static analysis (what is the program?)– Dynamic analysis: Measure behaviour at runtime, e.g. use of language

constructs, inheritance hierarchies, polymorphic call sites, etc.

• Code snippets (Qualitative)– Search for language constructs usage patterns– Read to understand how/why

• Programmers (Sociological)– Interview– Observe

12

Different Sources, different methods


What Have We Done?

• Modified the Python 2.6 interpreter to log information about running programs – class creation– method and function calls– instance member access– use of dynamic features

• Python programs selected from Source Forge• Programs run on a Debian machine

– interactive– tests– examples

• Program runs documented – tests– recordings– use cases 13


Dynamic Features in Python Programs

• Anomos, Bleachbit, Comix, ConvertAll, Exaile, Kodos, Mcomix, Pysolfc, Rednotebook, Retext, Sbackup, Solfege, Task coach, Torrent Search, Wikidpad, Zmail

• hasattr, eval, reload, getattr, __delattr__, __getattr__, execfile, __getattribute__, del attribute, __import__, exec, setattr, vars, __setattr__, delattr

0: Id-nummer1: the path, filename and row number from which the call was made, 2: Caller id.3: Caller type.4: Target Id5: Target type6: Feature name7: Argument types8: Results

14


What about Holkner & Harland’s “Evaluating the dynamic behaviour of

Python applications”?

15


Number of Features Used by Programs

16


Distribution of Dynamism for All Traces

17

Run-time Startup

Program 16% Program 15%

Library 29% Library 40%

• Distribution of dynamic features over libraries and program-specific code during start-up and run-time


Number of Programs Where Features Were Traced

18


Median, Average, Minimum and Maximum for All Features

19


Polymorphism in Python Programs

• Task Coach, SciPy, Pootle, Virtaal & the Translate Toolkit, PhotoFilmStrip, Brain Workshop, Eric4, PyMol, Childsplay, GNU Solfege, WikidPad, BleachBit, Mnemosyne, RedNotebook, DispcalGUI, Scikit Learn, Python parsing module, PDF-Shuffler, Link checker, Mcomix, Python megawidgets, Autocomplete for Notepad++, PyTruss, Idle, Radiotray, PyX, TorrentSearch, Diffuse, Timeline, GImageReader, PySolFC, PyPe, Requests, Youtube-dl, Docutils, Pychecker

0. Event ID1. Source file path2. Caller ID (current this at the call-site)3. Caller type4. Target ID (the receiver of the method call)5. Class name of target + : + class id 6. Name of called function/method7. Argument types8. Call line 9. A list of all super classes of the target type 20


Questions Asked

• How many unique call-sites?• How many call-sites are monomorphic?

– Trivially monomorphic vs. monomorphic• How many polymorphic call-sites?• Distribution of the degree of polymorphism seen

• For call-sites that saw several different types as receiver, what were the types and do they share a common supertype containing the method called?

21


Monomorphic Call Sites

• Trivially monomorphic: We have only recorded one single execution of this call site

• Monomorphic: We have recorded more than one execution of this call site, and the types seen were always the same

22

Polymorphic 4% Trivially mono 51%

Truly mono 45%


How Polymorphic are Python Call Sites?

23

Programs listed ordered by size from smallest to largestSaturday, May 31, 14

How Polymorphic are Python Call Sites?

24


25


References

• Phillip Heidegger and Peter Thiemann, “Recency types for analyzing scripting languages”, ECOOP 2010.

• John Aycock, “Aggressive Type Inference”, In Proceedings of the 8th International Python Conference, 2000.

• Michael Salib, “Faster than C: Static type inference with Starkiller”, In PyCon Proceedings, 2004.

• A. Holkner and J. Harland, “Evaluating the Dynamic Behaviour of Python Applications”, Proceedings of ACSC ’09, 2009.

• Beatrice Åkerblom and Tobias Wrigstad, “Tracing Dynamic Features in Python Programs”, Proceedings the 11th Working Conference on Mining Software Repositories, 2014.

26

http://wrigstad.com/http://wrigstad.com/

How do Python Programmers Use Python? - PyCon Sweden › 2014 › assets › slides ›...

Documents

Transcript of How do Python Programmers Use Python? - PyCon Sweden › 2014 › assets › slides ›...