Post on 12-Mar-2018
Rich Macfarlane, 2015
python Introduction
Python Teaching Team P
yth
on
In
tro
du
cti
on
Rich Macfarlane – Lecturer in CDCS Email: r.macfarlane@napier.ac.uk Twitter: @rjmacfarlane
http://www.iidi.napier.ac.uk/c/people/peopleid/13277237
Owen Lo – Researcher/Lecturer in CDCS Email: o.lo@napier.ac.uk
http://www.iidi.napier.ac.uk/c/people/peopleid/13280413
Python Schedule P
yth
on
In
tro
du
cti
on
Session Python Lectures/Labs
1 Scripting Languages/Python Overview, Environment,
Modules/Scripts, Flow Control, Variables, Numbers/Maths, Strings,
Lab: Python Cryptanalysis - Keyspace
2 Lists Collections, Code Development, Script Arguments, Exception
Handling, Lab: Python Hashing/Password Cracking
3 Collection Data Types: Tuples, Dictionaries, External Data - Files,
Exception Handling, Lab: Python Forensics Intro
3 Text manipulation, Regular Expressions, System Programming, More
External Data – Files/Web, Lab: Python Encryption
Python Schedule P
yth
on
In
tro
du
cti
on
• 4 Taught Lectures + 4 Practical Labs • Lectures - mix of talking about the language/theory +
some demoing of syntax/Python examples • Labs – examples of the theory from lectures + build
approx. 2 Python scripts per lab • Scripts used to build coursework • Must complete labs! – 4+ hours/wk
• Labs pretty detailed, so don’t need to take many notes
• Use Lecture notes + Moodle resources as reference in labs
• Can learn a lot of python in 4 weeks!
Rich Macfarlane
python scripting languages
What are Scripting Languages? Types
Scripting Languages What? P
yth
on
In
tro
du
cti
on
• What is a scripting language?
• High level language – higher than C#, Java
• Used to write ‘scripts’
• Scripts automate tasks such as command line OS programs, or web page creation
• Typically interpreted and not compiled
• Smaller programs – hundreds of lines max
• Dynamic Typing – Rapid development
Scripting Languages Types? P
yth
on
In
tro
du
cti
on
• OS Job control – JCL, Unix Shell Scripting, Bash
• Unix/Linux shell scripting – glue programs together: ps –aux | grep “snort” | more
• Text Processing – AWK, sed - Perl (started as)
• Web browser control – JavaScript • Scripts embedded in HTML – provide dynamic content
• Web pages - PHP
• Developed into application programming languages – Perl, Python, Ruby • Now Modular and Extensible, OO
Rich Macfarlane
python overview
Python History Why use Python? What is Python used for Who uses Python
Python History P
yth
on
In
tro
du
cti
on
• Python created in 1989 by Guido van Rossum
• Worked at Google in App Engine Team
• Now works for Dropbox – client/server written in Python
• Old language – has become popular more recently
• Versions:
• 2.x series 2000
• 3.x series 2008 • Not backward compatible
• 3.x is the future
• 2.x has most support now
• Use 2.7(.8) on this module Google > Dropbox
Python What? P
yth
on
In
tro
du
cti
on
• Powerful/Light High Level Language
• Scripting/general purpose programming language
• Between traditional scripting languages (TCL, Perl) and High Level Application Development Languages (C++, C#, Java)
• Similar to Perl, Ruby
• Top 5 most widely used
• Named after Monty Pythons
Flying Circus :D
Python Why? P
yth
on
In
tro
du
cti
on
Why are we learning Python? • Produces Good Quality Code
• Readable/consistent/minimal code compared to other scripting languages – more readable than Perl • Easy to learn and use – easier than C, C++ ,C#, Java
• Productivity • Rapid Prototyping/Proof of concept - hacking Take longer in other languages C,C++,Java • Less code to type/no compile stage
• Portability • Multi platform without any changes • Linux/Mac/Windows/Mobile
• Rich set of libraries/tools available • Standard Library/3rd party Libraries
• Can be used for wide range of application development
- Small scale applications
- OOP Can be used for strategic application development
Python Why? P
yth
on
In
tro
du
cti
on
Why are we learning Python? • Python Principles ‘easter egg’:
Python Platforms P
yth
on
In
tro
du
cti
on
Same code will run on: • Linux – Python installed on most
• Mac – pre installed
• Windows
• Mobile – Android/iOS
• Games Consoles
• Embedded Systems
Implementations: • Cpython – reference implementation – we will use
• IronPython – for scripting .NET and Mono
• Jpython – coded for Java Runtime
Python Features P
yth
on
In
tro
du
cti
on
• Interpreted Language • No compile phase/quick turnaround • Prototype code in Interpreter before adding to scripts
• Dynamically Typed • Typing at runtime - as apposed to static compile type typing:
• Strongly Typed • Dynamic type checking – reports misuse
• Automated Memory Management • Garbage collection
• Simplicity of Scripting – procedural • Object Oriented - optional • Language, with SE tools and libraries of high level compiled languages
Python Features P
yth
on
In
tro
du
cti
on
• Python Interactive Shell
Dynamic Typing – no declaration - quick
Strongly Typed
Dynamic Typing – now var is a integer
Python Applications P
yth
on
In
tro
du
cti
on
• Operating-Systems programming • OS Shell tools, scripting • Cross platform interfaces to OS
• Cyber Security and Forensics • Security –pen testing, exploit creation, automation of tools, text parsing,
log file analysis • Forensics – tools, scripting - digital media analysis
• Network scripting • Packet Sniffing, Traffic analysis – Scapy, pyNIDS
• Internet scripting • Client/server networking, XML, Fetch web pages, parse HTML • Web application development – django
• GUI programming - tkinter
• Database programming – cross db libraries
• Data Mining/Machine Learning, Robotics, Gaming • Numeric/Scientific Programming – NumPy, SciPy
Python Commercial User Base P
yth
on
In
tro
du
cti
on
• Intel, Cisco, HP
• Google, Yahoo
• YouTube
• Industrial Light & Magic
• BitTorrent
• NASA
• Rasberry Pi
• NSA
Python vs Perl P
yth
on
In
tro
du
cti
on
• ‘Conflict’ between Language Users
• Can do everything in Python that you can do in Perl
• Perl code shorter, quicker to code, but typically less readable, and harder to reuse
• Perl typically more ways to do same thing
• Python typically has only one(ish) way to do things
Python vs Perl P
yth
on
In
tro
du
cti
on
Rich Macfarlane
python Running Python Code
How Python is run How do we create Python code How do we run Python code
Running Python P
yth
on
In
tro
du
cti
on
Running Python Code P
yth
on
In
tro
du
cti
on
• 2 ways to run Python code:
1. Interactive Interpreter
Type Python Code at CLI >>> promt
2. Type Python
Type commands and they are evaluated/executed
Can type entire program into interpreter!
Type Python to run Interpreter
Python Interpreter prompt >>>
Running Python Code P
yth
on
In
tro
du
cti
on
1. Interactive Interpreter
Don’t have to use print at >>> Interpreter prompt - prints expressions
Use Interpreter for:
• Experimenting – executes code immediately, try out small examples of code, good error messages, can add code to script if works!
• Testing code in files – import code from a file and run to test. Test individual functions from code modules.
Running Python Code P
yth
on
In
tro
du
cti
on
2. Create Python Script File + Run through Interpreter 1. Create text file containing Python code; typically extension ‘.py’
Code file- Python statements.
(.py for importing)
2. Run ‘.py’ code file through the Python interpreter; It executes each
statement.
Use code files to reuse our code!
Running Python Code P
yth
on
In
tro
du
cti
on
GUI – IDLE
• Same 2 ways to run code:
• 1. Interactive Interpreter – “Python Shell”
• - Syntax Highlighting/Coloring – spot errors
• - Auto Completion with <TAB>
• - Code Statement History <ALT+P> <ALT+N>
• - function( and module. Popups
• - auto indent for code blocks
• - Debugger, Object Browser
Running Python Code P
yth
on
In
tro
du
cti
on
GUI – IDLE
2. Create .py Script/Module files & Run from IDLE File>New Window - opens editor - add code, save as .py file
Run>Run Module(F5) - runs .py file code in Interpreter window – easier and faster then import/reload()
Rich Macfarlane
python Syntax/Flow Control
Python statements Python differences to other languages What Python has removed! Python syntax for blocks of code Python if conditional statements
Python Code Syntax P
yth
on
In
tro
du
icti
on
• Python Statement • Line of Python code to do something – logic of program • Made up of Python Expressions
• Expressions create and manipulate Python Objects >>> if x == 5: print ‘Python is great’
• No Statement termination character • C# - statement termination character? • Python - End of line instead – don’t have to type!
>>> x = 5
>>> y = 2 Note: don’t have to use print in interpreter. ‘_’ prints last value
• \ can be used to continue lines
>>> z = x \
... + y
Assignment Statements Creates variable, and associated object
Python Code Syntax P
yth
on
In
tro
du
icti
on
• C# - what to group code statements into blocks?
• Python - Whitespace used to specify ‘blocks’ of code
def print_msg(msg):
print "hello " + msg
print "hello again " + msg
• Less cluttered than other languages
• Colon ‘:’ added to show code block/indentation expected
• You will forget this many times!!
• 4 spaces commonly used (2 – Google)
• DO NOT use mix of tabs and spaces!
Whitespace instead of { } Compound Statement - Embedded statements
Python Code Syntax P
yth
on
In
tro
du
icti
on
• Python - Whitespace used to specify ‘blocks’ of code
if True: if True:
print “true “ print “true “
print “do” print “do”
else: else:
print “false“ print “false“
print “do not“ print “do not“
print “more code” print “more code”
• Don’t have to type brackets round condition,
{} around code blocks, or ; at end of line
- Produces Runtime Error - Indentation must be correct - Unlike C style languages!
: is extra to type
Flow Control P
yth
on
In
tro
du
icti
on
• Boolean Data Type
Values: True False
b = True
b
True
b == False
False
Flow Control P
yth
on
In
tro
du
icti
on
• Comparison Operators
Compare 2 values and evaluate to True or False! == Equal to != Not Equal
< Less than > Greater than
<= Less than or equal to
>= Greater than or equal to
l = len(‘hack’)
l == 4
True
l >= 8
False
Boolean Expression Python Expression which equates to True or False
Flow Control - IF Statement P
yth
on
In
tro
du
icti
on
• Why do we need if statements? • Decides which code too run • Based on Boolean Tests being True or False
• IF Statement:
if <boolean test>:
<indented code block>
elif <boolean test>:
<indented code block> else:
<indented code block>
if speed >= 70: print ‘Driving License please' if mood == 'terrible' or speed >= 100: print 'You are in trouble!' elif mood == 'bad' or speed >= 90: print “You are getting a ticket" else: print "Let's try to keep the speed down sir"
Don’t forget the :
Multiple conditions and/or/not
spelt out
Indent With W/s
Multiple ifs Using elif (no switch/case)
Boolean Expression Python Expression which equates to True or False
Python Code Syntax P
yth
on
In
tro
du
icti
on
Comments • # used for single line comments
• Everything to end of line ignored
• End of the line = end of comment
#
# Script: Hello World
# Created: Oct 2013
#
print "Hello World" # Print the hello msg
""" triple quotes can sometimes be used for
multi-line comments also """
Rich Macfarlane
python scripts, modules & functions
Python Functions Python BIFs Python Modules Scripts and Libraries of Functions
Python Functions P
yth
on
In
tro
du
icti
on
• Reuse code using Python Functions Function Syntax: def <function_name>
• File containing functions called a Module
Function definition
Module
Function Doc String
Function code
Python Using Functions P
yth
on
In
tro
du
icti
on
Using Functions • To use a function, the following import syntax is used: from <module_name> import <function_name>
• Or typically use all code from a Module: import module_name
• On import all code in the .py file is run – function definitions are created.
• Functions can be run with module.function() syntax
• reload() can be used if changes are made to file
Module.Function
Module
Python Modules P
yth
on
In
tro
du
icti
on
• Modules are Python .py files containing code such as groups of related functions, and variables.
• Python program architecture – top level module, imports sub modules + built-in modules.
• Built-in Standard Library Modules: http://docs.python.org/library
>>> import os >>> os.getcwd()
'C:\\Python27‘
• Can also use External 3rd party Modules. • Create our own Modules on this ‘module’ … lab – create 2
modules per week, then use for coursework!
Python Built In Funtions (BIF) P
yth
on
In
tro
du
icti
on
• Python Standard Library • Python has many preloaded functions, which are
part of the Python Standard Library, and are available when the Python interpreter is run.
• BIFs are functions of the __builtin__ Module
http://docs.python.org/library/functions.html
__builtin__.
Python External Modules P
yth
on
In
tro
du
icti
on
• PyPI – The Python Package Index
• If the Standard Library does not contain functions which can help, there are many Python modules available on the web.
PyPI repository:
http://pypi.python.org/pypi
Python Module Types P
yth
on
In
tro
du
icti
on
• Modules are Python .py files containing reusable code.
Types:
• Script Modules
• Autonomous code which can be run standalone. Typically from command line with parameterised using arguments.
• Function Library Modules
• Variable and function definitions with can be imported and used by other code.
• Modules which are Both …
Python Modules P
yth
on
In
tro
du
icti
on
Mixed Use Python Modules
• Both a standalone Script or as a Library of Functions:
Boilerplate Code This code runs first! Calls main() if run as a script, and does not run any code if imported as a library of functions.
Special main() Function Can be used to test library of functions or Call Module code functions as a script (c/w)
Python Modules P
yth
on
In
tro
du
icti
on
Mixed Use Python Modules
• Standalone Script :
Run > Run Module (F5)
Python Modules P
yth
on
In
tro
du
icti
on
Mixed Use Python Modules
• As a Library of Functions:
Python Help on Modules P
yth
on
In
tro
du
cti
on
Python Modules Documentation
• Get help on a Module, what it does, what functions it contains
• Tech Method: From Interpreter use BIFs: • help(object/module)
• help(module.function)
Shows help on object/funtion • dir(object/module)
Shows the objects methods.
OR
• Easy way: Google Search for ‘Python Module/Function’
Typically bring back official Python doc first! python.org
Python fucntion without parenthesis refers to module rather than running
Python Help on Modules P
yth
on
In
tro
du
cti
on
Python Modules Documentation
Python Documentation P
yth
on
In
tro
du
cti
on
• Python Official Documentation:
http://www.python.org/doc/
• Python Quick Reference:
http://docs.python.org/reference/
http://rgruet.free.fr/PQR26/PQR2.6_modern_a4.pdf
• Python Cheat Sheet: http://sleet.aos.wisc.edu/~gpetty/wp/wp-
content/uploads/2011/10/Python_qr.pdf/
• Python Questions:
http://stackoverflow.com/questions/tagged/python
Rich Macfarlane
python Variables & Objects
Python Objects Python Built-in Object Types Dynamic Typing
Python Variables P
yth
on
In
tro
du
icti
on
• To create Python Programs we store code in Modules/Functions
• Functions contain Statements (& Expressions)
• Statements (& Expressions) can create/manipulate Python Variables
• Variables allow us to store data
• Python Variables are all Objects
• We will use Python Built-in Objects
• OOP – create your own Python Objects
Python Variables P
yth
on
In
tro
du
icti
on
• Python Object Data Types we will learn about: • Numbers:
• Integer, Boolean, Long, Float • String • Collections:
• List, Dictionary, Tuple • File • Module, Function - program units are
objects too • Can create very advanced applications with only
the built-in Object data types
Python Variables P
yth
on
In
tro
du
icti
on
Creating Python Objects • No need to declare variable or its data type • Assign a value with = operator, created/typed memory assigned >>> a = 6 # create integer obj
>>> a = “hello” # create string obj
>>> f = open(“x.txt”) # create file obj
• The syntax of the expression defines the type of Object created (dynamicaly typed)
• Variables are Case sensitive (like everything in Python) >>> print A
<run time error>
Python Variables P
yth
on
In
tro
du
icti
on
• Variables can be assigned values without defining a
type: >>> int = 3
>>> flt = 0.945
>>> x = y = z = “string value”
• Variables must be assigned a value before being
used >>> print int1
Traceback (most recent call last): File "<stdin>", line 1, in <module> NameError: name ' int1' is not defined
Python stops with Useful error (some languages do not)
Rich Macfarlane
python Numbers
Python Number Objects Numbers Object Types
Python Numbers P
yth
on
In
tro
du
icti
on
• Immutable • Can’t change
• Changing value of a number aactually produces a new number object
• 4 main numeric object types • int – Signed Integer
• long - Long Integer – for large numbers
• float - Floating Point Numbers
• bool – Boolean (0,1) & (False,True)
print type(var)
<type ‘int’>
Python Numbers P
yth
on
In
tro
du
icti
on
• Creation Examples:
• bool bored = False
• int lecture_marks_out_of_ten_so_far = 9
• long lng1 = -795875858954
• float flt1 = 3.1446546456
• Useful BIFs to Check type of variable: • Useful with Python’s dynamic typing
print type(l) IsInstance(l,long)
<type ‘long’> True
Python Numeric Ops/Functions P
yth
on
In
tro
du
icti
on
• Python Numeric Operators: • Addition, Subtraction: + -
• Multiplication, Division: * /
• Modulus: %
• x**y x to the power y
• Numeric BIFs:
• Conversion: int(), long(), float(), str()
• Math Module contains more advanced numeric functions math.sqrt(x) Square root of x
math.pow(x,y) x to the power y with scientific notation output
Rich Macfarlane
python Strings
String creation/literal content String Operators, BIFs and Methods Basic Text Formatting More Strings in Future Weeks!
Python Strings P
yth
on
In
tro
du
icti
on
• String Object is a Sequence of characters in quotes.
(an ordered sequence of characters)
str0 = '' # create empty string
str1 = 'parrot' # create string variable
str2 = "Don't Panic!! " # " and ' same
str3 = """Don't Panic!! # multiline
and carry on"""
str4 = " Don't Panic!! \
and carry on"
• Can use " or ' and embed one within the other
• Immutable – add 2 strings together get new string object
• Each Character are simply string object with only a single character in (not a different data type)
Python String Operators P
yth
on
In
tro
du
icti
on
• String sequence objects operators + Concatenation operator – join string together create a new string
* Repetition operator – create new string by repeating another
in Membership operator
Python String Slicing P
yth
on
In
tro
du
icti
on
• Slice Operators • Return contents from string (substrings)
[offset/index] returns single character from the offset
[start:end] returns Slice substring of chars
str1 = " Panic " Offsets: 0 1 2 3 4 5
-5 -4 -3 -2 -1
str1[0] # Return 1 char from offset 0 P str1[1:3] # Slice offset 1 to 3 (not incl) an str1[1:] # Slice offset 1 to last char anic
Q. s[:] ? S[-1]?
Python String Slicing P
yth
on
In
tro
du
icti
on
• Slice Operators - index backwards from end of string
str1 = " Panic " Offsets: 0 1 2 3 4 5
-5 -4 -3 -2 -1
print str1[-1] # Return 1 char from end of str c print str1[-4:4] # Slice offset -4 to offset 4 ani print str1[-4:] # Slice last 4 chars anic
print str1[len(str1)-4:] # Slice last 4 chars anic
Python String Functions P
yth
on
In
tro
du
cti
on
• String Sequence BIFs • Length: len()
• Conversion: str()
• Python does NOT perform automatic data type conversion!
• Dynamic typing, but strict typing, so you have to perform type conversion
Python String Functions P
yth
on
In
tro
du
cti
on
• String Object Methods • String type specific functions • Find string: String.find(‘searchstring’)
• Concatenation: .join(<list>)
• String Case: .upper() lower() .capitalize() .title()
.isupper() .islower()
.isalpha() .isdigit()
• To check which methods are
available on an object, type
the Object. then
<CTRL+SPACE>
Python String Formatting P
yth
on
In
tro
du
cti
on
• String formatting:
• Concatenation:
>>> ‘Python ’ + ‘has ’ + str(1) + ‘ way to do things‘ ‘Python has 1 way to do things’
>>> ‘Python’, ‘has ‘, str(1), ‘way to do things‘
‘Python has 1 way to do things’
• printf() style building/formatting strings
>>> ‘%s has %d ways to do things‘ % (‘Python’, 1)
‘Python has 1 way to do things’
>>> ‘%+.2f format float‘ % (3.1415)
‘+3.14 format float’
Format string
Values (tuple)
Python String Iteration P
yth
on
In
tro
du
icti
on
Iteration of string characters • for in built-in construct
for variable in list:
>>> passwd = ‘rangers’
>>> for char in passwd: print char r
a
n
g
e
r
s
char – string object is automatically assigned to each item in passwd String object we are iterating over
Rich Macfarlane
python Practical Lab
Python Keyspace Calculator P
yth
on
In
tro
du
icti
on
Lab - Script1:
Cryptanalysis – Code Breaking
Brute Force a Password
• Create a script to calculate Password combinations/keyspace for various length passwords – then avg time to brute force an ASCII password
• Calculate how many possible character combinations for a given password
• What would we need to know?
Python Keyspace Calculator P
yth
on
In
tro
du
icti
on
Code Breaking - Brute Force
• Calculate Password Key Space - possible combinations
• Chars = Number of characters in character set
• ASCII Characters?
• PassLen = Number of characters in a password
• Keyspace = Chars^PassLen
Python Keyspace Calculator P
yth
on
In
tro
du
icti
on
• Chars = Number of characters in character set
• ASCII Characters?
• 95 Printable
• PassLen = Number of characters in a password
• 1
• 8
• Keyspace = Chars^PassLen
• 95^1 = 95
• 95^8 = 6,634,204,312,890,625
Python Keyspace Calculator P
yth
on
In
tro
du
icti
on
• Lab – create script keyspace.py … lab1 p14
If called as a script call main()
Python Keyspace Calculator P
yth
on
In
tro
du
icti
on
• Lab – run script • What do we need to add? Where?
Python Keyspace Calculator P
yth
on
In
tro
du
icti
on
• Lab – run script
• Add code to get_keyspace() to calc keyspace
• How?
Python Keyspace Calculator P
yth
on
In
tro
du
icti
on
Code Breaking - Brute Force
• Calculate Average Attempts for a Key Space
• How many attempts before password recovered?
• Code breaking law of averages – code has equal chance of being found anywhere in keyspace
• Equal chance of being First or last combination in keyspace
• On average found half way through key space
• AverageAttempts = Keyspace/2
• 95/2 = 47.5 attempts on average to crack
• 6634204312890625 / 2 = 3317102156445312.5
Python Keyspace Calculator P
yth
on
In
tro
du
icti
on
Code Breaking - Brute Force
• Calculate Average Time to Find Password
• How much time before password recovered?
• How many passwords can system try in a hour?
• 100,000,000
• AverageTimeToCrack = AvgAttempts/PasswdsPerHour
• 47.5 / 100,000,000 = 0.000000475 hours
= 0.00171 secs (*60 *60)
• 3.11 x10^85 / 100,000,000 = 66342043 hours
= 7573 years (/24 /365)
Rich Macfarlane
python Introduction