Python - Napier40001507/CSN08115/PythonLecture1.pdf · Exception Handling, Lab: Python Forensics...

Rich Macfarlane, 2015

python Introduction

Python Teaching Team P

yth

on

In

tro

du

cti

on

Rich Macfarlane – Lecturer in CDCS Email: [email protected] Twitter: @rjmacfarlane

http://www.iidi.napier.ac.uk/c/people/peopleid/13277237

Owen Lo – Researcher/Lecturer in CDCS Email: [email protected]


mailto:[email protected]


mailto:[email protected]



Python Schedule P

yth

on

In

tro

du

cti

on

Session Python Lectures/Labs

1 Scripting Languages/Python Overview, Environment,

Modules/Scripts, Flow Control, Variables, Numbers/Maths, Strings,

Lab: Python Cryptanalysis - Keyspace

2 Lists Collections, Code Development, Script Arguments, Exception

Handling, Lab: Python Hashing/Password Cracking

3 Collection Data Types: Tuples, Dictionaries, External Data - Files,

Exception Handling, Lab: Python Forensics Intro

3 Text manipulation, Regular Expressions, System Programming, More

External Data – Files/Web, Lab: Python Encryption

Python Schedule P

yth

on

In

tro

du

cti

on

• 4 Taught Lectures + 4 Practical Labs • Lectures - mix of talking about the language/theory +

some demoing of syntax/Python examples • Labs – examples of the theory from lectures + build

approx. 2 Python scripts per lab • Scripts used to build coursework • Must complete labs! – 4+ hours/wk

• Labs pretty detailed, so don’t need to take many notes

• Use Lecture notes + Moodle resources as reference in labs

• Can learn a lot of python in 4 weeks!

Rich Macfarlane

python scripting languages

What are Scripting Languages? Types

Scripting Languages What? P

yth

on

In

tro

du

cti

on

• What is a scripting language?

• High level language – higher than C#, Java

• Used to write ‘scripts’

• Scripts automate tasks such as command line OS programs, or web page creation

• Typically interpreted and not compiled

• Smaller programs – hundreds of lines max

• Dynamic Typing – Rapid development

Scripting Languages Types? P

yth

on

In

tro

du

cti

on

• OS Job control – JCL, Unix Shell Scripting, Bash

• Unix/Linux shell scripting – glue programs together: ps –aux | grep “snort” | more

• Text Processing – AWK, sed - Perl (started as)

• Web browser control – JavaScript • Scripts embedded in HTML – provide dynamic content

• Web pages - PHP

• Developed into application programming languages – Perl, Python, Ruby • Now Modular and Extensible, OO

Rich Macfarlane

python overview

Python History Why use Python? What is Python used for Who uses Python

Python History P

yth

on

In

tro

du

cti

on

• Python created in 1989 by Guido van Rossum

• Worked at Google in App Engine Team

• Now works for Dropbox – client/server written in Python

• Old language – has become popular more recently

• Versions:

• 2.x series 2000

• 3.x series 2008 • Not backward compatible

• 3.x is the future

• 2.x has most support now

• Use 2.7(.8) on this module Google > Dropbox

Python What? P

yth

on

In

tro

du

cti

on

• Powerful/Light High Level Language

• Scripting/general purpose programming language

• Between traditional scripting languages (TCL, Perl) and High Level Application Development Languages (C++, C#, Java)

• Similar to Perl, Ruby

• Top 5 most widely used

• Named after Monty Pythons

Flying Circus :D

Python Why? P

yth

on

In

tro

du

cti

on

Why are we learning Python? • Produces Good Quality Code

• Readable/consistent/minimal code compared to other scripting languages – more readable than Perl • Easy to learn and use – easier than C, C++ ,C#, Java

• Productivity • Rapid Prototyping/Proof of concept - hacking Take longer in other languages C,C++,Java • Less code to type/no compile stage

• Portability • Multi platform without any changes • Linux/Mac/Windows/Mobile

• Rich set of libraries/tools available • Standard Library/3rd party Libraries

• Can be used for wide range of application development

- Small scale applications

- OOP Can be used for strategic application development

Python Why? P

yth

on

In

tro

du

cti

on

Why are we learning Python? • Python Principles ‘easter egg’:

Python Platforms P

yth

on

In

tro

du

cti

on

Same code will run on: • Linux – Python installed on most

• Mac – pre installed

• Windows

• Mobile – Android/iOS

• Games Consoles

• Embedded Systems

Implementations: • Cpython – reference implementation – we will use

• IronPython – for scripting .NET and Mono

• Jpython – coded for Java Runtime

Python Features P

yth

on

In

tro

du

cti

on

• Interpreted Language • No compile phase/quick turnaround • Prototype code in Interpreter before adding to scripts

• Dynamically Typed • Typing at runtime - as apposed to static compile type typing:

• Strongly Typed • Dynamic type checking – reports misuse

• Automated Memory Management • Garbage collection

• Simplicity of Scripting – procedural • Object Oriented - optional • Language, with SE tools and libraries of high level compiled languages

Python Features P

yth

on

In

tro

du

cti

on

• Python Interactive Shell

Dynamic Typing – no declaration - quick

Strongly Typed

Dynamic Typing – now var is a integer

Python Applications P

yth

on

In

tro

du

cti

on

• Operating-Systems programming • OS Shell tools, scripting • Cross platform interfaces to OS

• Cyber Security and Forensics • Security –pen testing, exploit creation, automation of tools, text parsing,

log file analysis • Forensics – tools, scripting - digital media analysis

• Network scripting • Packet Sniffing, Traffic analysis – Scapy, pyNIDS

• Internet scripting • Client/server networking, XML, Fetch web pages, parse HTML • Web application development – django

• GUI programming - tkinter

• Database programming – cross db libraries

• Data Mining/Machine Learning, Robotics, Gaming • Numeric/Scientific Programming – NumPy, SciPy

Python Commercial User Base P

yth

on

In

tro

du

cti

on

• Intel, Cisco, HP

• Google, Yahoo

• YouTube

• Industrial Light & Magic

• BitTorrent

• NASA

• Reddit

• Rasberry Pi

• NSA

Python vs Perl P

yth

on

In

tro

du

cti

on

• ‘Conflict’ between Language Users

• Can do everything in Python that you can do in Perl

• Perl code shorter, quicker to code, but typically less readable, and harder to reuse

• Perl typically more ways to do same thing

• Python typically has only one(ish) way to do things

Python vs Perl P

yth

on

In

tro

du

cti

on

Rich Macfarlane

python Running Python Code

How Python is run How do we create Python code How do we run Python code

Running Python P

yth

on

In

tro

du

cti

on

Running Python Code P

yth

on

In

tro

du

cti

on

• 2 ways to run Python code:

1. Interactive Interpreter

Type Python Code at CLI >>> promt

2. Type Python

Type commands and they are evaluated/executed

Can type entire program into interpreter!

Type Python to run Interpreter

Python Interpreter prompt >>>


yth

on

In

tro

du

cti

on

1. Interactive Interpreter

Don’t have to use print at >>> Interpreter prompt - prints expressions

Use Interpreter for:

• Experimenting – executes code immediately, try out small examples of code, good error messages, can add code to script if works!

• Testing code in files – import code from a file and run to test. Test individual functions from code modules.


yth

on

In

tro

du

cti

on

2. Create Python Script File + Run through Interpreter 1. Create text file containing Python code; typically extension ‘.py’

Code file- Python statements.

(.py for importing)

2. Run ‘.py’ code file through the Python interpreter; It executes each

statement.

Use code files to reuse our code!


yth

on

In

tro

du

cti

on

GUI – IDLE

• Same 2 ways to run code:

• 1. Interactive Interpreter – “Python Shell”

• - Syntax Highlighting/Coloring – spot errors

• - Auto Completion with <TAB>

• - Code Statement History <ALT+P> <ALT+N>

• - function( and module. Popups

• - auto indent for code blocks

• - Debugger, Object Browser


yth

on

In

tro

du

cti

on

GUI – IDLE

2. Create .py Script/Module files & Run from IDLE File>New Window - opens editor - add code, save as .py file

Run>Run Module(F5) - runs .py file code in Interpreter window – easier and faster then import/reload()

Rich Macfarlane

python Syntax/Flow Control

Python statements Python differences to other languages What Python has removed! Python syntax for blocks of code Python if conditional statements

Python Code Syntax P

yth

on

In

tro

du

icti

on

• Python Statement • Line of Python code to do something – logic of program • Made up of Python Expressions

• Expressions create and manipulate Python Objects >>> if x == 5: print ‘Python is great’

• No Statement termination character • C# - statement termination character? • Python - End of line instead – don’t have to type!

>>> x = 5

>>> y = 2 Note: don’t have to use print in interpreter. ‘_’ prints last value

• \ can be used to continue lines

>>> z = x \

... + y

Assignment Statements Creates variable, and associated object


yth

on

In

tro

du

icti

on

• C# - what to group code statements into blocks?

• Python - Whitespace used to specify ‘blocks’ of code

def print_msg(msg):

print "hello " + msg

print "hello again " + msg

• Less cluttered than other languages

• Colon ‘:’ added to show code block/indentation expected

• You will forget this many times!!

• 4 spaces commonly used (2 – Google)

• DO NOT use mix of tabs and spaces!

Whitespace instead of { } Compound Statement - Embedded statements


yth

on

In

tro

du

icti

on

• Python - Whitespace used to specify ‘blocks’ of code

if True: if True:

print “true “ print “true “

print “do” print “do”

else: else:

print “false“ print “false“

print “do not“ print “do not“

print “more code” print “more code”

• Don’t have to type brackets round condition,

{} around code blocks, or ; at end of line

- Produces Runtime Error - Indentation must be correct - Unlike C style languages!

: is extra to type

Flow Control P

yth

on

In

tro

du

icti

on

• Boolean Data Type

Values: True False

b = True

b

True

b == False

False

Flow Control P

yth

on

In

tro

du

icti

on

• Comparison Operators

Compare 2 values and evaluate to True or False! == Equal to != Not Equal

< Less than > Greater than

<= Less than or equal to

>= Greater than or equal to

l = len(‘hack’)

l == 4

True

l >= 8

False

Boolean Expression Python Expression which equates to True or False

Flow Control - IF Statement P

yth

on

In

tro

du

icti

on

• Why do we need if statements? • Decides which code too run • Based on Boolean Tests being True or False

• IF Statement:

if <boolean test>:

<indented code block>

elif <boolean test>:

<indented code block> else:

<indented code block>

if speed >= 70: print ‘Driving License please' if mood == 'terrible' or speed >= 100: print 'You are in trouble!' elif mood == 'bad' or speed >= 90: print “You are getting a ticket" else: print "Let's try to keep the speed down sir"

Don’t forget the :

Multiple conditions and/or/not

spelt out

Indent With W/s

Multiple ifs Using elif (no switch/case)

Boolean Expression Python Expression which equates to True or False


yth

on

In

tro

du

icti

on

Comments • # used for single line comments

• Everything to end of line ignored

• End of the line = end of comment

#

# Script: Hello World

# Created: Oct 2013

#

print "Hello World" # Print the hello msg

""" triple quotes can sometimes be used for

multi-line comments also """

Rich Macfarlane

python scripts, modules & functions

Python Functions Python BIFs Python Modules Scripts and Libraries of Functions

Python Functions P

yth

on

In

tro

du

icti

on

• Reuse code using Python Functions Function Syntax: def <function_name>

• File containing functions called a Module

Function definition

Module

Function Doc String

Function code

Python Using Functions P

yth

on

In

tro

du

icti

on

Using Functions • To use a function, the following import syntax is used: from <module_name> import <function_name>

• Or typically use all code from a Module: import module_name

• On import all code in the .py file is run – function definitions are created.

• Functions can be run with module.function() syntax

• reload() can be used if changes are made to file

Module.Function

Module

Python Modules P

yth

on

In

tro

du

icti

on

• Modules are Python .py files containing code such as groups of related functions, and variables.

• Python program architecture – top level module, imports sub modules + built-in modules.

• Built-in Standard Library Modules: http://docs.python.org/library

>>> import os >>> os.getcwd()

'C:\\Python27‘

• Can also use External 3rd party Modules. • Create our own Modules on this ‘module’ … lab – create 2

modules per week, then use for coursework!

http://docs.python.org/library

http://docs.python.org/library

Python Built In Funtions (BIF) P

yth

on

In

tro

du

icti

on

• Python Standard Library • Python has many preloaded functions, which are

part of the Python Standard Library, and are available when the Python interpreter is run.

• BIFs are functions of the __builtin__ Module

http://docs.python.org/library/functions.html

__builtin__.

http://docs.python.org/library/functions.html

Python External Modules P

yth

on

In

tro

du

icti

on

• PyPI – The Python Package Index

• If the Standard Library does not contain functions which can help, there are many Python modules available on the web.

PyPI repository:

http://pypi.python.org/pypi



Python Module Types P

yth

on

In

tro

du

icti

on

• Modules are Python .py files containing reusable code.

Types:

• Script Modules

• Autonomous code which can be run standalone. Typically from command line with parameterised using arguments.

• Function Library Modules

• Variable and function definitions with can be imported and used by other code.

• Modules which are Both …

Python Modules P

yth

on

In

tro

du

icti

on

Mixed Use Python Modules

• Both a standalone Script or as a Library of Functions:

Boilerplate Code This code runs first! Calls main() if run as a script, and does not run any code if imported as a library of functions.

Special main() Function Can be used to test library of functions or Call Module code functions as a script (c/w)

Python Modules P

yth

on

In

tro

du

icti

on


• Standalone Script :

Run > Run Module (F5)

Python Modules P

yth

on

In

tro

du

icti

on


• As a Library of Functions:

Python Help on Modules P

yth

on

In

tro

du

cti

on

Python Modules Documentation

• Get help on a Module, what it does, what functions it contains

• Tech Method: From Interpreter use BIFs: • help(object/module)

• help(module.function)

Shows help on object/funtion • dir(object/module)

Shows the objects methods.

OR

• Easy way: Google Search for ‘Python Module/Function’

Typically bring back official Python doc first! python.org

Python fucntion without parenthesis refers to module rather than running

Python Help on Modules P

yth

on

In

tro

du

cti

on

Python Modules Documentation

Python Documentation P

yth

on

In

tro

du

cti

on

• Python Official Documentation:

http://www.python.org/doc/

• Python Quick Reference:

http://docs.python.org/reference/

http://rgruet.free.fr/PQR26/PQR2.6_modern_a4.pdf

• Python Cheat Sheet: http://sleet.aos.wisc.edu/~gpetty/wp/wp-

content/uploads/2011/10/Python_qr.pdf/

• Python Questions:

http://stackoverflow.com/questions/tagged/python

http://www.python.org/doc/


http://rgruet.free.fr/PQR26/PQR2.6_modern_a4.pdf






Rich Macfarlane

python Variables & Objects

Python Objects Python Built-in Object Types Dynamic Typing

Python Variables P

yth

on

In

tro

du

icti

on

• To create Python Programs we store code in Modules/Functions

• Functions contain Statements (& Expressions)

• Statements (& Expressions) can create/manipulate Python Variables

• Variables allow us to store data

• Python Variables are all Objects

• We will use Python Built-in Objects

• OOP – create your own Python Objects

Python Variables P

yth

on

In

tro

du

icti

on

• Python Object Data Types we will learn about: • Numbers:

• Integer, Boolean, Long, Float • String • Collections:

• List, Dictionary, Tuple • File • Module, Function - program units are

objects too • Can create very advanced applications with only

the built-in Object data types

Python Variables P

yth

on

In

tro

du

icti

on

Creating Python Objects • No need to declare variable or its data type • Assign a value with = operator, created/typed memory assigned >>> a = 6 # create integer obj

>>> a = “hello” # create string obj

>>> f = open(“x.txt”) # create file obj

• The syntax of the expression defines the type of Object created (dynamicaly typed)

• Variables are Case sensitive (like everything in Python) >>> print A

<run time error>

Python Variables P

yth

on

In

tro

du

icti

on

• Variables can be assigned values without defining a

type: >>> int = 3

>>> flt = 0.945

>>> x = y = z = “string value”

• Variables must be assigned a value before being

used >>> print int1

Traceback (most recent call last): File "<stdin>", line 1, in <module> NameError: name ' int1' is not defined

Python stops with Useful error (some languages do not)

Rich Macfarlane

python Numbers

Python Number Objects Numbers Object Types

Python Numbers P

yth

on

In

tro

du

icti

on

• Immutable • Can’t change

• Changing value of a number aactually produces a new number object

• 4 main numeric object types • int – Signed Integer

• long - Long Integer – for large numbers

• float - Floating Point Numbers

• bool – Boolean (0,1) & (False,True)

print type(var)

<type ‘int’>

Python Numbers P

yth

on

In

tro

du

icti

on

• Creation Examples:

• bool bored = False

• int lecture_marks_out_of_ten_so_far = 9

• long lng1 = -795875858954

• float flt1 = 3.1446546456

• Useful BIFs to Check type of variable: • Useful with Python’s dynamic typing

print type(l) IsInstance(l,long)

<type ‘long’> True

Python Numeric Ops/Functions P

yth

on

In

tro

du

icti

on

• Python Numeric Operators: • Addition, Subtraction: + -

• Multiplication, Division: * /

• Modulus: %

• x**y x to the power y

• Numeric BIFs:

• Conversion: int(), long(), float(), str()

• Math Module contains more advanced numeric functions math.sqrt(x) Square root of x

math.pow(x,y) x to the power y with scientific notation output

Rich Macfarlane

python Strings

String creation/literal content String Operators, BIFs and Methods Basic Text Formatting More Strings in Future Weeks!

Python Strings P

yth

on

In

tro

du

icti

on

• String Object is a Sequence of characters in quotes.

(an ordered sequence of characters)

str0 = '' # create empty string

str1 = 'parrot' # create string variable

str2 = "Don't Panic!! " # " and ' same

str3 = """Don't Panic!! # multiline

and carry on"""

str4 = " Don't Panic!! \

and carry on"

• Can use " or ' and embed one within the other

• Immutable – add 2 strings together get new string object

• Each Character are simply string object with only a single character in (not a different data type)

Python String Operators P

yth

on

In

tro

du

icti

on

• String sequence objects operators + Concatenation operator – join string together create a new string

* Repetition operator – create new string by repeating another

in Membership operator

Python String Slicing P

yth

on

In

tro

du

icti

on

• Slice Operators • Return contents from string (substrings)

[offset/index] returns single character from the offset

[start:end] returns Slice substring of chars

str1 = " Panic " Offsets: 0 1 2 3 4 5

-5 -4 -3 -2 -1

str1[0] # Return 1 char from offset 0 P str1[1:3] # Slice offset 1 to 3 (not incl) an str1[1:] # Slice offset 1 to last char anic

Q. s[:] ? S[-1]?

Python String Slicing P

yth

on

In

tro

du

icti

on

• Slice Operators - index backwards from end of string

str1 = " Panic " Offsets: 0 1 2 3 4 5

-5 -4 -3 -2 -1

print str1[-1] # Return 1 char from end of str c print str1[-4:4] # Slice offset -4 to offset 4 ani print str1[-4:] # Slice last 4 chars anic

print str1[len(str1)-4:] # Slice last 4 chars anic

Python String Functions P

yth

on

In

tro

du

cti

on

• String Sequence BIFs • Length: len()

• Conversion: str()

• Python does NOT perform automatic data type conversion!

• Dynamic typing, but strict typing, so you have to perform type conversion

Python String Functions P

yth

on

In

tro

du

cti

on

• String Object Methods • String type specific functions • Find string: String.find(‘searchstring’)

• Concatenation: .join(<list>)

• String Case: .upper() lower() .capitalize() .title()

.isupper() .islower()

.isalpha() .isdigit()

• To check which methods are

available on an object, type

the Object. then

<CTRL+SPACE>

Python String Formatting P

yth

on

In

tro

du

cti

on

• String formatting:

• Concatenation:

>>> ‘Python ’ + ‘has ’ + str(1) + ‘ way to do things‘ ‘Python has 1 way to do things’

>>> ‘Python’, ‘has ‘, str(1), ‘way to do things‘

‘Python has 1 way to do things’

• printf() style building/formatting strings

>>> ‘%s has %d ways to do things‘ % (‘Python’, 1)

‘Python has 1 way to do things’

>>> ‘%+.2f format float‘ % (3.1415)

‘+3.14 format float’

Format string

Values (tuple)

Python String Iteration P

yth

on

In

tro

du

icti

on

Iteration of string characters • for in built-in construct

for variable in list:

>>> passwd = ‘rangers’

>>> for char in passwd: print char r

a

n

g

e

r

s

char – string object is automatically assigned to each item in passwd String object we are iterating over

Rich Macfarlane

python Practical Lab

Python Keyspace Calculator P

yth

on

In

tro

du

icti

on

Lab - Script1:

Cryptanalysis – Code Breaking

Brute Force a Password

• Create a script to calculate Password combinations/keyspace for various length passwords – then avg time to brute force an ASCII password

• Calculate how many possible character combinations for a given password

• What would we need to know?


yth

on

In

tro

du

icti

on

Code Breaking - Brute Force

• Calculate Password Key Space - possible combinations

• Chars = Number of characters in character set

• ASCII Characters?

• PassLen = Number of characters in a password

• Keyspace = Chars^PassLen


yth

on

In

tro

du

icti

on

• Chars = Number of characters in character set

• ASCII Characters?

• 95 Printable

• PassLen = Number of characters in a password

• 1

• 8

• Keyspace = Chars^PassLen

• 95^1 = 95

• 95^8 = 6,634,204,312,890,625


yth

on

In

tro

du

icti

on

• Lab – create script keyspace.py … lab1 p14

If called as a script call main()


yth

on

In

tro

du

icti

on

• Lab – run script • What do we need to add? Where?


yth

on

In

tro

du

icti

on

• Lab – run script

• Add code to get_keyspace() to calc keyspace

• How?


yth

on

In

tro

du

icti

on


• Calculate Average Attempts for a Key Space

• How many attempts before password recovered?

• Code breaking law of averages – code has equal chance of being found anywhere in keyspace

• Equal chance of being First or last combination in keyspace

• On average found half way through key space

• AverageAttempts = Keyspace/2

• 95/2 = 47.5 attempts on average to crack

• 6634204312890625 / 2 = 3317102156445312.5


yth

on

In

tro

du

icti

on


• Calculate Average Time to Find Password

• How much time before password recovered?

• How many passwords can system try in a hour?

• 100,000,000

• AverageTimeToCrack = AvgAttempts/PasswdsPerHour

• 47.5 / 100,000,000 = 0.000000475 hours

= 0.00171 secs (*60 *60)

• 3.11 x10^85 / 100,000,000 = 66342043 hours

= 7573 years (/24 /365)

Rich Macfarlane

python Introduction

Python - Napier40001507/CSN08115/PythonLecture1.pdf · Exception Handling, Lab: Python Forensics...

Documents

Transcript of Python - Napier40001507/CSN08115/PythonLecture1.pdf · Exception Handling, Lab: Python Forensics...