Python Crash Course File I/O Bachelors V1.0 dd 20-01-2015 Hour 2

Click here to load reader

  • date post

    17-Jan-2016
  • Category

    Documents

  • view

    216
  • download

    3

Embed Size (px)

Transcript of Python Crash Course File I/O Bachelors V1.0 dd 20-01-2015 Hour 2

  • Python Crash CourseFile I/OBachelorsV1.0dd 20-01-2015Hour 2

  • File I/OTypes of input/output availableInteractiveKeyboardScreenFilesAscii/texttxtcsvBinaryStructuredFITS > pyFITS, astropy.io.fitsURLPipes

  • Interactive I/O, fancy output>>> s = 'Hello, world.'>>> str(s)'Hello, world.'>>> repr(s)"'Hello, world.'">>> str(1.0/7.0)'0.142857142857'>>> repr(1.0/7.0)'0.14285714285714285'>>> x = 10 * 3.25>>> y = 200 * 200>>> s = 'The value of x is ' + repr(x) + ', and y is ' + repr(y) + '...'>>> print sThe value of x is 32.5, and y is 40000...>>> # The repr() of a string adds string quotes and backslashes:... hello = 'hello, world\n'>>> hellos = repr(hello)>>> print hellos'hello, world\n'>>> # The argument to repr() may be any Python object:... repr((x, y, ('spam', 'eggs')))"(32.5, 40000, ('spam', 'eggs'))"

  • Interactive I/O, fancy output>>> import math>>> print 'The value of PI is approximately %5.3f.' % math.piThe value of PI is approximately 3.142.Old string formatting>>> table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 7678}>>> for name, phone in table.items():... print '{0:10} ==> {1:10d}'.format(name, phone)...Jack ==> 4098Dcab ==> 7678Sjoerd ==> 4127New string formatting

  • Formatting I/OA conversion specifier contains two or more characters and has the following components, which must occur in this order: The "%" character, which marks the start of the specifier. Mapping key (optional), consisting of a parenthesised sequence of characters (for example, (somename)). Conversion flags (optional), which affect the result of some conversion types. Minimum field width (optional). If specified as an "*" (asterisk), the actual width is read from the next element of the tuple in values, and the object to convert comes after the minimum field width and optional precision. Precision (optional), given as a "." (dot) followed by the precision. If specified as "*" (an asterisk), the actual width is read from the next element of the tuple in values, and the value to convert comes after the precision. Length modifier (optional). Conversion type.

    >>> print '%(language)s has %(#)03d quote types.' % \ {'language': "Python", "#": 2}Python has 002 quote types.

  • The conversion types are:

    ConversionMeaningdSigned integer decimal.iSigned integer decimal.oUnsigned octal.uUnsigned decimal.xUnsigned hexadecimal (lowercase).XUnsigned hexadecimal (uppercase).eFloating point exponential format (lowercase).EFloating point exponential format (uppercase).fFloating point decimal format.FFloating point decimal format.gSame as "e" if exponent is greater than -4 or less than precision, "f" otherwise.GSame as "E" if exponent is greater than -4 or less than precision, "F" otherwise.cSingle character (accepts integer or single character string).rString (converts any python object using repr()).sString (converts any python object using str()).%No argument is converted, results in a "%" character in the result.

  • Interactive I/O>>> print Python is great,, isnt it?

    >>> str = raw_input( Enter your input: )>>> print Received input is: ,str

    Enter your input: Hello PythonReceived input is: Hello Python

    >>> str = input("Enter your input: ");>>> print "Received input is: ", str

    Enter your input: [x*5 for x in range(2,10,2)]Received input is: [10, 20, 30, 40]

    If the readline modules was loaded the raw_input() will use it to provide elaborate line editing and history features.

  • File I/O>>> fname = myfile.dat

    >>> f = file(fname) >>> lines = f.readlines() >>> f.close()

    >>> f = file(fname) >>> firstline = f.readline() >>> secondline = f.readline()

    >>> f = file(fname) >>> for l in f: ... print l.split()[1] >>> f.close()

    >>> outfname = myoutput >>> outf = file(outfname, w) # second argument denotes writable >>> outf.write(My very own file\n) >>> outf.close()

  • Read File I/O>>> f = open("test.txt")>>> # Read everything into single string:>>> content = f.read()>>> len(content)>>> print content>>> f.read() # At End Of File>>> f.close()>>> # f.read(20) reads (at most) 20 bytesUsing with block:>>> with open(test.txt, r) as f:... content = f.read()>>> f.closedCSV file:>>> import csv>>> ifile = open(photoz.csv, "r")>>> reader = csv.reader(ifile)>>> for row in reader:... print row,>>> ifile.close()

  • Read and write text file>>> from numpy import * >>> data = loadtxt("myfile.txt") # myfile.txt contains 4 columns of numbers >>> t,z = data[:,0], data[:,3] # data is a 2D numpy array, t is 1st col, z is 4th col

    >>> t,x,y,z = loadtxt("myfile.txt", unpack=True) # to automatically unpack all columns >>> t,z = loadtxt("myfile.txt", usecols = (0,3), unpack=True) # to select just a few columns

    >>> data = loadtxt("myfile.txt", skiprows = 7) # to skip 7 rows from top of file >>> data = loadtxt("myfile.txt", comments = '!') # use '!' as comment char instead of '#' >>> data = loadtxt("myfile.txt", delimiter=';') # use ';' as column separator instead of whitespace >>> data = loadtxt("myfile.txt", dtype = int) # file contains integers instead of floats>>> from numpy import * >>> savetxt("myfile.txt", data) # data is 2D array >>> savetxt("myfile.txt", x) # if x is 1D array then get 1 column in file. >>> savetxt("myfile.txt", (x,y)) # x,y are 1D arrays. 2 rows in file. >>> savetxt("myfile.txt", transpose((x,y))) # x,y are 1D arrays. 2 columns in file. >>> savetxt("myfile.txt", transpose((x,y)), fmt='%6.3f') # use new format instead of '%.18e' >>> savetxt("myfile.txt", data, delimiter = ';') # use ';' to separate columns instead of space

  • String formatting for output>>> sigma = 6.76/2.354>>> print(sigma is %5.3f metres%sigma) sigma is 2.872 metres >>> d = {bob: 1.87, fred: 1.768} >>> for name, height in d.items(): ... print(%s is %.2f metres tall%(name.capitalize(), height)) ... Bob is 1.87 metres tall Fred is 1.77 metres tall

    >>> nsweets = range(100) >>> calories = [i * 2.345 for i in nsweets] >>> fout = file(sweetinfo.txt, w) >>> for i in range(nsweets): ... fout.write(%5i %8.3f\n%(nsweets[i], calories[i])) ... >>> fout.close()

  • File I/O, CSV filesCSV (Comma Separated Values) format is the most common import and export format for spreadsheets and databases.Functionscsv.readercsv.writercsv.register_dialectcsv.unregister_dialectcsv.get_dialectcsv.list_dialectscsv.field_size_limit

  • File I/O, CSV filesReading CSV files

    Writing CSV filesimport csv # imports the csv module f = open('data1.csv', 'rb') # opens the csv filetry: reader = csv.reader(f) # creates the reader object for row in reader: # iterates the rows of the file in orders print row # prints each rowfinally: f.close() # closingimport csv ifile = open('test.csv', "rb")reader = csv.reader(ifile)ofile = open('ttest.csv', "wb")writer = csv.writer(ofile, delimiter='\t', quotechar='"', quoting=csv.QUOTE_ALL) for row in reader: writer.writerow(row) ifile.close()ofile.close()

  • File I/O, CSV filesThe csv module contains a the following quoting options. csv.QUOTE_ALL Quote everything, regardless of type.csv.QUOTE_MINIMALQuote fields with special characterscsv.QUOTE_NONNUMERICQuote all fields that are not integers or floatscsv.QUOTE_NONEDo not quote anything on output

  • File I/O, PicklePickle: powerful algorithm for serializing and de-serializing a Python object structurecan transform a complex object into a byte streamcan transform the byte stream into an object with the same internal structuremost obvious thing to do with these byte streams is to write them onto a filealso conceivable to send them across a network or store them in a databaseThe following types can be pickled:None, True, and Falseintegers, long integers, floating point numbers, complex numbersnormal and Unicode stringstuples, lists, sets, and dictionaries containing only picklable objectsfunctions defined at the top level of a modulebuilt-in functions defined at the top level of a moduleclasses that are defined at the top level of a moduleinstances of such classes whose __dict__ or the result of calling __getstate__() is picklable (see section The pickle protocol for details).

  • File I/O, PickleExample saveimport pickle

    data1 = {'a': [1, 2.0, 3, 4+6j], 'b': ('string', u'Unicode string'), 'c': None}

    selfref_list = [1, 2, 3]selfref_list.append(selfref_list)

    output = open('data.pkl', 'wb')

    # Pickle dictionary using protocol 0.pickle.dump(data1, output)

    # Pickle the list using the highest protocol available.pickle.dump(selfref_list, output, -1)

    output.close()

  • File I/O, PickleExample loadimport pprint, pickle

    pkl_file = open('data.pkl', 'rb')

    data1 = pickle.load(pkl_file)pprint.pprint(data1)

    data2 = pickle.load(pkl_file)pprint.pprint(data2)

    pkl_file.close()(dp0S'a'p1(lp2I1aF2.0aI3ac__builtin__complexp3(F4.0F6.0tp4Rp5asS'c'p6NsS'b'p7(S'string'p8VUnicode stringp9tp10s.]q

  • File I/O, PickleReal live example AstroWise cluster job submissionclient server mo