Input and Output Streams – part II - huji.ac.il · PDF fileSlide 10 Byte Streams To read...

Input and Output Streams – part II

Lesson OverviewTopics covered

(1) Short recap on Input/Output Streams

(2) Using streams to read structured data – the problem

(3) The Decorator Design Pattern – the solution

(4) Stream Concatenation – plumbing revisited

(5) Reading and Writing objects

The General Scenario

Application

Problem : How can we communicate with various devices using a single common interface? Notice that we do not want to know the details of the hardware devices. Instead we would like to have a common interface for accessing all of them them!

I/O AbstractionWe can achieve the separation by designing a common interface for reading any kind of data, and common interface for writing any kind of data.

This interface is implemented by the notion of input and output streams.

Any input/output can be represented as a sequence of

bits. For convenience we divide the sequence into a sequence of bytes.

Input/Output StreamsAn input/output stream is a sequence of bytes that is attached to some input/output source. You can read/write data from/to the stream in a sequential order. One byte at a time or several bytes at a time.

12 72 32 17 83 11 7 91 108

43 55 31 37 34 13 17 1 15

Input streamreading direction

writing direction Output stream

Basic reading/writing algorithmsNo matter where the data is coming from or going to and no matter what its type, the algorithms for sequentially reading and writing data are basically the same:

Reading : (1) open a stream

(2) while (more information)

(2.1) read information

(3) close the stream

Basic reading/writing algorithmsWriting : (1) open a stream

(2) while (more information)

(2.1) write information

(3) close the stream

Character StreamsReader and Writer are the abstract superclasses for character streams in java.io.

Reader provides the API and partial implementation for readers — streams that read 16-bit characters.

Writer provides the API and partial implementation for writers — streams that write 16-bit characters.

Character Streams Subclasses of Reader and Writer implement specialized streams and are divided into two categories: those that read from or write to data sinks and those that perform some sort of processing .

data sinks processing

Byte StreamsTo read and write 8-bit bytes, programs should use the byte streams, descendents of InputStream and OutputStream.

InputStream and OutputStream provide the API and partial implementation for input streams (streams that read 8-bit bytes) and output streams (streams that write 8-bit bytes).

These streams are typically used to read and write binary data such as images and sounds.

Byte StreamsSubclasses of InputStream and OutputStream provide specialized I/O that falls into two categories

Input SuperclassesReader and InputStream define similar APIs but for different data types. Both are abstract classes!

Reader contains these methods for reading characters and arrays of characters.

int read() //Read a single character,-1 is ends.

int read(char cbuf[]) //Read chars into an array

//read chars into a portion of the array.

int read(char cbuf[], int offset, int length)

Input SuperclassesInputStream defines the same methods but for reading bytes and arrays of bytes:

int read() //Read a single byte,-1 if end.

int read(byte cbuf[]) //Read several bytes...

int read(byte cbuf[], int offset, int length)

Both Reader and InputStream provide methods for marking a location in the stream, skipping input, and resetting the current position.

Output SuperclassesWriter and OutputStream are similarly parallel.

Writer defines these methods for writing characters and arrays of characters:

int write(int c) //write a single character.

int write(char cbuf[]) //write an array of chars.

//write a portion of an array of chars.

int write(char cbuf[], int offset, int length)

Output SuperclassesOutputStream defines the same methods but for bytes:

int write(int c) //writes a single byte

int write(byte cbuf[]) //writes an array of bytes

int write(byte cbuf[], int offset, int length)

All of the streams — readers, writers, input streams, and output streams — are automatically opened when created.

Close a stream by calling its close method. A program should close a stream as soon as it is done with it, in order to free up system resources.

Example 1: Typing a text fileimport java.io.*;public class MyTextType {public static void main(String[] args){

try{

FileReader fileIn= new FileReader("temp.txt");

int c;

while((c=fileIn.read()) != -1)

System.out.print((char)c);

fileIn.close();

}catch(FileNotFoundException fnfe){

System.err.println("file temp.text not found!!!");

}catch (IOException ioe){

System.err.println("error reading from file!");}

}

}

Streams – current summaryInputStream and OutputStream gives us a low level for reading and writing binary data. We can only read/write a single byte or an array of bytes.

Likewise, Reader and Writer gives us a low level for reading and writing text files. We can only read/write a single char or an array of chars.

Using Streams to Read Structured Data (The Problem)

In many cases, the data we want to read/write usually has a more complex structure.

Examples:

Textual data ordered in a table

A list of short values where every 2 bytes represent a single short value

We would like to be able to read/write the data in a structured way.

Case study I: Reading a file of integers

Suppose we are given a file which contains a list of integers.

We would like to read all number from the file and store them somewhere.

Let us assume that the number of integers n is known in advance (e.g. N=100).

Here is what the code would look like if we only make use of the low-level input streams we know so far:

Reading a file of integers (The hard way)public static int[] readNumbersFromFile

( String fileName, int n) throws IOException{

int[] arr = new int[n];

FileInputStream fileIn= new FileInputStream(fileName);

int currByte;

int currInt; int base = 8;

for(int i = 1; i < n; i++){

for (int j = 3; j >= 0; j--)

currInt = s |

( unsignedByteToInt(fileIn.read()) << base*j );

}

fileIn.close();

}

Reading a file of integers (The hard way)/* a small hack function required to deal with the sign bit of

the integer, which is used in the 2's complement notation.

*/public static int unsignedByteToInt(byte b) {

return (int) b & 0xFF;

}

The problem of reading structured dataThe example above demonstrates that even reading a primitive data type such as an integer using low-level streams is a rather hard task, which requires intricate bitwise operations.

However, this is merely an example, there are many types of structured data which we may encounter – numbers, lines in a text file etc.

However, we would like to be able to read/write structured data from various input and output devices.

A straightforward (and bad) solution would be ... (?)

Case study II: Efficiently reading bytes from a large file

Suppose we are given a very large file which contains a list of n bytes.

Each byte represents some code word in a secret language.

Our task is to read the bytes one after the other, and then use a decoder that we have been provided with to decode their meaning.

Here is what our (bad) code would look like:

Decoding a file of bytes (The bad way)public static int[] decodeFile

( String fileName, int n) throws IOException{

FileReader fileIn= new FileReader(fileName);

int currByte;

for(int i = 1; i < n; i++){

currByte = fileIn.read();

System.out.println(Decoder.decode((byte)currByte));

}

fileIn.close();

}

What is bad here?

Case study II: Efficiently reading bytes from a large fileAlthough the read command of the FileInputStream class can actually directly read bytes from a file this is very very inefficient.

Any read/write operation from the disk usually involves the use of the Operating System (OS) and is therefore very time consuming.

The basic reading mechanism of the OS is built on reading much bigger chunks of data from the disk at once.

These chunks are called “Pages”. A typical page is of size ~4k (IS THIS TRUE TODAY???)

In our previous code snippet, we read “from the disk” byte by byte!

Efficiently reading bytes from a large file (The Problem)

Efficiency is a feature that we would like to have in other similar scenarios:

Reading a large file of characters

Reading a large file of booleans

Reading a large file of words (Strings)

Reading a large text file line after line.

Once more: a bad solution would be ?

Are the 2 problems related?

We began by introducing a problem of reading structured data.

We then introduced the problem of efficiently reading from a file (or any other input source).

Are these two problems related to one another?

The answer lies in discussing both in terms of OO design:

The Design ProblemObjective: Have methods for reading/writing data on a higher level. (structured, efficient etc.)

Problems:

There are many enhancements for reading/writing data

There are many types of input/output streams

If we would include all enhancements in all types of streams we will end up with a lot of duplicated code and it would be hard to add new enhancements or new types of streams

The Design Solution – The Decorator Pattern

The Decorator pattern is another type of structural pattern (like the Strategy pattern some of you used in ex2)

Design Pattern?????!!!! hmmmmm

Sounds familiar, now where have I heard that phrase before?

“Here is a first hypothesis: the drawing is blind, if not the draftsman or draftswoman. As such, and in the moment proper to it, the operation of drawing would have something to do with blindness, would in some way regard blindness” (Jacques Derrida, Memoires of the Blind)

A Design Pattern (GoF)“A design pattern names, abstracts, and identifies key aspects of a common design structure that makes it useful for creating a reusable object-oriented design.”

The GoF design patterns are “descriptions of communicating objects and classes that are customized to solve a general design problem in a particular context.”

Classification of Design PatternsPurpose - what a pattern does

Creational Patterns - Concern the process of object creation. (Example: Factory Method)

Structural Patterns - Deal with the composition of classes and objects. (Examples: Decorator,Strategy)

Behavioral Patterns - Deal with the interaction of classes and objects. (Example: Iterator)

Classification of Design PatternsScope - what the pattern applies to

Class Patterns - Focus on the relationships between classes and their

subclasses. Involve inheritance reuse

Object Patterns Focus on the relationships between objects Involve composition reuse

The Decorator Design PatternIntent

The Decorator pattern is used for adding additional functionality (or repsonsibilities) to an object dynamically.

Decorators provide a flexible alternative to sublcassing for extending functionality.

Decorator Design Pattern (cont.)Motivation

It is easy to add functionality to an entire class of objects by subclassing an object, but it is impossible to extend a single object this way. With the Decorator Pattern, you can add functionality to a single object and leave others like it unmodified.

The component which we want to extend is wrapped (or enclosed) within another object that adds the required functionality. This enclosing object is called a decorator.

The decorator conforms to the interface of the component it encloses, so that its presence is transparent to the component's clients.

Decorator Design Pattern (cont.)Motivation (cont.)

The decorator forwards all requests to the component which it encloses and may perform additional actions (such as shifting bits!) before or after forwarding.

The transparency allows to nest decorators recursively, thereby allowing an unlimited number of added responsibilities.

Decorator Design Pattern (cont.)

The Decorator Design Pattern (cont.)Structure

IOstreams and the Decorator Pattern (The solution)

In order to allow various extensions or “decorations” that could be applied to various types of Input/Output streams, the IO package contains a set of classes which implement the decorator design pattern.

Example: Reader Decorators - Use a “decorator”: a class that is both derived from Reader, and encloses another Reader object as a data member (received by constructor of new class).

All Reader methods are “forwarded” (or delegated) to the inner Reader object.

New attributes (methods) use the inner Reader object as well.

IOstreams and the Decorator Pattern (The solution)

We gain two things: (1) The “old” interface is preserved – Every Reader decorator can

“pass” as a Reader object (i.e. is transparent).

(2) We can “chain” several functionalities – We can now decorate a specific Reader with several types of decorations. Example: adding both line reading capabilities and efficient reading capability (see example below).

The Same solution exists for Writer, InputStream and OutputStream classes.

In Java, “decorators” are called “Filters”, and the base class for adding attributes is FilterXXX.

Java Filter classes

In Java, “decorators” are called “Filters”, and the base class for adding attributes is FilterXXX.

For each enhancement for reading from an input stream there is a suitable filter input stream.

For each enhancement for writing to an output stream there is a suitable filter output stream.

Example: BufferedReader

Reader

read()

...

...

BufferedReader

readLine()

read()

...

...

Example: DataInputStream

InputStream

read()

...

...

DataInputStream

readShort()

read()

...

...

Plumbing Pitstop – Revisited

Water

Reservoir

Main Pipe Sec. Pipe Filter

Plumbing Pitstop – Getting Water ≡ Reading

Water

Reservoir

Main Pipe Sec. Pipe Filter

File (Source)

FileInputStream

DataInputStream

Plumbing Pitstop – where does my used water go to?

Sink (Drain)Source Pipe FilterDest. Pipe

Destination

Nachal Soreq?

Plumbing Pitstop – Spilling Water ≡ Writing

Sink (Drain)Source Pipe FilterDest. Pipe

Destination

ByteArray

byte[] arr;

ByteArrayOutputStream

BufferedOutputStream

FileOutputStream

File

Source vs. Filter streams

In the plumbing example we saw that there are 2 different kinds of Input/Output Streams:

(1) Source Streams: Streams that are designed to connect to a specific source such as: FileInputStream, ByteArrayInputStream, etc.

(2) Filter Streams: Streams that are designed to connect to other streams – that may offer some more functionality. For example: BufferedInputStream, DataInputStream, etc.

Source vs. Filter streams

All Filter streams implement the Decorator design pattern and can therefore be connected to any source stream.

This is a very neat and useful example of how a design pattern actually allows extended functionallity and resuability.

Note however, that a decorator can never be used on its own. It will always be dependant on its enclosed component (or stream in our case) for receiving/sending the actual bytes/chars.

Decorators can also be chained on to another – or recursively applied. This is called chaining

Example I: Chaining Streams

try { DataInputStream input = new DataInputStream( new BufferedInputStream( new FileInputStream(args[0])));} catch (FileNotFoundException fnfe) { // ...}

readShort()

read() read() read()

BufferedDataFile

Example II: Chaining Streams

try { BufferedReader input = new BufferedReader( new PushbackReader( new FileReader(args[0])));} catch (FileNotFoundException fnfe) { // ...}

readLine()

read() read() read()

PushbackBufferedFile

Reading a file of integers revisited (The right way)

public static int[] readNumbersFromFile( String fileName, int n) throws IOException{

int[] arr = new int[n];

DataInputStream dataIn = new DataInputStream(

new BufferedInputStream(

new FileInputStream(fileName)));

int currByte;

int currInt;

for(int i = 1; i < n; i++){

arr[i] = dataIn.readInt();

}

fileIn.close();

}

Reading a text file line by line (The good way)//reads an input text file and creates a new file which does not contain any empty lines in it.

public static void removeEmptyLines( String fileName, String destFileName) throws IOException{

BufferedReader bufIn = new BufferedReader(

new FileReader(fileName));

BufferedWriter bufOut = new BufferedWriter(

new FileWriter(destFileName));

//continued on next slide...

Reading a text file line by line (The good way)//read the file line by line – and remove empty lines

String line;

while( (line = bufIn.readLine()) != null )

if(!line.equals(“”))

bufOut.write(line,0,line.length());

bufOut.newLine();

}

//don't forget to flush and close streams!

bufOut.flush();

bufOut.close();

fileIn.close();

}

Stream OverviewStreams Description

Memory

Files

Type of I/O

CharArrayReaderCharArrayWriterByteArrayInputStreamByteArrayOutputStream

Use these streams to read from and write to memory. You create these streams on an existing array and then use the read and write methods to read from or write to the array.

StringReaderStringWriterStringBufferInputStream

Use StringReader to read characters from a String in memory. Use StringWriter to write to a String. StringWriter collects the characters written to it in a StringBuffer, which can then be converted to a String.StringBufferInputStream is similar to StringReader, except that it reads bytes from a StringBuffer.

FileReaderFileWriterFileInputStreamFileOutputStream

Collectively called file streams, these streams are used to read from or write to a file on the native file system.

Stream Overview (cont.)Type of I/O Streams Description

Buffering

Used to serialize objects.

BufferedReaderBufferedWriterBufferedInputStreamBufferedOutputStream

Buffer data while reading or writing, thereby reducing the number of accesses required on the original data source. Buffered streams are typically more efficient than similar nonbuffered streams and are often used with other streams.

ObjectSerialization

N/AObjectInputStreamObjectOutputStream

DataConversion

N/A DataInputStreamDataOutputStream

Read or write primitive data types in a machine-independent format.

Stream Overview (cont.)Type of I/O Streams Description

Printing

Filtering

PrintWriterPrintStream

Contain convenient printing methods. These are the easiest streams to write to, so you will often see other writable streams wrapped in one of these.

FilterReaderFilterWriterFilterInputStreamFilterOutputStream

These abstract classes define the interface for filter streams, which filter data as it's being read or written.

Converting between

Bytes and Characters

InputStreamReaderOutputStreamWriter

A reader and writer pair that forms the bridge between byte streams and character streams.An InputStreamReader reads bytes from an InputStream and converts them to characters, using the default character encoding or a character encoding specified by name.An OutputStreamWriter converts characters to bytes, using the default character encoding or a character encoding specified by name and then writes those bytes to an OutputStream.

Object Input & Output Streams

Java allows us to read and write whole objects into a binary file. The process of saving an objects state requires saving all non-static data members it holds.These data members may be primitive or non-primitive (ref to an object).This process is termed Serialization: serializing the object’s data members into a stream.

ObjectOutputStream

The class ObjectOutputStream allows writing an object into a stream. It contains the following method:

public void writeObject(Object obj)

throws IOException

ObjectInputStream

The class allows reading objects from an underlying InputStream. It contains the following method:

public Object readObject() throws IOException, ClassNotFoundException

Object Serialization

In order for an object to be written into an Object Stream it must implement the interface Serializable.The interface is part of the java.io package.The interface is an empty interface – i.e. no methods are included in it!

Serialization - Exampleimport java.io.*;

public class WordEntry implements Serializable{

private String word;private int counter;

public WordEntry(String word){this.word= word;counter=0;

}

//continued on next slide…

Example (cont.)public void setCounter(int counter){

if(counter >0 )this.counter= counter;

}

public int getCounter(){return(counter);

}

public String getWord(){return(word);

}//continued on next slide…

Example (cont.)public String toString(){

return("word= " + word + " counter= " + counter);}

public boolean equals(Object obj){if(!(obj instanceof WordEntry))

return(false);

WordEntry other= (WordEntry)obj;return(this.word.equalsIgnoreCase

(other.word));}

}//end of class.

Saving a WordCounter Objectpublic static void main(String[] args){WordCounter wCount= new WordCounter(“yes”,10”);

try{ObjectOutputStream objOut= new ObjectOutputStream( new BufferedOutputStream(

new FileOutputStream(args[0]))); objOut.writeObject(wCount); objOut.flush(); objOut.close();}catch(IOException ioe){ }

}

Loading a WordCounter Objectpublic static void main(String[] args){WordCounter wCount;

try{ObjectInputStream objIn= new ObjectInputStream( new FileInputStream(args[0]));

wCount= (WordCounter)objIn.readObject(); objIn.close();}catch(IOException ioe){ //do something}catch(ClassNotFoundException cnfe){

} }

Final Example: An easy way to detect foolish copying - Diffimport java.io.*;

public class Diff {public static void main(String[] args){

//check for arguments if(args.length !=2){ System.out.println("Usage: java Diff <file1> <file2>"); System.exit(0); }

//used for comparing the files textually line by line LineNumberReader file1In,file2In;

//set to false if the files are different boolean noDiff= true; //used to keep all differences found between the 2 files. StringBuffer output= new StringBuffer(); try{

file1In= new LineNumberReader(new FileReader(args[0]));file2In= new LineNumberReader(new FileReader(args[1]));

Diff (cont.)String line1,line2;line1= file1In.readLine();line2= file2In.readLine();

while( (line1!= null) && (line2!= null) ){

//compare the 2 lines. if(! line1.equals(line2) ){

output.append("line no. " + file1In.getLineNumber() + " :\n");

output.append(args[0] +" "+ line1 + "\n"); output.append(args[1] +" "+ line2 + "\n"); noDiff= false;

}

line1= file1In.readLine(); line2= file2In.readLine();}

Diff (cont.)//check if file1 has more lines!!!if(line1 != null){ StringBuffer output1= new StringBuffer(); output1.append(args[0] +" " +

"contains more lines: \n"); output1.append("line no.: " + file1In.getLineNumber()

+": " + line1 + "\n");

//now let's read all the lines left in file1! while( (line1=file1In.readLine()) != null) output1.append("line no.: " + file1In.getLineNumber()

+": " + line1 + "\n" );

output.append(output1.toString());}

if(line2 != null){//same code as in upper case only now we work on

line2!

Diff (cont.)}catch(FileNotFoundException fnfe){ System.out.println

("one of the files doe's could not be opened!");}catch(IOException ioe){ System.out.println

("an error occured when reading from one of the files.” + “aborting...");}

//check if differencs . if so print a message and the //stringBuffer with all differencesif(!noDiff){

System.out.println("Diff found some differences between the files: ");

System.out.println(output.toString());}else //no differences! System.out.println

("Diff found no differences between the files: ");}

}//end of main

Input and Output Streams – part II - huji.ac.il · PDF fileSlide 10 Byte Streams To read...

Documents

Transcript of Input and Output Streams – part II - huji.ac.il · PDF fileSlide 10 Byte Streams To read...