10/15/14  

1  

CSCI 1101A

Images Ch. 2 to 5

[Multimedia—Guzdial] Mohammad T. Irfan 9/30/14 – 10/9/14

Intro to Image & sound Chapter 2 [Guzdial]

10/15/14  

2  

Jython

u  Jython environment for students (JES)

Use up/down arrows to walk through the command history

Multimedia resource

u  http://www.cc.gatech.edu/~mark.guzdial/mediacomp/mediasources-py2ed.zip

u  This link is also available in the “Resources” section of the class website (http://www.bowdoin.edu/~mirfan/CSCI-1101A.html)

10/15/14  

3  

Using JES command area

>>> print 34 + 56 90 >>> print 34.1/46.5 0.7333333333333334 >>> print 22 * 33 726 >>> print 14 - 15 -1 >>> print “Hello” Hello >>> print “Hello ” + “World!” Hello World!

Adding integers

Dividing floats

Multiplying integers

Subtracting integers

Printing a string

Adding (concatenating) two strings

JES built-in functions

u  A bunch of functions are pre-defined in JES for sound and picture manipulations u  pickAFile()

u  makePicture()

u  makeSound()

u  show()

u  play()

u  Some of these functions accept input parameters

u  Refer to handout

10/15/14  

4  

Warm up: Show a picture

What to do to show a picture

u  1. Get the picture file name (with location in your hard drive)

u  2. Create a picture object

u  3. Show the picture object

10/15/14  

5  

pickAFile() leads to a file chooser! Returns the selected file name with location

Picture Functions

u  makePicture(filename) creates and returns a picture object corresponding to the JPEG filename

u  show(picObj) displays the picture object in a window

u  Weʼ’ll learn functions for manipulating pictures later, like getColor, setColor, and repaint

10/15/14  

6  

Program for showing a picture

#Let the user pick an image file

#Then show the image

#All the print statements are optional

def showPic():

fileName = pickAFile()

print fileName #prints location & name

picObj = makePicture(fileName)

print picObj #prints image info

y = show(picObj)

print y #prints None (nothing returned)

Alternative way

#Let the user pick an image file

#Then show the image

def showPic():

show ( makePicture( pickAFile() ) )

u  Recall: comments u  # single line comment

u ‘‘‘ multiple lines of comment

’’’

10/15/14  

7  

Running your program

u  Remember to load it first! u  Every time you make a change, you will need to

load your program

u  Enter the name of the function and the parameters in parenthesis in the command area u  showPic()

The Most Common JES mistake: Forgetting to Load

Don’t say def showpic(): in command area

Next: Playing sound

10/15/14  

8  

Sound Functions

u  makeSound(filename) creates and returns a sound object u  Allowed file types: wav, aiff, au

u  play(soundObj) makes the sound object play (but doesnʼ’t wait until itʼ’s done)

u  blockingPlay(soundObj) waits for the sound to finish

u  Weʼ’ll learn more later like getSample and setSample

Program

#Play a sound file

def playMusic():

fileName = pickAFile()

soundObj = makeSound(fileName)

play(soundObj)

10/15/14  

9  

A function with parameter

#Play a sound file

def playMusic(fileName):

soundObj = makeSound(fileName)

play(soundObj)

u  To run this program u  Load it first

u  In command area:

u  playMusic(‘c:/backpack.wav’) # For Windows

u  playMusic(‘Users/YourName/Desktop/backpack.wav’) # For Mac

Reading: Introduction to Computing and Programming in Python: A Multimedia Approach [Guzdial] Chapter 3: Modifying Pictures using Loops

Pictures, Pixels, and Colors

10/15/14  

10  

Colors

u  Our perception of color u  spectral vs. mixture

u  Luminance

Pixels

u  We digitize pictures into lots of little dots

u  Enough dots and it looks like a continuous whole to our eye u  Our eye has limited resolution

u  Each picture element is referred to as a pixel

10/15/14  

11  

Pixel objects

u Pixels are picture elements u  Each pixel object

knows its color

u  It also knows where it is in its picture

When we zoom the picture to 500%, we can see individual pixels.

New!!!

A Picture is a matrix of pixels

u  Itʼ’s not a continuous line of elements, that is, a one-dimensional array

u  A picture has two dimensions: Width and Height

u  We need a two-dimensional array: a matrix

10/15/14  

12  

Referencing a matrix

u  We talk about positions in a matrix as (x,y), or (horizontal, vertical)

u  Element (1,0) in the matrix at left is the value 12

u  Element (0,2) is 6

Encoding color

u  Each pixel encodes color at that position in the picture

u  Lots of encodings for color u  Printers use CMYK: Cyan, Magenta, Yellow, and blacK.

u  Others use HSB for Hue, Saturation, and Brightness (also called HSV for Hue, Saturation, and Value)

u  Weʼ’ll use the most common for computers u  RGB: Red, Green, Blue

10/15/14  

13  

Encoding RGB

u  Each component color (red, green, and blue) is encoded as a single byte

u  Colors go from (0,0,0) to (255,255,255) u  If all three components are the

same, the color is in greyscale

u  (200,200,200) at (3,1)

u  (0,0,0) (at position (3,0) in example) is black

u  (255,255,255) is white

How much can we encode in 8 bits?

u  Letʼ’s walk it through. u  If we have one bit, we can represent two patterns:

0 and 1.

u  If we have two bits, we can represent four patterns: 00, 01, 10, and 11.

u  If we have three bits, we can represent eight patterns: 000, 001, 010, 011, 100, 101, 110, 111

u  General rule: In n bits, we can have 2n patterns u  In 8 bits, we can have 28 patterns, or 256

u  If we make one pattern 0, then the highest value we can represent is 28-1, or 255

10/15/14  

14  

Is that enough?

u  Weʼ’re representing color in 24 (3 * 8) bits. u  Thatʼ’s 16,777,216 (224) possible colors

u  Our eye can discern millions of colors, so itʼ’s probably pretty close

u  Some graphics systems support 32 bits per pixel u  May be more pixels for color, or an additional 8

bits to represent 256 levels of transparency

Size of images

320 x 240 image

640 x 480 image

1024 x 768 monitor

24 bit color (3 bytes)

230,400 bytes

921,600 bytes 2,359,296 bytes

32 bit color (4 bytes)

307,200 bytes

1,228,800 bytes

3,145,728 bytes

10/15/14  

15  

Decomposition of a picture

Picture

Pixel

Color

show repaint getPixel getPixels getWidth getHeight

writePictureTo

getRed, getGreen, getBlue setRed, setGreen, setBlue

getColor setColor

getX, getY

makeColor pickAColor

makeDarker, makeLighter

Reminder: Manipulating Pictures

>>> file = pickAFile() >>> print file /Users/yourName/mediasources/barbara.jpg >>> picture = makePicture(file) # picture is a picture object >>> show(picture) >>> print picture Picture, filename /Users/yourName/mediasources/barbara.jpg height 294 width 222

10/15/14  

16  

What is a “picture object?”

u  An “encoding” that represents an image u  Knows its height and width

u  Knows its filename

u  Knows its window if itʼ’s opened (via show and repainted with repaint)

Getting pixels

>>> pixel=getPixel(picture,1,1) >>> print pixel Pixel, color=color r=168 g=131 b=105 >>> pixels=getPixels(picture) >>> print pixels[0] Pixel, color=color r=168 g=131 b=105

getPixel(picture,x,y) gets a single pixel

getPixels(picture) gets all of them in a list

(In the following code, picture is a picture object – previously defined.)

10/15/14  

17  

What can we do with a pixel?

•  getRed, getGreen, and getBlue are functions that take a pixel as input and return a value between 0 and 255

•  setRed, setGreen, and setBlue are functions that take a pixel as input and a value between 0 and 255

We can also get, set, and make Colors u  getColor takes a pixel as input and returns a

Color object with the color at that pixel u  setColor takes a pixel as input and a Color,

then sets the pixel to that color u  makeColor takes red, green, and blue values

(in that order) between 0 and 255, and returns a Color object

u  pickAColor lets you use a color chooser and returns the chosen color

u  We also have functions that can makeLighter and makeDarker an input color

10/15/14  

18  

Demo: pixels and colors (continued from previous slides)

>>> print getRed(pixel) 168 >>> setRed(pixel,255) >>> print getRed(pixel) 255 >>> color=getColor(pixel) >>> print color color r=255 g=131 b=105 >>> newColor=makeColor(0,100,0) >>> print newColor color r=0 g=100 b=0 >>> setColor(pixel,newColor) >>> print getColor(pixel) color r=0 g=100 b=0

Demo >>>color = makeColor(81,63,51) >>>newcolor = makeColor(255,51,51) >>> print color color r=81 g=63 b=51 >>> print newcolor color r=255 g=51 b=51 >>> print distance(color,newcolor) 174.41330224498358 >>> print color color r=168 g=131 b=105 >>> print makeDarker(color) color r=117 g=91 b=73 >>> print color color r=117 g=91 b=73 >>> newcolor=pickAColor() >>> print newcolor color r=255 g=51 b=51

Another example (not continued from the previous slide).

10/15/14  

19  

Distance between colors u  How close are two colors? u  Distance

u  Pretend itʼ’s cartesian coordinate system u  Distance between two points:

u  Distance between two colors: u  distance(color1, color2) function returns the following

We can change pixels directly…

>>> file="/Users/yourName/mediasources/barbara.jpg" >>> pict=makePicture(file) >>> show(pict) >>> setColor(getPixel(pict,10,100),yellow) >>> setColor(getPixel(pict,11,100),yellow) >>> setColor(getPixel(pict,12,100),yellow) >>> setColor(getPixel(pict,13,100),yellow) >>> repaint(pict)

But that’s really dull and boring to ���change each pixel at a time…���Isn’t there a better way?

10/15/14  

20  

Use a loop!

#Take a picture object as parameter���#Make every pixel 50% less red def decreaseRed(picture): for p in getPixels(picture): originalRed = getRed(p) setRed(p, originalRed*0.5)

In command area: >>> file="/Users/yourName/mediasources/barbara.jpg" >>> picture=makePicture(file) >>> show(picture) >>> decreaseRed(picture) >>> repaint(picture)

You can also write a function pixelDemo() with these lines

there New!

Converting to grayscale u  We know that if red=green=blue, we get gray

u  But what value do we set all three to?

u  What we need is a value representing the darkness of the color, the luminance

u  There are lots of ways of getting it, but one way that works reasonably well is dirt simple—simply take the average:

10/15/14  

21  

Converting to grayscale

def grayScale(picture): for p in getPixels(picture): intensity = (getRed(p)+getGreen(p)+getBlue(p))/3 setColor(p, makeColor(intensity,intensity,intensity))

Alternatives?

Weighted grayscale

u  newRed = originalRed * 0.299

u  newGreen = originalGreen * 0.589

u  newBlue = originalBlue * 0.114

u  intensity = newRed + newGreen + newBlue

def weightedGrayScale(picture):     for p in getPixels(picture):         intensity = getRed(p)* 0.299 + getGreen(p)* 0.589 + getBlue(p)* 0.114         setRed(p, intensity)         setGreen(p, intensity)         setBlue(p, intensity)     repaint(picture)

10/15/14  

22  

Can we reverse grayscale and get back the original colors? No.

u  Weʼ’ve lost information u  We no longer know what the ratios are between

the reds, the greens, and the blues

u  We no longer know any particular value.

Creating a negative Original negative negative-negative

10/15/14  

23  

Creating a negative

u  Inverse the colors u  newRed = 255 – originalRed

u  Similar for green and blue

def negative(picture): for px in getPixels(picture): r = getRed(px) g = getGreen(px) b = getBlue(px) negativeColor = makeColor(255-r, 255-g, 255-b) setColor(px, negativeColor)

Alternatives?

Class Participation Homework 2 (1 point) u  Practice the codes from sections 3.4

(creating a sunset) and 3.5 (lightening and darkening)

u  Submit the printouts this Thursday

10/15/14  

24  

Advanced Image Manipulation Techniques

Ch. 4 [Guzdial]

Problem: convert a rectangular part of a picture to grayscale u  Not all pixels will be converted to grayscale

u  How can we select pixels from a rectangular area of the picture? u  Top-left corner: (startX, startY)

u  Width of the rectangle

u  Height of the rectangle

10/15/14  

25  

Solution

Problem: Mirror a picture vertically

Issues: 1.  How to mirror a pixel? 2.  Select pixels to be mirrored

•  What would be the range of x values? •  What would be the range of y values?

10/15/14  

26  

Solution

Alternative way of creating picture objects u  setMediaPath()

u  select a “source” folder for pictures

u  fileName = getMediaPath(“jungle2.jpg”)

u  picObject = makePicture(fileName)

10/15/14  

27  

Problem: Crop part of an image into a newly created blank canvas

Solution

u  Note: following solution is a simpler version of textbook programs 27 and 28

Which folder? The folder selected

for setMediaPath()

10/15/14  

28  

Problem: Rotate a picture CCW and save it in a new canvas

Solution

u  Simpler (and different) version of textbook’s programs 32 and 33

10/15/14  

29  

Class participation HW 3

u  Class participation HW 3 (practice problems) u  Mirroring the temple and making collage (from Ch.

4).

u  Submit the printouts by Tuesday, Oct 21. Try to solve these in a different way.

Advanced Image Manipulation Techniques

Ch. 5 [Guzdial] 10/16

10/15/14  

30  

Announcements u  Class participation HW 3 (practice problems)

u  Mirroring the temple and making collage (from Ch. 4).

u  Chromakey (Ch. 5)

u  Submit the printouts by Tuesday, Oct 21. Try to solve these (especially, Ch 4) in a different way.

u  Lab Assignment 5 is due on Monday, Oct 20 (11:59pm) u  Note: The two versions of flipping will produce two

different flipped images. That is fine, but you still need to do both versions.

u  Lab Assignment 6 is due on Friday, Oct 24 (11:59pm)

u  Agenda for tomorrow’s lab u  Work on lab assignments 5 and 6

Problem: red-eye reduction

u  Write a function for red-eye reduction u  Parameters: picture object, coordinates for the

box, replacement color

u  Show the modified picture object

10/15/14  

31  

Algorithm

for every x coordinate within the box

for every y coordinate within the box

px = pixel at (x, y)

if distance(red, px’s color) < 165

change px’s color

show the modified picture

return the modified picture

Complexity?

Code

10/15/14  

32  

Problem: Blur a picture

u  How to blur?

u  Algorithm

for all reasonable x coordinates for all reasonable y coordinates Make pixel (x, y)’s r, g, b components the average of surrounding pixels’ r, g, b, resp.

Code

10/15/14  

33  

Problem: edge detection

Tracking moving objects

10/15/14  

34  

How to detect edges?

u  What is an edge?

u  There is an abrupt change of color

u  Algorithm

for each x coordinate

for each y coordinate

if the luminance of (x, y) pixel is very different from that of its neighboring pixels, (x, y) is part of an edge

u  In addition to above, we want to create and return a new picture that shows the edges

Code

10/15/14  

35  

Class participation HW3 u  Mirroring temple and collage (Ch. 4) u  Chromakey (Program 47)

u  Here, the source and bg parameters are picture objects. File names as arguments will not work. You will need to create the picture objects from the command area first. You can then supply those picture objects to the chromakey function. Here’s one way of creating these picture objects (type the following in command area):

u  setMediaPath()

u  file = getMediaPath(“blue-mark.jpg”)

u  source = makePicture(file)

u  file = getMediaPath(“moon-surface.jpg”)

u  bg = makePicture(file)

u  chromakey(source, bg) #call the chromakey function #with proper arguments