Multimedia Beta-version EjUAnPc2VA#t=20.
-
date post
19-Dec-2015 -
Category
Documents
-
view
216 -
download
1
Transcript of Multimedia Beta-version EjUAnPc2VA#t=20.
CS338
MultimediaBeta-version
httpwwwyoutubecomwatch_popupv=jEjUAnPc2VAt=20
Overview -1
Required background C C++ or Java CS240 (Data Structures) or CS212 (for
Engineers) This is NOT a graphics course (see
CS460560) Labs
Data compression Elements of graphics in OpenGL Photo manipulation
GIMP Steganography
Creating amp Rendering modelscopy DJ Foreman 2012 2
Topics (general order of coverage)
Compression EncodingDecoding Codecs Huffman encoding
example lab Working with Images
Image creation Photography Drawing RIB (Renderman Interface
Bytestream) Modeling
OpenGL ndash lab Makehuman 3DCanvas etc
lab Image manipulation
Gimp Photoshop etc lab Rendering Bitmaps
Ray tracing
HTML CSS amp Javascript - lab
Animation Hand drawn Program generated
Stop-motion Alice lab DirectX amp OpenGL
Hardware acceleration RIB revisited lab Shaders
Steganography lab Additional topics (time
permitting) Augmented reality Geospatial data systems
copy DJ Foreman 2012 3
Software we might use (all free)
Modelers and environments Ayam ndash ortho amp 3D 3DCanvas ndash 3D Blender ndash ortho amp 3D Makehuman - 3D
Renderers Aqsis Pixie ndash no GUI file input
copy DJ Foreman 2012 4
5
Image formats
Bitmap Also called ldquoraster imagesrdquo
Stored as individual pixels Resolution dependent Screen resolution = 100 ppi Printing requires 150-300 ppi
Vector Scalable primitives amp formulas Resolution independent
copy DJ Foreman 2012
6
Bitmap formats
BMP (program generated) GIF (ldquo but copy limits on use of format) JPEG JPG (photo) PNG (like GIF wo copy limits) PICT(ldquo Mac BMP+Vector) PCX (ldquo Personal Computer Exchange) PSD (Photoshop) TIFF (ldquo can also be a container)
copy DJ Foreman 2012
7
Vector formats
AI (Adobe Illustrator) CDR (CorelDRAW) CMX (Corel Exchange) CGM (Computer Graphics Metafile ) DXF (AutoCAD ) WMF (Windows Metafile )
copy DJ Foreman 2012
8copy DJ Foreman 2012
Compression
Codecs A device or program for encoding andor
decoding a digital data stream or signal Encoding A-gtD for storage or transmission DecodingD-gtA for playback Lossy vs lossless Raw uncompressed Pulse Code Modulation
(PCM) audio a digital representation of an analog signal where the magnitude of the signal is sampled regularly at uniform intervals Eg (441 kHz 16 bit stereo as represented on an
audio CD or in a wav or aiff file) is a standard across multiple platforms
copy DJ Foreman 2012 9
Codecs - 2
May emphasize aspects of data Video
motion vs color Audio
latency (cell phones) vs high-fidelity (music) Data size
transmission speed or storage size vs data loss
copy DJ Foreman 2012 10
Codec References
Forensic Computing A Practitioners Guide by T Sammes amp B Jenkinson (Springer 2000) ISBN 9781852332990
httpwwwgarykesslernetlibraryfile_sigshtml
copy DJ Foreman 2012 11
Considerations
Tradeoffs compression speed compressed data size quality (data loss)
Areas of study information theory rate-distortion theory
Popular algorithms (all lossless) Lempel-Ziv (LZ) LZW (fast decompression) LZR (LZ-Renau) (ZIP files) LZW (Lempel-Ziv-Welch) (for GIF files)
copy DJ Foreman 2012 12
Lossless Encoding
Huffman compression Bit string representing any symbol is never a
prefix of the bit string representing any other symbolie if 010 is the code for a symbol then no other symbol starts with 010
Frequency table must be known computable or included
Lossless ndash every character gets encoded Arithmetic encoding
Probabilistic algorithm Slightly superior to Huffman but often
patentedcopy DJ Foreman 2012 13
Methodology
Compute a model of the data eg a Huffman tree based on probability Probability determined from input file
Map the data according to the model Read 1 symbol Search the model (tree) for that symbol
If itrsquos a Huffman tree going left=0 right=1 Append each 0 or 1 to the code string When symbol is found you are done Note all symbols will have lt 8 bits (thus
compression)
copy DJ Foreman 2012 14
15
Huffman lab ndash more details Read write are always in bytes
You may need to pad You may have extraneous letters in the output
due to padding You donrsquot need to add a header to specify size
You have 2 weeks to do the lab Code may be in C C++ or Java The readme specifies how to compile the
program Encoded data is sent to a binary file or a simple
text file (but isnrsquot ldquoreadablerdquo as text) Encoded data can only be ldquocheckedrdquo with a
decodercopy DJ Foreman 2012
16
Sample of prefix coding amp compression
copy DJ Foreman 2012
B e a r c a t s (8 bytes of text)010101100110001110001111 encoded as123456781234567812345678 3 bytes
A simple Huffman-like code(wo probability basis) 0 1
A 110
B 010
C 0001
E 10
R 011
S 111
T 001
Compression-Model types
Static Read all the data Compute frequencies Re-read the data amp encode
Dynamic Build simple basic model Modify model as more data is processed
copy DJ Foreman 2012 17
Standard Compressed-File Content
Three parts 24-byte header with magic (defines
codec) variable-length annotation block contiguous segment of audio data
Storage methodology network (big-endian) byte order multi-byte audio data may require byte
reversal in order to operate on it by the arithmetic unit of certain processors
copy DJ Foreman 2012 18
Example 1 Header file for AU files
copy DJ Foreman 2012 19
typedef unsigned long u_32 unsigned 32-bit integer
typedef struct
u_32 magic the ldquomagic numberrdquo
u_32 hdr_size byte offset to start of data
u_32 data_size length (optional)
u_32 encoding data encoding enumeration
u_32 sample_rate samples per second
u_32 channels of interleaved channels
Audio_filehdr
Following is from include ltmultimedialibaudiohgt
AUDIO_FILE_MAGIC ((u_32)0x2e736e64) ldquosndrdquo
Audio Encoding EnumerationsAUDIO_FILE_ENCODING_MULAW_8 (1) 8-bit ISDN u-law
AUDIO_FILE_ENCODING_LINEAR_8 (2) 8-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_16 (3) 16-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_32 (5) 32-bit linear PCM
AUDIO_FILE_ENCODING_FLOAT (6) 32-bit IEEE floating point
AUDIO_FILE_ENCODING_DOUBLE (7) 64-bit IEEE floating point
AUDIO_FILE_ENCODING_ADPCM_G721 (23) 4-bit CCITT g721 ADPCM
AUDIO_FILE_ENCODING_ADPCM_G723_3 (25) CCITT g723 3-bit ADPCM
AUDIO_FILE_ENCODING_ALAW_8 (27) 8-bit ISDN A-law
copy DJ Foreman 2012 20
ldquoLinearrdquo values are SIGNED intrsquosFloats are signed zero-centered normalized to ( -10 lt= x lt= 10 )
Example 2 ZIP files (PkWare)Purpose Size in bytes
Signature header 4
Required version 2
GP flags 2
Method 2
Last mod time 2
Last mod date 2
CRC-32 4
Compressed size 4
Uncompressed size 4
Filename length 2
Extra field length 2
File name variable
extra variable
copy DJ Foreman 2012 21
Ref httplivedocsadobecomflex3htmlhelphtmlcontent=ByteArrays_3html
More info on encoding httpwwwpkwarecomdocumentscase
studiesAPPNOTETXT
httpwwwfileinfocomfiletypescompressed
copy DJ Foreman 2012 22
Recording Bit-Rate
Constant (CBR) rate at which a codecs output data should be
consumed is constant Max bit-rate matters not the average Uses all available bandwidth Not good for storage (lossy)
Variable (VBR) Quantity of datatime unit (for output) varies Better quality for audio and video Slower to encode Supported by most portable devices post 2006
copy DJ Foreman 2012 23
Lab assignment ndash 1 ndash part 1
Write a program with 1 parameter If you are on an encoding team
Open a test file (eg ldquoxyztxtrdquo) Compress it using simple Huffman compression using
the frequency table from my FTP site Output compressed file (eg ldquoxyzencrdquo) to SAME folder
as input
If you are on a decoding team Open compressed file ldquoxyzencrdquo Decompress the input file Output file ldquoxyzdecrdquo so it can be compared to ldquoxyztxtrdquo
The parameter is the full input file path amp name (ie file names MUST NOT be hard-coded)
Remember a code can be a single bit
copy DJ Foreman 2012 24
25
Lab assignment 1 ndash part 2
Use ONLY MY TABLE for the frequencies Use ONLY uppercase letters of the table Do NOT use the special character
tables Find someone in class with the
opposite program and TEST your program to see if it properly encodes or decodes a message
Grading tree construction tree following proper binary output (encoders) proper character output (decoders)
copy DJ Foreman 2012
Working with ImagesModeling
Containers
A container is a FILE format NOT a code scheme
Eg AVI amp WAV are container formats Formats for storage Independent of encoding or content
Others Ogg ASF QuickTime RealMedia Matroska
DivX and MP4
copy DJ Foreman 2012 27
28
Movie (metadata)
Track
Media
Media Information
Container file format (MP4QT)
Chunk map
Comp time
Chunk offset
Key frame
Desc
Decode time
Priority map
Size map
Sample Table
Media Handler
Data information
Media header
Video header
Track header
Movie header
copy DJ Foreman 2012
Pixels A pixel is 3 color-dots (squares) in a collection of dots
(squares) making up the picture Each color-dot is for one of the three primary colors
RGB Minimum of 1 byte per color (depending on bit-depth
which varies with camera) thus at least 3 bytes per pixel
A 10MP camera (30 million color-dots) needs 30M bytes This gets compressed by the camera (JPEG format) Final file size = 30 million bytes divided by the amount
of compression (compression ratio) An 81 compression ratio would reduce that 30 million
bytes to 375 megabytes (as done on a 4 MP camera) A RAW file has NO compression
copy DJ Foreman 2012 29
Image creation
Basic mechanisms Photography
Film Digital ndashPhotoshop Gimp
Drawing Line BrushModeling
Do it yourself OpenGL (2x and 3x)
Modelingrendering programs Breeze designer Ayam 3DCanvas Makehuman Alice
copy DJ Foreman 2012 30
Photography (film) Silver halide crystals (AgCl AgBr AgI)
Exposure to light turns them black Developer removes the loose (exposed) halide leaving
pure silver (a negative image) Fixing bath flushes out unexposed AgHalide
Greatest desire of photographers - a medium with High speed (gathers light quickly) Low noise (no undesired artifacts) High resolution (very detailed images)
Visible lines per mm (lpm) at a specified contrast level
Notes Larger ldquoclumpsrdquo are faster but limit resolution Smaller ldquoclumpsrdquo are hard to coat evenly Randomness of coating prevents Moireacute patternshttpenwikipediaorgwikiMoirC3A9_pattern
copy DJ Foreman 2012 31
FYI ndash a little chemistry
Ag+ X- + hv rarr Ag+ + X + e-
1 Radiation frees the Br2 1 Ag ion + 1 electron (Ag+ + e- rarr Ag0) -gt 1 Ag atom3 Neighbors are now ldquosensitizedrdquo 4 Developing (Reducing) converts exposed AgX to
black metallic Ag and free X5 Remaining AgX and free X are washed away (after
mixing with thiosulfate to make it soluble)
This is one silver (Ag) ldquograinrdquo in the ldquolatent imagerdquo X is any halide (Br Cl etc)
copy DJ Foreman 2012 32
crystal
Radiation (light)
Resolution 35 mm film for comparison
A 36 x 24 mm frame of ISO 100-speed film contains (asymp) the equivalent of 20 million pixels 1 There are NO pixels in film
Full-frame digital (36 x 24mm = 32 image ratio) Canon EOS 5D Nikon D3 (21MP) - $5K2
Medium format digital (6 x 45cm) Phase One P40+ (39MP) ($22K)
APS-C sized sensor (asymp24x16mm = 32 image ratio) (note 2009 data below)
Nikon D90 (123MP) - $900 Canon Rebel (15MP) $700 Used in most DSLRrsquos (note image ratio 32)
Other SONY Alpha (note image ratio 43)
1 Langford Michael Basic Photography (7th Ed 2000) Oxford Focal Press ISBN 0 240 51592 7
2 Prices as of 2009 given for comparison purposes onlycopy DJ Foreman 2012 33
Photography (digital) Image capture formats
RAW ndash 3 separate images (RGB) [12 bitspixel (4288 2848)]8=18318336 bytes per
picture JPEG ndash compressed (16x 8x 4x) Some cameras do BOTH (RAW + JPEG) for each image
Pixel notes Sensor size 35mm (or larger) vs APS-C Pixel count ndash any increase rarr
decrease in pixel size or increase in sensor size Pixel size ndash smaller pixels give us
Greater resolution (finer details) - Good More ldquonoiserdquo (incorrect values) - bad Possible ldquoadjacencyrdquo issues - bad Loss of sensitivity - bad
copy DJ Foreman 2012 34
35
Some Comparisons
Digital media vs film Pixels are the same (RGB) Pixel size varies with device (mfgr
dependency) Pixel density varies Zooming digitally does NOT give you more
detail (in spite of what they do on TV shows)
Optical zoom is useful Uses optical magnification of the image on
the target Expansion of light rays not multiplication of
pixelscopy DJ Foreman 2012
Working Definitions - 1
Graphic Model data structure nodes represent random variables Pairs of nodes connected by arcs correspond
to variables that are not independent Graphic modeling
using a Graphic Model to simulate a real-world object (this is not a formal definition)
Rendering generating a 2D image from a 3D graphic model
copy DJ Foreman 2012 36
37
Working Definitions - 2
Geometric primitives ndash Points line segments and polygons Described by vertices
Control points ndash Special points attached to a node on a
Beacutezier curve Alter the shape and angle of adjacent curve
segment Evaluators ndash
Functions that interpolate a set of control points
Produce a new set of control pointscopy DJ Foreman 2012
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
Overview -1
Required background C C++ or Java CS240 (Data Structures) or CS212 (for
Engineers) This is NOT a graphics course (see
CS460560) Labs
Data compression Elements of graphics in OpenGL Photo manipulation
GIMP Steganography
Creating amp Rendering modelscopy DJ Foreman 2012 2
Topics (general order of coverage)
Compression EncodingDecoding Codecs Huffman encoding
example lab Working with Images
Image creation Photography Drawing RIB (Renderman Interface
Bytestream) Modeling
OpenGL ndash lab Makehuman 3DCanvas etc
lab Image manipulation
Gimp Photoshop etc lab Rendering Bitmaps
Ray tracing
HTML CSS amp Javascript - lab
Animation Hand drawn Program generated
Stop-motion Alice lab DirectX amp OpenGL
Hardware acceleration RIB revisited lab Shaders
Steganography lab Additional topics (time
permitting) Augmented reality Geospatial data systems
copy DJ Foreman 2012 3
Software we might use (all free)
Modelers and environments Ayam ndash ortho amp 3D 3DCanvas ndash 3D Blender ndash ortho amp 3D Makehuman - 3D
Renderers Aqsis Pixie ndash no GUI file input
copy DJ Foreman 2012 4
5
Image formats
Bitmap Also called ldquoraster imagesrdquo
Stored as individual pixels Resolution dependent Screen resolution = 100 ppi Printing requires 150-300 ppi
Vector Scalable primitives amp formulas Resolution independent
copy DJ Foreman 2012
6
Bitmap formats
BMP (program generated) GIF (ldquo but copy limits on use of format) JPEG JPG (photo) PNG (like GIF wo copy limits) PICT(ldquo Mac BMP+Vector) PCX (ldquo Personal Computer Exchange) PSD (Photoshop) TIFF (ldquo can also be a container)
copy DJ Foreman 2012
7
Vector formats
AI (Adobe Illustrator) CDR (CorelDRAW) CMX (Corel Exchange) CGM (Computer Graphics Metafile ) DXF (AutoCAD ) WMF (Windows Metafile )
copy DJ Foreman 2012
8copy DJ Foreman 2012
Compression
Codecs A device or program for encoding andor
decoding a digital data stream or signal Encoding A-gtD for storage or transmission DecodingD-gtA for playback Lossy vs lossless Raw uncompressed Pulse Code Modulation
(PCM) audio a digital representation of an analog signal where the magnitude of the signal is sampled regularly at uniform intervals Eg (441 kHz 16 bit stereo as represented on an
audio CD or in a wav or aiff file) is a standard across multiple platforms
copy DJ Foreman 2012 9
Codecs - 2
May emphasize aspects of data Video
motion vs color Audio
latency (cell phones) vs high-fidelity (music) Data size
transmission speed or storage size vs data loss
copy DJ Foreman 2012 10
Codec References
Forensic Computing A Practitioners Guide by T Sammes amp B Jenkinson (Springer 2000) ISBN 9781852332990
httpwwwgarykesslernetlibraryfile_sigshtml
copy DJ Foreman 2012 11
Considerations
Tradeoffs compression speed compressed data size quality (data loss)
Areas of study information theory rate-distortion theory
Popular algorithms (all lossless) Lempel-Ziv (LZ) LZW (fast decompression) LZR (LZ-Renau) (ZIP files) LZW (Lempel-Ziv-Welch) (for GIF files)
copy DJ Foreman 2012 12
Lossless Encoding
Huffman compression Bit string representing any symbol is never a
prefix of the bit string representing any other symbolie if 010 is the code for a symbol then no other symbol starts with 010
Frequency table must be known computable or included
Lossless ndash every character gets encoded Arithmetic encoding
Probabilistic algorithm Slightly superior to Huffman but often
patentedcopy DJ Foreman 2012 13
Methodology
Compute a model of the data eg a Huffman tree based on probability Probability determined from input file
Map the data according to the model Read 1 symbol Search the model (tree) for that symbol
If itrsquos a Huffman tree going left=0 right=1 Append each 0 or 1 to the code string When symbol is found you are done Note all symbols will have lt 8 bits (thus
compression)
copy DJ Foreman 2012 14
15
Huffman lab ndash more details Read write are always in bytes
You may need to pad You may have extraneous letters in the output
due to padding You donrsquot need to add a header to specify size
You have 2 weeks to do the lab Code may be in C C++ or Java The readme specifies how to compile the
program Encoded data is sent to a binary file or a simple
text file (but isnrsquot ldquoreadablerdquo as text) Encoded data can only be ldquocheckedrdquo with a
decodercopy DJ Foreman 2012
16
Sample of prefix coding amp compression
copy DJ Foreman 2012
B e a r c a t s (8 bytes of text)010101100110001110001111 encoded as123456781234567812345678 3 bytes
A simple Huffman-like code(wo probability basis) 0 1
A 110
B 010
C 0001
E 10
R 011
S 111
T 001
Compression-Model types
Static Read all the data Compute frequencies Re-read the data amp encode
Dynamic Build simple basic model Modify model as more data is processed
copy DJ Foreman 2012 17
Standard Compressed-File Content
Three parts 24-byte header with magic (defines
codec) variable-length annotation block contiguous segment of audio data
Storage methodology network (big-endian) byte order multi-byte audio data may require byte
reversal in order to operate on it by the arithmetic unit of certain processors
copy DJ Foreman 2012 18
Example 1 Header file for AU files
copy DJ Foreman 2012 19
typedef unsigned long u_32 unsigned 32-bit integer
typedef struct
u_32 magic the ldquomagic numberrdquo
u_32 hdr_size byte offset to start of data
u_32 data_size length (optional)
u_32 encoding data encoding enumeration
u_32 sample_rate samples per second
u_32 channels of interleaved channels
Audio_filehdr
Following is from include ltmultimedialibaudiohgt
AUDIO_FILE_MAGIC ((u_32)0x2e736e64) ldquosndrdquo
Audio Encoding EnumerationsAUDIO_FILE_ENCODING_MULAW_8 (1) 8-bit ISDN u-law
AUDIO_FILE_ENCODING_LINEAR_8 (2) 8-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_16 (3) 16-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_32 (5) 32-bit linear PCM
AUDIO_FILE_ENCODING_FLOAT (6) 32-bit IEEE floating point
AUDIO_FILE_ENCODING_DOUBLE (7) 64-bit IEEE floating point
AUDIO_FILE_ENCODING_ADPCM_G721 (23) 4-bit CCITT g721 ADPCM
AUDIO_FILE_ENCODING_ADPCM_G723_3 (25) CCITT g723 3-bit ADPCM
AUDIO_FILE_ENCODING_ALAW_8 (27) 8-bit ISDN A-law
copy DJ Foreman 2012 20
ldquoLinearrdquo values are SIGNED intrsquosFloats are signed zero-centered normalized to ( -10 lt= x lt= 10 )
Example 2 ZIP files (PkWare)Purpose Size in bytes
Signature header 4
Required version 2
GP flags 2
Method 2
Last mod time 2
Last mod date 2
CRC-32 4
Compressed size 4
Uncompressed size 4
Filename length 2
Extra field length 2
File name variable
extra variable
copy DJ Foreman 2012 21
Ref httplivedocsadobecomflex3htmlhelphtmlcontent=ByteArrays_3html
More info on encoding httpwwwpkwarecomdocumentscase
studiesAPPNOTETXT
httpwwwfileinfocomfiletypescompressed
copy DJ Foreman 2012 22
Recording Bit-Rate
Constant (CBR) rate at which a codecs output data should be
consumed is constant Max bit-rate matters not the average Uses all available bandwidth Not good for storage (lossy)
Variable (VBR) Quantity of datatime unit (for output) varies Better quality for audio and video Slower to encode Supported by most portable devices post 2006
copy DJ Foreman 2012 23
Lab assignment ndash 1 ndash part 1
Write a program with 1 parameter If you are on an encoding team
Open a test file (eg ldquoxyztxtrdquo) Compress it using simple Huffman compression using
the frequency table from my FTP site Output compressed file (eg ldquoxyzencrdquo) to SAME folder
as input
If you are on a decoding team Open compressed file ldquoxyzencrdquo Decompress the input file Output file ldquoxyzdecrdquo so it can be compared to ldquoxyztxtrdquo
The parameter is the full input file path amp name (ie file names MUST NOT be hard-coded)
Remember a code can be a single bit
copy DJ Foreman 2012 24
25
Lab assignment 1 ndash part 2
Use ONLY MY TABLE for the frequencies Use ONLY uppercase letters of the table Do NOT use the special character
tables Find someone in class with the
opposite program and TEST your program to see if it properly encodes or decodes a message
Grading tree construction tree following proper binary output (encoders) proper character output (decoders)
copy DJ Foreman 2012
Working with ImagesModeling
Containers
A container is a FILE format NOT a code scheme
Eg AVI amp WAV are container formats Formats for storage Independent of encoding or content
Others Ogg ASF QuickTime RealMedia Matroska
DivX and MP4
copy DJ Foreman 2012 27
28
Movie (metadata)
Track
Media
Media Information
Container file format (MP4QT)
Chunk map
Comp time
Chunk offset
Key frame
Desc
Decode time
Priority map
Size map
Sample Table
Media Handler
Data information
Media header
Video header
Track header
Movie header
copy DJ Foreman 2012
Pixels A pixel is 3 color-dots (squares) in a collection of dots
(squares) making up the picture Each color-dot is for one of the three primary colors
RGB Minimum of 1 byte per color (depending on bit-depth
which varies with camera) thus at least 3 bytes per pixel
A 10MP camera (30 million color-dots) needs 30M bytes This gets compressed by the camera (JPEG format) Final file size = 30 million bytes divided by the amount
of compression (compression ratio) An 81 compression ratio would reduce that 30 million
bytes to 375 megabytes (as done on a 4 MP camera) A RAW file has NO compression
copy DJ Foreman 2012 29
Image creation
Basic mechanisms Photography
Film Digital ndashPhotoshop Gimp
Drawing Line BrushModeling
Do it yourself OpenGL (2x and 3x)
Modelingrendering programs Breeze designer Ayam 3DCanvas Makehuman Alice
copy DJ Foreman 2012 30
Photography (film) Silver halide crystals (AgCl AgBr AgI)
Exposure to light turns them black Developer removes the loose (exposed) halide leaving
pure silver (a negative image) Fixing bath flushes out unexposed AgHalide
Greatest desire of photographers - a medium with High speed (gathers light quickly) Low noise (no undesired artifacts) High resolution (very detailed images)
Visible lines per mm (lpm) at a specified contrast level
Notes Larger ldquoclumpsrdquo are faster but limit resolution Smaller ldquoclumpsrdquo are hard to coat evenly Randomness of coating prevents Moireacute patternshttpenwikipediaorgwikiMoirC3A9_pattern
copy DJ Foreman 2012 31
FYI ndash a little chemistry
Ag+ X- + hv rarr Ag+ + X + e-
1 Radiation frees the Br2 1 Ag ion + 1 electron (Ag+ + e- rarr Ag0) -gt 1 Ag atom3 Neighbors are now ldquosensitizedrdquo 4 Developing (Reducing) converts exposed AgX to
black metallic Ag and free X5 Remaining AgX and free X are washed away (after
mixing with thiosulfate to make it soluble)
This is one silver (Ag) ldquograinrdquo in the ldquolatent imagerdquo X is any halide (Br Cl etc)
copy DJ Foreman 2012 32
crystal
Radiation (light)
Resolution 35 mm film for comparison
A 36 x 24 mm frame of ISO 100-speed film contains (asymp) the equivalent of 20 million pixels 1 There are NO pixels in film
Full-frame digital (36 x 24mm = 32 image ratio) Canon EOS 5D Nikon D3 (21MP) - $5K2
Medium format digital (6 x 45cm) Phase One P40+ (39MP) ($22K)
APS-C sized sensor (asymp24x16mm = 32 image ratio) (note 2009 data below)
Nikon D90 (123MP) - $900 Canon Rebel (15MP) $700 Used in most DSLRrsquos (note image ratio 32)
Other SONY Alpha (note image ratio 43)
1 Langford Michael Basic Photography (7th Ed 2000) Oxford Focal Press ISBN 0 240 51592 7
2 Prices as of 2009 given for comparison purposes onlycopy DJ Foreman 2012 33
Photography (digital) Image capture formats
RAW ndash 3 separate images (RGB) [12 bitspixel (4288 2848)]8=18318336 bytes per
picture JPEG ndash compressed (16x 8x 4x) Some cameras do BOTH (RAW + JPEG) for each image
Pixel notes Sensor size 35mm (or larger) vs APS-C Pixel count ndash any increase rarr
decrease in pixel size or increase in sensor size Pixel size ndash smaller pixels give us
Greater resolution (finer details) - Good More ldquonoiserdquo (incorrect values) - bad Possible ldquoadjacencyrdquo issues - bad Loss of sensitivity - bad
copy DJ Foreman 2012 34
35
Some Comparisons
Digital media vs film Pixels are the same (RGB) Pixel size varies with device (mfgr
dependency) Pixel density varies Zooming digitally does NOT give you more
detail (in spite of what they do on TV shows)
Optical zoom is useful Uses optical magnification of the image on
the target Expansion of light rays not multiplication of
pixelscopy DJ Foreman 2012
Working Definitions - 1
Graphic Model data structure nodes represent random variables Pairs of nodes connected by arcs correspond
to variables that are not independent Graphic modeling
using a Graphic Model to simulate a real-world object (this is not a formal definition)
Rendering generating a 2D image from a 3D graphic model
copy DJ Foreman 2012 36
37
Working Definitions - 2
Geometric primitives ndash Points line segments and polygons Described by vertices
Control points ndash Special points attached to a node on a
Beacutezier curve Alter the shape and angle of adjacent curve
segment Evaluators ndash
Functions that interpolate a set of control points
Produce a new set of control pointscopy DJ Foreman 2012
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
Topics (general order of coverage)
Compression EncodingDecoding Codecs Huffman encoding
example lab Working with Images
Image creation Photography Drawing RIB (Renderman Interface
Bytestream) Modeling
OpenGL ndash lab Makehuman 3DCanvas etc
lab Image manipulation
Gimp Photoshop etc lab Rendering Bitmaps
Ray tracing
HTML CSS amp Javascript - lab
Animation Hand drawn Program generated
Stop-motion Alice lab DirectX amp OpenGL
Hardware acceleration RIB revisited lab Shaders
Steganography lab Additional topics (time
permitting) Augmented reality Geospatial data systems
copy DJ Foreman 2012 3
Software we might use (all free)
Modelers and environments Ayam ndash ortho amp 3D 3DCanvas ndash 3D Blender ndash ortho amp 3D Makehuman - 3D
Renderers Aqsis Pixie ndash no GUI file input
copy DJ Foreman 2012 4
5
Image formats
Bitmap Also called ldquoraster imagesrdquo
Stored as individual pixels Resolution dependent Screen resolution = 100 ppi Printing requires 150-300 ppi
Vector Scalable primitives amp formulas Resolution independent
copy DJ Foreman 2012
6
Bitmap formats
BMP (program generated) GIF (ldquo but copy limits on use of format) JPEG JPG (photo) PNG (like GIF wo copy limits) PICT(ldquo Mac BMP+Vector) PCX (ldquo Personal Computer Exchange) PSD (Photoshop) TIFF (ldquo can also be a container)
copy DJ Foreman 2012
7
Vector formats
AI (Adobe Illustrator) CDR (CorelDRAW) CMX (Corel Exchange) CGM (Computer Graphics Metafile ) DXF (AutoCAD ) WMF (Windows Metafile )
copy DJ Foreman 2012
8copy DJ Foreman 2012
Compression
Codecs A device or program for encoding andor
decoding a digital data stream or signal Encoding A-gtD for storage or transmission DecodingD-gtA for playback Lossy vs lossless Raw uncompressed Pulse Code Modulation
(PCM) audio a digital representation of an analog signal where the magnitude of the signal is sampled regularly at uniform intervals Eg (441 kHz 16 bit stereo as represented on an
audio CD or in a wav or aiff file) is a standard across multiple platforms
copy DJ Foreman 2012 9
Codecs - 2
May emphasize aspects of data Video
motion vs color Audio
latency (cell phones) vs high-fidelity (music) Data size
transmission speed or storage size vs data loss
copy DJ Foreman 2012 10
Codec References
Forensic Computing A Practitioners Guide by T Sammes amp B Jenkinson (Springer 2000) ISBN 9781852332990
httpwwwgarykesslernetlibraryfile_sigshtml
copy DJ Foreman 2012 11
Considerations
Tradeoffs compression speed compressed data size quality (data loss)
Areas of study information theory rate-distortion theory
Popular algorithms (all lossless) Lempel-Ziv (LZ) LZW (fast decompression) LZR (LZ-Renau) (ZIP files) LZW (Lempel-Ziv-Welch) (for GIF files)
copy DJ Foreman 2012 12
Lossless Encoding
Huffman compression Bit string representing any symbol is never a
prefix of the bit string representing any other symbolie if 010 is the code for a symbol then no other symbol starts with 010
Frequency table must be known computable or included
Lossless ndash every character gets encoded Arithmetic encoding
Probabilistic algorithm Slightly superior to Huffman but often
patentedcopy DJ Foreman 2012 13
Methodology
Compute a model of the data eg a Huffman tree based on probability Probability determined from input file
Map the data according to the model Read 1 symbol Search the model (tree) for that symbol
If itrsquos a Huffman tree going left=0 right=1 Append each 0 or 1 to the code string When symbol is found you are done Note all symbols will have lt 8 bits (thus
compression)
copy DJ Foreman 2012 14
15
Huffman lab ndash more details Read write are always in bytes
You may need to pad You may have extraneous letters in the output
due to padding You donrsquot need to add a header to specify size
You have 2 weeks to do the lab Code may be in C C++ or Java The readme specifies how to compile the
program Encoded data is sent to a binary file or a simple
text file (but isnrsquot ldquoreadablerdquo as text) Encoded data can only be ldquocheckedrdquo with a
decodercopy DJ Foreman 2012
16
Sample of prefix coding amp compression
copy DJ Foreman 2012
B e a r c a t s (8 bytes of text)010101100110001110001111 encoded as123456781234567812345678 3 bytes
A simple Huffman-like code(wo probability basis) 0 1
A 110
B 010
C 0001
E 10
R 011
S 111
T 001
Compression-Model types
Static Read all the data Compute frequencies Re-read the data amp encode
Dynamic Build simple basic model Modify model as more data is processed
copy DJ Foreman 2012 17
Standard Compressed-File Content
Three parts 24-byte header with magic (defines
codec) variable-length annotation block contiguous segment of audio data
Storage methodology network (big-endian) byte order multi-byte audio data may require byte
reversal in order to operate on it by the arithmetic unit of certain processors
copy DJ Foreman 2012 18
Example 1 Header file for AU files
copy DJ Foreman 2012 19
typedef unsigned long u_32 unsigned 32-bit integer
typedef struct
u_32 magic the ldquomagic numberrdquo
u_32 hdr_size byte offset to start of data
u_32 data_size length (optional)
u_32 encoding data encoding enumeration
u_32 sample_rate samples per second
u_32 channels of interleaved channels
Audio_filehdr
Following is from include ltmultimedialibaudiohgt
AUDIO_FILE_MAGIC ((u_32)0x2e736e64) ldquosndrdquo
Audio Encoding EnumerationsAUDIO_FILE_ENCODING_MULAW_8 (1) 8-bit ISDN u-law
AUDIO_FILE_ENCODING_LINEAR_8 (2) 8-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_16 (3) 16-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_32 (5) 32-bit linear PCM
AUDIO_FILE_ENCODING_FLOAT (6) 32-bit IEEE floating point
AUDIO_FILE_ENCODING_DOUBLE (7) 64-bit IEEE floating point
AUDIO_FILE_ENCODING_ADPCM_G721 (23) 4-bit CCITT g721 ADPCM
AUDIO_FILE_ENCODING_ADPCM_G723_3 (25) CCITT g723 3-bit ADPCM
AUDIO_FILE_ENCODING_ALAW_8 (27) 8-bit ISDN A-law
copy DJ Foreman 2012 20
ldquoLinearrdquo values are SIGNED intrsquosFloats are signed zero-centered normalized to ( -10 lt= x lt= 10 )
Example 2 ZIP files (PkWare)Purpose Size in bytes
Signature header 4
Required version 2
GP flags 2
Method 2
Last mod time 2
Last mod date 2
CRC-32 4
Compressed size 4
Uncompressed size 4
Filename length 2
Extra field length 2
File name variable
extra variable
copy DJ Foreman 2012 21
Ref httplivedocsadobecomflex3htmlhelphtmlcontent=ByteArrays_3html
More info on encoding httpwwwpkwarecomdocumentscase
studiesAPPNOTETXT
httpwwwfileinfocomfiletypescompressed
copy DJ Foreman 2012 22
Recording Bit-Rate
Constant (CBR) rate at which a codecs output data should be
consumed is constant Max bit-rate matters not the average Uses all available bandwidth Not good for storage (lossy)
Variable (VBR) Quantity of datatime unit (for output) varies Better quality for audio and video Slower to encode Supported by most portable devices post 2006
copy DJ Foreman 2012 23
Lab assignment ndash 1 ndash part 1
Write a program with 1 parameter If you are on an encoding team
Open a test file (eg ldquoxyztxtrdquo) Compress it using simple Huffman compression using
the frequency table from my FTP site Output compressed file (eg ldquoxyzencrdquo) to SAME folder
as input
If you are on a decoding team Open compressed file ldquoxyzencrdquo Decompress the input file Output file ldquoxyzdecrdquo so it can be compared to ldquoxyztxtrdquo
The parameter is the full input file path amp name (ie file names MUST NOT be hard-coded)
Remember a code can be a single bit
copy DJ Foreman 2012 24
25
Lab assignment 1 ndash part 2
Use ONLY MY TABLE for the frequencies Use ONLY uppercase letters of the table Do NOT use the special character
tables Find someone in class with the
opposite program and TEST your program to see if it properly encodes or decodes a message
Grading tree construction tree following proper binary output (encoders) proper character output (decoders)
copy DJ Foreman 2012
Working with ImagesModeling
Containers
A container is a FILE format NOT a code scheme
Eg AVI amp WAV are container formats Formats for storage Independent of encoding or content
Others Ogg ASF QuickTime RealMedia Matroska
DivX and MP4
copy DJ Foreman 2012 27
28
Movie (metadata)
Track
Media
Media Information
Container file format (MP4QT)
Chunk map
Comp time
Chunk offset
Key frame
Desc
Decode time
Priority map
Size map
Sample Table
Media Handler
Data information
Media header
Video header
Track header
Movie header
copy DJ Foreman 2012
Pixels A pixel is 3 color-dots (squares) in a collection of dots
(squares) making up the picture Each color-dot is for one of the three primary colors
RGB Minimum of 1 byte per color (depending on bit-depth
which varies with camera) thus at least 3 bytes per pixel
A 10MP camera (30 million color-dots) needs 30M bytes This gets compressed by the camera (JPEG format) Final file size = 30 million bytes divided by the amount
of compression (compression ratio) An 81 compression ratio would reduce that 30 million
bytes to 375 megabytes (as done on a 4 MP camera) A RAW file has NO compression
copy DJ Foreman 2012 29
Image creation
Basic mechanisms Photography
Film Digital ndashPhotoshop Gimp
Drawing Line BrushModeling
Do it yourself OpenGL (2x and 3x)
Modelingrendering programs Breeze designer Ayam 3DCanvas Makehuman Alice
copy DJ Foreman 2012 30
Photography (film) Silver halide crystals (AgCl AgBr AgI)
Exposure to light turns them black Developer removes the loose (exposed) halide leaving
pure silver (a negative image) Fixing bath flushes out unexposed AgHalide
Greatest desire of photographers - a medium with High speed (gathers light quickly) Low noise (no undesired artifacts) High resolution (very detailed images)
Visible lines per mm (lpm) at a specified contrast level
Notes Larger ldquoclumpsrdquo are faster but limit resolution Smaller ldquoclumpsrdquo are hard to coat evenly Randomness of coating prevents Moireacute patternshttpenwikipediaorgwikiMoirC3A9_pattern
copy DJ Foreman 2012 31
FYI ndash a little chemistry
Ag+ X- + hv rarr Ag+ + X + e-
1 Radiation frees the Br2 1 Ag ion + 1 electron (Ag+ + e- rarr Ag0) -gt 1 Ag atom3 Neighbors are now ldquosensitizedrdquo 4 Developing (Reducing) converts exposed AgX to
black metallic Ag and free X5 Remaining AgX and free X are washed away (after
mixing with thiosulfate to make it soluble)
This is one silver (Ag) ldquograinrdquo in the ldquolatent imagerdquo X is any halide (Br Cl etc)
copy DJ Foreman 2012 32
crystal
Radiation (light)
Resolution 35 mm film for comparison
A 36 x 24 mm frame of ISO 100-speed film contains (asymp) the equivalent of 20 million pixels 1 There are NO pixels in film
Full-frame digital (36 x 24mm = 32 image ratio) Canon EOS 5D Nikon D3 (21MP) - $5K2
Medium format digital (6 x 45cm) Phase One P40+ (39MP) ($22K)
APS-C sized sensor (asymp24x16mm = 32 image ratio) (note 2009 data below)
Nikon D90 (123MP) - $900 Canon Rebel (15MP) $700 Used in most DSLRrsquos (note image ratio 32)
Other SONY Alpha (note image ratio 43)
1 Langford Michael Basic Photography (7th Ed 2000) Oxford Focal Press ISBN 0 240 51592 7
2 Prices as of 2009 given for comparison purposes onlycopy DJ Foreman 2012 33
Photography (digital) Image capture formats
RAW ndash 3 separate images (RGB) [12 bitspixel (4288 2848)]8=18318336 bytes per
picture JPEG ndash compressed (16x 8x 4x) Some cameras do BOTH (RAW + JPEG) for each image
Pixel notes Sensor size 35mm (or larger) vs APS-C Pixel count ndash any increase rarr
decrease in pixel size or increase in sensor size Pixel size ndash smaller pixels give us
Greater resolution (finer details) - Good More ldquonoiserdquo (incorrect values) - bad Possible ldquoadjacencyrdquo issues - bad Loss of sensitivity - bad
copy DJ Foreman 2012 34
35
Some Comparisons
Digital media vs film Pixels are the same (RGB) Pixel size varies with device (mfgr
dependency) Pixel density varies Zooming digitally does NOT give you more
detail (in spite of what they do on TV shows)
Optical zoom is useful Uses optical magnification of the image on
the target Expansion of light rays not multiplication of
pixelscopy DJ Foreman 2012
Working Definitions - 1
Graphic Model data structure nodes represent random variables Pairs of nodes connected by arcs correspond
to variables that are not independent Graphic modeling
using a Graphic Model to simulate a real-world object (this is not a formal definition)
Rendering generating a 2D image from a 3D graphic model
copy DJ Foreman 2012 36
37
Working Definitions - 2
Geometric primitives ndash Points line segments and polygons Described by vertices
Control points ndash Special points attached to a node on a
Beacutezier curve Alter the shape and angle of adjacent curve
segment Evaluators ndash
Functions that interpolate a set of control points
Produce a new set of control pointscopy DJ Foreman 2012
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
Software we might use (all free)
Modelers and environments Ayam ndash ortho amp 3D 3DCanvas ndash 3D Blender ndash ortho amp 3D Makehuman - 3D
Renderers Aqsis Pixie ndash no GUI file input
copy DJ Foreman 2012 4
5
Image formats
Bitmap Also called ldquoraster imagesrdquo
Stored as individual pixels Resolution dependent Screen resolution = 100 ppi Printing requires 150-300 ppi
Vector Scalable primitives amp formulas Resolution independent
copy DJ Foreman 2012
6
Bitmap formats
BMP (program generated) GIF (ldquo but copy limits on use of format) JPEG JPG (photo) PNG (like GIF wo copy limits) PICT(ldquo Mac BMP+Vector) PCX (ldquo Personal Computer Exchange) PSD (Photoshop) TIFF (ldquo can also be a container)
copy DJ Foreman 2012
7
Vector formats
AI (Adobe Illustrator) CDR (CorelDRAW) CMX (Corel Exchange) CGM (Computer Graphics Metafile ) DXF (AutoCAD ) WMF (Windows Metafile )
copy DJ Foreman 2012
8copy DJ Foreman 2012
Compression
Codecs A device or program for encoding andor
decoding a digital data stream or signal Encoding A-gtD for storage or transmission DecodingD-gtA for playback Lossy vs lossless Raw uncompressed Pulse Code Modulation
(PCM) audio a digital representation of an analog signal where the magnitude of the signal is sampled regularly at uniform intervals Eg (441 kHz 16 bit stereo as represented on an
audio CD or in a wav or aiff file) is a standard across multiple platforms
copy DJ Foreman 2012 9
Codecs - 2
May emphasize aspects of data Video
motion vs color Audio
latency (cell phones) vs high-fidelity (music) Data size
transmission speed or storage size vs data loss
copy DJ Foreman 2012 10
Codec References
Forensic Computing A Practitioners Guide by T Sammes amp B Jenkinson (Springer 2000) ISBN 9781852332990
httpwwwgarykesslernetlibraryfile_sigshtml
copy DJ Foreman 2012 11
Considerations
Tradeoffs compression speed compressed data size quality (data loss)
Areas of study information theory rate-distortion theory
Popular algorithms (all lossless) Lempel-Ziv (LZ) LZW (fast decompression) LZR (LZ-Renau) (ZIP files) LZW (Lempel-Ziv-Welch) (for GIF files)
copy DJ Foreman 2012 12
Lossless Encoding
Huffman compression Bit string representing any symbol is never a
prefix of the bit string representing any other symbolie if 010 is the code for a symbol then no other symbol starts with 010
Frequency table must be known computable or included
Lossless ndash every character gets encoded Arithmetic encoding
Probabilistic algorithm Slightly superior to Huffman but often
patentedcopy DJ Foreman 2012 13
Methodology
Compute a model of the data eg a Huffman tree based on probability Probability determined from input file
Map the data according to the model Read 1 symbol Search the model (tree) for that symbol
If itrsquos a Huffman tree going left=0 right=1 Append each 0 or 1 to the code string When symbol is found you are done Note all symbols will have lt 8 bits (thus
compression)
copy DJ Foreman 2012 14
15
Huffman lab ndash more details Read write are always in bytes
You may need to pad You may have extraneous letters in the output
due to padding You donrsquot need to add a header to specify size
You have 2 weeks to do the lab Code may be in C C++ or Java The readme specifies how to compile the
program Encoded data is sent to a binary file or a simple
text file (but isnrsquot ldquoreadablerdquo as text) Encoded data can only be ldquocheckedrdquo with a
decodercopy DJ Foreman 2012
16
Sample of prefix coding amp compression
copy DJ Foreman 2012
B e a r c a t s (8 bytes of text)010101100110001110001111 encoded as123456781234567812345678 3 bytes
A simple Huffman-like code(wo probability basis) 0 1
A 110
B 010
C 0001
E 10
R 011
S 111
T 001
Compression-Model types
Static Read all the data Compute frequencies Re-read the data amp encode
Dynamic Build simple basic model Modify model as more data is processed
copy DJ Foreman 2012 17
Standard Compressed-File Content
Three parts 24-byte header with magic (defines
codec) variable-length annotation block contiguous segment of audio data
Storage methodology network (big-endian) byte order multi-byte audio data may require byte
reversal in order to operate on it by the arithmetic unit of certain processors
copy DJ Foreman 2012 18
Example 1 Header file for AU files
copy DJ Foreman 2012 19
typedef unsigned long u_32 unsigned 32-bit integer
typedef struct
u_32 magic the ldquomagic numberrdquo
u_32 hdr_size byte offset to start of data
u_32 data_size length (optional)
u_32 encoding data encoding enumeration
u_32 sample_rate samples per second
u_32 channels of interleaved channels
Audio_filehdr
Following is from include ltmultimedialibaudiohgt
AUDIO_FILE_MAGIC ((u_32)0x2e736e64) ldquosndrdquo
Audio Encoding EnumerationsAUDIO_FILE_ENCODING_MULAW_8 (1) 8-bit ISDN u-law
AUDIO_FILE_ENCODING_LINEAR_8 (2) 8-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_16 (3) 16-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_32 (5) 32-bit linear PCM
AUDIO_FILE_ENCODING_FLOAT (6) 32-bit IEEE floating point
AUDIO_FILE_ENCODING_DOUBLE (7) 64-bit IEEE floating point
AUDIO_FILE_ENCODING_ADPCM_G721 (23) 4-bit CCITT g721 ADPCM
AUDIO_FILE_ENCODING_ADPCM_G723_3 (25) CCITT g723 3-bit ADPCM
AUDIO_FILE_ENCODING_ALAW_8 (27) 8-bit ISDN A-law
copy DJ Foreman 2012 20
ldquoLinearrdquo values are SIGNED intrsquosFloats are signed zero-centered normalized to ( -10 lt= x lt= 10 )
Example 2 ZIP files (PkWare)Purpose Size in bytes
Signature header 4
Required version 2
GP flags 2
Method 2
Last mod time 2
Last mod date 2
CRC-32 4
Compressed size 4
Uncompressed size 4
Filename length 2
Extra field length 2
File name variable
extra variable
copy DJ Foreman 2012 21
Ref httplivedocsadobecomflex3htmlhelphtmlcontent=ByteArrays_3html
More info on encoding httpwwwpkwarecomdocumentscase
studiesAPPNOTETXT
httpwwwfileinfocomfiletypescompressed
copy DJ Foreman 2012 22
Recording Bit-Rate
Constant (CBR) rate at which a codecs output data should be
consumed is constant Max bit-rate matters not the average Uses all available bandwidth Not good for storage (lossy)
Variable (VBR) Quantity of datatime unit (for output) varies Better quality for audio and video Slower to encode Supported by most portable devices post 2006
copy DJ Foreman 2012 23
Lab assignment ndash 1 ndash part 1
Write a program with 1 parameter If you are on an encoding team
Open a test file (eg ldquoxyztxtrdquo) Compress it using simple Huffman compression using
the frequency table from my FTP site Output compressed file (eg ldquoxyzencrdquo) to SAME folder
as input
If you are on a decoding team Open compressed file ldquoxyzencrdquo Decompress the input file Output file ldquoxyzdecrdquo so it can be compared to ldquoxyztxtrdquo
The parameter is the full input file path amp name (ie file names MUST NOT be hard-coded)
Remember a code can be a single bit
copy DJ Foreman 2012 24
25
Lab assignment 1 ndash part 2
Use ONLY MY TABLE for the frequencies Use ONLY uppercase letters of the table Do NOT use the special character
tables Find someone in class with the
opposite program and TEST your program to see if it properly encodes or decodes a message
Grading tree construction tree following proper binary output (encoders) proper character output (decoders)
copy DJ Foreman 2012
Working with ImagesModeling
Containers
A container is a FILE format NOT a code scheme
Eg AVI amp WAV are container formats Formats for storage Independent of encoding or content
Others Ogg ASF QuickTime RealMedia Matroska
DivX and MP4
copy DJ Foreman 2012 27
28
Movie (metadata)
Track
Media
Media Information
Container file format (MP4QT)
Chunk map
Comp time
Chunk offset
Key frame
Desc
Decode time
Priority map
Size map
Sample Table
Media Handler
Data information
Media header
Video header
Track header
Movie header
copy DJ Foreman 2012
Pixels A pixel is 3 color-dots (squares) in a collection of dots
(squares) making up the picture Each color-dot is for one of the three primary colors
RGB Minimum of 1 byte per color (depending on bit-depth
which varies with camera) thus at least 3 bytes per pixel
A 10MP camera (30 million color-dots) needs 30M bytes This gets compressed by the camera (JPEG format) Final file size = 30 million bytes divided by the amount
of compression (compression ratio) An 81 compression ratio would reduce that 30 million
bytes to 375 megabytes (as done on a 4 MP camera) A RAW file has NO compression
copy DJ Foreman 2012 29
Image creation
Basic mechanisms Photography
Film Digital ndashPhotoshop Gimp
Drawing Line BrushModeling
Do it yourself OpenGL (2x and 3x)
Modelingrendering programs Breeze designer Ayam 3DCanvas Makehuman Alice
copy DJ Foreman 2012 30
Photography (film) Silver halide crystals (AgCl AgBr AgI)
Exposure to light turns them black Developer removes the loose (exposed) halide leaving
pure silver (a negative image) Fixing bath flushes out unexposed AgHalide
Greatest desire of photographers - a medium with High speed (gathers light quickly) Low noise (no undesired artifacts) High resolution (very detailed images)
Visible lines per mm (lpm) at a specified contrast level
Notes Larger ldquoclumpsrdquo are faster but limit resolution Smaller ldquoclumpsrdquo are hard to coat evenly Randomness of coating prevents Moireacute patternshttpenwikipediaorgwikiMoirC3A9_pattern
copy DJ Foreman 2012 31
FYI ndash a little chemistry
Ag+ X- + hv rarr Ag+ + X + e-
1 Radiation frees the Br2 1 Ag ion + 1 electron (Ag+ + e- rarr Ag0) -gt 1 Ag atom3 Neighbors are now ldquosensitizedrdquo 4 Developing (Reducing) converts exposed AgX to
black metallic Ag and free X5 Remaining AgX and free X are washed away (after
mixing with thiosulfate to make it soluble)
This is one silver (Ag) ldquograinrdquo in the ldquolatent imagerdquo X is any halide (Br Cl etc)
copy DJ Foreman 2012 32
crystal
Radiation (light)
Resolution 35 mm film for comparison
A 36 x 24 mm frame of ISO 100-speed film contains (asymp) the equivalent of 20 million pixels 1 There are NO pixels in film
Full-frame digital (36 x 24mm = 32 image ratio) Canon EOS 5D Nikon D3 (21MP) - $5K2
Medium format digital (6 x 45cm) Phase One P40+ (39MP) ($22K)
APS-C sized sensor (asymp24x16mm = 32 image ratio) (note 2009 data below)
Nikon D90 (123MP) - $900 Canon Rebel (15MP) $700 Used in most DSLRrsquos (note image ratio 32)
Other SONY Alpha (note image ratio 43)
1 Langford Michael Basic Photography (7th Ed 2000) Oxford Focal Press ISBN 0 240 51592 7
2 Prices as of 2009 given for comparison purposes onlycopy DJ Foreman 2012 33
Photography (digital) Image capture formats
RAW ndash 3 separate images (RGB) [12 bitspixel (4288 2848)]8=18318336 bytes per
picture JPEG ndash compressed (16x 8x 4x) Some cameras do BOTH (RAW + JPEG) for each image
Pixel notes Sensor size 35mm (or larger) vs APS-C Pixel count ndash any increase rarr
decrease in pixel size or increase in sensor size Pixel size ndash smaller pixels give us
Greater resolution (finer details) - Good More ldquonoiserdquo (incorrect values) - bad Possible ldquoadjacencyrdquo issues - bad Loss of sensitivity - bad
copy DJ Foreman 2012 34
35
Some Comparisons
Digital media vs film Pixels are the same (RGB) Pixel size varies with device (mfgr
dependency) Pixel density varies Zooming digitally does NOT give you more
detail (in spite of what they do on TV shows)
Optical zoom is useful Uses optical magnification of the image on
the target Expansion of light rays not multiplication of
pixelscopy DJ Foreman 2012
Working Definitions - 1
Graphic Model data structure nodes represent random variables Pairs of nodes connected by arcs correspond
to variables that are not independent Graphic modeling
using a Graphic Model to simulate a real-world object (this is not a formal definition)
Rendering generating a 2D image from a 3D graphic model
copy DJ Foreman 2012 36
37
Working Definitions - 2
Geometric primitives ndash Points line segments and polygons Described by vertices
Control points ndash Special points attached to a node on a
Beacutezier curve Alter the shape and angle of adjacent curve
segment Evaluators ndash
Functions that interpolate a set of control points
Produce a new set of control pointscopy DJ Foreman 2012
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
5
Image formats
Bitmap Also called ldquoraster imagesrdquo
Stored as individual pixels Resolution dependent Screen resolution = 100 ppi Printing requires 150-300 ppi
Vector Scalable primitives amp formulas Resolution independent
copy DJ Foreman 2012
6
Bitmap formats
BMP (program generated) GIF (ldquo but copy limits on use of format) JPEG JPG (photo) PNG (like GIF wo copy limits) PICT(ldquo Mac BMP+Vector) PCX (ldquo Personal Computer Exchange) PSD (Photoshop) TIFF (ldquo can also be a container)
copy DJ Foreman 2012
7
Vector formats
AI (Adobe Illustrator) CDR (CorelDRAW) CMX (Corel Exchange) CGM (Computer Graphics Metafile ) DXF (AutoCAD ) WMF (Windows Metafile )
copy DJ Foreman 2012
8copy DJ Foreman 2012
Compression
Codecs A device or program for encoding andor
decoding a digital data stream or signal Encoding A-gtD for storage or transmission DecodingD-gtA for playback Lossy vs lossless Raw uncompressed Pulse Code Modulation
(PCM) audio a digital representation of an analog signal where the magnitude of the signal is sampled regularly at uniform intervals Eg (441 kHz 16 bit stereo as represented on an
audio CD or in a wav or aiff file) is a standard across multiple platforms
copy DJ Foreman 2012 9
Codecs - 2
May emphasize aspects of data Video
motion vs color Audio
latency (cell phones) vs high-fidelity (music) Data size
transmission speed or storage size vs data loss
copy DJ Foreman 2012 10
Codec References
Forensic Computing A Practitioners Guide by T Sammes amp B Jenkinson (Springer 2000) ISBN 9781852332990
httpwwwgarykesslernetlibraryfile_sigshtml
copy DJ Foreman 2012 11
Considerations
Tradeoffs compression speed compressed data size quality (data loss)
Areas of study information theory rate-distortion theory
Popular algorithms (all lossless) Lempel-Ziv (LZ) LZW (fast decompression) LZR (LZ-Renau) (ZIP files) LZW (Lempel-Ziv-Welch) (for GIF files)
copy DJ Foreman 2012 12
Lossless Encoding
Huffman compression Bit string representing any symbol is never a
prefix of the bit string representing any other symbolie if 010 is the code for a symbol then no other symbol starts with 010
Frequency table must be known computable or included
Lossless ndash every character gets encoded Arithmetic encoding
Probabilistic algorithm Slightly superior to Huffman but often
patentedcopy DJ Foreman 2012 13
Methodology
Compute a model of the data eg a Huffman tree based on probability Probability determined from input file
Map the data according to the model Read 1 symbol Search the model (tree) for that symbol
If itrsquos a Huffman tree going left=0 right=1 Append each 0 or 1 to the code string When symbol is found you are done Note all symbols will have lt 8 bits (thus
compression)
copy DJ Foreman 2012 14
15
Huffman lab ndash more details Read write are always in bytes
You may need to pad You may have extraneous letters in the output
due to padding You donrsquot need to add a header to specify size
You have 2 weeks to do the lab Code may be in C C++ or Java The readme specifies how to compile the
program Encoded data is sent to a binary file or a simple
text file (but isnrsquot ldquoreadablerdquo as text) Encoded data can only be ldquocheckedrdquo with a
decodercopy DJ Foreman 2012
16
Sample of prefix coding amp compression
copy DJ Foreman 2012
B e a r c a t s (8 bytes of text)010101100110001110001111 encoded as123456781234567812345678 3 bytes
A simple Huffman-like code(wo probability basis) 0 1
A 110
B 010
C 0001
E 10
R 011
S 111
T 001
Compression-Model types
Static Read all the data Compute frequencies Re-read the data amp encode
Dynamic Build simple basic model Modify model as more data is processed
copy DJ Foreman 2012 17
Standard Compressed-File Content
Three parts 24-byte header with magic (defines
codec) variable-length annotation block contiguous segment of audio data
Storage methodology network (big-endian) byte order multi-byte audio data may require byte
reversal in order to operate on it by the arithmetic unit of certain processors
copy DJ Foreman 2012 18
Example 1 Header file for AU files
copy DJ Foreman 2012 19
typedef unsigned long u_32 unsigned 32-bit integer
typedef struct
u_32 magic the ldquomagic numberrdquo
u_32 hdr_size byte offset to start of data
u_32 data_size length (optional)
u_32 encoding data encoding enumeration
u_32 sample_rate samples per second
u_32 channels of interleaved channels
Audio_filehdr
Following is from include ltmultimedialibaudiohgt
AUDIO_FILE_MAGIC ((u_32)0x2e736e64) ldquosndrdquo
Audio Encoding EnumerationsAUDIO_FILE_ENCODING_MULAW_8 (1) 8-bit ISDN u-law
AUDIO_FILE_ENCODING_LINEAR_8 (2) 8-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_16 (3) 16-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_32 (5) 32-bit linear PCM
AUDIO_FILE_ENCODING_FLOAT (6) 32-bit IEEE floating point
AUDIO_FILE_ENCODING_DOUBLE (7) 64-bit IEEE floating point
AUDIO_FILE_ENCODING_ADPCM_G721 (23) 4-bit CCITT g721 ADPCM
AUDIO_FILE_ENCODING_ADPCM_G723_3 (25) CCITT g723 3-bit ADPCM
AUDIO_FILE_ENCODING_ALAW_8 (27) 8-bit ISDN A-law
copy DJ Foreman 2012 20
ldquoLinearrdquo values are SIGNED intrsquosFloats are signed zero-centered normalized to ( -10 lt= x lt= 10 )
Example 2 ZIP files (PkWare)Purpose Size in bytes
Signature header 4
Required version 2
GP flags 2
Method 2
Last mod time 2
Last mod date 2
CRC-32 4
Compressed size 4
Uncompressed size 4
Filename length 2
Extra field length 2
File name variable
extra variable
copy DJ Foreman 2012 21
Ref httplivedocsadobecomflex3htmlhelphtmlcontent=ByteArrays_3html
More info on encoding httpwwwpkwarecomdocumentscase
studiesAPPNOTETXT
httpwwwfileinfocomfiletypescompressed
copy DJ Foreman 2012 22
Recording Bit-Rate
Constant (CBR) rate at which a codecs output data should be
consumed is constant Max bit-rate matters not the average Uses all available bandwidth Not good for storage (lossy)
Variable (VBR) Quantity of datatime unit (for output) varies Better quality for audio and video Slower to encode Supported by most portable devices post 2006
copy DJ Foreman 2012 23
Lab assignment ndash 1 ndash part 1
Write a program with 1 parameter If you are on an encoding team
Open a test file (eg ldquoxyztxtrdquo) Compress it using simple Huffman compression using
the frequency table from my FTP site Output compressed file (eg ldquoxyzencrdquo) to SAME folder
as input
If you are on a decoding team Open compressed file ldquoxyzencrdquo Decompress the input file Output file ldquoxyzdecrdquo so it can be compared to ldquoxyztxtrdquo
The parameter is the full input file path amp name (ie file names MUST NOT be hard-coded)
Remember a code can be a single bit
copy DJ Foreman 2012 24
25
Lab assignment 1 ndash part 2
Use ONLY MY TABLE for the frequencies Use ONLY uppercase letters of the table Do NOT use the special character
tables Find someone in class with the
opposite program and TEST your program to see if it properly encodes or decodes a message
Grading tree construction tree following proper binary output (encoders) proper character output (decoders)
copy DJ Foreman 2012
Working with ImagesModeling
Containers
A container is a FILE format NOT a code scheme
Eg AVI amp WAV are container formats Formats for storage Independent of encoding or content
Others Ogg ASF QuickTime RealMedia Matroska
DivX and MP4
copy DJ Foreman 2012 27
28
Movie (metadata)
Track
Media
Media Information
Container file format (MP4QT)
Chunk map
Comp time
Chunk offset
Key frame
Desc
Decode time
Priority map
Size map
Sample Table
Media Handler
Data information
Media header
Video header
Track header
Movie header
copy DJ Foreman 2012
Pixels A pixel is 3 color-dots (squares) in a collection of dots
(squares) making up the picture Each color-dot is for one of the three primary colors
RGB Minimum of 1 byte per color (depending on bit-depth
which varies with camera) thus at least 3 bytes per pixel
A 10MP camera (30 million color-dots) needs 30M bytes This gets compressed by the camera (JPEG format) Final file size = 30 million bytes divided by the amount
of compression (compression ratio) An 81 compression ratio would reduce that 30 million
bytes to 375 megabytes (as done on a 4 MP camera) A RAW file has NO compression
copy DJ Foreman 2012 29
Image creation
Basic mechanisms Photography
Film Digital ndashPhotoshop Gimp
Drawing Line BrushModeling
Do it yourself OpenGL (2x and 3x)
Modelingrendering programs Breeze designer Ayam 3DCanvas Makehuman Alice
copy DJ Foreman 2012 30
Photography (film) Silver halide crystals (AgCl AgBr AgI)
Exposure to light turns them black Developer removes the loose (exposed) halide leaving
pure silver (a negative image) Fixing bath flushes out unexposed AgHalide
Greatest desire of photographers - a medium with High speed (gathers light quickly) Low noise (no undesired artifacts) High resolution (very detailed images)
Visible lines per mm (lpm) at a specified contrast level
Notes Larger ldquoclumpsrdquo are faster but limit resolution Smaller ldquoclumpsrdquo are hard to coat evenly Randomness of coating prevents Moireacute patternshttpenwikipediaorgwikiMoirC3A9_pattern
copy DJ Foreman 2012 31
FYI ndash a little chemistry
Ag+ X- + hv rarr Ag+ + X + e-
1 Radiation frees the Br2 1 Ag ion + 1 electron (Ag+ + e- rarr Ag0) -gt 1 Ag atom3 Neighbors are now ldquosensitizedrdquo 4 Developing (Reducing) converts exposed AgX to
black metallic Ag and free X5 Remaining AgX and free X are washed away (after
mixing with thiosulfate to make it soluble)
This is one silver (Ag) ldquograinrdquo in the ldquolatent imagerdquo X is any halide (Br Cl etc)
copy DJ Foreman 2012 32
crystal
Radiation (light)
Resolution 35 mm film for comparison
A 36 x 24 mm frame of ISO 100-speed film contains (asymp) the equivalent of 20 million pixels 1 There are NO pixels in film
Full-frame digital (36 x 24mm = 32 image ratio) Canon EOS 5D Nikon D3 (21MP) - $5K2
Medium format digital (6 x 45cm) Phase One P40+ (39MP) ($22K)
APS-C sized sensor (asymp24x16mm = 32 image ratio) (note 2009 data below)
Nikon D90 (123MP) - $900 Canon Rebel (15MP) $700 Used in most DSLRrsquos (note image ratio 32)
Other SONY Alpha (note image ratio 43)
1 Langford Michael Basic Photography (7th Ed 2000) Oxford Focal Press ISBN 0 240 51592 7
2 Prices as of 2009 given for comparison purposes onlycopy DJ Foreman 2012 33
Photography (digital) Image capture formats
RAW ndash 3 separate images (RGB) [12 bitspixel (4288 2848)]8=18318336 bytes per
picture JPEG ndash compressed (16x 8x 4x) Some cameras do BOTH (RAW + JPEG) for each image
Pixel notes Sensor size 35mm (or larger) vs APS-C Pixel count ndash any increase rarr
decrease in pixel size or increase in sensor size Pixel size ndash smaller pixels give us
Greater resolution (finer details) - Good More ldquonoiserdquo (incorrect values) - bad Possible ldquoadjacencyrdquo issues - bad Loss of sensitivity - bad
copy DJ Foreman 2012 34
35
Some Comparisons
Digital media vs film Pixels are the same (RGB) Pixel size varies with device (mfgr
dependency) Pixel density varies Zooming digitally does NOT give you more
detail (in spite of what they do on TV shows)
Optical zoom is useful Uses optical magnification of the image on
the target Expansion of light rays not multiplication of
pixelscopy DJ Foreman 2012
Working Definitions - 1
Graphic Model data structure nodes represent random variables Pairs of nodes connected by arcs correspond
to variables that are not independent Graphic modeling
using a Graphic Model to simulate a real-world object (this is not a formal definition)
Rendering generating a 2D image from a 3D graphic model
copy DJ Foreman 2012 36
37
Working Definitions - 2
Geometric primitives ndash Points line segments and polygons Described by vertices
Control points ndash Special points attached to a node on a
Beacutezier curve Alter the shape and angle of adjacent curve
segment Evaluators ndash
Functions that interpolate a set of control points
Produce a new set of control pointscopy DJ Foreman 2012
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
6
Bitmap formats
BMP (program generated) GIF (ldquo but copy limits on use of format) JPEG JPG (photo) PNG (like GIF wo copy limits) PICT(ldquo Mac BMP+Vector) PCX (ldquo Personal Computer Exchange) PSD (Photoshop) TIFF (ldquo can also be a container)
copy DJ Foreman 2012
7
Vector formats
AI (Adobe Illustrator) CDR (CorelDRAW) CMX (Corel Exchange) CGM (Computer Graphics Metafile ) DXF (AutoCAD ) WMF (Windows Metafile )
copy DJ Foreman 2012
8copy DJ Foreman 2012
Compression
Codecs A device or program for encoding andor
decoding a digital data stream or signal Encoding A-gtD for storage or transmission DecodingD-gtA for playback Lossy vs lossless Raw uncompressed Pulse Code Modulation
(PCM) audio a digital representation of an analog signal where the magnitude of the signal is sampled regularly at uniform intervals Eg (441 kHz 16 bit stereo as represented on an
audio CD or in a wav or aiff file) is a standard across multiple platforms
copy DJ Foreman 2012 9
Codecs - 2
May emphasize aspects of data Video
motion vs color Audio
latency (cell phones) vs high-fidelity (music) Data size
transmission speed or storage size vs data loss
copy DJ Foreman 2012 10
Codec References
Forensic Computing A Practitioners Guide by T Sammes amp B Jenkinson (Springer 2000) ISBN 9781852332990
httpwwwgarykesslernetlibraryfile_sigshtml
copy DJ Foreman 2012 11
Considerations
Tradeoffs compression speed compressed data size quality (data loss)
Areas of study information theory rate-distortion theory
Popular algorithms (all lossless) Lempel-Ziv (LZ) LZW (fast decompression) LZR (LZ-Renau) (ZIP files) LZW (Lempel-Ziv-Welch) (for GIF files)
copy DJ Foreman 2012 12
Lossless Encoding
Huffman compression Bit string representing any symbol is never a
prefix of the bit string representing any other symbolie if 010 is the code for a symbol then no other symbol starts with 010
Frequency table must be known computable or included
Lossless ndash every character gets encoded Arithmetic encoding
Probabilistic algorithm Slightly superior to Huffman but often
patentedcopy DJ Foreman 2012 13
Methodology
Compute a model of the data eg a Huffman tree based on probability Probability determined from input file
Map the data according to the model Read 1 symbol Search the model (tree) for that symbol
If itrsquos a Huffman tree going left=0 right=1 Append each 0 or 1 to the code string When symbol is found you are done Note all symbols will have lt 8 bits (thus
compression)
copy DJ Foreman 2012 14
15
Huffman lab ndash more details Read write are always in bytes
You may need to pad You may have extraneous letters in the output
due to padding You donrsquot need to add a header to specify size
You have 2 weeks to do the lab Code may be in C C++ or Java The readme specifies how to compile the
program Encoded data is sent to a binary file or a simple
text file (but isnrsquot ldquoreadablerdquo as text) Encoded data can only be ldquocheckedrdquo with a
decodercopy DJ Foreman 2012
16
Sample of prefix coding amp compression
copy DJ Foreman 2012
B e a r c a t s (8 bytes of text)010101100110001110001111 encoded as123456781234567812345678 3 bytes
A simple Huffman-like code(wo probability basis) 0 1
A 110
B 010
C 0001
E 10
R 011
S 111
T 001
Compression-Model types
Static Read all the data Compute frequencies Re-read the data amp encode
Dynamic Build simple basic model Modify model as more data is processed
copy DJ Foreman 2012 17
Standard Compressed-File Content
Three parts 24-byte header with magic (defines
codec) variable-length annotation block contiguous segment of audio data
Storage methodology network (big-endian) byte order multi-byte audio data may require byte
reversal in order to operate on it by the arithmetic unit of certain processors
copy DJ Foreman 2012 18
Example 1 Header file for AU files
copy DJ Foreman 2012 19
typedef unsigned long u_32 unsigned 32-bit integer
typedef struct
u_32 magic the ldquomagic numberrdquo
u_32 hdr_size byte offset to start of data
u_32 data_size length (optional)
u_32 encoding data encoding enumeration
u_32 sample_rate samples per second
u_32 channels of interleaved channels
Audio_filehdr
Following is from include ltmultimedialibaudiohgt
AUDIO_FILE_MAGIC ((u_32)0x2e736e64) ldquosndrdquo
Audio Encoding EnumerationsAUDIO_FILE_ENCODING_MULAW_8 (1) 8-bit ISDN u-law
AUDIO_FILE_ENCODING_LINEAR_8 (2) 8-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_16 (3) 16-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_32 (5) 32-bit linear PCM
AUDIO_FILE_ENCODING_FLOAT (6) 32-bit IEEE floating point
AUDIO_FILE_ENCODING_DOUBLE (7) 64-bit IEEE floating point
AUDIO_FILE_ENCODING_ADPCM_G721 (23) 4-bit CCITT g721 ADPCM
AUDIO_FILE_ENCODING_ADPCM_G723_3 (25) CCITT g723 3-bit ADPCM
AUDIO_FILE_ENCODING_ALAW_8 (27) 8-bit ISDN A-law
copy DJ Foreman 2012 20
ldquoLinearrdquo values are SIGNED intrsquosFloats are signed zero-centered normalized to ( -10 lt= x lt= 10 )
Example 2 ZIP files (PkWare)Purpose Size in bytes
Signature header 4
Required version 2
GP flags 2
Method 2
Last mod time 2
Last mod date 2
CRC-32 4
Compressed size 4
Uncompressed size 4
Filename length 2
Extra field length 2
File name variable
extra variable
copy DJ Foreman 2012 21
Ref httplivedocsadobecomflex3htmlhelphtmlcontent=ByteArrays_3html
More info on encoding httpwwwpkwarecomdocumentscase
studiesAPPNOTETXT
httpwwwfileinfocomfiletypescompressed
copy DJ Foreman 2012 22
Recording Bit-Rate
Constant (CBR) rate at which a codecs output data should be
consumed is constant Max bit-rate matters not the average Uses all available bandwidth Not good for storage (lossy)
Variable (VBR) Quantity of datatime unit (for output) varies Better quality for audio and video Slower to encode Supported by most portable devices post 2006
copy DJ Foreman 2012 23
Lab assignment ndash 1 ndash part 1
Write a program with 1 parameter If you are on an encoding team
Open a test file (eg ldquoxyztxtrdquo) Compress it using simple Huffman compression using
the frequency table from my FTP site Output compressed file (eg ldquoxyzencrdquo) to SAME folder
as input
If you are on a decoding team Open compressed file ldquoxyzencrdquo Decompress the input file Output file ldquoxyzdecrdquo so it can be compared to ldquoxyztxtrdquo
The parameter is the full input file path amp name (ie file names MUST NOT be hard-coded)
Remember a code can be a single bit
copy DJ Foreman 2012 24
25
Lab assignment 1 ndash part 2
Use ONLY MY TABLE for the frequencies Use ONLY uppercase letters of the table Do NOT use the special character
tables Find someone in class with the
opposite program and TEST your program to see if it properly encodes or decodes a message
Grading tree construction tree following proper binary output (encoders) proper character output (decoders)
copy DJ Foreman 2012
Working with ImagesModeling
Containers
A container is a FILE format NOT a code scheme
Eg AVI amp WAV are container formats Formats for storage Independent of encoding or content
Others Ogg ASF QuickTime RealMedia Matroska
DivX and MP4
copy DJ Foreman 2012 27
28
Movie (metadata)
Track
Media
Media Information
Container file format (MP4QT)
Chunk map
Comp time
Chunk offset
Key frame
Desc
Decode time
Priority map
Size map
Sample Table
Media Handler
Data information
Media header
Video header
Track header
Movie header
copy DJ Foreman 2012
Pixels A pixel is 3 color-dots (squares) in a collection of dots
(squares) making up the picture Each color-dot is for one of the three primary colors
RGB Minimum of 1 byte per color (depending on bit-depth
which varies with camera) thus at least 3 bytes per pixel
A 10MP camera (30 million color-dots) needs 30M bytes This gets compressed by the camera (JPEG format) Final file size = 30 million bytes divided by the amount
of compression (compression ratio) An 81 compression ratio would reduce that 30 million
bytes to 375 megabytes (as done on a 4 MP camera) A RAW file has NO compression
copy DJ Foreman 2012 29
Image creation
Basic mechanisms Photography
Film Digital ndashPhotoshop Gimp
Drawing Line BrushModeling
Do it yourself OpenGL (2x and 3x)
Modelingrendering programs Breeze designer Ayam 3DCanvas Makehuman Alice
copy DJ Foreman 2012 30
Photography (film) Silver halide crystals (AgCl AgBr AgI)
Exposure to light turns them black Developer removes the loose (exposed) halide leaving
pure silver (a negative image) Fixing bath flushes out unexposed AgHalide
Greatest desire of photographers - a medium with High speed (gathers light quickly) Low noise (no undesired artifacts) High resolution (very detailed images)
Visible lines per mm (lpm) at a specified contrast level
Notes Larger ldquoclumpsrdquo are faster but limit resolution Smaller ldquoclumpsrdquo are hard to coat evenly Randomness of coating prevents Moireacute patternshttpenwikipediaorgwikiMoirC3A9_pattern
copy DJ Foreman 2012 31
FYI ndash a little chemistry
Ag+ X- + hv rarr Ag+ + X + e-
1 Radiation frees the Br2 1 Ag ion + 1 electron (Ag+ + e- rarr Ag0) -gt 1 Ag atom3 Neighbors are now ldquosensitizedrdquo 4 Developing (Reducing) converts exposed AgX to
black metallic Ag and free X5 Remaining AgX and free X are washed away (after
mixing with thiosulfate to make it soluble)
This is one silver (Ag) ldquograinrdquo in the ldquolatent imagerdquo X is any halide (Br Cl etc)
copy DJ Foreman 2012 32
crystal
Radiation (light)
Resolution 35 mm film for comparison
A 36 x 24 mm frame of ISO 100-speed film contains (asymp) the equivalent of 20 million pixels 1 There are NO pixels in film
Full-frame digital (36 x 24mm = 32 image ratio) Canon EOS 5D Nikon D3 (21MP) - $5K2
Medium format digital (6 x 45cm) Phase One P40+ (39MP) ($22K)
APS-C sized sensor (asymp24x16mm = 32 image ratio) (note 2009 data below)
Nikon D90 (123MP) - $900 Canon Rebel (15MP) $700 Used in most DSLRrsquos (note image ratio 32)
Other SONY Alpha (note image ratio 43)
1 Langford Michael Basic Photography (7th Ed 2000) Oxford Focal Press ISBN 0 240 51592 7
2 Prices as of 2009 given for comparison purposes onlycopy DJ Foreman 2012 33
Photography (digital) Image capture formats
RAW ndash 3 separate images (RGB) [12 bitspixel (4288 2848)]8=18318336 bytes per
picture JPEG ndash compressed (16x 8x 4x) Some cameras do BOTH (RAW + JPEG) for each image
Pixel notes Sensor size 35mm (or larger) vs APS-C Pixel count ndash any increase rarr
decrease in pixel size or increase in sensor size Pixel size ndash smaller pixels give us
Greater resolution (finer details) - Good More ldquonoiserdquo (incorrect values) - bad Possible ldquoadjacencyrdquo issues - bad Loss of sensitivity - bad
copy DJ Foreman 2012 34
35
Some Comparisons
Digital media vs film Pixels are the same (RGB) Pixel size varies with device (mfgr
dependency) Pixel density varies Zooming digitally does NOT give you more
detail (in spite of what they do on TV shows)
Optical zoom is useful Uses optical magnification of the image on
the target Expansion of light rays not multiplication of
pixelscopy DJ Foreman 2012
Working Definitions - 1
Graphic Model data structure nodes represent random variables Pairs of nodes connected by arcs correspond
to variables that are not independent Graphic modeling
using a Graphic Model to simulate a real-world object (this is not a formal definition)
Rendering generating a 2D image from a 3D graphic model
copy DJ Foreman 2012 36
37
Working Definitions - 2
Geometric primitives ndash Points line segments and polygons Described by vertices
Control points ndash Special points attached to a node on a
Beacutezier curve Alter the shape and angle of adjacent curve
segment Evaluators ndash
Functions that interpolate a set of control points
Produce a new set of control pointscopy DJ Foreman 2012
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
7
Vector formats
AI (Adobe Illustrator) CDR (CorelDRAW) CMX (Corel Exchange) CGM (Computer Graphics Metafile ) DXF (AutoCAD ) WMF (Windows Metafile )
copy DJ Foreman 2012
8copy DJ Foreman 2012
Compression
Codecs A device or program for encoding andor
decoding a digital data stream or signal Encoding A-gtD for storage or transmission DecodingD-gtA for playback Lossy vs lossless Raw uncompressed Pulse Code Modulation
(PCM) audio a digital representation of an analog signal where the magnitude of the signal is sampled regularly at uniform intervals Eg (441 kHz 16 bit stereo as represented on an
audio CD or in a wav or aiff file) is a standard across multiple platforms
copy DJ Foreman 2012 9
Codecs - 2
May emphasize aspects of data Video
motion vs color Audio
latency (cell phones) vs high-fidelity (music) Data size
transmission speed or storage size vs data loss
copy DJ Foreman 2012 10
Codec References
Forensic Computing A Practitioners Guide by T Sammes amp B Jenkinson (Springer 2000) ISBN 9781852332990
httpwwwgarykesslernetlibraryfile_sigshtml
copy DJ Foreman 2012 11
Considerations
Tradeoffs compression speed compressed data size quality (data loss)
Areas of study information theory rate-distortion theory
Popular algorithms (all lossless) Lempel-Ziv (LZ) LZW (fast decompression) LZR (LZ-Renau) (ZIP files) LZW (Lempel-Ziv-Welch) (for GIF files)
copy DJ Foreman 2012 12
Lossless Encoding
Huffman compression Bit string representing any symbol is never a
prefix of the bit string representing any other symbolie if 010 is the code for a symbol then no other symbol starts with 010
Frequency table must be known computable or included
Lossless ndash every character gets encoded Arithmetic encoding
Probabilistic algorithm Slightly superior to Huffman but often
patentedcopy DJ Foreman 2012 13
Methodology
Compute a model of the data eg a Huffman tree based on probability Probability determined from input file
Map the data according to the model Read 1 symbol Search the model (tree) for that symbol
If itrsquos a Huffman tree going left=0 right=1 Append each 0 or 1 to the code string When symbol is found you are done Note all symbols will have lt 8 bits (thus
compression)
copy DJ Foreman 2012 14
15
Huffman lab ndash more details Read write are always in bytes
You may need to pad You may have extraneous letters in the output
due to padding You donrsquot need to add a header to specify size
You have 2 weeks to do the lab Code may be in C C++ or Java The readme specifies how to compile the
program Encoded data is sent to a binary file or a simple
text file (but isnrsquot ldquoreadablerdquo as text) Encoded data can only be ldquocheckedrdquo with a
decodercopy DJ Foreman 2012
16
Sample of prefix coding amp compression
copy DJ Foreman 2012
B e a r c a t s (8 bytes of text)010101100110001110001111 encoded as123456781234567812345678 3 bytes
A simple Huffman-like code(wo probability basis) 0 1
A 110
B 010
C 0001
E 10
R 011
S 111
T 001
Compression-Model types
Static Read all the data Compute frequencies Re-read the data amp encode
Dynamic Build simple basic model Modify model as more data is processed
copy DJ Foreman 2012 17
Standard Compressed-File Content
Three parts 24-byte header with magic (defines
codec) variable-length annotation block contiguous segment of audio data
Storage methodology network (big-endian) byte order multi-byte audio data may require byte
reversal in order to operate on it by the arithmetic unit of certain processors
copy DJ Foreman 2012 18
Example 1 Header file for AU files
copy DJ Foreman 2012 19
typedef unsigned long u_32 unsigned 32-bit integer
typedef struct
u_32 magic the ldquomagic numberrdquo
u_32 hdr_size byte offset to start of data
u_32 data_size length (optional)
u_32 encoding data encoding enumeration
u_32 sample_rate samples per second
u_32 channels of interleaved channels
Audio_filehdr
Following is from include ltmultimedialibaudiohgt
AUDIO_FILE_MAGIC ((u_32)0x2e736e64) ldquosndrdquo
Audio Encoding EnumerationsAUDIO_FILE_ENCODING_MULAW_8 (1) 8-bit ISDN u-law
AUDIO_FILE_ENCODING_LINEAR_8 (2) 8-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_16 (3) 16-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_32 (5) 32-bit linear PCM
AUDIO_FILE_ENCODING_FLOAT (6) 32-bit IEEE floating point
AUDIO_FILE_ENCODING_DOUBLE (7) 64-bit IEEE floating point
AUDIO_FILE_ENCODING_ADPCM_G721 (23) 4-bit CCITT g721 ADPCM
AUDIO_FILE_ENCODING_ADPCM_G723_3 (25) CCITT g723 3-bit ADPCM
AUDIO_FILE_ENCODING_ALAW_8 (27) 8-bit ISDN A-law
copy DJ Foreman 2012 20
ldquoLinearrdquo values are SIGNED intrsquosFloats are signed zero-centered normalized to ( -10 lt= x lt= 10 )
Example 2 ZIP files (PkWare)Purpose Size in bytes
Signature header 4
Required version 2
GP flags 2
Method 2
Last mod time 2
Last mod date 2
CRC-32 4
Compressed size 4
Uncompressed size 4
Filename length 2
Extra field length 2
File name variable
extra variable
copy DJ Foreman 2012 21
Ref httplivedocsadobecomflex3htmlhelphtmlcontent=ByteArrays_3html
More info on encoding httpwwwpkwarecomdocumentscase
studiesAPPNOTETXT
httpwwwfileinfocomfiletypescompressed
copy DJ Foreman 2012 22
Recording Bit-Rate
Constant (CBR) rate at which a codecs output data should be
consumed is constant Max bit-rate matters not the average Uses all available bandwidth Not good for storage (lossy)
Variable (VBR) Quantity of datatime unit (for output) varies Better quality for audio and video Slower to encode Supported by most portable devices post 2006
copy DJ Foreman 2012 23
Lab assignment ndash 1 ndash part 1
Write a program with 1 parameter If you are on an encoding team
Open a test file (eg ldquoxyztxtrdquo) Compress it using simple Huffman compression using
the frequency table from my FTP site Output compressed file (eg ldquoxyzencrdquo) to SAME folder
as input
If you are on a decoding team Open compressed file ldquoxyzencrdquo Decompress the input file Output file ldquoxyzdecrdquo so it can be compared to ldquoxyztxtrdquo
The parameter is the full input file path amp name (ie file names MUST NOT be hard-coded)
Remember a code can be a single bit
copy DJ Foreman 2012 24
25
Lab assignment 1 ndash part 2
Use ONLY MY TABLE for the frequencies Use ONLY uppercase letters of the table Do NOT use the special character
tables Find someone in class with the
opposite program and TEST your program to see if it properly encodes or decodes a message
Grading tree construction tree following proper binary output (encoders) proper character output (decoders)
copy DJ Foreman 2012
Working with ImagesModeling
Containers
A container is a FILE format NOT a code scheme
Eg AVI amp WAV are container formats Formats for storage Independent of encoding or content
Others Ogg ASF QuickTime RealMedia Matroska
DivX and MP4
copy DJ Foreman 2012 27
28
Movie (metadata)
Track
Media
Media Information
Container file format (MP4QT)
Chunk map
Comp time
Chunk offset
Key frame
Desc
Decode time
Priority map
Size map
Sample Table
Media Handler
Data information
Media header
Video header
Track header
Movie header
copy DJ Foreman 2012
Pixels A pixel is 3 color-dots (squares) in a collection of dots
(squares) making up the picture Each color-dot is for one of the three primary colors
RGB Minimum of 1 byte per color (depending on bit-depth
which varies with camera) thus at least 3 bytes per pixel
A 10MP camera (30 million color-dots) needs 30M bytes This gets compressed by the camera (JPEG format) Final file size = 30 million bytes divided by the amount
of compression (compression ratio) An 81 compression ratio would reduce that 30 million
bytes to 375 megabytes (as done on a 4 MP camera) A RAW file has NO compression
copy DJ Foreman 2012 29
Image creation
Basic mechanisms Photography
Film Digital ndashPhotoshop Gimp
Drawing Line BrushModeling
Do it yourself OpenGL (2x and 3x)
Modelingrendering programs Breeze designer Ayam 3DCanvas Makehuman Alice
copy DJ Foreman 2012 30
Photography (film) Silver halide crystals (AgCl AgBr AgI)
Exposure to light turns them black Developer removes the loose (exposed) halide leaving
pure silver (a negative image) Fixing bath flushes out unexposed AgHalide
Greatest desire of photographers - a medium with High speed (gathers light quickly) Low noise (no undesired artifacts) High resolution (very detailed images)
Visible lines per mm (lpm) at a specified contrast level
Notes Larger ldquoclumpsrdquo are faster but limit resolution Smaller ldquoclumpsrdquo are hard to coat evenly Randomness of coating prevents Moireacute patternshttpenwikipediaorgwikiMoirC3A9_pattern
copy DJ Foreman 2012 31
FYI ndash a little chemistry
Ag+ X- + hv rarr Ag+ + X + e-
1 Radiation frees the Br2 1 Ag ion + 1 electron (Ag+ + e- rarr Ag0) -gt 1 Ag atom3 Neighbors are now ldquosensitizedrdquo 4 Developing (Reducing) converts exposed AgX to
black metallic Ag and free X5 Remaining AgX and free X are washed away (after
mixing with thiosulfate to make it soluble)
This is one silver (Ag) ldquograinrdquo in the ldquolatent imagerdquo X is any halide (Br Cl etc)
copy DJ Foreman 2012 32
crystal
Radiation (light)
Resolution 35 mm film for comparison
A 36 x 24 mm frame of ISO 100-speed film contains (asymp) the equivalent of 20 million pixels 1 There are NO pixels in film
Full-frame digital (36 x 24mm = 32 image ratio) Canon EOS 5D Nikon D3 (21MP) - $5K2
Medium format digital (6 x 45cm) Phase One P40+ (39MP) ($22K)
APS-C sized sensor (asymp24x16mm = 32 image ratio) (note 2009 data below)
Nikon D90 (123MP) - $900 Canon Rebel (15MP) $700 Used in most DSLRrsquos (note image ratio 32)
Other SONY Alpha (note image ratio 43)
1 Langford Michael Basic Photography (7th Ed 2000) Oxford Focal Press ISBN 0 240 51592 7
2 Prices as of 2009 given for comparison purposes onlycopy DJ Foreman 2012 33
Photography (digital) Image capture formats
RAW ndash 3 separate images (RGB) [12 bitspixel (4288 2848)]8=18318336 bytes per
picture JPEG ndash compressed (16x 8x 4x) Some cameras do BOTH (RAW + JPEG) for each image
Pixel notes Sensor size 35mm (or larger) vs APS-C Pixel count ndash any increase rarr
decrease in pixel size or increase in sensor size Pixel size ndash smaller pixels give us
Greater resolution (finer details) - Good More ldquonoiserdquo (incorrect values) - bad Possible ldquoadjacencyrdquo issues - bad Loss of sensitivity - bad
copy DJ Foreman 2012 34
35
Some Comparisons
Digital media vs film Pixels are the same (RGB) Pixel size varies with device (mfgr
dependency) Pixel density varies Zooming digitally does NOT give you more
detail (in spite of what they do on TV shows)
Optical zoom is useful Uses optical magnification of the image on
the target Expansion of light rays not multiplication of
pixelscopy DJ Foreman 2012
Working Definitions - 1
Graphic Model data structure nodes represent random variables Pairs of nodes connected by arcs correspond
to variables that are not independent Graphic modeling
using a Graphic Model to simulate a real-world object (this is not a formal definition)
Rendering generating a 2D image from a 3D graphic model
copy DJ Foreman 2012 36
37
Working Definitions - 2
Geometric primitives ndash Points line segments and polygons Described by vertices
Control points ndash Special points attached to a node on a
Beacutezier curve Alter the shape and angle of adjacent curve
segment Evaluators ndash
Functions that interpolate a set of control points
Produce a new set of control pointscopy DJ Foreman 2012
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
8copy DJ Foreman 2012
Compression
Codecs A device or program for encoding andor
decoding a digital data stream or signal Encoding A-gtD for storage or transmission DecodingD-gtA for playback Lossy vs lossless Raw uncompressed Pulse Code Modulation
(PCM) audio a digital representation of an analog signal where the magnitude of the signal is sampled regularly at uniform intervals Eg (441 kHz 16 bit stereo as represented on an
audio CD or in a wav or aiff file) is a standard across multiple platforms
copy DJ Foreman 2012 9
Codecs - 2
May emphasize aspects of data Video
motion vs color Audio
latency (cell phones) vs high-fidelity (music) Data size
transmission speed or storage size vs data loss
copy DJ Foreman 2012 10
Codec References
Forensic Computing A Practitioners Guide by T Sammes amp B Jenkinson (Springer 2000) ISBN 9781852332990
httpwwwgarykesslernetlibraryfile_sigshtml
copy DJ Foreman 2012 11
Considerations
Tradeoffs compression speed compressed data size quality (data loss)
Areas of study information theory rate-distortion theory
Popular algorithms (all lossless) Lempel-Ziv (LZ) LZW (fast decompression) LZR (LZ-Renau) (ZIP files) LZW (Lempel-Ziv-Welch) (for GIF files)
copy DJ Foreman 2012 12
Lossless Encoding
Huffman compression Bit string representing any symbol is never a
prefix of the bit string representing any other symbolie if 010 is the code for a symbol then no other symbol starts with 010
Frequency table must be known computable or included
Lossless ndash every character gets encoded Arithmetic encoding
Probabilistic algorithm Slightly superior to Huffman but often
patentedcopy DJ Foreman 2012 13
Methodology
Compute a model of the data eg a Huffman tree based on probability Probability determined from input file
Map the data according to the model Read 1 symbol Search the model (tree) for that symbol
If itrsquos a Huffman tree going left=0 right=1 Append each 0 or 1 to the code string When symbol is found you are done Note all symbols will have lt 8 bits (thus
compression)
copy DJ Foreman 2012 14
15
Huffman lab ndash more details Read write are always in bytes
You may need to pad You may have extraneous letters in the output
due to padding You donrsquot need to add a header to specify size
You have 2 weeks to do the lab Code may be in C C++ or Java The readme specifies how to compile the
program Encoded data is sent to a binary file or a simple
text file (but isnrsquot ldquoreadablerdquo as text) Encoded data can only be ldquocheckedrdquo with a
decodercopy DJ Foreman 2012
16
Sample of prefix coding amp compression
copy DJ Foreman 2012
B e a r c a t s (8 bytes of text)010101100110001110001111 encoded as123456781234567812345678 3 bytes
A simple Huffman-like code(wo probability basis) 0 1
A 110
B 010
C 0001
E 10
R 011
S 111
T 001
Compression-Model types
Static Read all the data Compute frequencies Re-read the data amp encode
Dynamic Build simple basic model Modify model as more data is processed
copy DJ Foreman 2012 17
Standard Compressed-File Content
Three parts 24-byte header with magic (defines
codec) variable-length annotation block contiguous segment of audio data
Storage methodology network (big-endian) byte order multi-byte audio data may require byte
reversal in order to operate on it by the arithmetic unit of certain processors
copy DJ Foreman 2012 18
Example 1 Header file for AU files
copy DJ Foreman 2012 19
typedef unsigned long u_32 unsigned 32-bit integer
typedef struct
u_32 magic the ldquomagic numberrdquo
u_32 hdr_size byte offset to start of data
u_32 data_size length (optional)
u_32 encoding data encoding enumeration
u_32 sample_rate samples per second
u_32 channels of interleaved channels
Audio_filehdr
Following is from include ltmultimedialibaudiohgt
AUDIO_FILE_MAGIC ((u_32)0x2e736e64) ldquosndrdquo
Audio Encoding EnumerationsAUDIO_FILE_ENCODING_MULAW_8 (1) 8-bit ISDN u-law
AUDIO_FILE_ENCODING_LINEAR_8 (2) 8-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_16 (3) 16-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_32 (5) 32-bit linear PCM
AUDIO_FILE_ENCODING_FLOAT (6) 32-bit IEEE floating point
AUDIO_FILE_ENCODING_DOUBLE (7) 64-bit IEEE floating point
AUDIO_FILE_ENCODING_ADPCM_G721 (23) 4-bit CCITT g721 ADPCM
AUDIO_FILE_ENCODING_ADPCM_G723_3 (25) CCITT g723 3-bit ADPCM
AUDIO_FILE_ENCODING_ALAW_8 (27) 8-bit ISDN A-law
copy DJ Foreman 2012 20
ldquoLinearrdquo values are SIGNED intrsquosFloats are signed zero-centered normalized to ( -10 lt= x lt= 10 )
Example 2 ZIP files (PkWare)Purpose Size in bytes
Signature header 4
Required version 2
GP flags 2
Method 2
Last mod time 2
Last mod date 2
CRC-32 4
Compressed size 4
Uncompressed size 4
Filename length 2
Extra field length 2
File name variable
extra variable
copy DJ Foreman 2012 21
Ref httplivedocsadobecomflex3htmlhelphtmlcontent=ByteArrays_3html
More info on encoding httpwwwpkwarecomdocumentscase
studiesAPPNOTETXT
httpwwwfileinfocomfiletypescompressed
copy DJ Foreman 2012 22
Recording Bit-Rate
Constant (CBR) rate at which a codecs output data should be
consumed is constant Max bit-rate matters not the average Uses all available bandwidth Not good for storage (lossy)
Variable (VBR) Quantity of datatime unit (for output) varies Better quality for audio and video Slower to encode Supported by most portable devices post 2006
copy DJ Foreman 2012 23
Lab assignment ndash 1 ndash part 1
Write a program with 1 parameter If you are on an encoding team
Open a test file (eg ldquoxyztxtrdquo) Compress it using simple Huffman compression using
the frequency table from my FTP site Output compressed file (eg ldquoxyzencrdquo) to SAME folder
as input
If you are on a decoding team Open compressed file ldquoxyzencrdquo Decompress the input file Output file ldquoxyzdecrdquo so it can be compared to ldquoxyztxtrdquo
The parameter is the full input file path amp name (ie file names MUST NOT be hard-coded)
Remember a code can be a single bit
copy DJ Foreman 2012 24
25
Lab assignment 1 ndash part 2
Use ONLY MY TABLE for the frequencies Use ONLY uppercase letters of the table Do NOT use the special character
tables Find someone in class with the
opposite program and TEST your program to see if it properly encodes or decodes a message
Grading tree construction tree following proper binary output (encoders) proper character output (decoders)
copy DJ Foreman 2012
Working with ImagesModeling
Containers
A container is a FILE format NOT a code scheme
Eg AVI amp WAV are container formats Formats for storage Independent of encoding or content
Others Ogg ASF QuickTime RealMedia Matroska
DivX and MP4
copy DJ Foreman 2012 27
28
Movie (metadata)
Track
Media
Media Information
Container file format (MP4QT)
Chunk map
Comp time
Chunk offset
Key frame
Desc
Decode time
Priority map
Size map
Sample Table
Media Handler
Data information
Media header
Video header
Track header
Movie header
copy DJ Foreman 2012
Pixels A pixel is 3 color-dots (squares) in a collection of dots
(squares) making up the picture Each color-dot is for one of the three primary colors
RGB Minimum of 1 byte per color (depending on bit-depth
which varies with camera) thus at least 3 bytes per pixel
A 10MP camera (30 million color-dots) needs 30M bytes This gets compressed by the camera (JPEG format) Final file size = 30 million bytes divided by the amount
of compression (compression ratio) An 81 compression ratio would reduce that 30 million
bytes to 375 megabytes (as done on a 4 MP camera) A RAW file has NO compression
copy DJ Foreman 2012 29
Image creation
Basic mechanisms Photography
Film Digital ndashPhotoshop Gimp
Drawing Line BrushModeling
Do it yourself OpenGL (2x and 3x)
Modelingrendering programs Breeze designer Ayam 3DCanvas Makehuman Alice
copy DJ Foreman 2012 30
Photography (film) Silver halide crystals (AgCl AgBr AgI)
Exposure to light turns them black Developer removes the loose (exposed) halide leaving
pure silver (a negative image) Fixing bath flushes out unexposed AgHalide
Greatest desire of photographers - a medium with High speed (gathers light quickly) Low noise (no undesired artifacts) High resolution (very detailed images)
Visible lines per mm (lpm) at a specified contrast level
Notes Larger ldquoclumpsrdquo are faster but limit resolution Smaller ldquoclumpsrdquo are hard to coat evenly Randomness of coating prevents Moireacute patternshttpenwikipediaorgwikiMoirC3A9_pattern
copy DJ Foreman 2012 31
FYI ndash a little chemistry
Ag+ X- + hv rarr Ag+ + X + e-
1 Radiation frees the Br2 1 Ag ion + 1 electron (Ag+ + e- rarr Ag0) -gt 1 Ag atom3 Neighbors are now ldquosensitizedrdquo 4 Developing (Reducing) converts exposed AgX to
black metallic Ag and free X5 Remaining AgX and free X are washed away (after
mixing with thiosulfate to make it soluble)
This is one silver (Ag) ldquograinrdquo in the ldquolatent imagerdquo X is any halide (Br Cl etc)
copy DJ Foreman 2012 32
crystal
Radiation (light)
Resolution 35 mm film for comparison
A 36 x 24 mm frame of ISO 100-speed film contains (asymp) the equivalent of 20 million pixels 1 There are NO pixels in film
Full-frame digital (36 x 24mm = 32 image ratio) Canon EOS 5D Nikon D3 (21MP) - $5K2
Medium format digital (6 x 45cm) Phase One P40+ (39MP) ($22K)
APS-C sized sensor (asymp24x16mm = 32 image ratio) (note 2009 data below)
Nikon D90 (123MP) - $900 Canon Rebel (15MP) $700 Used in most DSLRrsquos (note image ratio 32)
Other SONY Alpha (note image ratio 43)
1 Langford Michael Basic Photography (7th Ed 2000) Oxford Focal Press ISBN 0 240 51592 7
2 Prices as of 2009 given for comparison purposes onlycopy DJ Foreman 2012 33
Photography (digital) Image capture formats
RAW ndash 3 separate images (RGB) [12 bitspixel (4288 2848)]8=18318336 bytes per
picture JPEG ndash compressed (16x 8x 4x) Some cameras do BOTH (RAW + JPEG) for each image
Pixel notes Sensor size 35mm (or larger) vs APS-C Pixel count ndash any increase rarr
decrease in pixel size or increase in sensor size Pixel size ndash smaller pixels give us
Greater resolution (finer details) - Good More ldquonoiserdquo (incorrect values) - bad Possible ldquoadjacencyrdquo issues - bad Loss of sensitivity - bad
copy DJ Foreman 2012 34
35
Some Comparisons
Digital media vs film Pixels are the same (RGB) Pixel size varies with device (mfgr
dependency) Pixel density varies Zooming digitally does NOT give you more
detail (in spite of what they do on TV shows)
Optical zoom is useful Uses optical magnification of the image on
the target Expansion of light rays not multiplication of
pixelscopy DJ Foreman 2012
Working Definitions - 1
Graphic Model data structure nodes represent random variables Pairs of nodes connected by arcs correspond
to variables that are not independent Graphic modeling
using a Graphic Model to simulate a real-world object (this is not a formal definition)
Rendering generating a 2D image from a 3D graphic model
copy DJ Foreman 2012 36
37
Working Definitions - 2
Geometric primitives ndash Points line segments and polygons Described by vertices
Control points ndash Special points attached to a node on a
Beacutezier curve Alter the shape and angle of adjacent curve
segment Evaluators ndash
Functions that interpolate a set of control points
Produce a new set of control pointscopy DJ Foreman 2012
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
Codecs A device or program for encoding andor
decoding a digital data stream or signal Encoding A-gtD for storage or transmission DecodingD-gtA for playback Lossy vs lossless Raw uncompressed Pulse Code Modulation
(PCM) audio a digital representation of an analog signal where the magnitude of the signal is sampled regularly at uniform intervals Eg (441 kHz 16 bit stereo as represented on an
audio CD or in a wav or aiff file) is a standard across multiple platforms
copy DJ Foreman 2012 9
Codecs - 2
May emphasize aspects of data Video
motion vs color Audio
latency (cell phones) vs high-fidelity (music) Data size
transmission speed or storage size vs data loss
copy DJ Foreman 2012 10
Codec References
Forensic Computing A Practitioners Guide by T Sammes amp B Jenkinson (Springer 2000) ISBN 9781852332990
httpwwwgarykesslernetlibraryfile_sigshtml
copy DJ Foreman 2012 11
Considerations
Tradeoffs compression speed compressed data size quality (data loss)
Areas of study information theory rate-distortion theory
Popular algorithms (all lossless) Lempel-Ziv (LZ) LZW (fast decompression) LZR (LZ-Renau) (ZIP files) LZW (Lempel-Ziv-Welch) (for GIF files)
copy DJ Foreman 2012 12
Lossless Encoding
Huffman compression Bit string representing any symbol is never a
prefix of the bit string representing any other symbolie if 010 is the code for a symbol then no other symbol starts with 010
Frequency table must be known computable or included
Lossless ndash every character gets encoded Arithmetic encoding
Probabilistic algorithm Slightly superior to Huffman but often
patentedcopy DJ Foreman 2012 13
Methodology
Compute a model of the data eg a Huffman tree based on probability Probability determined from input file
Map the data according to the model Read 1 symbol Search the model (tree) for that symbol
If itrsquos a Huffman tree going left=0 right=1 Append each 0 or 1 to the code string When symbol is found you are done Note all symbols will have lt 8 bits (thus
compression)
copy DJ Foreman 2012 14
15
Huffman lab ndash more details Read write are always in bytes
You may need to pad You may have extraneous letters in the output
due to padding You donrsquot need to add a header to specify size
You have 2 weeks to do the lab Code may be in C C++ or Java The readme specifies how to compile the
program Encoded data is sent to a binary file or a simple
text file (but isnrsquot ldquoreadablerdquo as text) Encoded data can only be ldquocheckedrdquo with a
decodercopy DJ Foreman 2012
16
Sample of prefix coding amp compression
copy DJ Foreman 2012
B e a r c a t s (8 bytes of text)010101100110001110001111 encoded as123456781234567812345678 3 bytes
A simple Huffman-like code(wo probability basis) 0 1
A 110
B 010
C 0001
E 10
R 011
S 111
T 001
Compression-Model types
Static Read all the data Compute frequencies Re-read the data amp encode
Dynamic Build simple basic model Modify model as more data is processed
copy DJ Foreman 2012 17
Standard Compressed-File Content
Three parts 24-byte header with magic (defines
codec) variable-length annotation block contiguous segment of audio data
Storage methodology network (big-endian) byte order multi-byte audio data may require byte
reversal in order to operate on it by the arithmetic unit of certain processors
copy DJ Foreman 2012 18
Example 1 Header file for AU files
copy DJ Foreman 2012 19
typedef unsigned long u_32 unsigned 32-bit integer
typedef struct
u_32 magic the ldquomagic numberrdquo
u_32 hdr_size byte offset to start of data
u_32 data_size length (optional)
u_32 encoding data encoding enumeration
u_32 sample_rate samples per second
u_32 channels of interleaved channels
Audio_filehdr
Following is from include ltmultimedialibaudiohgt
AUDIO_FILE_MAGIC ((u_32)0x2e736e64) ldquosndrdquo
Audio Encoding EnumerationsAUDIO_FILE_ENCODING_MULAW_8 (1) 8-bit ISDN u-law
AUDIO_FILE_ENCODING_LINEAR_8 (2) 8-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_16 (3) 16-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_32 (5) 32-bit linear PCM
AUDIO_FILE_ENCODING_FLOAT (6) 32-bit IEEE floating point
AUDIO_FILE_ENCODING_DOUBLE (7) 64-bit IEEE floating point
AUDIO_FILE_ENCODING_ADPCM_G721 (23) 4-bit CCITT g721 ADPCM
AUDIO_FILE_ENCODING_ADPCM_G723_3 (25) CCITT g723 3-bit ADPCM
AUDIO_FILE_ENCODING_ALAW_8 (27) 8-bit ISDN A-law
copy DJ Foreman 2012 20
ldquoLinearrdquo values are SIGNED intrsquosFloats are signed zero-centered normalized to ( -10 lt= x lt= 10 )
Example 2 ZIP files (PkWare)Purpose Size in bytes
Signature header 4
Required version 2
GP flags 2
Method 2
Last mod time 2
Last mod date 2
CRC-32 4
Compressed size 4
Uncompressed size 4
Filename length 2
Extra field length 2
File name variable
extra variable
copy DJ Foreman 2012 21
Ref httplivedocsadobecomflex3htmlhelphtmlcontent=ByteArrays_3html
More info on encoding httpwwwpkwarecomdocumentscase
studiesAPPNOTETXT
httpwwwfileinfocomfiletypescompressed
copy DJ Foreman 2012 22
Recording Bit-Rate
Constant (CBR) rate at which a codecs output data should be
consumed is constant Max bit-rate matters not the average Uses all available bandwidth Not good for storage (lossy)
Variable (VBR) Quantity of datatime unit (for output) varies Better quality for audio and video Slower to encode Supported by most portable devices post 2006
copy DJ Foreman 2012 23
Lab assignment ndash 1 ndash part 1
Write a program with 1 parameter If you are on an encoding team
Open a test file (eg ldquoxyztxtrdquo) Compress it using simple Huffman compression using
the frequency table from my FTP site Output compressed file (eg ldquoxyzencrdquo) to SAME folder
as input
If you are on a decoding team Open compressed file ldquoxyzencrdquo Decompress the input file Output file ldquoxyzdecrdquo so it can be compared to ldquoxyztxtrdquo
The parameter is the full input file path amp name (ie file names MUST NOT be hard-coded)
Remember a code can be a single bit
copy DJ Foreman 2012 24
25
Lab assignment 1 ndash part 2
Use ONLY MY TABLE for the frequencies Use ONLY uppercase letters of the table Do NOT use the special character
tables Find someone in class with the
opposite program and TEST your program to see if it properly encodes or decodes a message
Grading tree construction tree following proper binary output (encoders) proper character output (decoders)
copy DJ Foreman 2012
Working with ImagesModeling
Containers
A container is a FILE format NOT a code scheme
Eg AVI amp WAV are container formats Formats for storage Independent of encoding or content
Others Ogg ASF QuickTime RealMedia Matroska
DivX and MP4
copy DJ Foreman 2012 27
28
Movie (metadata)
Track
Media
Media Information
Container file format (MP4QT)
Chunk map
Comp time
Chunk offset
Key frame
Desc
Decode time
Priority map
Size map
Sample Table
Media Handler
Data information
Media header
Video header
Track header
Movie header
copy DJ Foreman 2012
Pixels A pixel is 3 color-dots (squares) in a collection of dots
(squares) making up the picture Each color-dot is for one of the three primary colors
RGB Minimum of 1 byte per color (depending on bit-depth
which varies with camera) thus at least 3 bytes per pixel
A 10MP camera (30 million color-dots) needs 30M bytes This gets compressed by the camera (JPEG format) Final file size = 30 million bytes divided by the amount
of compression (compression ratio) An 81 compression ratio would reduce that 30 million
bytes to 375 megabytes (as done on a 4 MP camera) A RAW file has NO compression
copy DJ Foreman 2012 29
Image creation
Basic mechanisms Photography
Film Digital ndashPhotoshop Gimp
Drawing Line BrushModeling
Do it yourself OpenGL (2x and 3x)
Modelingrendering programs Breeze designer Ayam 3DCanvas Makehuman Alice
copy DJ Foreman 2012 30
Photography (film) Silver halide crystals (AgCl AgBr AgI)
Exposure to light turns them black Developer removes the loose (exposed) halide leaving
pure silver (a negative image) Fixing bath flushes out unexposed AgHalide
Greatest desire of photographers - a medium with High speed (gathers light quickly) Low noise (no undesired artifacts) High resolution (very detailed images)
Visible lines per mm (lpm) at a specified contrast level
Notes Larger ldquoclumpsrdquo are faster but limit resolution Smaller ldquoclumpsrdquo are hard to coat evenly Randomness of coating prevents Moireacute patternshttpenwikipediaorgwikiMoirC3A9_pattern
copy DJ Foreman 2012 31
FYI ndash a little chemistry
Ag+ X- + hv rarr Ag+ + X + e-
1 Radiation frees the Br2 1 Ag ion + 1 electron (Ag+ + e- rarr Ag0) -gt 1 Ag atom3 Neighbors are now ldquosensitizedrdquo 4 Developing (Reducing) converts exposed AgX to
black metallic Ag and free X5 Remaining AgX and free X are washed away (after
mixing with thiosulfate to make it soluble)
This is one silver (Ag) ldquograinrdquo in the ldquolatent imagerdquo X is any halide (Br Cl etc)
copy DJ Foreman 2012 32
crystal
Radiation (light)
Resolution 35 mm film for comparison
A 36 x 24 mm frame of ISO 100-speed film contains (asymp) the equivalent of 20 million pixels 1 There are NO pixels in film
Full-frame digital (36 x 24mm = 32 image ratio) Canon EOS 5D Nikon D3 (21MP) - $5K2
Medium format digital (6 x 45cm) Phase One P40+ (39MP) ($22K)
APS-C sized sensor (asymp24x16mm = 32 image ratio) (note 2009 data below)
Nikon D90 (123MP) - $900 Canon Rebel (15MP) $700 Used in most DSLRrsquos (note image ratio 32)
Other SONY Alpha (note image ratio 43)
1 Langford Michael Basic Photography (7th Ed 2000) Oxford Focal Press ISBN 0 240 51592 7
2 Prices as of 2009 given for comparison purposes onlycopy DJ Foreman 2012 33
Photography (digital) Image capture formats
RAW ndash 3 separate images (RGB) [12 bitspixel (4288 2848)]8=18318336 bytes per
picture JPEG ndash compressed (16x 8x 4x) Some cameras do BOTH (RAW + JPEG) for each image
Pixel notes Sensor size 35mm (or larger) vs APS-C Pixel count ndash any increase rarr
decrease in pixel size or increase in sensor size Pixel size ndash smaller pixels give us
Greater resolution (finer details) - Good More ldquonoiserdquo (incorrect values) - bad Possible ldquoadjacencyrdquo issues - bad Loss of sensitivity - bad
copy DJ Foreman 2012 34
35
Some Comparisons
Digital media vs film Pixels are the same (RGB) Pixel size varies with device (mfgr
dependency) Pixel density varies Zooming digitally does NOT give you more
detail (in spite of what they do on TV shows)
Optical zoom is useful Uses optical magnification of the image on
the target Expansion of light rays not multiplication of
pixelscopy DJ Foreman 2012
Working Definitions - 1
Graphic Model data structure nodes represent random variables Pairs of nodes connected by arcs correspond
to variables that are not independent Graphic modeling
using a Graphic Model to simulate a real-world object (this is not a formal definition)
Rendering generating a 2D image from a 3D graphic model
copy DJ Foreman 2012 36
37
Working Definitions - 2
Geometric primitives ndash Points line segments and polygons Described by vertices
Control points ndash Special points attached to a node on a
Beacutezier curve Alter the shape and angle of adjacent curve
segment Evaluators ndash
Functions that interpolate a set of control points
Produce a new set of control pointscopy DJ Foreman 2012
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
Codecs - 2
May emphasize aspects of data Video
motion vs color Audio
latency (cell phones) vs high-fidelity (music) Data size
transmission speed or storage size vs data loss
copy DJ Foreman 2012 10
Codec References
Forensic Computing A Practitioners Guide by T Sammes amp B Jenkinson (Springer 2000) ISBN 9781852332990
httpwwwgarykesslernetlibraryfile_sigshtml
copy DJ Foreman 2012 11
Considerations
Tradeoffs compression speed compressed data size quality (data loss)
Areas of study information theory rate-distortion theory
Popular algorithms (all lossless) Lempel-Ziv (LZ) LZW (fast decompression) LZR (LZ-Renau) (ZIP files) LZW (Lempel-Ziv-Welch) (for GIF files)
copy DJ Foreman 2012 12
Lossless Encoding
Huffman compression Bit string representing any symbol is never a
prefix of the bit string representing any other symbolie if 010 is the code for a symbol then no other symbol starts with 010
Frequency table must be known computable or included
Lossless ndash every character gets encoded Arithmetic encoding
Probabilistic algorithm Slightly superior to Huffman but often
patentedcopy DJ Foreman 2012 13
Methodology
Compute a model of the data eg a Huffman tree based on probability Probability determined from input file
Map the data according to the model Read 1 symbol Search the model (tree) for that symbol
If itrsquos a Huffman tree going left=0 right=1 Append each 0 or 1 to the code string When symbol is found you are done Note all symbols will have lt 8 bits (thus
compression)
copy DJ Foreman 2012 14
15
Huffman lab ndash more details Read write are always in bytes
You may need to pad You may have extraneous letters in the output
due to padding You donrsquot need to add a header to specify size
You have 2 weeks to do the lab Code may be in C C++ or Java The readme specifies how to compile the
program Encoded data is sent to a binary file or a simple
text file (but isnrsquot ldquoreadablerdquo as text) Encoded data can only be ldquocheckedrdquo with a
decodercopy DJ Foreman 2012
16
Sample of prefix coding amp compression
copy DJ Foreman 2012
B e a r c a t s (8 bytes of text)010101100110001110001111 encoded as123456781234567812345678 3 bytes
A simple Huffman-like code(wo probability basis) 0 1
A 110
B 010
C 0001
E 10
R 011
S 111
T 001
Compression-Model types
Static Read all the data Compute frequencies Re-read the data amp encode
Dynamic Build simple basic model Modify model as more data is processed
copy DJ Foreman 2012 17
Standard Compressed-File Content
Three parts 24-byte header with magic (defines
codec) variable-length annotation block contiguous segment of audio data
Storage methodology network (big-endian) byte order multi-byte audio data may require byte
reversal in order to operate on it by the arithmetic unit of certain processors
copy DJ Foreman 2012 18
Example 1 Header file for AU files
copy DJ Foreman 2012 19
typedef unsigned long u_32 unsigned 32-bit integer
typedef struct
u_32 magic the ldquomagic numberrdquo
u_32 hdr_size byte offset to start of data
u_32 data_size length (optional)
u_32 encoding data encoding enumeration
u_32 sample_rate samples per second
u_32 channels of interleaved channels
Audio_filehdr
Following is from include ltmultimedialibaudiohgt
AUDIO_FILE_MAGIC ((u_32)0x2e736e64) ldquosndrdquo
Audio Encoding EnumerationsAUDIO_FILE_ENCODING_MULAW_8 (1) 8-bit ISDN u-law
AUDIO_FILE_ENCODING_LINEAR_8 (2) 8-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_16 (3) 16-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_32 (5) 32-bit linear PCM
AUDIO_FILE_ENCODING_FLOAT (6) 32-bit IEEE floating point
AUDIO_FILE_ENCODING_DOUBLE (7) 64-bit IEEE floating point
AUDIO_FILE_ENCODING_ADPCM_G721 (23) 4-bit CCITT g721 ADPCM
AUDIO_FILE_ENCODING_ADPCM_G723_3 (25) CCITT g723 3-bit ADPCM
AUDIO_FILE_ENCODING_ALAW_8 (27) 8-bit ISDN A-law
copy DJ Foreman 2012 20
ldquoLinearrdquo values are SIGNED intrsquosFloats are signed zero-centered normalized to ( -10 lt= x lt= 10 )
Example 2 ZIP files (PkWare)Purpose Size in bytes
Signature header 4
Required version 2
GP flags 2
Method 2
Last mod time 2
Last mod date 2
CRC-32 4
Compressed size 4
Uncompressed size 4
Filename length 2
Extra field length 2
File name variable
extra variable
copy DJ Foreman 2012 21
Ref httplivedocsadobecomflex3htmlhelphtmlcontent=ByteArrays_3html
More info on encoding httpwwwpkwarecomdocumentscase
studiesAPPNOTETXT
httpwwwfileinfocomfiletypescompressed
copy DJ Foreman 2012 22
Recording Bit-Rate
Constant (CBR) rate at which a codecs output data should be
consumed is constant Max bit-rate matters not the average Uses all available bandwidth Not good for storage (lossy)
Variable (VBR) Quantity of datatime unit (for output) varies Better quality for audio and video Slower to encode Supported by most portable devices post 2006
copy DJ Foreman 2012 23
Lab assignment ndash 1 ndash part 1
Write a program with 1 parameter If you are on an encoding team
Open a test file (eg ldquoxyztxtrdquo) Compress it using simple Huffman compression using
the frequency table from my FTP site Output compressed file (eg ldquoxyzencrdquo) to SAME folder
as input
If you are on a decoding team Open compressed file ldquoxyzencrdquo Decompress the input file Output file ldquoxyzdecrdquo so it can be compared to ldquoxyztxtrdquo
The parameter is the full input file path amp name (ie file names MUST NOT be hard-coded)
Remember a code can be a single bit
copy DJ Foreman 2012 24
25
Lab assignment 1 ndash part 2
Use ONLY MY TABLE for the frequencies Use ONLY uppercase letters of the table Do NOT use the special character
tables Find someone in class with the
opposite program and TEST your program to see if it properly encodes or decodes a message
Grading tree construction tree following proper binary output (encoders) proper character output (decoders)
copy DJ Foreman 2012
Working with ImagesModeling
Containers
A container is a FILE format NOT a code scheme
Eg AVI amp WAV are container formats Formats for storage Independent of encoding or content
Others Ogg ASF QuickTime RealMedia Matroska
DivX and MP4
copy DJ Foreman 2012 27
28
Movie (metadata)
Track
Media
Media Information
Container file format (MP4QT)
Chunk map
Comp time
Chunk offset
Key frame
Desc
Decode time
Priority map
Size map
Sample Table
Media Handler
Data information
Media header
Video header
Track header
Movie header
copy DJ Foreman 2012
Pixels A pixel is 3 color-dots (squares) in a collection of dots
(squares) making up the picture Each color-dot is for one of the three primary colors
RGB Minimum of 1 byte per color (depending on bit-depth
which varies with camera) thus at least 3 bytes per pixel
A 10MP camera (30 million color-dots) needs 30M bytes This gets compressed by the camera (JPEG format) Final file size = 30 million bytes divided by the amount
of compression (compression ratio) An 81 compression ratio would reduce that 30 million
bytes to 375 megabytes (as done on a 4 MP camera) A RAW file has NO compression
copy DJ Foreman 2012 29
Image creation
Basic mechanisms Photography
Film Digital ndashPhotoshop Gimp
Drawing Line BrushModeling
Do it yourself OpenGL (2x and 3x)
Modelingrendering programs Breeze designer Ayam 3DCanvas Makehuman Alice
copy DJ Foreman 2012 30
Photography (film) Silver halide crystals (AgCl AgBr AgI)
Exposure to light turns them black Developer removes the loose (exposed) halide leaving
pure silver (a negative image) Fixing bath flushes out unexposed AgHalide
Greatest desire of photographers - a medium with High speed (gathers light quickly) Low noise (no undesired artifacts) High resolution (very detailed images)
Visible lines per mm (lpm) at a specified contrast level
Notes Larger ldquoclumpsrdquo are faster but limit resolution Smaller ldquoclumpsrdquo are hard to coat evenly Randomness of coating prevents Moireacute patternshttpenwikipediaorgwikiMoirC3A9_pattern
copy DJ Foreman 2012 31
FYI ndash a little chemistry
Ag+ X- + hv rarr Ag+ + X + e-
1 Radiation frees the Br2 1 Ag ion + 1 electron (Ag+ + e- rarr Ag0) -gt 1 Ag atom3 Neighbors are now ldquosensitizedrdquo 4 Developing (Reducing) converts exposed AgX to
black metallic Ag and free X5 Remaining AgX and free X are washed away (after
mixing with thiosulfate to make it soluble)
This is one silver (Ag) ldquograinrdquo in the ldquolatent imagerdquo X is any halide (Br Cl etc)
copy DJ Foreman 2012 32
crystal
Radiation (light)
Resolution 35 mm film for comparison
A 36 x 24 mm frame of ISO 100-speed film contains (asymp) the equivalent of 20 million pixels 1 There are NO pixels in film
Full-frame digital (36 x 24mm = 32 image ratio) Canon EOS 5D Nikon D3 (21MP) - $5K2
Medium format digital (6 x 45cm) Phase One P40+ (39MP) ($22K)
APS-C sized sensor (asymp24x16mm = 32 image ratio) (note 2009 data below)
Nikon D90 (123MP) - $900 Canon Rebel (15MP) $700 Used in most DSLRrsquos (note image ratio 32)
Other SONY Alpha (note image ratio 43)
1 Langford Michael Basic Photography (7th Ed 2000) Oxford Focal Press ISBN 0 240 51592 7
2 Prices as of 2009 given for comparison purposes onlycopy DJ Foreman 2012 33
Photography (digital) Image capture formats
RAW ndash 3 separate images (RGB) [12 bitspixel (4288 2848)]8=18318336 bytes per
picture JPEG ndash compressed (16x 8x 4x) Some cameras do BOTH (RAW + JPEG) for each image
Pixel notes Sensor size 35mm (or larger) vs APS-C Pixel count ndash any increase rarr
decrease in pixel size or increase in sensor size Pixel size ndash smaller pixels give us
Greater resolution (finer details) - Good More ldquonoiserdquo (incorrect values) - bad Possible ldquoadjacencyrdquo issues - bad Loss of sensitivity - bad
copy DJ Foreman 2012 34
35
Some Comparisons
Digital media vs film Pixels are the same (RGB) Pixel size varies with device (mfgr
dependency) Pixel density varies Zooming digitally does NOT give you more
detail (in spite of what they do on TV shows)
Optical zoom is useful Uses optical magnification of the image on
the target Expansion of light rays not multiplication of
pixelscopy DJ Foreman 2012
Working Definitions - 1
Graphic Model data structure nodes represent random variables Pairs of nodes connected by arcs correspond
to variables that are not independent Graphic modeling
using a Graphic Model to simulate a real-world object (this is not a formal definition)
Rendering generating a 2D image from a 3D graphic model
copy DJ Foreman 2012 36
37
Working Definitions - 2
Geometric primitives ndash Points line segments and polygons Described by vertices
Control points ndash Special points attached to a node on a
Beacutezier curve Alter the shape and angle of adjacent curve
segment Evaluators ndash
Functions that interpolate a set of control points
Produce a new set of control pointscopy DJ Foreman 2012
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
Codec References
Forensic Computing A Practitioners Guide by T Sammes amp B Jenkinson (Springer 2000) ISBN 9781852332990
httpwwwgarykesslernetlibraryfile_sigshtml
copy DJ Foreman 2012 11
Considerations
Tradeoffs compression speed compressed data size quality (data loss)
Areas of study information theory rate-distortion theory
Popular algorithms (all lossless) Lempel-Ziv (LZ) LZW (fast decompression) LZR (LZ-Renau) (ZIP files) LZW (Lempel-Ziv-Welch) (for GIF files)
copy DJ Foreman 2012 12
Lossless Encoding
Huffman compression Bit string representing any symbol is never a
prefix of the bit string representing any other symbolie if 010 is the code for a symbol then no other symbol starts with 010
Frequency table must be known computable or included
Lossless ndash every character gets encoded Arithmetic encoding
Probabilistic algorithm Slightly superior to Huffman but often
patentedcopy DJ Foreman 2012 13
Methodology
Compute a model of the data eg a Huffman tree based on probability Probability determined from input file
Map the data according to the model Read 1 symbol Search the model (tree) for that symbol
If itrsquos a Huffman tree going left=0 right=1 Append each 0 or 1 to the code string When symbol is found you are done Note all symbols will have lt 8 bits (thus
compression)
copy DJ Foreman 2012 14
15
Huffman lab ndash more details Read write are always in bytes
You may need to pad You may have extraneous letters in the output
due to padding You donrsquot need to add a header to specify size
You have 2 weeks to do the lab Code may be in C C++ or Java The readme specifies how to compile the
program Encoded data is sent to a binary file or a simple
text file (but isnrsquot ldquoreadablerdquo as text) Encoded data can only be ldquocheckedrdquo with a
decodercopy DJ Foreman 2012
16
Sample of prefix coding amp compression
copy DJ Foreman 2012
B e a r c a t s (8 bytes of text)010101100110001110001111 encoded as123456781234567812345678 3 bytes
A simple Huffman-like code(wo probability basis) 0 1
A 110
B 010
C 0001
E 10
R 011
S 111
T 001
Compression-Model types
Static Read all the data Compute frequencies Re-read the data amp encode
Dynamic Build simple basic model Modify model as more data is processed
copy DJ Foreman 2012 17
Standard Compressed-File Content
Three parts 24-byte header with magic (defines
codec) variable-length annotation block contiguous segment of audio data
Storage methodology network (big-endian) byte order multi-byte audio data may require byte
reversal in order to operate on it by the arithmetic unit of certain processors
copy DJ Foreman 2012 18
Example 1 Header file for AU files
copy DJ Foreman 2012 19
typedef unsigned long u_32 unsigned 32-bit integer
typedef struct
u_32 magic the ldquomagic numberrdquo
u_32 hdr_size byte offset to start of data
u_32 data_size length (optional)
u_32 encoding data encoding enumeration
u_32 sample_rate samples per second
u_32 channels of interleaved channels
Audio_filehdr
Following is from include ltmultimedialibaudiohgt
AUDIO_FILE_MAGIC ((u_32)0x2e736e64) ldquosndrdquo
Audio Encoding EnumerationsAUDIO_FILE_ENCODING_MULAW_8 (1) 8-bit ISDN u-law
AUDIO_FILE_ENCODING_LINEAR_8 (2) 8-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_16 (3) 16-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_32 (5) 32-bit linear PCM
AUDIO_FILE_ENCODING_FLOAT (6) 32-bit IEEE floating point
AUDIO_FILE_ENCODING_DOUBLE (7) 64-bit IEEE floating point
AUDIO_FILE_ENCODING_ADPCM_G721 (23) 4-bit CCITT g721 ADPCM
AUDIO_FILE_ENCODING_ADPCM_G723_3 (25) CCITT g723 3-bit ADPCM
AUDIO_FILE_ENCODING_ALAW_8 (27) 8-bit ISDN A-law
copy DJ Foreman 2012 20
ldquoLinearrdquo values are SIGNED intrsquosFloats are signed zero-centered normalized to ( -10 lt= x lt= 10 )
Example 2 ZIP files (PkWare)Purpose Size in bytes
Signature header 4
Required version 2
GP flags 2
Method 2
Last mod time 2
Last mod date 2
CRC-32 4
Compressed size 4
Uncompressed size 4
Filename length 2
Extra field length 2
File name variable
extra variable
copy DJ Foreman 2012 21
Ref httplivedocsadobecomflex3htmlhelphtmlcontent=ByteArrays_3html
More info on encoding httpwwwpkwarecomdocumentscase
studiesAPPNOTETXT
httpwwwfileinfocomfiletypescompressed
copy DJ Foreman 2012 22
Recording Bit-Rate
Constant (CBR) rate at which a codecs output data should be
consumed is constant Max bit-rate matters not the average Uses all available bandwidth Not good for storage (lossy)
Variable (VBR) Quantity of datatime unit (for output) varies Better quality for audio and video Slower to encode Supported by most portable devices post 2006
copy DJ Foreman 2012 23
Lab assignment ndash 1 ndash part 1
Write a program with 1 parameter If you are on an encoding team
Open a test file (eg ldquoxyztxtrdquo) Compress it using simple Huffman compression using
the frequency table from my FTP site Output compressed file (eg ldquoxyzencrdquo) to SAME folder
as input
If you are on a decoding team Open compressed file ldquoxyzencrdquo Decompress the input file Output file ldquoxyzdecrdquo so it can be compared to ldquoxyztxtrdquo
The parameter is the full input file path amp name (ie file names MUST NOT be hard-coded)
Remember a code can be a single bit
copy DJ Foreman 2012 24
25
Lab assignment 1 ndash part 2
Use ONLY MY TABLE for the frequencies Use ONLY uppercase letters of the table Do NOT use the special character
tables Find someone in class with the
opposite program and TEST your program to see if it properly encodes or decodes a message
Grading tree construction tree following proper binary output (encoders) proper character output (decoders)
copy DJ Foreman 2012
Working with ImagesModeling
Containers
A container is a FILE format NOT a code scheme
Eg AVI amp WAV are container formats Formats for storage Independent of encoding or content
Others Ogg ASF QuickTime RealMedia Matroska
DivX and MP4
copy DJ Foreman 2012 27
28
Movie (metadata)
Track
Media
Media Information
Container file format (MP4QT)
Chunk map
Comp time
Chunk offset
Key frame
Desc
Decode time
Priority map
Size map
Sample Table
Media Handler
Data information
Media header
Video header
Track header
Movie header
copy DJ Foreman 2012
Pixels A pixel is 3 color-dots (squares) in a collection of dots
(squares) making up the picture Each color-dot is for one of the three primary colors
RGB Minimum of 1 byte per color (depending on bit-depth
which varies with camera) thus at least 3 bytes per pixel
A 10MP camera (30 million color-dots) needs 30M bytes This gets compressed by the camera (JPEG format) Final file size = 30 million bytes divided by the amount
of compression (compression ratio) An 81 compression ratio would reduce that 30 million
bytes to 375 megabytes (as done on a 4 MP camera) A RAW file has NO compression
copy DJ Foreman 2012 29
Image creation
Basic mechanisms Photography
Film Digital ndashPhotoshop Gimp
Drawing Line BrushModeling
Do it yourself OpenGL (2x and 3x)
Modelingrendering programs Breeze designer Ayam 3DCanvas Makehuman Alice
copy DJ Foreman 2012 30
Photography (film) Silver halide crystals (AgCl AgBr AgI)
Exposure to light turns them black Developer removes the loose (exposed) halide leaving
pure silver (a negative image) Fixing bath flushes out unexposed AgHalide
Greatest desire of photographers - a medium with High speed (gathers light quickly) Low noise (no undesired artifacts) High resolution (very detailed images)
Visible lines per mm (lpm) at a specified contrast level
Notes Larger ldquoclumpsrdquo are faster but limit resolution Smaller ldquoclumpsrdquo are hard to coat evenly Randomness of coating prevents Moireacute patternshttpenwikipediaorgwikiMoirC3A9_pattern
copy DJ Foreman 2012 31
FYI ndash a little chemistry
Ag+ X- + hv rarr Ag+ + X + e-
1 Radiation frees the Br2 1 Ag ion + 1 electron (Ag+ + e- rarr Ag0) -gt 1 Ag atom3 Neighbors are now ldquosensitizedrdquo 4 Developing (Reducing) converts exposed AgX to
black metallic Ag and free X5 Remaining AgX and free X are washed away (after
mixing with thiosulfate to make it soluble)
This is one silver (Ag) ldquograinrdquo in the ldquolatent imagerdquo X is any halide (Br Cl etc)
copy DJ Foreman 2012 32
crystal
Radiation (light)
Resolution 35 mm film for comparison
A 36 x 24 mm frame of ISO 100-speed film contains (asymp) the equivalent of 20 million pixels 1 There are NO pixels in film
Full-frame digital (36 x 24mm = 32 image ratio) Canon EOS 5D Nikon D3 (21MP) - $5K2
Medium format digital (6 x 45cm) Phase One P40+ (39MP) ($22K)
APS-C sized sensor (asymp24x16mm = 32 image ratio) (note 2009 data below)
Nikon D90 (123MP) - $900 Canon Rebel (15MP) $700 Used in most DSLRrsquos (note image ratio 32)
Other SONY Alpha (note image ratio 43)
1 Langford Michael Basic Photography (7th Ed 2000) Oxford Focal Press ISBN 0 240 51592 7
2 Prices as of 2009 given for comparison purposes onlycopy DJ Foreman 2012 33
Photography (digital) Image capture formats
RAW ndash 3 separate images (RGB) [12 bitspixel (4288 2848)]8=18318336 bytes per
picture JPEG ndash compressed (16x 8x 4x) Some cameras do BOTH (RAW + JPEG) for each image
Pixel notes Sensor size 35mm (or larger) vs APS-C Pixel count ndash any increase rarr
decrease in pixel size or increase in sensor size Pixel size ndash smaller pixels give us
Greater resolution (finer details) - Good More ldquonoiserdquo (incorrect values) - bad Possible ldquoadjacencyrdquo issues - bad Loss of sensitivity - bad
copy DJ Foreman 2012 34
35
Some Comparisons
Digital media vs film Pixels are the same (RGB) Pixel size varies with device (mfgr
dependency) Pixel density varies Zooming digitally does NOT give you more
detail (in spite of what they do on TV shows)
Optical zoom is useful Uses optical magnification of the image on
the target Expansion of light rays not multiplication of
pixelscopy DJ Foreman 2012
Working Definitions - 1
Graphic Model data structure nodes represent random variables Pairs of nodes connected by arcs correspond
to variables that are not independent Graphic modeling
using a Graphic Model to simulate a real-world object (this is not a formal definition)
Rendering generating a 2D image from a 3D graphic model
copy DJ Foreman 2012 36
37
Working Definitions - 2
Geometric primitives ndash Points line segments and polygons Described by vertices
Control points ndash Special points attached to a node on a
Beacutezier curve Alter the shape and angle of adjacent curve
segment Evaluators ndash
Functions that interpolate a set of control points
Produce a new set of control pointscopy DJ Foreman 2012
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
Considerations
Tradeoffs compression speed compressed data size quality (data loss)
Areas of study information theory rate-distortion theory
Popular algorithms (all lossless) Lempel-Ziv (LZ) LZW (fast decompression) LZR (LZ-Renau) (ZIP files) LZW (Lempel-Ziv-Welch) (for GIF files)
copy DJ Foreman 2012 12
Lossless Encoding
Huffman compression Bit string representing any symbol is never a
prefix of the bit string representing any other symbolie if 010 is the code for a symbol then no other symbol starts with 010
Frequency table must be known computable or included
Lossless ndash every character gets encoded Arithmetic encoding
Probabilistic algorithm Slightly superior to Huffman but often
patentedcopy DJ Foreman 2012 13
Methodology
Compute a model of the data eg a Huffman tree based on probability Probability determined from input file
Map the data according to the model Read 1 symbol Search the model (tree) for that symbol
If itrsquos a Huffman tree going left=0 right=1 Append each 0 or 1 to the code string When symbol is found you are done Note all symbols will have lt 8 bits (thus
compression)
copy DJ Foreman 2012 14
15
Huffman lab ndash more details Read write are always in bytes
You may need to pad You may have extraneous letters in the output
due to padding You donrsquot need to add a header to specify size
You have 2 weeks to do the lab Code may be in C C++ or Java The readme specifies how to compile the
program Encoded data is sent to a binary file or a simple
text file (but isnrsquot ldquoreadablerdquo as text) Encoded data can only be ldquocheckedrdquo with a
decodercopy DJ Foreman 2012
16
Sample of prefix coding amp compression
copy DJ Foreman 2012
B e a r c a t s (8 bytes of text)010101100110001110001111 encoded as123456781234567812345678 3 bytes
A simple Huffman-like code(wo probability basis) 0 1
A 110
B 010
C 0001
E 10
R 011
S 111
T 001
Compression-Model types
Static Read all the data Compute frequencies Re-read the data amp encode
Dynamic Build simple basic model Modify model as more data is processed
copy DJ Foreman 2012 17
Standard Compressed-File Content
Three parts 24-byte header with magic (defines
codec) variable-length annotation block contiguous segment of audio data
Storage methodology network (big-endian) byte order multi-byte audio data may require byte
reversal in order to operate on it by the arithmetic unit of certain processors
copy DJ Foreman 2012 18
Example 1 Header file for AU files
copy DJ Foreman 2012 19
typedef unsigned long u_32 unsigned 32-bit integer
typedef struct
u_32 magic the ldquomagic numberrdquo
u_32 hdr_size byte offset to start of data
u_32 data_size length (optional)
u_32 encoding data encoding enumeration
u_32 sample_rate samples per second
u_32 channels of interleaved channels
Audio_filehdr
Following is from include ltmultimedialibaudiohgt
AUDIO_FILE_MAGIC ((u_32)0x2e736e64) ldquosndrdquo
Audio Encoding EnumerationsAUDIO_FILE_ENCODING_MULAW_8 (1) 8-bit ISDN u-law
AUDIO_FILE_ENCODING_LINEAR_8 (2) 8-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_16 (3) 16-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_32 (5) 32-bit linear PCM
AUDIO_FILE_ENCODING_FLOAT (6) 32-bit IEEE floating point
AUDIO_FILE_ENCODING_DOUBLE (7) 64-bit IEEE floating point
AUDIO_FILE_ENCODING_ADPCM_G721 (23) 4-bit CCITT g721 ADPCM
AUDIO_FILE_ENCODING_ADPCM_G723_3 (25) CCITT g723 3-bit ADPCM
AUDIO_FILE_ENCODING_ALAW_8 (27) 8-bit ISDN A-law
copy DJ Foreman 2012 20
ldquoLinearrdquo values are SIGNED intrsquosFloats are signed zero-centered normalized to ( -10 lt= x lt= 10 )
Example 2 ZIP files (PkWare)Purpose Size in bytes
Signature header 4
Required version 2
GP flags 2
Method 2
Last mod time 2
Last mod date 2
CRC-32 4
Compressed size 4
Uncompressed size 4
Filename length 2
Extra field length 2
File name variable
extra variable
copy DJ Foreman 2012 21
Ref httplivedocsadobecomflex3htmlhelphtmlcontent=ByteArrays_3html
More info on encoding httpwwwpkwarecomdocumentscase
studiesAPPNOTETXT
httpwwwfileinfocomfiletypescompressed
copy DJ Foreman 2012 22
Recording Bit-Rate
Constant (CBR) rate at which a codecs output data should be
consumed is constant Max bit-rate matters not the average Uses all available bandwidth Not good for storage (lossy)
Variable (VBR) Quantity of datatime unit (for output) varies Better quality for audio and video Slower to encode Supported by most portable devices post 2006
copy DJ Foreman 2012 23
Lab assignment ndash 1 ndash part 1
Write a program with 1 parameter If you are on an encoding team
Open a test file (eg ldquoxyztxtrdquo) Compress it using simple Huffman compression using
the frequency table from my FTP site Output compressed file (eg ldquoxyzencrdquo) to SAME folder
as input
If you are on a decoding team Open compressed file ldquoxyzencrdquo Decompress the input file Output file ldquoxyzdecrdquo so it can be compared to ldquoxyztxtrdquo
The parameter is the full input file path amp name (ie file names MUST NOT be hard-coded)
Remember a code can be a single bit
copy DJ Foreman 2012 24
25
Lab assignment 1 ndash part 2
Use ONLY MY TABLE for the frequencies Use ONLY uppercase letters of the table Do NOT use the special character
tables Find someone in class with the
opposite program and TEST your program to see if it properly encodes or decodes a message
Grading tree construction tree following proper binary output (encoders) proper character output (decoders)
copy DJ Foreman 2012
Working with ImagesModeling
Containers
A container is a FILE format NOT a code scheme
Eg AVI amp WAV are container formats Formats for storage Independent of encoding or content
Others Ogg ASF QuickTime RealMedia Matroska
DivX and MP4
copy DJ Foreman 2012 27
28
Movie (metadata)
Track
Media
Media Information
Container file format (MP4QT)
Chunk map
Comp time
Chunk offset
Key frame
Desc
Decode time
Priority map
Size map
Sample Table
Media Handler
Data information
Media header
Video header
Track header
Movie header
copy DJ Foreman 2012
Pixels A pixel is 3 color-dots (squares) in a collection of dots
(squares) making up the picture Each color-dot is for one of the three primary colors
RGB Minimum of 1 byte per color (depending on bit-depth
which varies with camera) thus at least 3 bytes per pixel
A 10MP camera (30 million color-dots) needs 30M bytes This gets compressed by the camera (JPEG format) Final file size = 30 million bytes divided by the amount
of compression (compression ratio) An 81 compression ratio would reduce that 30 million
bytes to 375 megabytes (as done on a 4 MP camera) A RAW file has NO compression
copy DJ Foreman 2012 29
Image creation
Basic mechanisms Photography
Film Digital ndashPhotoshop Gimp
Drawing Line BrushModeling
Do it yourself OpenGL (2x and 3x)
Modelingrendering programs Breeze designer Ayam 3DCanvas Makehuman Alice
copy DJ Foreman 2012 30
Photography (film) Silver halide crystals (AgCl AgBr AgI)
Exposure to light turns them black Developer removes the loose (exposed) halide leaving
pure silver (a negative image) Fixing bath flushes out unexposed AgHalide
Greatest desire of photographers - a medium with High speed (gathers light quickly) Low noise (no undesired artifacts) High resolution (very detailed images)
Visible lines per mm (lpm) at a specified contrast level
Notes Larger ldquoclumpsrdquo are faster but limit resolution Smaller ldquoclumpsrdquo are hard to coat evenly Randomness of coating prevents Moireacute patternshttpenwikipediaorgwikiMoirC3A9_pattern
copy DJ Foreman 2012 31
FYI ndash a little chemistry
Ag+ X- + hv rarr Ag+ + X + e-
1 Radiation frees the Br2 1 Ag ion + 1 electron (Ag+ + e- rarr Ag0) -gt 1 Ag atom3 Neighbors are now ldquosensitizedrdquo 4 Developing (Reducing) converts exposed AgX to
black metallic Ag and free X5 Remaining AgX and free X are washed away (after
mixing with thiosulfate to make it soluble)
This is one silver (Ag) ldquograinrdquo in the ldquolatent imagerdquo X is any halide (Br Cl etc)
copy DJ Foreman 2012 32
crystal
Radiation (light)
Resolution 35 mm film for comparison
A 36 x 24 mm frame of ISO 100-speed film contains (asymp) the equivalent of 20 million pixels 1 There are NO pixels in film
Full-frame digital (36 x 24mm = 32 image ratio) Canon EOS 5D Nikon D3 (21MP) - $5K2
Medium format digital (6 x 45cm) Phase One P40+ (39MP) ($22K)
APS-C sized sensor (asymp24x16mm = 32 image ratio) (note 2009 data below)
Nikon D90 (123MP) - $900 Canon Rebel (15MP) $700 Used in most DSLRrsquos (note image ratio 32)
Other SONY Alpha (note image ratio 43)
1 Langford Michael Basic Photography (7th Ed 2000) Oxford Focal Press ISBN 0 240 51592 7
2 Prices as of 2009 given for comparison purposes onlycopy DJ Foreman 2012 33
Photography (digital) Image capture formats
RAW ndash 3 separate images (RGB) [12 bitspixel (4288 2848)]8=18318336 bytes per
picture JPEG ndash compressed (16x 8x 4x) Some cameras do BOTH (RAW + JPEG) for each image
Pixel notes Sensor size 35mm (or larger) vs APS-C Pixel count ndash any increase rarr
decrease in pixel size or increase in sensor size Pixel size ndash smaller pixels give us
Greater resolution (finer details) - Good More ldquonoiserdquo (incorrect values) - bad Possible ldquoadjacencyrdquo issues - bad Loss of sensitivity - bad
copy DJ Foreman 2012 34
35
Some Comparisons
Digital media vs film Pixels are the same (RGB) Pixel size varies with device (mfgr
dependency) Pixel density varies Zooming digitally does NOT give you more
detail (in spite of what they do on TV shows)
Optical zoom is useful Uses optical magnification of the image on
the target Expansion of light rays not multiplication of
pixelscopy DJ Foreman 2012
Working Definitions - 1
Graphic Model data structure nodes represent random variables Pairs of nodes connected by arcs correspond
to variables that are not independent Graphic modeling
using a Graphic Model to simulate a real-world object (this is not a formal definition)
Rendering generating a 2D image from a 3D graphic model
copy DJ Foreman 2012 36
37
Working Definitions - 2
Geometric primitives ndash Points line segments and polygons Described by vertices
Control points ndash Special points attached to a node on a
Beacutezier curve Alter the shape and angle of adjacent curve
segment Evaluators ndash
Functions that interpolate a set of control points
Produce a new set of control pointscopy DJ Foreman 2012
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
Lossless Encoding
Huffman compression Bit string representing any symbol is never a
prefix of the bit string representing any other symbolie if 010 is the code for a symbol then no other symbol starts with 010
Frequency table must be known computable or included
Lossless ndash every character gets encoded Arithmetic encoding
Probabilistic algorithm Slightly superior to Huffman but often
patentedcopy DJ Foreman 2012 13
Methodology
Compute a model of the data eg a Huffman tree based on probability Probability determined from input file
Map the data according to the model Read 1 symbol Search the model (tree) for that symbol
If itrsquos a Huffman tree going left=0 right=1 Append each 0 or 1 to the code string When symbol is found you are done Note all symbols will have lt 8 bits (thus
compression)
copy DJ Foreman 2012 14
15
Huffman lab ndash more details Read write are always in bytes
You may need to pad You may have extraneous letters in the output
due to padding You donrsquot need to add a header to specify size
You have 2 weeks to do the lab Code may be in C C++ or Java The readme specifies how to compile the
program Encoded data is sent to a binary file or a simple
text file (but isnrsquot ldquoreadablerdquo as text) Encoded data can only be ldquocheckedrdquo with a
decodercopy DJ Foreman 2012
16
Sample of prefix coding amp compression
copy DJ Foreman 2012
B e a r c a t s (8 bytes of text)010101100110001110001111 encoded as123456781234567812345678 3 bytes
A simple Huffman-like code(wo probability basis) 0 1
A 110
B 010
C 0001
E 10
R 011
S 111
T 001
Compression-Model types
Static Read all the data Compute frequencies Re-read the data amp encode
Dynamic Build simple basic model Modify model as more data is processed
copy DJ Foreman 2012 17
Standard Compressed-File Content
Three parts 24-byte header with magic (defines
codec) variable-length annotation block contiguous segment of audio data
Storage methodology network (big-endian) byte order multi-byte audio data may require byte
reversal in order to operate on it by the arithmetic unit of certain processors
copy DJ Foreman 2012 18
Example 1 Header file for AU files
copy DJ Foreman 2012 19
typedef unsigned long u_32 unsigned 32-bit integer
typedef struct
u_32 magic the ldquomagic numberrdquo
u_32 hdr_size byte offset to start of data
u_32 data_size length (optional)
u_32 encoding data encoding enumeration
u_32 sample_rate samples per second
u_32 channels of interleaved channels
Audio_filehdr
Following is from include ltmultimedialibaudiohgt
AUDIO_FILE_MAGIC ((u_32)0x2e736e64) ldquosndrdquo
Audio Encoding EnumerationsAUDIO_FILE_ENCODING_MULAW_8 (1) 8-bit ISDN u-law
AUDIO_FILE_ENCODING_LINEAR_8 (2) 8-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_16 (3) 16-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_32 (5) 32-bit linear PCM
AUDIO_FILE_ENCODING_FLOAT (6) 32-bit IEEE floating point
AUDIO_FILE_ENCODING_DOUBLE (7) 64-bit IEEE floating point
AUDIO_FILE_ENCODING_ADPCM_G721 (23) 4-bit CCITT g721 ADPCM
AUDIO_FILE_ENCODING_ADPCM_G723_3 (25) CCITT g723 3-bit ADPCM
AUDIO_FILE_ENCODING_ALAW_8 (27) 8-bit ISDN A-law
copy DJ Foreman 2012 20
ldquoLinearrdquo values are SIGNED intrsquosFloats are signed zero-centered normalized to ( -10 lt= x lt= 10 )
Example 2 ZIP files (PkWare)Purpose Size in bytes
Signature header 4
Required version 2
GP flags 2
Method 2
Last mod time 2
Last mod date 2
CRC-32 4
Compressed size 4
Uncompressed size 4
Filename length 2
Extra field length 2
File name variable
extra variable
copy DJ Foreman 2012 21
Ref httplivedocsadobecomflex3htmlhelphtmlcontent=ByteArrays_3html
More info on encoding httpwwwpkwarecomdocumentscase
studiesAPPNOTETXT
httpwwwfileinfocomfiletypescompressed
copy DJ Foreman 2012 22
Recording Bit-Rate
Constant (CBR) rate at which a codecs output data should be
consumed is constant Max bit-rate matters not the average Uses all available bandwidth Not good for storage (lossy)
Variable (VBR) Quantity of datatime unit (for output) varies Better quality for audio and video Slower to encode Supported by most portable devices post 2006
copy DJ Foreman 2012 23
Lab assignment ndash 1 ndash part 1
Write a program with 1 parameter If you are on an encoding team
Open a test file (eg ldquoxyztxtrdquo) Compress it using simple Huffman compression using
the frequency table from my FTP site Output compressed file (eg ldquoxyzencrdquo) to SAME folder
as input
If you are on a decoding team Open compressed file ldquoxyzencrdquo Decompress the input file Output file ldquoxyzdecrdquo so it can be compared to ldquoxyztxtrdquo
The parameter is the full input file path amp name (ie file names MUST NOT be hard-coded)
Remember a code can be a single bit
copy DJ Foreman 2012 24
25
Lab assignment 1 ndash part 2
Use ONLY MY TABLE for the frequencies Use ONLY uppercase letters of the table Do NOT use the special character
tables Find someone in class with the
opposite program and TEST your program to see if it properly encodes or decodes a message
Grading tree construction tree following proper binary output (encoders) proper character output (decoders)
copy DJ Foreman 2012
Working with ImagesModeling
Containers
A container is a FILE format NOT a code scheme
Eg AVI amp WAV are container formats Formats for storage Independent of encoding or content
Others Ogg ASF QuickTime RealMedia Matroska
DivX and MP4
copy DJ Foreman 2012 27
28
Movie (metadata)
Track
Media
Media Information
Container file format (MP4QT)
Chunk map
Comp time
Chunk offset
Key frame
Desc
Decode time
Priority map
Size map
Sample Table
Media Handler
Data information
Media header
Video header
Track header
Movie header
copy DJ Foreman 2012
Pixels A pixel is 3 color-dots (squares) in a collection of dots
(squares) making up the picture Each color-dot is for one of the three primary colors
RGB Minimum of 1 byte per color (depending on bit-depth
which varies with camera) thus at least 3 bytes per pixel
A 10MP camera (30 million color-dots) needs 30M bytes This gets compressed by the camera (JPEG format) Final file size = 30 million bytes divided by the amount
of compression (compression ratio) An 81 compression ratio would reduce that 30 million
bytes to 375 megabytes (as done on a 4 MP camera) A RAW file has NO compression
copy DJ Foreman 2012 29
Image creation
Basic mechanisms Photography
Film Digital ndashPhotoshop Gimp
Drawing Line BrushModeling
Do it yourself OpenGL (2x and 3x)
Modelingrendering programs Breeze designer Ayam 3DCanvas Makehuman Alice
copy DJ Foreman 2012 30
Photography (film) Silver halide crystals (AgCl AgBr AgI)
Exposure to light turns them black Developer removes the loose (exposed) halide leaving
pure silver (a negative image) Fixing bath flushes out unexposed AgHalide
Greatest desire of photographers - a medium with High speed (gathers light quickly) Low noise (no undesired artifacts) High resolution (very detailed images)
Visible lines per mm (lpm) at a specified contrast level
Notes Larger ldquoclumpsrdquo are faster but limit resolution Smaller ldquoclumpsrdquo are hard to coat evenly Randomness of coating prevents Moireacute patternshttpenwikipediaorgwikiMoirC3A9_pattern
copy DJ Foreman 2012 31
FYI ndash a little chemistry
Ag+ X- + hv rarr Ag+ + X + e-
1 Radiation frees the Br2 1 Ag ion + 1 electron (Ag+ + e- rarr Ag0) -gt 1 Ag atom3 Neighbors are now ldquosensitizedrdquo 4 Developing (Reducing) converts exposed AgX to
black metallic Ag and free X5 Remaining AgX and free X are washed away (after
mixing with thiosulfate to make it soluble)
This is one silver (Ag) ldquograinrdquo in the ldquolatent imagerdquo X is any halide (Br Cl etc)
copy DJ Foreman 2012 32
crystal
Radiation (light)
Resolution 35 mm film for comparison
A 36 x 24 mm frame of ISO 100-speed film contains (asymp) the equivalent of 20 million pixels 1 There are NO pixels in film
Full-frame digital (36 x 24mm = 32 image ratio) Canon EOS 5D Nikon D3 (21MP) - $5K2
Medium format digital (6 x 45cm) Phase One P40+ (39MP) ($22K)
APS-C sized sensor (asymp24x16mm = 32 image ratio) (note 2009 data below)
Nikon D90 (123MP) - $900 Canon Rebel (15MP) $700 Used in most DSLRrsquos (note image ratio 32)
Other SONY Alpha (note image ratio 43)
1 Langford Michael Basic Photography (7th Ed 2000) Oxford Focal Press ISBN 0 240 51592 7
2 Prices as of 2009 given for comparison purposes onlycopy DJ Foreman 2012 33
Photography (digital) Image capture formats
RAW ndash 3 separate images (RGB) [12 bitspixel (4288 2848)]8=18318336 bytes per
picture JPEG ndash compressed (16x 8x 4x) Some cameras do BOTH (RAW + JPEG) for each image
Pixel notes Sensor size 35mm (or larger) vs APS-C Pixel count ndash any increase rarr
decrease in pixel size or increase in sensor size Pixel size ndash smaller pixels give us
Greater resolution (finer details) - Good More ldquonoiserdquo (incorrect values) - bad Possible ldquoadjacencyrdquo issues - bad Loss of sensitivity - bad
copy DJ Foreman 2012 34
35
Some Comparisons
Digital media vs film Pixels are the same (RGB) Pixel size varies with device (mfgr
dependency) Pixel density varies Zooming digitally does NOT give you more
detail (in spite of what they do on TV shows)
Optical zoom is useful Uses optical magnification of the image on
the target Expansion of light rays not multiplication of
pixelscopy DJ Foreman 2012
Working Definitions - 1
Graphic Model data structure nodes represent random variables Pairs of nodes connected by arcs correspond
to variables that are not independent Graphic modeling
using a Graphic Model to simulate a real-world object (this is not a formal definition)
Rendering generating a 2D image from a 3D graphic model
copy DJ Foreman 2012 36
37
Working Definitions - 2
Geometric primitives ndash Points line segments and polygons Described by vertices
Control points ndash Special points attached to a node on a
Beacutezier curve Alter the shape and angle of adjacent curve
segment Evaluators ndash
Functions that interpolate a set of control points
Produce a new set of control pointscopy DJ Foreman 2012
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
Methodology
Compute a model of the data eg a Huffman tree based on probability Probability determined from input file
Map the data according to the model Read 1 symbol Search the model (tree) for that symbol
If itrsquos a Huffman tree going left=0 right=1 Append each 0 or 1 to the code string When symbol is found you are done Note all symbols will have lt 8 bits (thus
compression)
copy DJ Foreman 2012 14
15
Huffman lab ndash more details Read write are always in bytes
You may need to pad You may have extraneous letters in the output
due to padding You donrsquot need to add a header to specify size
You have 2 weeks to do the lab Code may be in C C++ or Java The readme specifies how to compile the
program Encoded data is sent to a binary file or a simple
text file (but isnrsquot ldquoreadablerdquo as text) Encoded data can only be ldquocheckedrdquo with a
decodercopy DJ Foreman 2012
16
Sample of prefix coding amp compression
copy DJ Foreman 2012
B e a r c a t s (8 bytes of text)010101100110001110001111 encoded as123456781234567812345678 3 bytes
A simple Huffman-like code(wo probability basis) 0 1
A 110
B 010
C 0001
E 10
R 011
S 111
T 001
Compression-Model types
Static Read all the data Compute frequencies Re-read the data amp encode
Dynamic Build simple basic model Modify model as more data is processed
copy DJ Foreman 2012 17
Standard Compressed-File Content
Three parts 24-byte header with magic (defines
codec) variable-length annotation block contiguous segment of audio data
Storage methodology network (big-endian) byte order multi-byte audio data may require byte
reversal in order to operate on it by the arithmetic unit of certain processors
copy DJ Foreman 2012 18
Example 1 Header file for AU files
copy DJ Foreman 2012 19
typedef unsigned long u_32 unsigned 32-bit integer
typedef struct
u_32 magic the ldquomagic numberrdquo
u_32 hdr_size byte offset to start of data
u_32 data_size length (optional)
u_32 encoding data encoding enumeration
u_32 sample_rate samples per second
u_32 channels of interleaved channels
Audio_filehdr
Following is from include ltmultimedialibaudiohgt
AUDIO_FILE_MAGIC ((u_32)0x2e736e64) ldquosndrdquo
Audio Encoding EnumerationsAUDIO_FILE_ENCODING_MULAW_8 (1) 8-bit ISDN u-law
AUDIO_FILE_ENCODING_LINEAR_8 (2) 8-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_16 (3) 16-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_32 (5) 32-bit linear PCM
AUDIO_FILE_ENCODING_FLOAT (6) 32-bit IEEE floating point
AUDIO_FILE_ENCODING_DOUBLE (7) 64-bit IEEE floating point
AUDIO_FILE_ENCODING_ADPCM_G721 (23) 4-bit CCITT g721 ADPCM
AUDIO_FILE_ENCODING_ADPCM_G723_3 (25) CCITT g723 3-bit ADPCM
AUDIO_FILE_ENCODING_ALAW_8 (27) 8-bit ISDN A-law
copy DJ Foreman 2012 20
ldquoLinearrdquo values are SIGNED intrsquosFloats are signed zero-centered normalized to ( -10 lt= x lt= 10 )
Example 2 ZIP files (PkWare)Purpose Size in bytes
Signature header 4
Required version 2
GP flags 2
Method 2
Last mod time 2
Last mod date 2
CRC-32 4
Compressed size 4
Uncompressed size 4
Filename length 2
Extra field length 2
File name variable
extra variable
copy DJ Foreman 2012 21
Ref httplivedocsadobecomflex3htmlhelphtmlcontent=ByteArrays_3html
More info on encoding httpwwwpkwarecomdocumentscase
studiesAPPNOTETXT
httpwwwfileinfocomfiletypescompressed
copy DJ Foreman 2012 22
Recording Bit-Rate
Constant (CBR) rate at which a codecs output data should be
consumed is constant Max bit-rate matters not the average Uses all available bandwidth Not good for storage (lossy)
Variable (VBR) Quantity of datatime unit (for output) varies Better quality for audio and video Slower to encode Supported by most portable devices post 2006
copy DJ Foreman 2012 23
Lab assignment ndash 1 ndash part 1
Write a program with 1 parameter If you are on an encoding team
Open a test file (eg ldquoxyztxtrdquo) Compress it using simple Huffman compression using
the frequency table from my FTP site Output compressed file (eg ldquoxyzencrdquo) to SAME folder
as input
If you are on a decoding team Open compressed file ldquoxyzencrdquo Decompress the input file Output file ldquoxyzdecrdquo so it can be compared to ldquoxyztxtrdquo
The parameter is the full input file path amp name (ie file names MUST NOT be hard-coded)
Remember a code can be a single bit
copy DJ Foreman 2012 24
25
Lab assignment 1 ndash part 2
Use ONLY MY TABLE for the frequencies Use ONLY uppercase letters of the table Do NOT use the special character
tables Find someone in class with the
opposite program and TEST your program to see if it properly encodes or decodes a message
Grading tree construction tree following proper binary output (encoders) proper character output (decoders)
copy DJ Foreman 2012
Working with ImagesModeling
Containers
A container is a FILE format NOT a code scheme
Eg AVI amp WAV are container formats Formats for storage Independent of encoding or content
Others Ogg ASF QuickTime RealMedia Matroska
DivX and MP4
copy DJ Foreman 2012 27
28
Movie (metadata)
Track
Media
Media Information
Container file format (MP4QT)
Chunk map
Comp time
Chunk offset
Key frame
Desc
Decode time
Priority map
Size map
Sample Table
Media Handler
Data information
Media header
Video header
Track header
Movie header
copy DJ Foreman 2012
Pixels A pixel is 3 color-dots (squares) in a collection of dots
(squares) making up the picture Each color-dot is for one of the three primary colors
RGB Minimum of 1 byte per color (depending on bit-depth
which varies with camera) thus at least 3 bytes per pixel
A 10MP camera (30 million color-dots) needs 30M bytes This gets compressed by the camera (JPEG format) Final file size = 30 million bytes divided by the amount
of compression (compression ratio) An 81 compression ratio would reduce that 30 million
bytes to 375 megabytes (as done on a 4 MP camera) A RAW file has NO compression
copy DJ Foreman 2012 29
Image creation
Basic mechanisms Photography
Film Digital ndashPhotoshop Gimp
Drawing Line BrushModeling
Do it yourself OpenGL (2x and 3x)
Modelingrendering programs Breeze designer Ayam 3DCanvas Makehuman Alice
copy DJ Foreman 2012 30
Photography (film) Silver halide crystals (AgCl AgBr AgI)
Exposure to light turns them black Developer removes the loose (exposed) halide leaving
pure silver (a negative image) Fixing bath flushes out unexposed AgHalide
Greatest desire of photographers - a medium with High speed (gathers light quickly) Low noise (no undesired artifacts) High resolution (very detailed images)
Visible lines per mm (lpm) at a specified contrast level
Notes Larger ldquoclumpsrdquo are faster but limit resolution Smaller ldquoclumpsrdquo are hard to coat evenly Randomness of coating prevents Moireacute patternshttpenwikipediaorgwikiMoirC3A9_pattern
copy DJ Foreman 2012 31
FYI ndash a little chemistry
Ag+ X- + hv rarr Ag+ + X + e-
1 Radiation frees the Br2 1 Ag ion + 1 electron (Ag+ + e- rarr Ag0) -gt 1 Ag atom3 Neighbors are now ldquosensitizedrdquo 4 Developing (Reducing) converts exposed AgX to
black metallic Ag and free X5 Remaining AgX and free X are washed away (after
mixing with thiosulfate to make it soluble)
This is one silver (Ag) ldquograinrdquo in the ldquolatent imagerdquo X is any halide (Br Cl etc)
copy DJ Foreman 2012 32
crystal
Radiation (light)
Resolution 35 mm film for comparison
A 36 x 24 mm frame of ISO 100-speed film contains (asymp) the equivalent of 20 million pixels 1 There are NO pixels in film
Full-frame digital (36 x 24mm = 32 image ratio) Canon EOS 5D Nikon D3 (21MP) - $5K2
Medium format digital (6 x 45cm) Phase One P40+ (39MP) ($22K)
APS-C sized sensor (asymp24x16mm = 32 image ratio) (note 2009 data below)
Nikon D90 (123MP) - $900 Canon Rebel (15MP) $700 Used in most DSLRrsquos (note image ratio 32)
Other SONY Alpha (note image ratio 43)
1 Langford Michael Basic Photography (7th Ed 2000) Oxford Focal Press ISBN 0 240 51592 7
2 Prices as of 2009 given for comparison purposes onlycopy DJ Foreman 2012 33
Photography (digital) Image capture formats
RAW ndash 3 separate images (RGB) [12 bitspixel (4288 2848)]8=18318336 bytes per
picture JPEG ndash compressed (16x 8x 4x) Some cameras do BOTH (RAW + JPEG) for each image
Pixel notes Sensor size 35mm (or larger) vs APS-C Pixel count ndash any increase rarr
decrease in pixel size or increase in sensor size Pixel size ndash smaller pixels give us
Greater resolution (finer details) - Good More ldquonoiserdquo (incorrect values) - bad Possible ldquoadjacencyrdquo issues - bad Loss of sensitivity - bad
copy DJ Foreman 2012 34
35
Some Comparisons
Digital media vs film Pixels are the same (RGB) Pixel size varies with device (mfgr
dependency) Pixel density varies Zooming digitally does NOT give you more
detail (in spite of what they do on TV shows)
Optical zoom is useful Uses optical magnification of the image on
the target Expansion of light rays not multiplication of
pixelscopy DJ Foreman 2012
Working Definitions - 1
Graphic Model data structure nodes represent random variables Pairs of nodes connected by arcs correspond
to variables that are not independent Graphic modeling
using a Graphic Model to simulate a real-world object (this is not a formal definition)
Rendering generating a 2D image from a 3D graphic model
copy DJ Foreman 2012 36
37
Working Definitions - 2
Geometric primitives ndash Points line segments and polygons Described by vertices
Control points ndash Special points attached to a node on a
Beacutezier curve Alter the shape and angle of adjacent curve
segment Evaluators ndash
Functions that interpolate a set of control points
Produce a new set of control pointscopy DJ Foreman 2012
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
15
Huffman lab ndash more details Read write are always in bytes
You may need to pad You may have extraneous letters in the output
due to padding You donrsquot need to add a header to specify size
You have 2 weeks to do the lab Code may be in C C++ or Java The readme specifies how to compile the
program Encoded data is sent to a binary file or a simple
text file (but isnrsquot ldquoreadablerdquo as text) Encoded data can only be ldquocheckedrdquo with a
decodercopy DJ Foreman 2012
16
Sample of prefix coding amp compression
copy DJ Foreman 2012
B e a r c a t s (8 bytes of text)010101100110001110001111 encoded as123456781234567812345678 3 bytes
A simple Huffman-like code(wo probability basis) 0 1
A 110
B 010
C 0001
E 10
R 011
S 111
T 001
Compression-Model types
Static Read all the data Compute frequencies Re-read the data amp encode
Dynamic Build simple basic model Modify model as more data is processed
copy DJ Foreman 2012 17
Standard Compressed-File Content
Three parts 24-byte header with magic (defines
codec) variable-length annotation block contiguous segment of audio data
Storage methodology network (big-endian) byte order multi-byte audio data may require byte
reversal in order to operate on it by the arithmetic unit of certain processors
copy DJ Foreman 2012 18
Example 1 Header file for AU files
copy DJ Foreman 2012 19
typedef unsigned long u_32 unsigned 32-bit integer
typedef struct
u_32 magic the ldquomagic numberrdquo
u_32 hdr_size byte offset to start of data
u_32 data_size length (optional)
u_32 encoding data encoding enumeration
u_32 sample_rate samples per second
u_32 channels of interleaved channels
Audio_filehdr
Following is from include ltmultimedialibaudiohgt
AUDIO_FILE_MAGIC ((u_32)0x2e736e64) ldquosndrdquo
Audio Encoding EnumerationsAUDIO_FILE_ENCODING_MULAW_8 (1) 8-bit ISDN u-law
AUDIO_FILE_ENCODING_LINEAR_8 (2) 8-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_16 (3) 16-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_32 (5) 32-bit linear PCM
AUDIO_FILE_ENCODING_FLOAT (6) 32-bit IEEE floating point
AUDIO_FILE_ENCODING_DOUBLE (7) 64-bit IEEE floating point
AUDIO_FILE_ENCODING_ADPCM_G721 (23) 4-bit CCITT g721 ADPCM
AUDIO_FILE_ENCODING_ADPCM_G723_3 (25) CCITT g723 3-bit ADPCM
AUDIO_FILE_ENCODING_ALAW_8 (27) 8-bit ISDN A-law
copy DJ Foreman 2012 20
ldquoLinearrdquo values are SIGNED intrsquosFloats are signed zero-centered normalized to ( -10 lt= x lt= 10 )
Example 2 ZIP files (PkWare)Purpose Size in bytes
Signature header 4
Required version 2
GP flags 2
Method 2
Last mod time 2
Last mod date 2
CRC-32 4
Compressed size 4
Uncompressed size 4
Filename length 2
Extra field length 2
File name variable
extra variable
copy DJ Foreman 2012 21
Ref httplivedocsadobecomflex3htmlhelphtmlcontent=ByteArrays_3html
More info on encoding httpwwwpkwarecomdocumentscase
studiesAPPNOTETXT
httpwwwfileinfocomfiletypescompressed
copy DJ Foreman 2012 22
Recording Bit-Rate
Constant (CBR) rate at which a codecs output data should be
consumed is constant Max bit-rate matters not the average Uses all available bandwidth Not good for storage (lossy)
Variable (VBR) Quantity of datatime unit (for output) varies Better quality for audio and video Slower to encode Supported by most portable devices post 2006
copy DJ Foreman 2012 23
Lab assignment ndash 1 ndash part 1
Write a program with 1 parameter If you are on an encoding team
Open a test file (eg ldquoxyztxtrdquo) Compress it using simple Huffman compression using
the frequency table from my FTP site Output compressed file (eg ldquoxyzencrdquo) to SAME folder
as input
If you are on a decoding team Open compressed file ldquoxyzencrdquo Decompress the input file Output file ldquoxyzdecrdquo so it can be compared to ldquoxyztxtrdquo
The parameter is the full input file path amp name (ie file names MUST NOT be hard-coded)
Remember a code can be a single bit
copy DJ Foreman 2012 24
25
Lab assignment 1 ndash part 2
Use ONLY MY TABLE for the frequencies Use ONLY uppercase letters of the table Do NOT use the special character
tables Find someone in class with the
opposite program and TEST your program to see if it properly encodes or decodes a message
Grading tree construction tree following proper binary output (encoders) proper character output (decoders)
copy DJ Foreman 2012
Working with ImagesModeling
Containers
A container is a FILE format NOT a code scheme
Eg AVI amp WAV are container formats Formats for storage Independent of encoding or content
Others Ogg ASF QuickTime RealMedia Matroska
DivX and MP4
copy DJ Foreman 2012 27
28
Movie (metadata)
Track
Media
Media Information
Container file format (MP4QT)
Chunk map
Comp time
Chunk offset
Key frame
Desc
Decode time
Priority map
Size map
Sample Table
Media Handler
Data information
Media header
Video header
Track header
Movie header
copy DJ Foreman 2012
Pixels A pixel is 3 color-dots (squares) in a collection of dots
(squares) making up the picture Each color-dot is for one of the three primary colors
RGB Minimum of 1 byte per color (depending on bit-depth
which varies with camera) thus at least 3 bytes per pixel
A 10MP camera (30 million color-dots) needs 30M bytes This gets compressed by the camera (JPEG format) Final file size = 30 million bytes divided by the amount
of compression (compression ratio) An 81 compression ratio would reduce that 30 million
bytes to 375 megabytes (as done on a 4 MP camera) A RAW file has NO compression
copy DJ Foreman 2012 29
Image creation
Basic mechanisms Photography
Film Digital ndashPhotoshop Gimp
Drawing Line BrushModeling
Do it yourself OpenGL (2x and 3x)
Modelingrendering programs Breeze designer Ayam 3DCanvas Makehuman Alice
copy DJ Foreman 2012 30
Photography (film) Silver halide crystals (AgCl AgBr AgI)
Exposure to light turns them black Developer removes the loose (exposed) halide leaving
pure silver (a negative image) Fixing bath flushes out unexposed AgHalide
Greatest desire of photographers - a medium with High speed (gathers light quickly) Low noise (no undesired artifacts) High resolution (very detailed images)
Visible lines per mm (lpm) at a specified contrast level
Notes Larger ldquoclumpsrdquo are faster but limit resolution Smaller ldquoclumpsrdquo are hard to coat evenly Randomness of coating prevents Moireacute patternshttpenwikipediaorgwikiMoirC3A9_pattern
copy DJ Foreman 2012 31
FYI ndash a little chemistry
Ag+ X- + hv rarr Ag+ + X + e-
1 Radiation frees the Br2 1 Ag ion + 1 electron (Ag+ + e- rarr Ag0) -gt 1 Ag atom3 Neighbors are now ldquosensitizedrdquo 4 Developing (Reducing) converts exposed AgX to
black metallic Ag and free X5 Remaining AgX and free X are washed away (after
mixing with thiosulfate to make it soluble)
This is one silver (Ag) ldquograinrdquo in the ldquolatent imagerdquo X is any halide (Br Cl etc)
copy DJ Foreman 2012 32
crystal
Radiation (light)
Resolution 35 mm film for comparison
A 36 x 24 mm frame of ISO 100-speed film contains (asymp) the equivalent of 20 million pixels 1 There are NO pixels in film
Full-frame digital (36 x 24mm = 32 image ratio) Canon EOS 5D Nikon D3 (21MP) - $5K2
Medium format digital (6 x 45cm) Phase One P40+ (39MP) ($22K)
APS-C sized sensor (asymp24x16mm = 32 image ratio) (note 2009 data below)
Nikon D90 (123MP) - $900 Canon Rebel (15MP) $700 Used in most DSLRrsquos (note image ratio 32)
Other SONY Alpha (note image ratio 43)
1 Langford Michael Basic Photography (7th Ed 2000) Oxford Focal Press ISBN 0 240 51592 7
2 Prices as of 2009 given for comparison purposes onlycopy DJ Foreman 2012 33
Photography (digital) Image capture formats
RAW ndash 3 separate images (RGB) [12 bitspixel (4288 2848)]8=18318336 bytes per
picture JPEG ndash compressed (16x 8x 4x) Some cameras do BOTH (RAW + JPEG) for each image
Pixel notes Sensor size 35mm (or larger) vs APS-C Pixel count ndash any increase rarr
decrease in pixel size or increase in sensor size Pixel size ndash smaller pixels give us
Greater resolution (finer details) - Good More ldquonoiserdquo (incorrect values) - bad Possible ldquoadjacencyrdquo issues - bad Loss of sensitivity - bad
copy DJ Foreman 2012 34
35
Some Comparisons
Digital media vs film Pixels are the same (RGB) Pixel size varies with device (mfgr
dependency) Pixel density varies Zooming digitally does NOT give you more
detail (in spite of what they do on TV shows)
Optical zoom is useful Uses optical magnification of the image on
the target Expansion of light rays not multiplication of
pixelscopy DJ Foreman 2012
Working Definitions - 1
Graphic Model data structure nodes represent random variables Pairs of nodes connected by arcs correspond
to variables that are not independent Graphic modeling
using a Graphic Model to simulate a real-world object (this is not a formal definition)
Rendering generating a 2D image from a 3D graphic model
copy DJ Foreman 2012 36
37
Working Definitions - 2
Geometric primitives ndash Points line segments and polygons Described by vertices
Control points ndash Special points attached to a node on a
Beacutezier curve Alter the shape and angle of adjacent curve
segment Evaluators ndash
Functions that interpolate a set of control points
Produce a new set of control pointscopy DJ Foreman 2012
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
16
Sample of prefix coding amp compression
copy DJ Foreman 2012
B e a r c a t s (8 bytes of text)010101100110001110001111 encoded as123456781234567812345678 3 bytes
A simple Huffman-like code(wo probability basis) 0 1
A 110
B 010
C 0001
E 10
R 011
S 111
T 001
Compression-Model types
Static Read all the data Compute frequencies Re-read the data amp encode
Dynamic Build simple basic model Modify model as more data is processed
copy DJ Foreman 2012 17
Standard Compressed-File Content
Three parts 24-byte header with magic (defines
codec) variable-length annotation block contiguous segment of audio data
Storage methodology network (big-endian) byte order multi-byte audio data may require byte
reversal in order to operate on it by the arithmetic unit of certain processors
copy DJ Foreman 2012 18
Example 1 Header file for AU files
copy DJ Foreman 2012 19
typedef unsigned long u_32 unsigned 32-bit integer
typedef struct
u_32 magic the ldquomagic numberrdquo
u_32 hdr_size byte offset to start of data
u_32 data_size length (optional)
u_32 encoding data encoding enumeration
u_32 sample_rate samples per second
u_32 channels of interleaved channels
Audio_filehdr
Following is from include ltmultimedialibaudiohgt
AUDIO_FILE_MAGIC ((u_32)0x2e736e64) ldquosndrdquo
Audio Encoding EnumerationsAUDIO_FILE_ENCODING_MULAW_8 (1) 8-bit ISDN u-law
AUDIO_FILE_ENCODING_LINEAR_8 (2) 8-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_16 (3) 16-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_32 (5) 32-bit linear PCM
AUDIO_FILE_ENCODING_FLOAT (6) 32-bit IEEE floating point
AUDIO_FILE_ENCODING_DOUBLE (7) 64-bit IEEE floating point
AUDIO_FILE_ENCODING_ADPCM_G721 (23) 4-bit CCITT g721 ADPCM
AUDIO_FILE_ENCODING_ADPCM_G723_3 (25) CCITT g723 3-bit ADPCM
AUDIO_FILE_ENCODING_ALAW_8 (27) 8-bit ISDN A-law
copy DJ Foreman 2012 20
ldquoLinearrdquo values are SIGNED intrsquosFloats are signed zero-centered normalized to ( -10 lt= x lt= 10 )
Example 2 ZIP files (PkWare)Purpose Size in bytes
Signature header 4
Required version 2
GP flags 2
Method 2
Last mod time 2
Last mod date 2
CRC-32 4
Compressed size 4
Uncompressed size 4
Filename length 2
Extra field length 2
File name variable
extra variable
copy DJ Foreman 2012 21
Ref httplivedocsadobecomflex3htmlhelphtmlcontent=ByteArrays_3html
More info on encoding httpwwwpkwarecomdocumentscase
studiesAPPNOTETXT
httpwwwfileinfocomfiletypescompressed
copy DJ Foreman 2012 22
Recording Bit-Rate
Constant (CBR) rate at which a codecs output data should be
consumed is constant Max bit-rate matters not the average Uses all available bandwidth Not good for storage (lossy)
Variable (VBR) Quantity of datatime unit (for output) varies Better quality for audio and video Slower to encode Supported by most portable devices post 2006
copy DJ Foreman 2012 23
Lab assignment ndash 1 ndash part 1
Write a program with 1 parameter If you are on an encoding team
Open a test file (eg ldquoxyztxtrdquo) Compress it using simple Huffman compression using
the frequency table from my FTP site Output compressed file (eg ldquoxyzencrdquo) to SAME folder
as input
If you are on a decoding team Open compressed file ldquoxyzencrdquo Decompress the input file Output file ldquoxyzdecrdquo so it can be compared to ldquoxyztxtrdquo
The parameter is the full input file path amp name (ie file names MUST NOT be hard-coded)
Remember a code can be a single bit
copy DJ Foreman 2012 24
25
Lab assignment 1 ndash part 2
Use ONLY MY TABLE for the frequencies Use ONLY uppercase letters of the table Do NOT use the special character
tables Find someone in class with the
opposite program and TEST your program to see if it properly encodes or decodes a message
Grading tree construction tree following proper binary output (encoders) proper character output (decoders)
copy DJ Foreman 2012
Working with ImagesModeling
Containers
A container is a FILE format NOT a code scheme
Eg AVI amp WAV are container formats Formats for storage Independent of encoding or content
Others Ogg ASF QuickTime RealMedia Matroska
DivX and MP4
copy DJ Foreman 2012 27
28
Movie (metadata)
Track
Media
Media Information
Container file format (MP4QT)
Chunk map
Comp time
Chunk offset
Key frame
Desc
Decode time
Priority map
Size map
Sample Table
Media Handler
Data information
Media header
Video header
Track header
Movie header
copy DJ Foreman 2012
Pixels A pixel is 3 color-dots (squares) in a collection of dots
(squares) making up the picture Each color-dot is for one of the three primary colors
RGB Minimum of 1 byte per color (depending on bit-depth
which varies with camera) thus at least 3 bytes per pixel
A 10MP camera (30 million color-dots) needs 30M bytes This gets compressed by the camera (JPEG format) Final file size = 30 million bytes divided by the amount
of compression (compression ratio) An 81 compression ratio would reduce that 30 million
bytes to 375 megabytes (as done on a 4 MP camera) A RAW file has NO compression
copy DJ Foreman 2012 29
Image creation
Basic mechanisms Photography
Film Digital ndashPhotoshop Gimp
Drawing Line BrushModeling
Do it yourself OpenGL (2x and 3x)
Modelingrendering programs Breeze designer Ayam 3DCanvas Makehuman Alice
copy DJ Foreman 2012 30
Photography (film) Silver halide crystals (AgCl AgBr AgI)
Exposure to light turns them black Developer removes the loose (exposed) halide leaving
pure silver (a negative image) Fixing bath flushes out unexposed AgHalide
Greatest desire of photographers - a medium with High speed (gathers light quickly) Low noise (no undesired artifacts) High resolution (very detailed images)
Visible lines per mm (lpm) at a specified contrast level
Notes Larger ldquoclumpsrdquo are faster but limit resolution Smaller ldquoclumpsrdquo are hard to coat evenly Randomness of coating prevents Moireacute patternshttpenwikipediaorgwikiMoirC3A9_pattern
copy DJ Foreman 2012 31
FYI ndash a little chemistry
Ag+ X- + hv rarr Ag+ + X + e-
1 Radiation frees the Br2 1 Ag ion + 1 electron (Ag+ + e- rarr Ag0) -gt 1 Ag atom3 Neighbors are now ldquosensitizedrdquo 4 Developing (Reducing) converts exposed AgX to
black metallic Ag and free X5 Remaining AgX and free X are washed away (after
mixing with thiosulfate to make it soluble)
This is one silver (Ag) ldquograinrdquo in the ldquolatent imagerdquo X is any halide (Br Cl etc)
copy DJ Foreman 2012 32
crystal
Radiation (light)
Resolution 35 mm film for comparison
A 36 x 24 mm frame of ISO 100-speed film contains (asymp) the equivalent of 20 million pixels 1 There are NO pixels in film
Full-frame digital (36 x 24mm = 32 image ratio) Canon EOS 5D Nikon D3 (21MP) - $5K2
Medium format digital (6 x 45cm) Phase One P40+ (39MP) ($22K)
APS-C sized sensor (asymp24x16mm = 32 image ratio) (note 2009 data below)
Nikon D90 (123MP) - $900 Canon Rebel (15MP) $700 Used in most DSLRrsquos (note image ratio 32)
Other SONY Alpha (note image ratio 43)
1 Langford Michael Basic Photography (7th Ed 2000) Oxford Focal Press ISBN 0 240 51592 7
2 Prices as of 2009 given for comparison purposes onlycopy DJ Foreman 2012 33
Photography (digital) Image capture formats
RAW ndash 3 separate images (RGB) [12 bitspixel (4288 2848)]8=18318336 bytes per
picture JPEG ndash compressed (16x 8x 4x) Some cameras do BOTH (RAW + JPEG) for each image
Pixel notes Sensor size 35mm (or larger) vs APS-C Pixel count ndash any increase rarr
decrease in pixel size or increase in sensor size Pixel size ndash smaller pixels give us
Greater resolution (finer details) - Good More ldquonoiserdquo (incorrect values) - bad Possible ldquoadjacencyrdquo issues - bad Loss of sensitivity - bad
copy DJ Foreman 2012 34
35
Some Comparisons
Digital media vs film Pixels are the same (RGB) Pixel size varies with device (mfgr
dependency) Pixel density varies Zooming digitally does NOT give you more
detail (in spite of what they do on TV shows)
Optical zoom is useful Uses optical magnification of the image on
the target Expansion of light rays not multiplication of
pixelscopy DJ Foreman 2012
Working Definitions - 1
Graphic Model data structure nodes represent random variables Pairs of nodes connected by arcs correspond
to variables that are not independent Graphic modeling
using a Graphic Model to simulate a real-world object (this is not a formal definition)
Rendering generating a 2D image from a 3D graphic model
copy DJ Foreman 2012 36
37
Working Definitions - 2
Geometric primitives ndash Points line segments and polygons Described by vertices
Control points ndash Special points attached to a node on a
Beacutezier curve Alter the shape and angle of adjacent curve
segment Evaluators ndash
Functions that interpolate a set of control points
Produce a new set of control pointscopy DJ Foreman 2012
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
Compression-Model types
Static Read all the data Compute frequencies Re-read the data amp encode
Dynamic Build simple basic model Modify model as more data is processed
copy DJ Foreman 2012 17
Standard Compressed-File Content
Three parts 24-byte header with magic (defines
codec) variable-length annotation block contiguous segment of audio data
Storage methodology network (big-endian) byte order multi-byte audio data may require byte
reversal in order to operate on it by the arithmetic unit of certain processors
copy DJ Foreman 2012 18
Example 1 Header file for AU files
copy DJ Foreman 2012 19
typedef unsigned long u_32 unsigned 32-bit integer
typedef struct
u_32 magic the ldquomagic numberrdquo
u_32 hdr_size byte offset to start of data
u_32 data_size length (optional)
u_32 encoding data encoding enumeration
u_32 sample_rate samples per second
u_32 channels of interleaved channels
Audio_filehdr
Following is from include ltmultimedialibaudiohgt
AUDIO_FILE_MAGIC ((u_32)0x2e736e64) ldquosndrdquo
Audio Encoding EnumerationsAUDIO_FILE_ENCODING_MULAW_8 (1) 8-bit ISDN u-law
AUDIO_FILE_ENCODING_LINEAR_8 (2) 8-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_16 (3) 16-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_32 (5) 32-bit linear PCM
AUDIO_FILE_ENCODING_FLOAT (6) 32-bit IEEE floating point
AUDIO_FILE_ENCODING_DOUBLE (7) 64-bit IEEE floating point
AUDIO_FILE_ENCODING_ADPCM_G721 (23) 4-bit CCITT g721 ADPCM
AUDIO_FILE_ENCODING_ADPCM_G723_3 (25) CCITT g723 3-bit ADPCM
AUDIO_FILE_ENCODING_ALAW_8 (27) 8-bit ISDN A-law
copy DJ Foreman 2012 20
ldquoLinearrdquo values are SIGNED intrsquosFloats are signed zero-centered normalized to ( -10 lt= x lt= 10 )
Example 2 ZIP files (PkWare)Purpose Size in bytes
Signature header 4
Required version 2
GP flags 2
Method 2
Last mod time 2
Last mod date 2
CRC-32 4
Compressed size 4
Uncompressed size 4
Filename length 2
Extra field length 2
File name variable
extra variable
copy DJ Foreman 2012 21
Ref httplivedocsadobecomflex3htmlhelphtmlcontent=ByteArrays_3html
More info on encoding httpwwwpkwarecomdocumentscase
studiesAPPNOTETXT
httpwwwfileinfocomfiletypescompressed
copy DJ Foreman 2012 22
Recording Bit-Rate
Constant (CBR) rate at which a codecs output data should be
consumed is constant Max bit-rate matters not the average Uses all available bandwidth Not good for storage (lossy)
Variable (VBR) Quantity of datatime unit (for output) varies Better quality for audio and video Slower to encode Supported by most portable devices post 2006
copy DJ Foreman 2012 23
Lab assignment ndash 1 ndash part 1
Write a program with 1 parameter If you are on an encoding team
Open a test file (eg ldquoxyztxtrdquo) Compress it using simple Huffman compression using
the frequency table from my FTP site Output compressed file (eg ldquoxyzencrdquo) to SAME folder
as input
If you are on a decoding team Open compressed file ldquoxyzencrdquo Decompress the input file Output file ldquoxyzdecrdquo so it can be compared to ldquoxyztxtrdquo
The parameter is the full input file path amp name (ie file names MUST NOT be hard-coded)
Remember a code can be a single bit
copy DJ Foreman 2012 24
25
Lab assignment 1 ndash part 2
Use ONLY MY TABLE for the frequencies Use ONLY uppercase letters of the table Do NOT use the special character
tables Find someone in class with the
opposite program and TEST your program to see if it properly encodes or decodes a message
Grading tree construction tree following proper binary output (encoders) proper character output (decoders)
copy DJ Foreman 2012
Working with ImagesModeling
Containers
A container is a FILE format NOT a code scheme
Eg AVI amp WAV are container formats Formats for storage Independent of encoding or content
Others Ogg ASF QuickTime RealMedia Matroska
DivX and MP4
copy DJ Foreman 2012 27
28
Movie (metadata)
Track
Media
Media Information
Container file format (MP4QT)
Chunk map
Comp time
Chunk offset
Key frame
Desc
Decode time
Priority map
Size map
Sample Table
Media Handler
Data information
Media header
Video header
Track header
Movie header
copy DJ Foreman 2012
Pixels A pixel is 3 color-dots (squares) in a collection of dots
(squares) making up the picture Each color-dot is for one of the three primary colors
RGB Minimum of 1 byte per color (depending on bit-depth
which varies with camera) thus at least 3 bytes per pixel
A 10MP camera (30 million color-dots) needs 30M bytes This gets compressed by the camera (JPEG format) Final file size = 30 million bytes divided by the amount
of compression (compression ratio) An 81 compression ratio would reduce that 30 million
bytes to 375 megabytes (as done on a 4 MP camera) A RAW file has NO compression
copy DJ Foreman 2012 29
Image creation
Basic mechanisms Photography
Film Digital ndashPhotoshop Gimp
Drawing Line BrushModeling
Do it yourself OpenGL (2x and 3x)
Modelingrendering programs Breeze designer Ayam 3DCanvas Makehuman Alice
copy DJ Foreman 2012 30
Photography (film) Silver halide crystals (AgCl AgBr AgI)
Exposure to light turns them black Developer removes the loose (exposed) halide leaving
pure silver (a negative image) Fixing bath flushes out unexposed AgHalide
Greatest desire of photographers - a medium with High speed (gathers light quickly) Low noise (no undesired artifacts) High resolution (very detailed images)
Visible lines per mm (lpm) at a specified contrast level
Notes Larger ldquoclumpsrdquo are faster but limit resolution Smaller ldquoclumpsrdquo are hard to coat evenly Randomness of coating prevents Moireacute patternshttpenwikipediaorgwikiMoirC3A9_pattern
copy DJ Foreman 2012 31
FYI ndash a little chemistry
Ag+ X- + hv rarr Ag+ + X + e-
1 Radiation frees the Br2 1 Ag ion + 1 electron (Ag+ + e- rarr Ag0) -gt 1 Ag atom3 Neighbors are now ldquosensitizedrdquo 4 Developing (Reducing) converts exposed AgX to
black metallic Ag and free X5 Remaining AgX and free X are washed away (after
mixing with thiosulfate to make it soluble)
This is one silver (Ag) ldquograinrdquo in the ldquolatent imagerdquo X is any halide (Br Cl etc)
copy DJ Foreman 2012 32
crystal
Radiation (light)
Resolution 35 mm film for comparison
A 36 x 24 mm frame of ISO 100-speed film contains (asymp) the equivalent of 20 million pixels 1 There are NO pixels in film
Full-frame digital (36 x 24mm = 32 image ratio) Canon EOS 5D Nikon D3 (21MP) - $5K2
Medium format digital (6 x 45cm) Phase One P40+ (39MP) ($22K)
APS-C sized sensor (asymp24x16mm = 32 image ratio) (note 2009 data below)
Nikon D90 (123MP) - $900 Canon Rebel (15MP) $700 Used in most DSLRrsquos (note image ratio 32)
Other SONY Alpha (note image ratio 43)
1 Langford Michael Basic Photography (7th Ed 2000) Oxford Focal Press ISBN 0 240 51592 7
2 Prices as of 2009 given for comparison purposes onlycopy DJ Foreman 2012 33
Photography (digital) Image capture formats
RAW ndash 3 separate images (RGB) [12 bitspixel (4288 2848)]8=18318336 bytes per
picture JPEG ndash compressed (16x 8x 4x) Some cameras do BOTH (RAW + JPEG) for each image
Pixel notes Sensor size 35mm (or larger) vs APS-C Pixel count ndash any increase rarr
decrease in pixel size or increase in sensor size Pixel size ndash smaller pixels give us
Greater resolution (finer details) - Good More ldquonoiserdquo (incorrect values) - bad Possible ldquoadjacencyrdquo issues - bad Loss of sensitivity - bad
copy DJ Foreman 2012 34
35
Some Comparisons
Digital media vs film Pixels are the same (RGB) Pixel size varies with device (mfgr
dependency) Pixel density varies Zooming digitally does NOT give you more
detail (in spite of what they do on TV shows)
Optical zoom is useful Uses optical magnification of the image on
the target Expansion of light rays not multiplication of
pixelscopy DJ Foreman 2012
Working Definitions - 1
Graphic Model data structure nodes represent random variables Pairs of nodes connected by arcs correspond
to variables that are not independent Graphic modeling
using a Graphic Model to simulate a real-world object (this is not a formal definition)
Rendering generating a 2D image from a 3D graphic model
copy DJ Foreman 2012 36
37
Working Definitions - 2
Geometric primitives ndash Points line segments and polygons Described by vertices
Control points ndash Special points attached to a node on a
Beacutezier curve Alter the shape and angle of adjacent curve
segment Evaluators ndash
Functions that interpolate a set of control points
Produce a new set of control pointscopy DJ Foreman 2012
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
Standard Compressed-File Content
Three parts 24-byte header with magic (defines
codec) variable-length annotation block contiguous segment of audio data
Storage methodology network (big-endian) byte order multi-byte audio data may require byte
reversal in order to operate on it by the arithmetic unit of certain processors
copy DJ Foreman 2012 18
Example 1 Header file for AU files
copy DJ Foreman 2012 19
typedef unsigned long u_32 unsigned 32-bit integer
typedef struct
u_32 magic the ldquomagic numberrdquo
u_32 hdr_size byte offset to start of data
u_32 data_size length (optional)
u_32 encoding data encoding enumeration
u_32 sample_rate samples per second
u_32 channels of interleaved channels
Audio_filehdr
Following is from include ltmultimedialibaudiohgt
AUDIO_FILE_MAGIC ((u_32)0x2e736e64) ldquosndrdquo
Audio Encoding EnumerationsAUDIO_FILE_ENCODING_MULAW_8 (1) 8-bit ISDN u-law
AUDIO_FILE_ENCODING_LINEAR_8 (2) 8-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_16 (3) 16-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_32 (5) 32-bit linear PCM
AUDIO_FILE_ENCODING_FLOAT (6) 32-bit IEEE floating point
AUDIO_FILE_ENCODING_DOUBLE (7) 64-bit IEEE floating point
AUDIO_FILE_ENCODING_ADPCM_G721 (23) 4-bit CCITT g721 ADPCM
AUDIO_FILE_ENCODING_ADPCM_G723_3 (25) CCITT g723 3-bit ADPCM
AUDIO_FILE_ENCODING_ALAW_8 (27) 8-bit ISDN A-law
copy DJ Foreman 2012 20
ldquoLinearrdquo values are SIGNED intrsquosFloats are signed zero-centered normalized to ( -10 lt= x lt= 10 )
Example 2 ZIP files (PkWare)Purpose Size in bytes
Signature header 4
Required version 2
GP flags 2
Method 2
Last mod time 2
Last mod date 2
CRC-32 4
Compressed size 4
Uncompressed size 4
Filename length 2
Extra field length 2
File name variable
extra variable
copy DJ Foreman 2012 21
Ref httplivedocsadobecomflex3htmlhelphtmlcontent=ByteArrays_3html
More info on encoding httpwwwpkwarecomdocumentscase
studiesAPPNOTETXT
httpwwwfileinfocomfiletypescompressed
copy DJ Foreman 2012 22
Recording Bit-Rate
Constant (CBR) rate at which a codecs output data should be
consumed is constant Max bit-rate matters not the average Uses all available bandwidth Not good for storage (lossy)
Variable (VBR) Quantity of datatime unit (for output) varies Better quality for audio and video Slower to encode Supported by most portable devices post 2006
copy DJ Foreman 2012 23
Lab assignment ndash 1 ndash part 1
Write a program with 1 parameter If you are on an encoding team
Open a test file (eg ldquoxyztxtrdquo) Compress it using simple Huffman compression using
the frequency table from my FTP site Output compressed file (eg ldquoxyzencrdquo) to SAME folder
as input
If you are on a decoding team Open compressed file ldquoxyzencrdquo Decompress the input file Output file ldquoxyzdecrdquo so it can be compared to ldquoxyztxtrdquo
The parameter is the full input file path amp name (ie file names MUST NOT be hard-coded)
Remember a code can be a single bit
copy DJ Foreman 2012 24
25
Lab assignment 1 ndash part 2
Use ONLY MY TABLE for the frequencies Use ONLY uppercase letters of the table Do NOT use the special character
tables Find someone in class with the
opposite program and TEST your program to see if it properly encodes or decodes a message
Grading tree construction tree following proper binary output (encoders) proper character output (decoders)
copy DJ Foreman 2012
Working with ImagesModeling
Containers
A container is a FILE format NOT a code scheme
Eg AVI amp WAV are container formats Formats for storage Independent of encoding or content
Others Ogg ASF QuickTime RealMedia Matroska
DivX and MP4
copy DJ Foreman 2012 27
28
Movie (metadata)
Track
Media
Media Information
Container file format (MP4QT)
Chunk map
Comp time
Chunk offset
Key frame
Desc
Decode time
Priority map
Size map
Sample Table
Media Handler
Data information
Media header
Video header
Track header
Movie header
copy DJ Foreman 2012
Pixels A pixel is 3 color-dots (squares) in a collection of dots
(squares) making up the picture Each color-dot is for one of the three primary colors
RGB Minimum of 1 byte per color (depending on bit-depth
which varies with camera) thus at least 3 bytes per pixel
A 10MP camera (30 million color-dots) needs 30M bytes This gets compressed by the camera (JPEG format) Final file size = 30 million bytes divided by the amount
of compression (compression ratio) An 81 compression ratio would reduce that 30 million
bytes to 375 megabytes (as done on a 4 MP camera) A RAW file has NO compression
copy DJ Foreman 2012 29
Image creation
Basic mechanisms Photography
Film Digital ndashPhotoshop Gimp
Drawing Line BrushModeling
Do it yourself OpenGL (2x and 3x)
Modelingrendering programs Breeze designer Ayam 3DCanvas Makehuman Alice
copy DJ Foreman 2012 30
Photography (film) Silver halide crystals (AgCl AgBr AgI)
Exposure to light turns them black Developer removes the loose (exposed) halide leaving
pure silver (a negative image) Fixing bath flushes out unexposed AgHalide
Greatest desire of photographers - a medium with High speed (gathers light quickly) Low noise (no undesired artifacts) High resolution (very detailed images)
Visible lines per mm (lpm) at a specified contrast level
Notes Larger ldquoclumpsrdquo are faster but limit resolution Smaller ldquoclumpsrdquo are hard to coat evenly Randomness of coating prevents Moireacute patternshttpenwikipediaorgwikiMoirC3A9_pattern
copy DJ Foreman 2012 31
FYI ndash a little chemistry
Ag+ X- + hv rarr Ag+ + X + e-
1 Radiation frees the Br2 1 Ag ion + 1 electron (Ag+ + e- rarr Ag0) -gt 1 Ag atom3 Neighbors are now ldquosensitizedrdquo 4 Developing (Reducing) converts exposed AgX to
black metallic Ag and free X5 Remaining AgX and free X are washed away (after
mixing with thiosulfate to make it soluble)
This is one silver (Ag) ldquograinrdquo in the ldquolatent imagerdquo X is any halide (Br Cl etc)
copy DJ Foreman 2012 32
crystal
Radiation (light)
Resolution 35 mm film for comparison
A 36 x 24 mm frame of ISO 100-speed film contains (asymp) the equivalent of 20 million pixels 1 There are NO pixels in film
Full-frame digital (36 x 24mm = 32 image ratio) Canon EOS 5D Nikon D3 (21MP) - $5K2
Medium format digital (6 x 45cm) Phase One P40+ (39MP) ($22K)
APS-C sized sensor (asymp24x16mm = 32 image ratio) (note 2009 data below)
Nikon D90 (123MP) - $900 Canon Rebel (15MP) $700 Used in most DSLRrsquos (note image ratio 32)
Other SONY Alpha (note image ratio 43)
1 Langford Michael Basic Photography (7th Ed 2000) Oxford Focal Press ISBN 0 240 51592 7
2 Prices as of 2009 given for comparison purposes onlycopy DJ Foreman 2012 33
Photography (digital) Image capture formats
RAW ndash 3 separate images (RGB) [12 bitspixel (4288 2848)]8=18318336 bytes per
picture JPEG ndash compressed (16x 8x 4x) Some cameras do BOTH (RAW + JPEG) for each image
Pixel notes Sensor size 35mm (or larger) vs APS-C Pixel count ndash any increase rarr
decrease in pixel size or increase in sensor size Pixel size ndash smaller pixels give us
Greater resolution (finer details) - Good More ldquonoiserdquo (incorrect values) - bad Possible ldquoadjacencyrdquo issues - bad Loss of sensitivity - bad
copy DJ Foreman 2012 34
35
Some Comparisons
Digital media vs film Pixels are the same (RGB) Pixel size varies with device (mfgr
dependency) Pixel density varies Zooming digitally does NOT give you more
detail (in spite of what they do on TV shows)
Optical zoom is useful Uses optical magnification of the image on
the target Expansion of light rays not multiplication of
pixelscopy DJ Foreman 2012
Working Definitions - 1
Graphic Model data structure nodes represent random variables Pairs of nodes connected by arcs correspond
to variables that are not independent Graphic modeling
using a Graphic Model to simulate a real-world object (this is not a formal definition)
Rendering generating a 2D image from a 3D graphic model
copy DJ Foreman 2012 36
37
Working Definitions - 2
Geometric primitives ndash Points line segments and polygons Described by vertices
Control points ndash Special points attached to a node on a
Beacutezier curve Alter the shape and angle of adjacent curve
segment Evaluators ndash
Functions that interpolate a set of control points
Produce a new set of control pointscopy DJ Foreman 2012
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
Example 1 Header file for AU files
copy DJ Foreman 2012 19
typedef unsigned long u_32 unsigned 32-bit integer
typedef struct
u_32 magic the ldquomagic numberrdquo
u_32 hdr_size byte offset to start of data
u_32 data_size length (optional)
u_32 encoding data encoding enumeration
u_32 sample_rate samples per second
u_32 channels of interleaved channels
Audio_filehdr
Following is from include ltmultimedialibaudiohgt
AUDIO_FILE_MAGIC ((u_32)0x2e736e64) ldquosndrdquo
Audio Encoding EnumerationsAUDIO_FILE_ENCODING_MULAW_8 (1) 8-bit ISDN u-law
AUDIO_FILE_ENCODING_LINEAR_8 (2) 8-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_16 (3) 16-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_32 (5) 32-bit linear PCM
AUDIO_FILE_ENCODING_FLOAT (6) 32-bit IEEE floating point
AUDIO_FILE_ENCODING_DOUBLE (7) 64-bit IEEE floating point
AUDIO_FILE_ENCODING_ADPCM_G721 (23) 4-bit CCITT g721 ADPCM
AUDIO_FILE_ENCODING_ADPCM_G723_3 (25) CCITT g723 3-bit ADPCM
AUDIO_FILE_ENCODING_ALAW_8 (27) 8-bit ISDN A-law
copy DJ Foreman 2012 20
ldquoLinearrdquo values are SIGNED intrsquosFloats are signed zero-centered normalized to ( -10 lt= x lt= 10 )
Example 2 ZIP files (PkWare)Purpose Size in bytes
Signature header 4
Required version 2
GP flags 2
Method 2
Last mod time 2
Last mod date 2
CRC-32 4
Compressed size 4
Uncompressed size 4
Filename length 2
Extra field length 2
File name variable
extra variable
copy DJ Foreman 2012 21
Ref httplivedocsadobecomflex3htmlhelphtmlcontent=ByteArrays_3html
More info on encoding httpwwwpkwarecomdocumentscase
studiesAPPNOTETXT
httpwwwfileinfocomfiletypescompressed
copy DJ Foreman 2012 22
Recording Bit-Rate
Constant (CBR) rate at which a codecs output data should be
consumed is constant Max bit-rate matters not the average Uses all available bandwidth Not good for storage (lossy)
Variable (VBR) Quantity of datatime unit (for output) varies Better quality for audio and video Slower to encode Supported by most portable devices post 2006
copy DJ Foreman 2012 23
Lab assignment ndash 1 ndash part 1
Write a program with 1 parameter If you are on an encoding team
Open a test file (eg ldquoxyztxtrdquo) Compress it using simple Huffman compression using
the frequency table from my FTP site Output compressed file (eg ldquoxyzencrdquo) to SAME folder
as input
If you are on a decoding team Open compressed file ldquoxyzencrdquo Decompress the input file Output file ldquoxyzdecrdquo so it can be compared to ldquoxyztxtrdquo
The parameter is the full input file path amp name (ie file names MUST NOT be hard-coded)
Remember a code can be a single bit
copy DJ Foreman 2012 24
25
Lab assignment 1 ndash part 2
Use ONLY MY TABLE for the frequencies Use ONLY uppercase letters of the table Do NOT use the special character
tables Find someone in class with the
opposite program and TEST your program to see if it properly encodes or decodes a message
Grading tree construction tree following proper binary output (encoders) proper character output (decoders)
copy DJ Foreman 2012
Working with ImagesModeling
Containers
A container is a FILE format NOT a code scheme
Eg AVI amp WAV are container formats Formats for storage Independent of encoding or content
Others Ogg ASF QuickTime RealMedia Matroska
DivX and MP4
copy DJ Foreman 2012 27
28
Movie (metadata)
Track
Media
Media Information
Container file format (MP4QT)
Chunk map
Comp time
Chunk offset
Key frame
Desc
Decode time
Priority map
Size map
Sample Table
Media Handler
Data information
Media header
Video header
Track header
Movie header
copy DJ Foreman 2012
Pixels A pixel is 3 color-dots (squares) in a collection of dots
(squares) making up the picture Each color-dot is for one of the three primary colors
RGB Minimum of 1 byte per color (depending on bit-depth
which varies with camera) thus at least 3 bytes per pixel
A 10MP camera (30 million color-dots) needs 30M bytes This gets compressed by the camera (JPEG format) Final file size = 30 million bytes divided by the amount
of compression (compression ratio) An 81 compression ratio would reduce that 30 million
bytes to 375 megabytes (as done on a 4 MP camera) A RAW file has NO compression
copy DJ Foreman 2012 29
Image creation
Basic mechanisms Photography
Film Digital ndashPhotoshop Gimp
Drawing Line BrushModeling
Do it yourself OpenGL (2x and 3x)
Modelingrendering programs Breeze designer Ayam 3DCanvas Makehuman Alice
copy DJ Foreman 2012 30
Photography (film) Silver halide crystals (AgCl AgBr AgI)
Exposure to light turns them black Developer removes the loose (exposed) halide leaving
pure silver (a negative image) Fixing bath flushes out unexposed AgHalide
Greatest desire of photographers - a medium with High speed (gathers light quickly) Low noise (no undesired artifacts) High resolution (very detailed images)
Visible lines per mm (lpm) at a specified contrast level
Notes Larger ldquoclumpsrdquo are faster but limit resolution Smaller ldquoclumpsrdquo are hard to coat evenly Randomness of coating prevents Moireacute patternshttpenwikipediaorgwikiMoirC3A9_pattern
copy DJ Foreman 2012 31
FYI ndash a little chemistry
Ag+ X- + hv rarr Ag+ + X + e-
1 Radiation frees the Br2 1 Ag ion + 1 electron (Ag+ + e- rarr Ag0) -gt 1 Ag atom3 Neighbors are now ldquosensitizedrdquo 4 Developing (Reducing) converts exposed AgX to
black metallic Ag and free X5 Remaining AgX and free X are washed away (after
mixing with thiosulfate to make it soluble)
This is one silver (Ag) ldquograinrdquo in the ldquolatent imagerdquo X is any halide (Br Cl etc)
copy DJ Foreman 2012 32
crystal
Radiation (light)
Resolution 35 mm film for comparison
A 36 x 24 mm frame of ISO 100-speed film contains (asymp) the equivalent of 20 million pixels 1 There are NO pixels in film
Full-frame digital (36 x 24mm = 32 image ratio) Canon EOS 5D Nikon D3 (21MP) - $5K2
Medium format digital (6 x 45cm) Phase One P40+ (39MP) ($22K)
APS-C sized sensor (asymp24x16mm = 32 image ratio) (note 2009 data below)
Nikon D90 (123MP) - $900 Canon Rebel (15MP) $700 Used in most DSLRrsquos (note image ratio 32)
Other SONY Alpha (note image ratio 43)
1 Langford Michael Basic Photography (7th Ed 2000) Oxford Focal Press ISBN 0 240 51592 7
2 Prices as of 2009 given for comparison purposes onlycopy DJ Foreman 2012 33
Photography (digital) Image capture formats
RAW ndash 3 separate images (RGB) [12 bitspixel (4288 2848)]8=18318336 bytes per
picture JPEG ndash compressed (16x 8x 4x) Some cameras do BOTH (RAW + JPEG) for each image
Pixel notes Sensor size 35mm (or larger) vs APS-C Pixel count ndash any increase rarr
decrease in pixel size or increase in sensor size Pixel size ndash smaller pixels give us
Greater resolution (finer details) - Good More ldquonoiserdquo (incorrect values) - bad Possible ldquoadjacencyrdquo issues - bad Loss of sensitivity - bad
copy DJ Foreman 2012 34
35
Some Comparisons
Digital media vs film Pixels are the same (RGB) Pixel size varies with device (mfgr
dependency) Pixel density varies Zooming digitally does NOT give you more
detail (in spite of what they do on TV shows)
Optical zoom is useful Uses optical magnification of the image on
the target Expansion of light rays not multiplication of
pixelscopy DJ Foreman 2012
Working Definitions - 1
Graphic Model data structure nodes represent random variables Pairs of nodes connected by arcs correspond
to variables that are not independent Graphic modeling
using a Graphic Model to simulate a real-world object (this is not a formal definition)
Rendering generating a 2D image from a 3D graphic model
copy DJ Foreman 2012 36
37
Working Definitions - 2
Geometric primitives ndash Points line segments and polygons Described by vertices
Control points ndash Special points attached to a node on a
Beacutezier curve Alter the shape and angle of adjacent curve
segment Evaluators ndash
Functions that interpolate a set of control points
Produce a new set of control pointscopy DJ Foreman 2012
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
Audio Encoding EnumerationsAUDIO_FILE_ENCODING_MULAW_8 (1) 8-bit ISDN u-law
AUDIO_FILE_ENCODING_LINEAR_8 (2) 8-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_16 (3) 16-bit linear PCM
AUDIO_FILE_ENCODING_LINEAR_32 (5) 32-bit linear PCM
AUDIO_FILE_ENCODING_FLOAT (6) 32-bit IEEE floating point
AUDIO_FILE_ENCODING_DOUBLE (7) 64-bit IEEE floating point
AUDIO_FILE_ENCODING_ADPCM_G721 (23) 4-bit CCITT g721 ADPCM
AUDIO_FILE_ENCODING_ADPCM_G723_3 (25) CCITT g723 3-bit ADPCM
AUDIO_FILE_ENCODING_ALAW_8 (27) 8-bit ISDN A-law
copy DJ Foreman 2012 20
ldquoLinearrdquo values are SIGNED intrsquosFloats are signed zero-centered normalized to ( -10 lt= x lt= 10 )
Example 2 ZIP files (PkWare)Purpose Size in bytes
Signature header 4
Required version 2
GP flags 2
Method 2
Last mod time 2
Last mod date 2
CRC-32 4
Compressed size 4
Uncompressed size 4
Filename length 2
Extra field length 2
File name variable
extra variable
copy DJ Foreman 2012 21
Ref httplivedocsadobecomflex3htmlhelphtmlcontent=ByteArrays_3html
More info on encoding httpwwwpkwarecomdocumentscase
studiesAPPNOTETXT
httpwwwfileinfocomfiletypescompressed
copy DJ Foreman 2012 22
Recording Bit-Rate
Constant (CBR) rate at which a codecs output data should be
consumed is constant Max bit-rate matters not the average Uses all available bandwidth Not good for storage (lossy)
Variable (VBR) Quantity of datatime unit (for output) varies Better quality for audio and video Slower to encode Supported by most portable devices post 2006
copy DJ Foreman 2012 23
Lab assignment ndash 1 ndash part 1
Write a program with 1 parameter If you are on an encoding team
Open a test file (eg ldquoxyztxtrdquo) Compress it using simple Huffman compression using
the frequency table from my FTP site Output compressed file (eg ldquoxyzencrdquo) to SAME folder
as input
If you are on a decoding team Open compressed file ldquoxyzencrdquo Decompress the input file Output file ldquoxyzdecrdquo so it can be compared to ldquoxyztxtrdquo
The parameter is the full input file path amp name (ie file names MUST NOT be hard-coded)
Remember a code can be a single bit
copy DJ Foreman 2012 24
25
Lab assignment 1 ndash part 2
Use ONLY MY TABLE for the frequencies Use ONLY uppercase letters of the table Do NOT use the special character
tables Find someone in class with the
opposite program and TEST your program to see if it properly encodes or decodes a message
Grading tree construction tree following proper binary output (encoders) proper character output (decoders)
copy DJ Foreman 2012
Working with ImagesModeling
Containers
A container is a FILE format NOT a code scheme
Eg AVI amp WAV are container formats Formats for storage Independent of encoding or content
Others Ogg ASF QuickTime RealMedia Matroska
DivX and MP4
copy DJ Foreman 2012 27
28
Movie (metadata)
Track
Media
Media Information
Container file format (MP4QT)
Chunk map
Comp time
Chunk offset
Key frame
Desc
Decode time
Priority map
Size map
Sample Table
Media Handler
Data information
Media header
Video header
Track header
Movie header
copy DJ Foreman 2012
Pixels A pixel is 3 color-dots (squares) in a collection of dots
(squares) making up the picture Each color-dot is for one of the three primary colors
RGB Minimum of 1 byte per color (depending on bit-depth
which varies with camera) thus at least 3 bytes per pixel
A 10MP camera (30 million color-dots) needs 30M bytes This gets compressed by the camera (JPEG format) Final file size = 30 million bytes divided by the amount
of compression (compression ratio) An 81 compression ratio would reduce that 30 million
bytes to 375 megabytes (as done on a 4 MP camera) A RAW file has NO compression
copy DJ Foreman 2012 29
Image creation
Basic mechanisms Photography
Film Digital ndashPhotoshop Gimp
Drawing Line BrushModeling
Do it yourself OpenGL (2x and 3x)
Modelingrendering programs Breeze designer Ayam 3DCanvas Makehuman Alice
copy DJ Foreman 2012 30
Photography (film) Silver halide crystals (AgCl AgBr AgI)
Exposure to light turns them black Developer removes the loose (exposed) halide leaving
pure silver (a negative image) Fixing bath flushes out unexposed AgHalide
Greatest desire of photographers - a medium with High speed (gathers light quickly) Low noise (no undesired artifacts) High resolution (very detailed images)
Visible lines per mm (lpm) at a specified contrast level
Notes Larger ldquoclumpsrdquo are faster but limit resolution Smaller ldquoclumpsrdquo are hard to coat evenly Randomness of coating prevents Moireacute patternshttpenwikipediaorgwikiMoirC3A9_pattern
copy DJ Foreman 2012 31
FYI ndash a little chemistry
Ag+ X- + hv rarr Ag+ + X + e-
1 Radiation frees the Br2 1 Ag ion + 1 electron (Ag+ + e- rarr Ag0) -gt 1 Ag atom3 Neighbors are now ldquosensitizedrdquo 4 Developing (Reducing) converts exposed AgX to
black metallic Ag and free X5 Remaining AgX and free X are washed away (after
mixing with thiosulfate to make it soluble)
This is one silver (Ag) ldquograinrdquo in the ldquolatent imagerdquo X is any halide (Br Cl etc)
copy DJ Foreman 2012 32
crystal
Radiation (light)
Resolution 35 mm film for comparison
A 36 x 24 mm frame of ISO 100-speed film contains (asymp) the equivalent of 20 million pixels 1 There are NO pixels in film
Full-frame digital (36 x 24mm = 32 image ratio) Canon EOS 5D Nikon D3 (21MP) - $5K2
Medium format digital (6 x 45cm) Phase One P40+ (39MP) ($22K)
APS-C sized sensor (asymp24x16mm = 32 image ratio) (note 2009 data below)
Nikon D90 (123MP) - $900 Canon Rebel (15MP) $700 Used in most DSLRrsquos (note image ratio 32)
Other SONY Alpha (note image ratio 43)
1 Langford Michael Basic Photography (7th Ed 2000) Oxford Focal Press ISBN 0 240 51592 7
2 Prices as of 2009 given for comparison purposes onlycopy DJ Foreman 2012 33
Photography (digital) Image capture formats
RAW ndash 3 separate images (RGB) [12 bitspixel (4288 2848)]8=18318336 bytes per
picture JPEG ndash compressed (16x 8x 4x) Some cameras do BOTH (RAW + JPEG) for each image
Pixel notes Sensor size 35mm (or larger) vs APS-C Pixel count ndash any increase rarr
decrease in pixel size or increase in sensor size Pixel size ndash smaller pixels give us
Greater resolution (finer details) - Good More ldquonoiserdquo (incorrect values) - bad Possible ldquoadjacencyrdquo issues - bad Loss of sensitivity - bad
copy DJ Foreman 2012 34
35
Some Comparisons
Digital media vs film Pixels are the same (RGB) Pixel size varies with device (mfgr
dependency) Pixel density varies Zooming digitally does NOT give you more
detail (in spite of what they do on TV shows)
Optical zoom is useful Uses optical magnification of the image on
the target Expansion of light rays not multiplication of
pixelscopy DJ Foreman 2012
Working Definitions - 1
Graphic Model data structure nodes represent random variables Pairs of nodes connected by arcs correspond
to variables that are not independent Graphic modeling
using a Graphic Model to simulate a real-world object (this is not a formal definition)
Rendering generating a 2D image from a 3D graphic model
copy DJ Foreman 2012 36
37
Working Definitions - 2
Geometric primitives ndash Points line segments and polygons Described by vertices
Control points ndash Special points attached to a node on a
Beacutezier curve Alter the shape and angle of adjacent curve
segment Evaluators ndash
Functions that interpolate a set of control points
Produce a new set of control pointscopy DJ Foreman 2012
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
Example 2 ZIP files (PkWare)Purpose Size in bytes
Signature header 4
Required version 2
GP flags 2
Method 2
Last mod time 2
Last mod date 2
CRC-32 4
Compressed size 4
Uncompressed size 4
Filename length 2
Extra field length 2
File name variable
extra variable
copy DJ Foreman 2012 21
Ref httplivedocsadobecomflex3htmlhelphtmlcontent=ByteArrays_3html
More info on encoding httpwwwpkwarecomdocumentscase
studiesAPPNOTETXT
httpwwwfileinfocomfiletypescompressed
copy DJ Foreman 2012 22
Recording Bit-Rate
Constant (CBR) rate at which a codecs output data should be
consumed is constant Max bit-rate matters not the average Uses all available bandwidth Not good for storage (lossy)
Variable (VBR) Quantity of datatime unit (for output) varies Better quality for audio and video Slower to encode Supported by most portable devices post 2006
copy DJ Foreman 2012 23
Lab assignment ndash 1 ndash part 1
Write a program with 1 parameter If you are on an encoding team
Open a test file (eg ldquoxyztxtrdquo) Compress it using simple Huffman compression using
the frequency table from my FTP site Output compressed file (eg ldquoxyzencrdquo) to SAME folder
as input
If you are on a decoding team Open compressed file ldquoxyzencrdquo Decompress the input file Output file ldquoxyzdecrdquo so it can be compared to ldquoxyztxtrdquo
The parameter is the full input file path amp name (ie file names MUST NOT be hard-coded)
Remember a code can be a single bit
copy DJ Foreman 2012 24
25
Lab assignment 1 ndash part 2
Use ONLY MY TABLE for the frequencies Use ONLY uppercase letters of the table Do NOT use the special character
tables Find someone in class with the
opposite program and TEST your program to see if it properly encodes or decodes a message
Grading tree construction tree following proper binary output (encoders) proper character output (decoders)
copy DJ Foreman 2012
Working with ImagesModeling
Containers
A container is a FILE format NOT a code scheme
Eg AVI amp WAV are container formats Formats for storage Independent of encoding or content
Others Ogg ASF QuickTime RealMedia Matroska
DivX and MP4
copy DJ Foreman 2012 27
28
Movie (metadata)
Track
Media
Media Information
Container file format (MP4QT)
Chunk map
Comp time
Chunk offset
Key frame
Desc
Decode time
Priority map
Size map
Sample Table
Media Handler
Data information
Media header
Video header
Track header
Movie header
copy DJ Foreman 2012
Pixels A pixel is 3 color-dots (squares) in a collection of dots
(squares) making up the picture Each color-dot is for one of the three primary colors
RGB Minimum of 1 byte per color (depending on bit-depth
which varies with camera) thus at least 3 bytes per pixel
A 10MP camera (30 million color-dots) needs 30M bytes This gets compressed by the camera (JPEG format) Final file size = 30 million bytes divided by the amount
of compression (compression ratio) An 81 compression ratio would reduce that 30 million
bytes to 375 megabytes (as done on a 4 MP camera) A RAW file has NO compression
copy DJ Foreman 2012 29
Image creation
Basic mechanisms Photography
Film Digital ndashPhotoshop Gimp
Drawing Line BrushModeling
Do it yourself OpenGL (2x and 3x)
Modelingrendering programs Breeze designer Ayam 3DCanvas Makehuman Alice
copy DJ Foreman 2012 30
Photography (film) Silver halide crystals (AgCl AgBr AgI)
Exposure to light turns them black Developer removes the loose (exposed) halide leaving
pure silver (a negative image) Fixing bath flushes out unexposed AgHalide
Greatest desire of photographers - a medium with High speed (gathers light quickly) Low noise (no undesired artifacts) High resolution (very detailed images)
Visible lines per mm (lpm) at a specified contrast level
Notes Larger ldquoclumpsrdquo are faster but limit resolution Smaller ldquoclumpsrdquo are hard to coat evenly Randomness of coating prevents Moireacute patternshttpenwikipediaorgwikiMoirC3A9_pattern
copy DJ Foreman 2012 31
FYI ndash a little chemistry
Ag+ X- + hv rarr Ag+ + X + e-
1 Radiation frees the Br2 1 Ag ion + 1 electron (Ag+ + e- rarr Ag0) -gt 1 Ag atom3 Neighbors are now ldquosensitizedrdquo 4 Developing (Reducing) converts exposed AgX to
black metallic Ag and free X5 Remaining AgX and free X are washed away (after
mixing with thiosulfate to make it soluble)
This is one silver (Ag) ldquograinrdquo in the ldquolatent imagerdquo X is any halide (Br Cl etc)
copy DJ Foreman 2012 32
crystal
Radiation (light)
Resolution 35 mm film for comparison
A 36 x 24 mm frame of ISO 100-speed film contains (asymp) the equivalent of 20 million pixels 1 There are NO pixels in film
Full-frame digital (36 x 24mm = 32 image ratio) Canon EOS 5D Nikon D3 (21MP) - $5K2
Medium format digital (6 x 45cm) Phase One P40+ (39MP) ($22K)
APS-C sized sensor (asymp24x16mm = 32 image ratio) (note 2009 data below)
Nikon D90 (123MP) - $900 Canon Rebel (15MP) $700 Used in most DSLRrsquos (note image ratio 32)
Other SONY Alpha (note image ratio 43)
1 Langford Michael Basic Photography (7th Ed 2000) Oxford Focal Press ISBN 0 240 51592 7
2 Prices as of 2009 given for comparison purposes onlycopy DJ Foreman 2012 33
Photography (digital) Image capture formats
RAW ndash 3 separate images (RGB) [12 bitspixel (4288 2848)]8=18318336 bytes per
picture JPEG ndash compressed (16x 8x 4x) Some cameras do BOTH (RAW + JPEG) for each image
Pixel notes Sensor size 35mm (or larger) vs APS-C Pixel count ndash any increase rarr
decrease in pixel size or increase in sensor size Pixel size ndash smaller pixels give us
Greater resolution (finer details) - Good More ldquonoiserdquo (incorrect values) - bad Possible ldquoadjacencyrdquo issues - bad Loss of sensitivity - bad
copy DJ Foreman 2012 34
35
Some Comparisons
Digital media vs film Pixels are the same (RGB) Pixel size varies with device (mfgr
dependency) Pixel density varies Zooming digitally does NOT give you more
detail (in spite of what they do on TV shows)
Optical zoom is useful Uses optical magnification of the image on
the target Expansion of light rays not multiplication of
pixelscopy DJ Foreman 2012
Working Definitions - 1
Graphic Model data structure nodes represent random variables Pairs of nodes connected by arcs correspond
to variables that are not independent Graphic modeling
using a Graphic Model to simulate a real-world object (this is not a formal definition)
Rendering generating a 2D image from a 3D graphic model
copy DJ Foreman 2012 36
37
Working Definitions - 2
Geometric primitives ndash Points line segments and polygons Described by vertices
Control points ndash Special points attached to a node on a
Beacutezier curve Alter the shape and angle of adjacent curve
segment Evaluators ndash
Functions that interpolate a set of control points
Produce a new set of control pointscopy DJ Foreman 2012
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
More info on encoding httpwwwpkwarecomdocumentscase
studiesAPPNOTETXT
httpwwwfileinfocomfiletypescompressed
copy DJ Foreman 2012 22
Recording Bit-Rate
Constant (CBR) rate at which a codecs output data should be
consumed is constant Max bit-rate matters not the average Uses all available bandwidth Not good for storage (lossy)
Variable (VBR) Quantity of datatime unit (for output) varies Better quality for audio and video Slower to encode Supported by most portable devices post 2006
copy DJ Foreman 2012 23
Lab assignment ndash 1 ndash part 1
Write a program with 1 parameter If you are on an encoding team
Open a test file (eg ldquoxyztxtrdquo) Compress it using simple Huffman compression using
the frequency table from my FTP site Output compressed file (eg ldquoxyzencrdquo) to SAME folder
as input
If you are on a decoding team Open compressed file ldquoxyzencrdquo Decompress the input file Output file ldquoxyzdecrdquo so it can be compared to ldquoxyztxtrdquo
The parameter is the full input file path amp name (ie file names MUST NOT be hard-coded)
Remember a code can be a single bit
copy DJ Foreman 2012 24
25
Lab assignment 1 ndash part 2
Use ONLY MY TABLE for the frequencies Use ONLY uppercase letters of the table Do NOT use the special character
tables Find someone in class with the
opposite program and TEST your program to see if it properly encodes or decodes a message
Grading tree construction tree following proper binary output (encoders) proper character output (decoders)
copy DJ Foreman 2012
Working with ImagesModeling
Containers
A container is a FILE format NOT a code scheme
Eg AVI amp WAV are container formats Formats for storage Independent of encoding or content
Others Ogg ASF QuickTime RealMedia Matroska
DivX and MP4
copy DJ Foreman 2012 27
28
Movie (metadata)
Track
Media
Media Information
Container file format (MP4QT)
Chunk map
Comp time
Chunk offset
Key frame
Desc
Decode time
Priority map
Size map
Sample Table
Media Handler
Data information
Media header
Video header
Track header
Movie header
copy DJ Foreman 2012
Pixels A pixel is 3 color-dots (squares) in a collection of dots
(squares) making up the picture Each color-dot is for one of the three primary colors
RGB Minimum of 1 byte per color (depending on bit-depth
which varies with camera) thus at least 3 bytes per pixel
A 10MP camera (30 million color-dots) needs 30M bytes This gets compressed by the camera (JPEG format) Final file size = 30 million bytes divided by the amount
of compression (compression ratio) An 81 compression ratio would reduce that 30 million
bytes to 375 megabytes (as done on a 4 MP camera) A RAW file has NO compression
copy DJ Foreman 2012 29
Image creation
Basic mechanisms Photography
Film Digital ndashPhotoshop Gimp
Drawing Line BrushModeling
Do it yourself OpenGL (2x and 3x)
Modelingrendering programs Breeze designer Ayam 3DCanvas Makehuman Alice
copy DJ Foreman 2012 30
Photography (film) Silver halide crystals (AgCl AgBr AgI)
Exposure to light turns them black Developer removes the loose (exposed) halide leaving
pure silver (a negative image) Fixing bath flushes out unexposed AgHalide
Greatest desire of photographers - a medium with High speed (gathers light quickly) Low noise (no undesired artifacts) High resolution (very detailed images)
Visible lines per mm (lpm) at a specified contrast level
Notes Larger ldquoclumpsrdquo are faster but limit resolution Smaller ldquoclumpsrdquo are hard to coat evenly Randomness of coating prevents Moireacute patternshttpenwikipediaorgwikiMoirC3A9_pattern
copy DJ Foreman 2012 31
FYI ndash a little chemistry
Ag+ X- + hv rarr Ag+ + X + e-
1 Radiation frees the Br2 1 Ag ion + 1 electron (Ag+ + e- rarr Ag0) -gt 1 Ag atom3 Neighbors are now ldquosensitizedrdquo 4 Developing (Reducing) converts exposed AgX to
black metallic Ag and free X5 Remaining AgX and free X are washed away (after
mixing with thiosulfate to make it soluble)
This is one silver (Ag) ldquograinrdquo in the ldquolatent imagerdquo X is any halide (Br Cl etc)
copy DJ Foreman 2012 32
crystal
Radiation (light)
Resolution 35 mm film for comparison
A 36 x 24 mm frame of ISO 100-speed film contains (asymp) the equivalent of 20 million pixels 1 There are NO pixels in film
Full-frame digital (36 x 24mm = 32 image ratio) Canon EOS 5D Nikon D3 (21MP) - $5K2
Medium format digital (6 x 45cm) Phase One P40+ (39MP) ($22K)
APS-C sized sensor (asymp24x16mm = 32 image ratio) (note 2009 data below)
Nikon D90 (123MP) - $900 Canon Rebel (15MP) $700 Used in most DSLRrsquos (note image ratio 32)
Other SONY Alpha (note image ratio 43)
1 Langford Michael Basic Photography (7th Ed 2000) Oxford Focal Press ISBN 0 240 51592 7
2 Prices as of 2009 given for comparison purposes onlycopy DJ Foreman 2012 33
Photography (digital) Image capture formats
RAW ndash 3 separate images (RGB) [12 bitspixel (4288 2848)]8=18318336 bytes per
picture JPEG ndash compressed (16x 8x 4x) Some cameras do BOTH (RAW + JPEG) for each image
Pixel notes Sensor size 35mm (or larger) vs APS-C Pixel count ndash any increase rarr
decrease in pixel size or increase in sensor size Pixel size ndash smaller pixels give us
Greater resolution (finer details) - Good More ldquonoiserdquo (incorrect values) - bad Possible ldquoadjacencyrdquo issues - bad Loss of sensitivity - bad
copy DJ Foreman 2012 34
35
Some Comparisons
Digital media vs film Pixels are the same (RGB) Pixel size varies with device (mfgr
dependency) Pixel density varies Zooming digitally does NOT give you more
detail (in spite of what they do on TV shows)
Optical zoom is useful Uses optical magnification of the image on
the target Expansion of light rays not multiplication of
pixelscopy DJ Foreman 2012
Working Definitions - 1
Graphic Model data structure nodes represent random variables Pairs of nodes connected by arcs correspond
to variables that are not independent Graphic modeling
using a Graphic Model to simulate a real-world object (this is not a formal definition)
Rendering generating a 2D image from a 3D graphic model
copy DJ Foreman 2012 36
37
Working Definitions - 2
Geometric primitives ndash Points line segments and polygons Described by vertices
Control points ndash Special points attached to a node on a
Beacutezier curve Alter the shape and angle of adjacent curve
segment Evaluators ndash
Functions that interpolate a set of control points
Produce a new set of control pointscopy DJ Foreman 2012
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
Recording Bit-Rate
Constant (CBR) rate at which a codecs output data should be
consumed is constant Max bit-rate matters not the average Uses all available bandwidth Not good for storage (lossy)
Variable (VBR) Quantity of datatime unit (for output) varies Better quality for audio and video Slower to encode Supported by most portable devices post 2006
copy DJ Foreman 2012 23
Lab assignment ndash 1 ndash part 1
Write a program with 1 parameter If you are on an encoding team
Open a test file (eg ldquoxyztxtrdquo) Compress it using simple Huffman compression using
the frequency table from my FTP site Output compressed file (eg ldquoxyzencrdquo) to SAME folder
as input
If you are on a decoding team Open compressed file ldquoxyzencrdquo Decompress the input file Output file ldquoxyzdecrdquo so it can be compared to ldquoxyztxtrdquo
The parameter is the full input file path amp name (ie file names MUST NOT be hard-coded)
Remember a code can be a single bit
copy DJ Foreman 2012 24
25
Lab assignment 1 ndash part 2
Use ONLY MY TABLE for the frequencies Use ONLY uppercase letters of the table Do NOT use the special character
tables Find someone in class with the
opposite program and TEST your program to see if it properly encodes or decodes a message
Grading tree construction tree following proper binary output (encoders) proper character output (decoders)
copy DJ Foreman 2012
Working with ImagesModeling
Containers
A container is a FILE format NOT a code scheme
Eg AVI amp WAV are container formats Formats for storage Independent of encoding or content
Others Ogg ASF QuickTime RealMedia Matroska
DivX and MP4
copy DJ Foreman 2012 27
28
Movie (metadata)
Track
Media
Media Information
Container file format (MP4QT)
Chunk map
Comp time
Chunk offset
Key frame
Desc
Decode time
Priority map
Size map
Sample Table
Media Handler
Data information
Media header
Video header
Track header
Movie header
copy DJ Foreman 2012
Pixels A pixel is 3 color-dots (squares) in a collection of dots
(squares) making up the picture Each color-dot is for one of the three primary colors
RGB Minimum of 1 byte per color (depending on bit-depth
which varies with camera) thus at least 3 bytes per pixel
A 10MP camera (30 million color-dots) needs 30M bytes This gets compressed by the camera (JPEG format) Final file size = 30 million bytes divided by the amount
of compression (compression ratio) An 81 compression ratio would reduce that 30 million
bytes to 375 megabytes (as done on a 4 MP camera) A RAW file has NO compression
copy DJ Foreman 2012 29
Image creation
Basic mechanisms Photography
Film Digital ndashPhotoshop Gimp
Drawing Line BrushModeling
Do it yourself OpenGL (2x and 3x)
Modelingrendering programs Breeze designer Ayam 3DCanvas Makehuman Alice
copy DJ Foreman 2012 30
Photography (film) Silver halide crystals (AgCl AgBr AgI)
Exposure to light turns them black Developer removes the loose (exposed) halide leaving
pure silver (a negative image) Fixing bath flushes out unexposed AgHalide
Greatest desire of photographers - a medium with High speed (gathers light quickly) Low noise (no undesired artifacts) High resolution (very detailed images)
Visible lines per mm (lpm) at a specified contrast level
Notes Larger ldquoclumpsrdquo are faster but limit resolution Smaller ldquoclumpsrdquo are hard to coat evenly Randomness of coating prevents Moireacute patternshttpenwikipediaorgwikiMoirC3A9_pattern
copy DJ Foreman 2012 31
FYI ndash a little chemistry
Ag+ X- + hv rarr Ag+ + X + e-
1 Radiation frees the Br2 1 Ag ion + 1 electron (Ag+ + e- rarr Ag0) -gt 1 Ag atom3 Neighbors are now ldquosensitizedrdquo 4 Developing (Reducing) converts exposed AgX to
black metallic Ag and free X5 Remaining AgX and free X are washed away (after
mixing with thiosulfate to make it soluble)
This is one silver (Ag) ldquograinrdquo in the ldquolatent imagerdquo X is any halide (Br Cl etc)
copy DJ Foreman 2012 32
crystal
Radiation (light)
Resolution 35 mm film for comparison
A 36 x 24 mm frame of ISO 100-speed film contains (asymp) the equivalent of 20 million pixels 1 There are NO pixels in film
Full-frame digital (36 x 24mm = 32 image ratio) Canon EOS 5D Nikon D3 (21MP) - $5K2
Medium format digital (6 x 45cm) Phase One P40+ (39MP) ($22K)
APS-C sized sensor (asymp24x16mm = 32 image ratio) (note 2009 data below)
Nikon D90 (123MP) - $900 Canon Rebel (15MP) $700 Used in most DSLRrsquos (note image ratio 32)
Other SONY Alpha (note image ratio 43)
1 Langford Michael Basic Photography (7th Ed 2000) Oxford Focal Press ISBN 0 240 51592 7
2 Prices as of 2009 given for comparison purposes onlycopy DJ Foreman 2012 33
Photography (digital) Image capture formats
RAW ndash 3 separate images (RGB) [12 bitspixel (4288 2848)]8=18318336 bytes per
picture JPEG ndash compressed (16x 8x 4x) Some cameras do BOTH (RAW + JPEG) for each image
Pixel notes Sensor size 35mm (or larger) vs APS-C Pixel count ndash any increase rarr
decrease in pixel size or increase in sensor size Pixel size ndash smaller pixels give us
Greater resolution (finer details) - Good More ldquonoiserdquo (incorrect values) - bad Possible ldquoadjacencyrdquo issues - bad Loss of sensitivity - bad
copy DJ Foreman 2012 34
35
Some Comparisons
Digital media vs film Pixels are the same (RGB) Pixel size varies with device (mfgr
dependency) Pixel density varies Zooming digitally does NOT give you more
detail (in spite of what they do on TV shows)
Optical zoom is useful Uses optical magnification of the image on
the target Expansion of light rays not multiplication of
pixelscopy DJ Foreman 2012
Working Definitions - 1
Graphic Model data structure nodes represent random variables Pairs of nodes connected by arcs correspond
to variables that are not independent Graphic modeling
using a Graphic Model to simulate a real-world object (this is not a formal definition)
Rendering generating a 2D image from a 3D graphic model
copy DJ Foreman 2012 36
37
Working Definitions - 2
Geometric primitives ndash Points line segments and polygons Described by vertices
Control points ndash Special points attached to a node on a
Beacutezier curve Alter the shape and angle of adjacent curve
segment Evaluators ndash
Functions that interpolate a set of control points
Produce a new set of control pointscopy DJ Foreman 2012
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
Lab assignment ndash 1 ndash part 1
Write a program with 1 parameter If you are on an encoding team
Open a test file (eg ldquoxyztxtrdquo) Compress it using simple Huffman compression using
the frequency table from my FTP site Output compressed file (eg ldquoxyzencrdquo) to SAME folder
as input
If you are on a decoding team Open compressed file ldquoxyzencrdquo Decompress the input file Output file ldquoxyzdecrdquo so it can be compared to ldquoxyztxtrdquo
The parameter is the full input file path amp name (ie file names MUST NOT be hard-coded)
Remember a code can be a single bit
copy DJ Foreman 2012 24
25
Lab assignment 1 ndash part 2
Use ONLY MY TABLE for the frequencies Use ONLY uppercase letters of the table Do NOT use the special character
tables Find someone in class with the
opposite program and TEST your program to see if it properly encodes or decodes a message
Grading tree construction tree following proper binary output (encoders) proper character output (decoders)
copy DJ Foreman 2012
Working with ImagesModeling
Containers
A container is a FILE format NOT a code scheme
Eg AVI amp WAV are container formats Formats for storage Independent of encoding or content
Others Ogg ASF QuickTime RealMedia Matroska
DivX and MP4
copy DJ Foreman 2012 27
28
Movie (metadata)
Track
Media
Media Information
Container file format (MP4QT)
Chunk map
Comp time
Chunk offset
Key frame
Desc
Decode time
Priority map
Size map
Sample Table
Media Handler
Data information
Media header
Video header
Track header
Movie header
copy DJ Foreman 2012
Pixels A pixel is 3 color-dots (squares) in a collection of dots
(squares) making up the picture Each color-dot is for one of the three primary colors
RGB Minimum of 1 byte per color (depending on bit-depth
which varies with camera) thus at least 3 bytes per pixel
A 10MP camera (30 million color-dots) needs 30M bytes This gets compressed by the camera (JPEG format) Final file size = 30 million bytes divided by the amount
of compression (compression ratio) An 81 compression ratio would reduce that 30 million
bytes to 375 megabytes (as done on a 4 MP camera) A RAW file has NO compression
copy DJ Foreman 2012 29
Image creation
Basic mechanisms Photography
Film Digital ndashPhotoshop Gimp
Drawing Line BrushModeling
Do it yourself OpenGL (2x and 3x)
Modelingrendering programs Breeze designer Ayam 3DCanvas Makehuman Alice
copy DJ Foreman 2012 30
Photography (film) Silver halide crystals (AgCl AgBr AgI)
Exposure to light turns them black Developer removes the loose (exposed) halide leaving
pure silver (a negative image) Fixing bath flushes out unexposed AgHalide
Greatest desire of photographers - a medium with High speed (gathers light quickly) Low noise (no undesired artifacts) High resolution (very detailed images)
Visible lines per mm (lpm) at a specified contrast level
Notes Larger ldquoclumpsrdquo are faster but limit resolution Smaller ldquoclumpsrdquo are hard to coat evenly Randomness of coating prevents Moireacute patternshttpenwikipediaorgwikiMoirC3A9_pattern
copy DJ Foreman 2012 31
FYI ndash a little chemistry
Ag+ X- + hv rarr Ag+ + X + e-
1 Radiation frees the Br2 1 Ag ion + 1 electron (Ag+ + e- rarr Ag0) -gt 1 Ag atom3 Neighbors are now ldquosensitizedrdquo 4 Developing (Reducing) converts exposed AgX to
black metallic Ag and free X5 Remaining AgX and free X are washed away (after
mixing with thiosulfate to make it soluble)
This is one silver (Ag) ldquograinrdquo in the ldquolatent imagerdquo X is any halide (Br Cl etc)
copy DJ Foreman 2012 32
crystal
Radiation (light)
Resolution 35 mm film for comparison
A 36 x 24 mm frame of ISO 100-speed film contains (asymp) the equivalent of 20 million pixels 1 There are NO pixels in film
Full-frame digital (36 x 24mm = 32 image ratio) Canon EOS 5D Nikon D3 (21MP) - $5K2
Medium format digital (6 x 45cm) Phase One P40+ (39MP) ($22K)
APS-C sized sensor (asymp24x16mm = 32 image ratio) (note 2009 data below)
Nikon D90 (123MP) - $900 Canon Rebel (15MP) $700 Used in most DSLRrsquos (note image ratio 32)
Other SONY Alpha (note image ratio 43)
1 Langford Michael Basic Photography (7th Ed 2000) Oxford Focal Press ISBN 0 240 51592 7
2 Prices as of 2009 given for comparison purposes onlycopy DJ Foreman 2012 33
Photography (digital) Image capture formats
RAW ndash 3 separate images (RGB) [12 bitspixel (4288 2848)]8=18318336 bytes per
picture JPEG ndash compressed (16x 8x 4x) Some cameras do BOTH (RAW + JPEG) for each image
Pixel notes Sensor size 35mm (or larger) vs APS-C Pixel count ndash any increase rarr
decrease in pixel size or increase in sensor size Pixel size ndash smaller pixels give us
Greater resolution (finer details) - Good More ldquonoiserdquo (incorrect values) - bad Possible ldquoadjacencyrdquo issues - bad Loss of sensitivity - bad
copy DJ Foreman 2012 34
35
Some Comparisons
Digital media vs film Pixels are the same (RGB) Pixel size varies with device (mfgr
dependency) Pixel density varies Zooming digitally does NOT give you more
detail (in spite of what they do on TV shows)
Optical zoom is useful Uses optical magnification of the image on
the target Expansion of light rays not multiplication of
pixelscopy DJ Foreman 2012
Working Definitions - 1
Graphic Model data structure nodes represent random variables Pairs of nodes connected by arcs correspond
to variables that are not independent Graphic modeling
using a Graphic Model to simulate a real-world object (this is not a formal definition)
Rendering generating a 2D image from a 3D graphic model
copy DJ Foreman 2012 36
37
Working Definitions - 2
Geometric primitives ndash Points line segments and polygons Described by vertices
Control points ndash Special points attached to a node on a
Beacutezier curve Alter the shape and angle of adjacent curve
segment Evaluators ndash
Functions that interpolate a set of control points
Produce a new set of control pointscopy DJ Foreman 2012
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
25
Lab assignment 1 ndash part 2
Use ONLY MY TABLE for the frequencies Use ONLY uppercase letters of the table Do NOT use the special character
tables Find someone in class with the
opposite program and TEST your program to see if it properly encodes or decodes a message
Grading tree construction tree following proper binary output (encoders) proper character output (decoders)
copy DJ Foreman 2012
Working with ImagesModeling
Containers
A container is a FILE format NOT a code scheme
Eg AVI amp WAV are container formats Formats for storage Independent of encoding or content
Others Ogg ASF QuickTime RealMedia Matroska
DivX and MP4
copy DJ Foreman 2012 27
28
Movie (metadata)
Track
Media
Media Information
Container file format (MP4QT)
Chunk map
Comp time
Chunk offset
Key frame
Desc
Decode time
Priority map
Size map
Sample Table
Media Handler
Data information
Media header
Video header
Track header
Movie header
copy DJ Foreman 2012
Pixels A pixel is 3 color-dots (squares) in a collection of dots
(squares) making up the picture Each color-dot is for one of the three primary colors
RGB Minimum of 1 byte per color (depending on bit-depth
which varies with camera) thus at least 3 bytes per pixel
A 10MP camera (30 million color-dots) needs 30M bytes This gets compressed by the camera (JPEG format) Final file size = 30 million bytes divided by the amount
of compression (compression ratio) An 81 compression ratio would reduce that 30 million
bytes to 375 megabytes (as done on a 4 MP camera) A RAW file has NO compression
copy DJ Foreman 2012 29
Image creation
Basic mechanisms Photography
Film Digital ndashPhotoshop Gimp
Drawing Line BrushModeling
Do it yourself OpenGL (2x and 3x)
Modelingrendering programs Breeze designer Ayam 3DCanvas Makehuman Alice
copy DJ Foreman 2012 30
Photography (film) Silver halide crystals (AgCl AgBr AgI)
Exposure to light turns them black Developer removes the loose (exposed) halide leaving
pure silver (a negative image) Fixing bath flushes out unexposed AgHalide
Greatest desire of photographers - a medium with High speed (gathers light quickly) Low noise (no undesired artifacts) High resolution (very detailed images)
Visible lines per mm (lpm) at a specified contrast level
Notes Larger ldquoclumpsrdquo are faster but limit resolution Smaller ldquoclumpsrdquo are hard to coat evenly Randomness of coating prevents Moireacute patternshttpenwikipediaorgwikiMoirC3A9_pattern
copy DJ Foreman 2012 31
FYI ndash a little chemistry
Ag+ X- + hv rarr Ag+ + X + e-
1 Radiation frees the Br2 1 Ag ion + 1 electron (Ag+ + e- rarr Ag0) -gt 1 Ag atom3 Neighbors are now ldquosensitizedrdquo 4 Developing (Reducing) converts exposed AgX to
black metallic Ag and free X5 Remaining AgX and free X are washed away (after
mixing with thiosulfate to make it soluble)
This is one silver (Ag) ldquograinrdquo in the ldquolatent imagerdquo X is any halide (Br Cl etc)
copy DJ Foreman 2012 32
crystal
Radiation (light)
Resolution 35 mm film for comparison
A 36 x 24 mm frame of ISO 100-speed film contains (asymp) the equivalent of 20 million pixels 1 There are NO pixels in film
Full-frame digital (36 x 24mm = 32 image ratio) Canon EOS 5D Nikon D3 (21MP) - $5K2
Medium format digital (6 x 45cm) Phase One P40+ (39MP) ($22K)
APS-C sized sensor (asymp24x16mm = 32 image ratio) (note 2009 data below)
Nikon D90 (123MP) - $900 Canon Rebel (15MP) $700 Used in most DSLRrsquos (note image ratio 32)
Other SONY Alpha (note image ratio 43)
1 Langford Michael Basic Photography (7th Ed 2000) Oxford Focal Press ISBN 0 240 51592 7
2 Prices as of 2009 given for comparison purposes onlycopy DJ Foreman 2012 33
Photography (digital) Image capture formats
RAW ndash 3 separate images (RGB) [12 bitspixel (4288 2848)]8=18318336 bytes per
picture JPEG ndash compressed (16x 8x 4x) Some cameras do BOTH (RAW + JPEG) for each image
Pixel notes Sensor size 35mm (or larger) vs APS-C Pixel count ndash any increase rarr
decrease in pixel size or increase in sensor size Pixel size ndash smaller pixels give us
Greater resolution (finer details) - Good More ldquonoiserdquo (incorrect values) - bad Possible ldquoadjacencyrdquo issues - bad Loss of sensitivity - bad
copy DJ Foreman 2012 34
35
Some Comparisons
Digital media vs film Pixels are the same (RGB) Pixel size varies with device (mfgr
dependency) Pixel density varies Zooming digitally does NOT give you more
detail (in spite of what they do on TV shows)
Optical zoom is useful Uses optical magnification of the image on
the target Expansion of light rays not multiplication of
pixelscopy DJ Foreman 2012
Working Definitions - 1
Graphic Model data structure nodes represent random variables Pairs of nodes connected by arcs correspond
to variables that are not independent Graphic modeling
using a Graphic Model to simulate a real-world object (this is not a formal definition)
Rendering generating a 2D image from a 3D graphic model
copy DJ Foreman 2012 36
37
Working Definitions - 2
Geometric primitives ndash Points line segments and polygons Described by vertices
Control points ndash Special points attached to a node on a
Beacutezier curve Alter the shape and angle of adjacent curve
segment Evaluators ndash
Functions that interpolate a set of control points
Produce a new set of control pointscopy DJ Foreman 2012
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
Working with ImagesModeling
Containers
A container is a FILE format NOT a code scheme
Eg AVI amp WAV are container formats Formats for storage Independent of encoding or content
Others Ogg ASF QuickTime RealMedia Matroska
DivX and MP4
copy DJ Foreman 2012 27
28
Movie (metadata)
Track
Media
Media Information
Container file format (MP4QT)
Chunk map
Comp time
Chunk offset
Key frame
Desc
Decode time
Priority map
Size map
Sample Table
Media Handler
Data information
Media header
Video header
Track header
Movie header
copy DJ Foreman 2012
Pixels A pixel is 3 color-dots (squares) in a collection of dots
(squares) making up the picture Each color-dot is for one of the three primary colors
RGB Minimum of 1 byte per color (depending on bit-depth
which varies with camera) thus at least 3 bytes per pixel
A 10MP camera (30 million color-dots) needs 30M bytes This gets compressed by the camera (JPEG format) Final file size = 30 million bytes divided by the amount
of compression (compression ratio) An 81 compression ratio would reduce that 30 million
bytes to 375 megabytes (as done on a 4 MP camera) A RAW file has NO compression
copy DJ Foreman 2012 29
Image creation
Basic mechanisms Photography
Film Digital ndashPhotoshop Gimp
Drawing Line BrushModeling
Do it yourself OpenGL (2x and 3x)
Modelingrendering programs Breeze designer Ayam 3DCanvas Makehuman Alice
copy DJ Foreman 2012 30
Photography (film) Silver halide crystals (AgCl AgBr AgI)
Exposure to light turns them black Developer removes the loose (exposed) halide leaving
pure silver (a negative image) Fixing bath flushes out unexposed AgHalide
Greatest desire of photographers - a medium with High speed (gathers light quickly) Low noise (no undesired artifacts) High resolution (very detailed images)
Visible lines per mm (lpm) at a specified contrast level
Notes Larger ldquoclumpsrdquo are faster but limit resolution Smaller ldquoclumpsrdquo are hard to coat evenly Randomness of coating prevents Moireacute patternshttpenwikipediaorgwikiMoirC3A9_pattern
copy DJ Foreman 2012 31
FYI ndash a little chemistry
Ag+ X- + hv rarr Ag+ + X + e-
1 Radiation frees the Br2 1 Ag ion + 1 electron (Ag+ + e- rarr Ag0) -gt 1 Ag atom3 Neighbors are now ldquosensitizedrdquo 4 Developing (Reducing) converts exposed AgX to
black metallic Ag and free X5 Remaining AgX and free X are washed away (after
mixing with thiosulfate to make it soluble)
This is one silver (Ag) ldquograinrdquo in the ldquolatent imagerdquo X is any halide (Br Cl etc)
copy DJ Foreman 2012 32
crystal
Radiation (light)
Resolution 35 mm film for comparison
A 36 x 24 mm frame of ISO 100-speed film contains (asymp) the equivalent of 20 million pixels 1 There are NO pixels in film
Full-frame digital (36 x 24mm = 32 image ratio) Canon EOS 5D Nikon D3 (21MP) - $5K2
Medium format digital (6 x 45cm) Phase One P40+ (39MP) ($22K)
APS-C sized sensor (asymp24x16mm = 32 image ratio) (note 2009 data below)
Nikon D90 (123MP) - $900 Canon Rebel (15MP) $700 Used in most DSLRrsquos (note image ratio 32)
Other SONY Alpha (note image ratio 43)
1 Langford Michael Basic Photography (7th Ed 2000) Oxford Focal Press ISBN 0 240 51592 7
2 Prices as of 2009 given for comparison purposes onlycopy DJ Foreman 2012 33
Photography (digital) Image capture formats
RAW ndash 3 separate images (RGB) [12 bitspixel (4288 2848)]8=18318336 bytes per
picture JPEG ndash compressed (16x 8x 4x) Some cameras do BOTH (RAW + JPEG) for each image
Pixel notes Sensor size 35mm (or larger) vs APS-C Pixel count ndash any increase rarr
decrease in pixel size or increase in sensor size Pixel size ndash smaller pixels give us
Greater resolution (finer details) - Good More ldquonoiserdquo (incorrect values) - bad Possible ldquoadjacencyrdquo issues - bad Loss of sensitivity - bad
copy DJ Foreman 2012 34
35
Some Comparisons
Digital media vs film Pixels are the same (RGB) Pixel size varies with device (mfgr
dependency) Pixel density varies Zooming digitally does NOT give you more
detail (in spite of what they do on TV shows)
Optical zoom is useful Uses optical magnification of the image on
the target Expansion of light rays not multiplication of
pixelscopy DJ Foreman 2012
Working Definitions - 1
Graphic Model data structure nodes represent random variables Pairs of nodes connected by arcs correspond
to variables that are not independent Graphic modeling
using a Graphic Model to simulate a real-world object (this is not a formal definition)
Rendering generating a 2D image from a 3D graphic model
copy DJ Foreman 2012 36
37
Working Definitions - 2
Geometric primitives ndash Points line segments and polygons Described by vertices
Control points ndash Special points attached to a node on a
Beacutezier curve Alter the shape and angle of adjacent curve
segment Evaluators ndash
Functions that interpolate a set of control points
Produce a new set of control pointscopy DJ Foreman 2012
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
Containers
A container is a FILE format NOT a code scheme
Eg AVI amp WAV are container formats Formats for storage Independent of encoding or content
Others Ogg ASF QuickTime RealMedia Matroska
DivX and MP4
copy DJ Foreman 2012 27
28
Movie (metadata)
Track
Media
Media Information
Container file format (MP4QT)
Chunk map
Comp time
Chunk offset
Key frame
Desc
Decode time
Priority map
Size map
Sample Table
Media Handler
Data information
Media header
Video header
Track header
Movie header
copy DJ Foreman 2012
Pixels A pixel is 3 color-dots (squares) in a collection of dots
(squares) making up the picture Each color-dot is for one of the three primary colors
RGB Minimum of 1 byte per color (depending on bit-depth
which varies with camera) thus at least 3 bytes per pixel
A 10MP camera (30 million color-dots) needs 30M bytes This gets compressed by the camera (JPEG format) Final file size = 30 million bytes divided by the amount
of compression (compression ratio) An 81 compression ratio would reduce that 30 million
bytes to 375 megabytes (as done on a 4 MP camera) A RAW file has NO compression
copy DJ Foreman 2012 29
Image creation
Basic mechanisms Photography
Film Digital ndashPhotoshop Gimp
Drawing Line BrushModeling
Do it yourself OpenGL (2x and 3x)
Modelingrendering programs Breeze designer Ayam 3DCanvas Makehuman Alice
copy DJ Foreman 2012 30
Photography (film) Silver halide crystals (AgCl AgBr AgI)
Exposure to light turns them black Developer removes the loose (exposed) halide leaving
pure silver (a negative image) Fixing bath flushes out unexposed AgHalide
Greatest desire of photographers - a medium with High speed (gathers light quickly) Low noise (no undesired artifacts) High resolution (very detailed images)
Visible lines per mm (lpm) at a specified contrast level
Notes Larger ldquoclumpsrdquo are faster but limit resolution Smaller ldquoclumpsrdquo are hard to coat evenly Randomness of coating prevents Moireacute patternshttpenwikipediaorgwikiMoirC3A9_pattern
copy DJ Foreman 2012 31
FYI ndash a little chemistry
Ag+ X- + hv rarr Ag+ + X + e-
1 Radiation frees the Br2 1 Ag ion + 1 electron (Ag+ + e- rarr Ag0) -gt 1 Ag atom3 Neighbors are now ldquosensitizedrdquo 4 Developing (Reducing) converts exposed AgX to
black metallic Ag and free X5 Remaining AgX and free X are washed away (after
mixing with thiosulfate to make it soluble)
This is one silver (Ag) ldquograinrdquo in the ldquolatent imagerdquo X is any halide (Br Cl etc)
copy DJ Foreman 2012 32
crystal
Radiation (light)
Resolution 35 mm film for comparison
A 36 x 24 mm frame of ISO 100-speed film contains (asymp) the equivalent of 20 million pixels 1 There are NO pixels in film
Full-frame digital (36 x 24mm = 32 image ratio) Canon EOS 5D Nikon D3 (21MP) - $5K2
Medium format digital (6 x 45cm) Phase One P40+ (39MP) ($22K)
APS-C sized sensor (asymp24x16mm = 32 image ratio) (note 2009 data below)
Nikon D90 (123MP) - $900 Canon Rebel (15MP) $700 Used in most DSLRrsquos (note image ratio 32)
Other SONY Alpha (note image ratio 43)
1 Langford Michael Basic Photography (7th Ed 2000) Oxford Focal Press ISBN 0 240 51592 7
2 Prices as of 2009 given for comparison purposes onlycopy DJ Foreman 2012 33
Photography (digital) Image capture formats
RAW ndash 3 separate images (RGB) [12 bitspixel (4288 2848)]8=18318336 bytes per
picture JPEG ndash compressed (16x 8x 4x) Some cameras do BOTH (RAW + JPEG) for each image
Pixel notes Sensor size 35mm (or larger) vs APS-C Pixel count ndash any increase rarr
decrease in pixel size or increase in sensor size Pixel size ndash smaller pixels give us
Greater resolution (finer details) - Good More ldquonoiserdquo (incorrect values) - bad Possible ldquoadjacencyrdquo issues - bad Loss of sensitivity - bad
copy DJ Foreman 2012 34
35
Some Comparisons
Digital media vs film Pixels are the same (RGB) Pixel size varies with device (mfgr
dependency) Pixel density varies Zooming digitally does NOT give you more
detail (in spite of what they do on TV shows)
Optical zoom is useful Uses optical magnification of the image on
the target Expansion of light rays not multiplication of
pixelscopy DJ Foreman 2012
Working Definitions - 1
Graphic Model data structure nodes represent random variables Pairs of nodes connected by arcs correspond
to variables that are not independent Graphic modeling
using a Graphic Model to simulate a real-world object (this is not a formal definition)
Rendering generating a 2D image from a 3D graphic model
copy DJ Foreman 2012 36
37
Working Definitions - 2
Geometric primitives ndash Points line segments and polygons Described by vertices
Control points ndash Special points attached to a node on a
Beacutezier curve Alter the shape and angle of adjacent curve
segment Evaluators ndash
Functions that interpolate a set of control points
Produce a new set of control pointscopy DJ Foreman 2012
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
28
Movie (metadata)
Track
Media
Media Information
Container file format (MP4QT)
Chunk map
Comp time
Chunk offset
Key frame
Desc
Decode time
Priority map
Size map
Sample Table
Media Handler
Data information
Media header
Video header
Track header
Movie header
copy DJ Foreman 2012
Pixels A pixel is 3 color-dots (squares) in a collection of dots
(squares) making up the picture Each color-dot is for one of the three primary colors
RGB Minimum of 1 byte per color (depending on bit-depth
which varies with camera) thus at least 3 bytes per pixel
A 10MP camera (30 million color-dots) needs 30M bytes This gets compressed by the camera (JPEG format) Final file size = 30 million bytes divided by the amount
of compression (compression ratio) An 81 compression ratio would reduce that 30 million
bytes to 375 megabytes (as done on a 4 MP camera) A RAW file has NO compression
copy DJ Foreman 2012 29
Image creation
Basic mechanisms Photography
Film Digital ndashPhotoshop Gimp
Drawing Line BrushModeling
Do it yourself OpenGL (2x and 3x)
Modelingrendering programs Breeze designer Ayam 3DCanvas Makehuman Alice
copy DJ Foreman 2012 30
Photography (film) Silver halide crystals (AgCl AgBr AgI)
Exposure to light turns them black Developer removes the loose (exposed) halide leaving
pure silver (a negative image) Fixing bath flushes out unexposed AgHalide
Greatest desire of photographers - a medium with High speed (gathers light quickly) Low noise (no undesired artifacts) High resolution (very detailed images)
Visible lines per mm (lpm) at a specified contrast level
Notes Larger ldquoclumpsrdquo are faster but limit resolution Smaller ldquoclumpsrdquo are hard to coat evenly Randomness of coating prevents Moireacute patternshttpenwikipediaorgwikiMoirC3A9_pattern
copy DJ Foreman 2012 31
FYI ndash a little chemistry
Ag+ X- + hv rarr Ag+ + X + e-
1 Radiation frees the Br2 1 Ag ion + 1 electron (Ag+ + e- rarr Ag0) -gt 1 Ag atom3 Neighbors are now ldquosensitizedrdquo 4 Developing (Reducing) converts exposed AgX to
black metallic Ag and free X5 Remaining AgX and free X are washed away (after
mixing with thiosulfate to make it soluble)
This is one silver (Ag) ldquograinrdquo in the ldquolatent imagerdquo X is any halide (Br Cl etc)
copy DJ Foreman 2012 32
crystal
Radiation (light)
Resolution 35 mm film for comparison
A 36 x 24 mm frame of ISO 100-speed film contains (asymp) the equivalent of 20 million pixels 1 There are NO pixels in film
Full-frame digital (36 x 24mm = 32 image ratio) Canon EOS 5D Nikon D3 (21MP) - $5K2
Medium format digital (6 x 45cm) Phase One P40+ (39MP) ($22K)
APS-C sized sensor (asymp24x16mm = 32 image ratio) (note 2009 data below)
Nikon D90 (123MP) - $900 Canon Rebel (15MP) $700 Used in most DSLRrsquos (note image ratio 32)
Other SONY Alpha (note image ratio 43)
1 Langford Michael Basic Photography (7th Ed 2000) Oxford Focal Press ISBN 0 240 51592 7
2 Prices as of 2009 given for comparison purposes onlycopy DJ Foreman 2012 33
Photography (digital) Image capture formats
RAW ndash 3 separate images (RGB) [12 bitspixel (4288 2848)]8=18318336 bytes per
picture JPEG ndash compressed (16x 8x 4x) Some cameras do BOTH (RAW + JPEG) for each image
Pixel notes Sensor size 35mm (or larger) vs APS-C Pixel count ndash any increase rarr
decrease in pixel size or increase in sensor size Pixel size ndash smaller pixels give us
Greater resolution (finer details) - Good More ldquonoiserdquo (incorrect values) - bad Possible ldquoadjacencyrdquo issues - bad Loss of sensitivity - bad
copy DJ Foreman 2012 34
35
Some Comparisons
Digital media vs film Pixels are the same (RGB) Pixel size varies with device (mfgr
dependency) Pixel density varies Zooming digitally does NOT give you more
detail (in spite of what they do on TV shows)
Optical zoom is useful Uses optical magnification of the image on
the target Expansion of light rays not multiplication of
pixelscopy DJ Foreman 2012
Working Definitions - 1
Graphic Model data structure nodes represent random variables Pairs of nodes connected by arcs correspond
to variables that are not independent Graphic modeling
using a Graphic Model to simulate a real-world object (this is not a formal definition)
Rendering generating a 2D image from a 3D graphic model
copy DJ Foreman 2012 36
37
Working Definitions - 2
Geometric primitives ndash Points line segments and polygons Described by vertices
Control points ndash Special points attached to a node on a
Beacutezier curve Alter the shape and angle of adjacent curve
segment Evaluators ndash
Functions that interpolate a set of control points
Produce a new set of control pointscopy DJ Foreman 2012
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
Pixels A pixel is 3 color-dots (squares) in a collection of dots
(squares) making up the picture Each color-dot is for one of the three primary colors
RGB Minimum of 1 byte per color (depending on bit-depth
which varies with camera) thus at least 3 bytes per pixel
A 10MP camera (30 million color-dots) needs 30M bytes This gets compressed by the camera (JPEG format) Final file size = 30 million bytes divided by the amount
of compression (compression ratio) An 81 compression ratio would reduce that 30 million
bytes to 375 megabytes (as done on a 4 MP camera) A RAW file has NO compression
copy DJ Foreman 2012 29
Image creation
Basic mechanisms Photography
Film Digital ndashPhotoshop Gimp
Drawing Line BrushModeling
Do it yourself OpenGL (2x and 3x)
Modelingrendering programs Breeze designer Ayam 3DCanvas Makehuman Alice
copy DJ Foreman 2012 30
Photography (film) Silver halide crystals (AgCl AgBr AgI)
Exposure to light turns them black Developer removes the loose (exposed) halide leaving
pure silver (a negative image) Fixing bath flushes out unexposed AgHalide
Greatest desire of photographers - a medium with High speed (gathers light quickly) Low noise (no undesired artifacts) High resolution (very detailed images)
Visible lines per mm (lpm) at a specified contrast level
Notes Larger ldquoclumpsrdquo are faster but limit resolution Smaller ldquoclumpsrdquo are hard to coat evenly Randomness of coating prevents Moireacute patternshttpenwikipediaorgwikiMoirC3A9_pattern
copy DJ Foreman 2012 31
FYI ndash a little chemistry
Ag+ X- + hv rarr Ag+ + X + e-
1 Radiation frees the Br2 1 Ag ion + 1 electron (Ag+ + e- rarr Ag0) -gt 1 Ag atom3 Neighbors are now ldquosensitizedrdquo 4 Developing (Reducing) converts exposed AgX to
black metallic Ag and free X5 Remaining AgX and free X are washed away (after
mixing with thiosulfate to make it soluble)
This is one silver (Ag) ldquograinrdquo in the ldquolatent imagerdquo X is any halide (Br Cl etc)
copy DJ Foreman 2012 32
crystal
Radiation (light)
Resolution 35 mm film for comparison
A 36 x 24 mm frame of ISO 100-speed film contains (asymp) the equivalent of 20 million pixels 1 There are NO pixels in film
Full-frame digital (36 x 24mm = 32 image ratio) Canon EOS 5D Nikon D3 (21MP) - $5K2
Medium format digital (6 x 45cm) Phase One P40+ (39MP) ($22K)
APS-C sized sensor (asymp24x16mm = 32 image ratio) (note 2009 data below)
Nikon D90 (123MP) - $900 Canon Rebel (15MP) $700 Used in most DSLRrsquos (note image ratio 32)
Other SONY Alpha (note image ratio 43)
1 Langford Michael Basic Photography (7th Ed 2000) Oxford Focal Press ISBN 0 240 51592 7
2 Prices as of 2009 given for comparison purposes onlycopy DJ Foreman 2012 33
Photography (digital) Image capture formats
RAW ndash 3 separate images (RGB) [12 bitspixel (4288 2848)]8=18318336 bytes per
picture JPEG ndash compressed (16x 8x 4x) Some cameras do BOTH (RAW + JPEG) for each image
Pixel notes Sensor size 35mm (or larger) vs APS-C Pixel count ndash any increase rarr
decrease in pixel size or increase in sensor size Pixel size ndash smaller pixels give us
Greater resolution (finer details) - Good More ldquonoiserdquo (incorrect values) - bad Possible ldquoadjacencyrdquo issues - bad Loss of sensitivity - bad
copy DJ Foreman 2012 34
35
Some Comparisons
Digital media vs film Pixels are the same (RGB) Pixel size varies with device (mfgr
dependency) Pixel density varies Zooming digitally does NOT give you more
detail (in spite of what they do on TV shows)
Optical zoom is useful Uses optical magnification of the image on
the target Expansion of light rays not multiplication of
pixelscopy DJ Foreman 2012
Working Definitions - 1
Graphic Model data structure nodes represent random variables Pairs of nodes connected by arcs correspond
to variables that are not independent Graphic modeling
using a Graphic Model to simulate a real-world object (this is not a formal definition)
Rendering generating a 2D image from a 3D graphic model
copy DJ Foreman 2012 36
37
Working Definitions - 2
Geometric primitives ndash Points line segments and polygons Described by vertices
Control points ndash Special points attached to a node on a
Beacutezier curve Alter the shape and angle of adjacent curve
segment Evaluators ndash
Functions that interpolate a set of control points
Produce a new set of control pointscopy DJ Foreman 2012
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
Image creation
Basic mechanisms Photography
Film Digital ndashPhotoshop Gimp
Drawing Line BrushModeling
Do it yourself OpenGL (2x and 3x)
Modelingrendering programs Breeze designer Ayam 3DCanvas Makehuman Alice
copy DJ Foreman 2012 30
Photography (film) Silver halide crystals (AgCl AgBr AgI)
Exposure to light turns them black Developer removes the loose (exposed) halide leaving
pure silver (a negative image) Fixing bath flushes out unexposed AgHalide
Greatest desire of photographers - a medium with High speed (gathers light quickly) Low noise (no undesired artifacts) High resolution (very detailed images)
Visible lines per mm (lpm) at a specified contrast level
Notes Larger ldquoclumpsrdquo are faster but limit resolution Smaller ldquoclumpsrdquo are hard to coat evenly Randomness of coating prevents Moireacute patternshttpenwikipediaorgwikiMoirC3A9_pattern
copy DJ Foreman 2012 31
FYI ndash a little chemistry
Ag+ X- + hv rarr Ag+ + X + e-
1 Radiation frees the Br2 1 Ag ion + 1 electron (Ag+ + e- rarr Ag0) -gt 1 Ag atom3 Neighbors are now ldquosensitizedrdquo 4 Developing (Reducing) converts exposed AgX to
black metallic Ag and free X5 Remaining AgX and free X are washed away (after
mixing with thiosulfate to make it soluble)
This is one silver (Ag) ldquograinrdquo in the ldquolatent imagerdquo X is any halide (Br Cl etc)
copy DJ Foreman 2012 32
crystal
Radiation (light)
Resolution 35 mm film for comparison
A 36 x 24 mm frame of ISO 100-speed film contains (asymp) the equivalent of 20 million pixels 1 There are NO pixels in film
Full-frame digital (36 x 24mm = 32 image ratio) Canon EOS 5D Nikon D3 (21MP) - $5K2
Medium format digital (6 x 45cm) Phase One P40+ (39MP) ($22K)
APS-C sized sensor (asymp24x16mm = 32 image ratio) (note 2009 data below)
Nikon D90 (123MP) - $900 Canon Rebel (15MP) $700 Used in most DSLRrsquos (note image ratio 32)
Other SONY Alpha (note image ratio 43)
1 Langford Michael Basic Photography (7th Ed 2000) Oxford Focal Press ISBN 0 240 51592 7
2 Prices as of 2009 given for comparison purposes onlycopy DJ Foreman 2012 33
Photography (digital) Image capture formats
RAW ndash 3 separate images (RGB) [12 bitspixel (4288 2848)]8=18318336 bytes per
picture JPEG ndash compressed (16x 8x 4x) Some cameras do BOTH (RAW + JPEG) for each image
Pixel notes Sensor size 35mm (or larger) vs APS-C Pixel count ndash any increase rarr
decrease in pixel size or increase in sensor size Pixel size ndash smaller pixels give us
Greater resolution (finer details) - Good More ldquonoiserdquo (incorrect values) - bad Possible ldquoadjacencyrdquo issues - bad Loss of sensitivity - bad
copy DJ Foreman 2012 34
35
Some Comparisons
Digital media vs film Pixels are the same (RGB) Pixel size varies with device (mfgr
dependency) Pixel density varies Zooming digitally does NOT give you more
detail (in spite of what they do on TV shows)
Optical zoom is useful Uses optical magnification of the image on
the target Expansion of light rays not multiplication of
pixelscopy DJ Foreman 2012
Working Definitions - 1
Graphic Model data structure nodes represent random variables Pairs of nodes connected by arcs correspond
to variables that are not independent Graphic modeling
using a Graphic Model to simulate a real-world object (this is not a formal definition)
Rendering generating a 2D image from a 3D graphic model
copy DJ Foreman 2012 36
37
Working Definitions - 2
Geometric primitives ndash Points line segments and polygons Described by vertices
Control points ndash Special points attached to a node on a
Beacutezier curve Alter the shape and angle of adjacent curve
segment Evaluators ndash
Functions that interpolate a set of control points
Produce a new set of control pointscopy DJ Foreman 2012
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
Photography (film) Silver halide crystals (AgCl AgBr AgI)
Exposure to light turns them black Developer removes the loose (exposed) halide leaving
pure silver (a negative image) Fixing bath flushes out unexposed AgHalide
Greatest desire of photographers - a medium with High speed (gathers light quickly) Low noise (no undesired artifacts) High resolution (very detailed images)
Visible lines per mm (lpm) at a specified contrast level
Notes Larger ldquoclumpsrdquo are faster but limit resolution Smaller ldquoclumpsrdquo are hard to coat evenly Randomness of coating prevents Moireacute patternshttpenwikipediaorgwikiMoirC3A9_pattern
copy DJ Foreman 2012 31
FYI ndash a little chemistry
Ag+ X- + hv rarr Ag+ + X + e-
1 Radiation frees the Br2 1 Ag ion + 1 electron (Ag+ + e- rarr Ag0) -gt 1 Ag atom3 Neighbors are now ldquosensitizedrdquo 4 Developing (Reducing) converts exposed AgX to
black metallic Ag and free X5 Remaining AgX and free X are washed away (after
mixing with thiosulfate to make it soluble)
This is one silver (Ag) ldquograinrdquo in the ldquolatent imagerdquo X is any halide (Br Cl etc)
copy DJ Foreman 2012 32
crystal
Radiation (light)
Resolution 35 mm film for comparison
A 36 x 24 mm frame of ISO 100-speed film contains (asymp) the equivalent of 20 million pixels 1 There are NO pixels in film
Full-frame digital (36 x 24mm = 32 image ratio) Canon EOS 5D Nikon D3 (21MP) - $5K2
Medium format digital (6 x 45cm) Phase One P40+ (39MP) ($22K)
APS-C sized sensor (asymp24x16mm = 32 image ratio) (note 2009 data below)
Nikon D90 (123MP) - $900 Canon Rebel (15MP) $700 Used in most DSLRrsquos (note image ratio 32)
Other SONY Alpha (note image ratio 43)
1 Langford Michael Basic Photography (7th Ed 2000) Oxford Focal Press ISBN 0 240 51592 7
2 Prices as of 2009 given for comparison purposes onlycopy DJ Foreman 2012 33
Photography (digital) Image capture formats
RAW ndash 3 separate images (RGB) [12 bitspixel (4288 2848)]8=18318336 bytes per
picture JPEG ndash compressed (16x 8x 4x) Some cameras do BOTH (RAW + JPEG) for each image
Pixel notes Sensor size 35mm (or larger) vs APS-C Pixel count ndash any increase rarr
decrease in pixel size or increase in sensor size Pixel size ndash smaller pixels give us
Greater resolution (finer details) - Good More ldquonoiserdquo (incorrect values) - bad Possible ldquoadjacencyrdquo issues - bad Loss of sensitivity - bad
copy DJ Foreman 2012 34
35
Some Comparisons
Digital media vs film Pixels are the same (RGB) Pixel size varies with device (mfgr
dependency) Pixel density varies Zooming digitally does NOT give you more
detail (in spite of what they do on TV shows)
Optical zoom is useful Uses optical magnification of the image on
the target Expansion of light rays not multiplication of
pixelscopy DJ Foreman 2012
Working Definitions - 1
Graphic Model data structure nodes represent random variables Pairs of nodes connected by arcs correspond
to variables that are not independent Graphic modeling
using a Graphic Model to simulate a real-world object (this is not a formal definition)
Rendering generating a 2D image from a 3D graphic model
copy DJ Foreman 2012 36
37
Working Definitions - 2
Geometric primitives ndash Points line segments and polygons Described by vertices
Control points ndash Special points attached to a node on a
Beacutezier curve Alter the shape and angle of adjacent curve
segment Evaluators ndash
Functions that interpolate a set of control points
Produce a new set of control pointscopy DJ Foreman 2012
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
FYI ndash a little chemistry
Ag+ X- + hv rarr Ag+ + X + e-
1 Radiation frees the Br2 1 Ag ion + 1 electron (Ag+ + e- rarr Ag0) -gt 1 Ag atom3 Neighbors are now ldquosensitizedrdquo 4 Developing (Reducing) converts exposed AgX to
black metallic Ag and free X5 Remaining AgX and free X are washed away (after
mixing with thiosulfate to make it soluble)
This is one silver (Ag) ldquograinrdquo in the ldquolatent imagerdquo X is any halide (Br Cl etc)
copy DJ Foreman 2012 32
crystal
Radiation (light)
Resolution 35 mm film for comparison
A 36 x 24 mm frame of ISO 100-speed film contains (asymp) the equivalent of 20 million pixels 1 There are NO pixels in film
Full-frame digital (36 x 24mm = 32 image ratio) Canon EOS 5D Nikon D3 (21MP) - $5K2
Medium format digital (6 x 45cm) Phase One P40+ (39MP) ($22K)
APS-C sized sensor (asymp24x16mm = 32 image ratio) (note 2009 data below)
Nikon D90 (123MP) - $900 Canon Rebel (15MP) $700 Used in most DSLRrsquos (note image ratio 32)
Other SONY Alpha (note image ratio 43)
1 Langford Michael Basic Photography (7th Ed 2000) Oxford Focal Press ISBN 0 240 51592 7
2 Prices as of 2009 given for comparison purposes onlycopy DJ Foreman 2012 33
Photography (digital) Image capture formats
RAW ndash 3 separate images (RGB) [12 bitspixel (4288 2848)]8=18318336 bytes per
picture JPEG ndash compressed (16x 8x 4x) Some cameras do BOTH (RAW + JPEG) for each image
Pixel notes Sensor size 35mm (or larger) vs APS-C Pixel count ndash any increase rarr
decrease in pixel size or increase in sensor size Pixel size ndash smaller pixels give us
Greater resolution (finer details) - Good More ldquonoiserdquo (incorrect values) - bad Possible ldquoadjacencyrdquo issues - bad Loss of sensitivity - bad
copy DJ Foreman 2012 34
35
Some Comparisons
Digital media vs film Pixels are the same (RGB) Pixel size varies with device (mfgr
dependency) Pixel density varies Zooming digitally does NOT give you more
detail (in spite of what they do on TV shows)
Optical zoom is useful Uses optical magnification of the image on
the target Expansion of light rays not multiplication of
pixelscopy DJ Foreman 2012
Working Definitions - 1
Graphic Model data structure nodes represent random variables Pairs of nodes connected by arcs correspond
to variables that are not independent Graphic modeling
using a Graphic Model to simulate a real-world object (this is not a formal definition)
Rendering generating a 2D image from a 3D graphic model
copy DJ Foreman 2012 36
37
Working Definitions - 2
Geometric primitives ndash Points line segments and polygons Described by vertices
Control points ndash Special points attached to a node on a
Beacutezier curve Alter the shape and angle of adjacent curve
segment Evaluators ndash
Functions that interpolate a set of control points
Produce a new set of control pointscopy DJ Foreman 2012
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
Resolution 35 mm film for comparison
A 36 x 24 mm frame of ISO 100-speed film contains (asymp) the equivalent of 20 million pixels 1 There are NO pixels in film
Full-frame digital (36 x 24mm = 32 image ratio) Canon EOS 5D Nikon D3 (21MP) - $5K2
Medium format digital (6 x 45cm) Phase One P40+ (39MP) ($22K)
APS-C sized sensor (asymp24x16mm = 32 image ratio) (note 2009 data below)
Nikon D90 (123MP) - $900 Canon Rebel (15MP) $700 Used in most DSLRrsquos (note image ratio 32)
Other SONY Alpha (note image ratio 43)
1 Langford Michael Basic Photography (7th Ed 2000) Oxford Focal Press ISBN 0 240 51592 7
2 Prices as of 2009 given for comparison purposes onlycopy DJ Foreman 2012 33
Photography (digital) Image capture formats
RAW ndash 3 separate images (RGB) [12 bitspixel (4288 2848)]8=18318336 bytes per
picture JPEG ndash compressed (16x 8x 4x) Some cameras do BOTH (RAW + JPEG) for each image
Pixel notes Sensor size 35mm (or larger) vs APS-C Pixel count ndash any increase rarr
decrease in pixel size or increase in sensor size Pixel size ndash smaller pixels give us
Greater resolution (finer details) - Good More ldquonoiserdquo (incorrect values) - bad Possible ldquoadjacencyrdquo issues - bad Loss of sensitivity - bad
copy DJ Foreman 2012 34
35
Some Comparisons
Digital media vs film Pixels are the same (RGB) Pixel size varies with device (mfgr
dependency) Pixel density varies Zooming digitally does NOT give you more
detail (in spite of what they do on TV shows)
Optical zoom is useful Uses optical magnification of the image on
the target Expansion of light rays not multiplication of
pixelscopy DJ Foreman 2012
Working Definitions - 1
Graphic Model data structure nodes represent random variables Pairs of nodes connected by arcs correspond
to variables that are not independent Graphic modeling
using a Graphic Model to simulate a real-world object (this is not a formal definition)
Rendering generating a 2D image from a 3D graphic model
copy DJ Foreman 2012 36
37
Working Definitions - 2
Geometric primitives ndash Points line segments and polygons Described by vertices
Control points ndash Special points attached to a node on a
Beacutezier curve Alter the shape and angle of adjacent curve
segment Evaluators ndash
Functions that interpolate a set of control points
Produce a new set of control pointscopy DJ Foreman 2012
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
Photography (digital) Image capture formats
RAW ndash 3 separate images (RGB) [12 bitspixel (4288 2848)]8=18318336 bytes per
picture JPEG ndash compressed (16x 8x 4x) Some cameras do BOTH (RAW + JPEG) for each image
Pixel notes Sensor size 35mm (or larger) vs APS-C Pixel count ndash any increase rarr
decrease in pixel size or increase in sensor size Pixel size ndash smaller pixels give us
Greater resolution (finer details) - Good More ldquonoiserdquo (incorrect values) - bad Possible ldquoadjacencyrdquo issues - bad Loss of sensitivity - bad
copy DJ Foreman 2012 34
35
Some Comparisons
Digital media vs film Pixels are the same (RGB) Pixel size varies with device (mfgr
dependency) Pixel density varies Zooming digitally does NOT give you more
detail (in spite of what they do on TV shows)
Optical zoom is useful Uses optical magnification of the image on
the target Expansion of light rays not multiplication of
pixelscopy DJ Foreman 2012
Working Definitions - 1
Graphic Model data structure nodes represent random variables Pairs of nodes connected by arcs correspond
to variables that are not independent Graphic modeling
using a Graphic Model to simulate a real-world object (this is not a formal definition)
Rendering generating a 2D image from a 3D graphic model
copy DJ Foreman 2012 36
37
Working Definitions - 2
Geometric primitives ndash Points line segments and polygons Described by vertices
Control points ndash Special points attached to a node on a
Beacutezier curve Alter the shape and angle of adjacent curve
segment Evaluators ndash
Functions that interpolate a set of control points
Produce a new set of control pointscopy DJ Foreman 2012
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
35
Some Comparisons
Digital media vs film Pixels are the same (RGB) Pixel size varies with device (mfgr
dependency) Pixel density varies Zooming digitally does NOT give you more
detail (in spite of what they do on TV shows)
Optical zoom is useful Uses optical magnification of the image on
the target Expansion of light rays not multiplication of
pixelscopy DJ Foreman 2012
Working Definitions - 1
Graphic Model data structure nodes represent random variables Pairs of nodes connected by arcs correspond
to variables that are not independent Graphic modeling
using a Graphic Model to simulate a real-world object (this is not a formal definition)
Rendering generating a 2D image from a 3D graphic model
copy DJ Foreman 2012 36
37
Working Definitions - 2
Geometric primitives ndash Points line segments and polygons Described by vertices
Control points ndash Special points attached to a node on a
Beacutezier curve Alter the shape and angle of adjacent curve
segment Evaluators ndash
Functions that interpolate a set of control points
Produce a new set of control pointscopy DJ Foreman 2012
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
Working Definitions - 1
Graphic Model data structure nodes represent random variables Pairs of nodes connected by arcs correspond
to variables that are not independent Graphic modeling
using a Graphic Model to simulate a real-world object (this is not a formal definition)
Rendering generating a 2D image from a 3D graphic model
copy DJ Foreman 2012 36
37
Working Definitions - 2
Geometric primitives ndash Points line segments and polygons Described by vertices
Control points ndash Special points attached to a node on a
Beacutezier curve Alter the shape and angle of adjacent curve
segment Evaluators ndash
Functions that interpolate a set of control points
Produce a new set of control pointscopy DJ Foreman 2012
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
37
Working Definitions - 2
Geometric primitives ndash Points line segments and polygons Described by vertices
Control points ndash Special points attached to a node on a
Beacutezier curve Alter the shape and angle of adjacent curve
segment Evaluators ndash
Functions that interpolate a set of control points
Produce a new set of control pointscopy DJ Foreman 2012
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
38
Renderman Interface Bytestream (RIB) copy Pixar
A standardized interface httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
A plaintext file created by modeling programs
Input to rendering programs
copy DJ Foreman 2012
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
Context
Context is the ldquoversionrdquo of OpenGL in use OpenGL 2x (current to 2009) OpenGL 30 (20 + deprecated functions) OpenGL 31 only newer graphics cards
Many old (lt= 30) functions are deprecated Many actions now done via shaders 1st create a context (like the chicken amp the egg)
create an old (eg 30) context activate it create the new context deactivate old context
Old context needed to create new one
copy DJ Foreman 2012 39
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
40
OpenGL 20 Fixed Function Pipeline
copy DJ Foreman 2012
Vertex Data
Frame Buffer
Vertex Processor
Fragment Processor
Pixel Data
Per-vertex operationsbull Eye-space
coordinatesbull Colorsbull Texture
coordinatesbull Fog coordinatesbull Point size
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
41
The Vertex Processor Lights object vertices V
Determine color to assign Pass color to pixel processor
This is then interpolated across the polygon
Transforms object vertices V Construct projection matrix
transform V from world space to camera space Transform V from camera space to screen space
Multiply vectors with matrices ie vectora column-vectorb
[x0 y0z0] [x1y1z1] x0x1 + y0y1 + z0z1
Has no information regarding connectivity Therefore canrsquot do back-face clipping
copy DJ Foreman 2012
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
42
OpenGL 20 Vertex Processor Pipeline
copy DJ Foreman 2012
Vertex Coordinates
Normal Vector
Color Values
Texture Coordinates
Fog Coordinates
Model View Matrix
Model View Matrix
Texture Matrix
Projection Matrix
Primitive Setup
Clipping
Vertex Shadersreplace everything in the dotted box
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
43
DirectX vs OpenGL
Direct3D designed as a hw
interface Application
manages hw Mostly for games
Opengl ldquomayrdquo be hardware
accelerated Implementation
manages hw Mostly for pro use
(more general-purpose)
copy DJ Foreman 2012
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
44
The Model-view Matrix
Used to transform vertices that enter the 3D API before they are rendered to the screen
Called the ldquoworld and viewrdquo matrix in DirectX
copy DJ Foreman 2012
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
45
Graphic fragment
The data needed to shade a pixel The data needed to test whether the
fragment survives Based on front-to-back relations
copy DJ Foreman 2012
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
46
Fragment Processing
copy DJ Foreman 2012
Color Summation
Per-pixel Fogging
Fragment Tests(survival etc)Frame Buffer
Texture Mapping
BitmapsPixelRectangles
From primitive setup
fragment shader replaces these
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
Lab 2- basic modeling Write a program using the OpenGL 20 interface
Create 3 figures Triangle (2D) Rectangle (2D) Irregular pentahedron (5-sided 3D figure) (a pyramid)
See httpenwikipediaorgwikiPentahedron for examples ANY one Vertex pinned at 000 Allowed to be non-equilateral
Draw all 3 axes (optional but HIGHLY recommended) The triangle is within the bounds of the rectangle
(or vice-versa) both objects in same plane different colors The pyramid must have different colors on all 5 sides Create movement controls for the pyramid
Cursor keys larrrarruarrdarr change camera location for world view L amp R keys rotate pyramid (CWCCW) about Z-axis Pinned pyramid Vertex STAYS at 000 The pyramid rotates the ldquocamerardquo position remains fixed
copy DJ Foreman 2012 47
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
48
Lab 2 grading
Point values Rectangle and triangle both exist
5 Pyramid exists 10 All have colors applied 5 View rotation (updown amp leftright) 10 pts
EACH Pyramid can be rotated (+ and -)
20 pts
copy DJ Foreman 2012
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
49
Some Examples for lab 2
copy DJ Foreman 2012
000
001
000
001
6-vertex pentahedronValid for lab
Still has 5 sides
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
OpenGL
PC Requirements OPENGL library - for manipulating the
model opengl32lib opengl32dll (comes with
WindowsXP amp 7) glu32dll ldquo gluh and glh ldquo
FreeGlut (newer ndash Open Source)
copy DJ Foreman 2012 50
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
51
Window Control
ONE of the following Glut - note the ldquotrdquo after the ldquoglurdquo
httpwwwopenglorgresourceslibrariesglutglut_downloadsphpwindows glut32dll rdquo glut32lib rdquo gluth
FreeGlut (newer ndash Open Source)httpfreeglutsourceforgenet freeglutdll freeglutlib gluth
gluth does a include of glh and gluhcopy DJ Foreman 2012
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
Programming in OpenGL
Define window Define objects Initialize ldquocallbackrdquo functions Initialize window Run the gl main loop
Useful tutorialhttpwwwthepcspycomreadopengl__introduction_and_getting_started
Note double underscore in name abovecopy DJ Foreman 2012 52
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
OpenGL Program Structure
Two parts1 Modeling coloring etc
Use standard C code with calls to OpenGL functions
Your program runs as subroutines WITHIN the OpenGL MainLoop
2 Callback functions Called by OpenGL from OUTSIDE your program Display Reshape
Purpose of ldquoregisteringrdquo callbacks passes a pointer to your functions allows OpenGL to call them
copy DJ Foreman 2012 53
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
Views Orthographic
All views are 2-dimensional Not good for games ignores the z-axis
Perspective See model from ldquocamera positionrdquo via
ldquoviewportrdquo
copy DJ Foreman 2012 54
cameraviewport
Φ
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
55
Image Rotation
Left-handed Right-handedtX2 + c tXY - sZ tXZ +
sY0 tX2 + c tXY + sZ tXZ - sY 0
tXY+sZ tY2 + c tYZ - sX 0 tXY-sZ tY2 + c tYZ + sX
0
tXZ - sY
tYZ + sX
tZ2 + c 0 tXY + sY
tYZ - sX tZ2 + c 0
0 0 0 1 0 0 0 1
copy DJ Foreman 2012
c = Cos (theta) s = Sin (theta)t = 1-Cos (theta) ltXYZgt is the unit vector for your arbitrary axis
By using the axisangle representation for your rotations it is possible to avoid gimbal lock
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
Program structure
Movement functions Shape defining functions (lines
polygons etc) Reshape function (for window re-sizing) Init function Display function Main (includes call to glutMainLoop)
NOTE gl glu amp glut prefixes on functionsBe careful
copy DJ Foreman 2012 56
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
57
GlutReshapeFunc
Transforms geometry for new window size
Used after every frame is drawn or after a re-size
void reshape (int w int h)
glViewport (0 0 w h)glMatrixMode (GL_PROJECTION)glLoadIdentity ( )glOrtho(hellip)glMatrixMode (GL_MODELVIEW)
copy DJ Foreman 2012
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
58
GL_Load_Identity
Replace current matrix with Identity matrix
Prevents applying multiple perspectives
copy DJ Foreman 2012
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
59
gluLookAt
Creates a viewing matrix set the camera position and orientation derived from
an eye point ndash where the viewer is located a reference point for the center of the scene an UP vector ndash maps the given vector TO the
y axis Describes any ldquotiltrdquo of the camera Defines the new value for ldquouprdquo as seen by the
viewer
copy DJ Foreman 2012
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
60
Matrix Modes
GL_PROJECTION sets up our camera GL_MODELVIEW for drawing GL_TEXTURE adjusts how textures are
drawn onto shapes
copy DJ Foreman 2012
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
61
Keyboard-related functions
glutSpecialFunc glutKeyboardFunc
copy DJ Foreman 2012
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
Basic Functions in ldquomainrdquo
Register your ldquocallbackrdquo functionsglutSpecialFunc (your function name)glutKeyboardFunc (your function name)display reshape
Run the programdj_init() your init routine (if any)glutMainLoop()
copy DJ Foreman 2012 62
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
63
Skeleton of program ndash slide 1include ltGLgluthgtinclude ltstdiohgtinclude ltmathhgtinclude ltstdlibhgtfloat exeyezthetavoid leftMotion (int x int y)void rightMotion (int x int y) GLfloat pvertices [][3]= and then their colors GLfloat pcolors[][3]=
now define the polygon that USES those vertices define any ONE face of the pyramid at a time
void pface(int a int b int c) pface is a name I made up for ldquopyramid facerdquo
3 vertices define a faceglBegin(GL_POLYGON)glEnd()
copy DJ Foreman 2012
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
64
Skeleton of program ndash slide 2void draw_pyramid() implement the faces of the figure
glPolygonMode (GL_FRONT_AND_BACK GL_FILL)
void draw_square()void draw_triangle() void draw_axes()
glPushMatrix ()glBegin(GL_LINES) as pairs of points (default action of GL_LINES)glEnd()glPopMatrix ()
copy DJ Foreman 2012
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
65
Skeleton of program ndash slide 3void dj_display() callback for glutDisplayFunc this is the function that draws the graphic object in the
pre-created window glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT) clear window
glLoadIdentity() eye at upgluLookAt(ex ey ez 000 010) draw_square()draw_triangle()draw_axes() last in code means display on topdraw_pyramid() now draw the pyramidglFlush()glutSwapBuffers ()
copy DJ Foreman 2012
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
66
Initialize the program ndash slide 4void dj_init() define the palette amp background
glutInitDisplayMode(GLUT_RGB | GLUT_DOUBLE)
glClearColor(00 00 00 00) set clear color to black
now specify that parameters will be RGB values amp opacity
glColor4f(00 04 05 05)
copy DJ Foreman 2012
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
67
Create a colored line ndash slide 5Void draw_a_line()
glPushMatrix ()glBegin(GL_LINES)
glColor3f(075 00 00) RGBglVertex3f(00 00 00) point 1 (xyz
coordinates) glVertex3f(150 00 00) point 2
glEnd()glPopMatrix ()
NOTE function names end in ldquo3frdquo
3=number of argumentsf=floating point values
copy DJ Foreman 2012
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
68
Skeleton of program ndash slide 6int main(int argc charargv[])
glutInit(ampargc argv) donrsquot leave this out
glutInitWindowSize(wdht)
glutInitWindowPosition(wxwy)
glutCreateWindow(DJs 1st) define name on window
glutSpecialFunc(myspecialkeys)
register the call-back functions with Glut
glutKeyboardFunc(mykey)
glutDisplayFunc(dj_display) more setup
glutReshapeFunc(dj_reshaper)
dj_init()
Some texts show init being called AFTER the glutDisplayFunc call
but it doesnt actually CAUSE display action it just sets up the environment
glutMainLoop() the only real executable stmts in program
return 0 (except if for etc)
copy DJ Foreman 2012
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
Basic Display Function
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
glLoadIdentity()
eye_point at(xyz) up (xyz)
gluLookAt (ex ey ez 000 010)
draw_trianglerectanglepyramid etc
draw_axes
glutSwapBuffers () if double-buffered back buffer moves to front at retrace Also does a glFlush
glFlush() only needed if glutSwapBuffers is not used
copy DJ Foreman 2012 69
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
openGL 31
Using shaders amp Vertex Array ObjectsglBindVertexArray(my_vao_ID[0])
select 1st VAOglDrawArrays(GL_TRIANGLES 0 3) draw
1st object note difference from glBegin (GL_POLYGON)
hellip Note name of out(put) variable in
vertex shader must be the same as in(put) variable in fragment shader ie vertex -gt fragment (as in original
pipeline)copy DJ Foreman 2012 70
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
71
The Vertex Shader
Transform vertexs virtual 3D position to 2D screen coordinate
Manipulate position color and texture coordinate
Output to Geometry shader
generate new graphics primitives from those at beginning of pipeline
Rasterizer
copy DJ Foreman 2012
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
Drawing
Last drawn object is in ldquoFRONTrdquo MUST set bit depth AND enable bit
depth test Rotation
Around a vector starting at 000 Use glRotate(Ѳ xyz)
Around any vector NOT anchored at 000Requires pre-multiplication of matrices with the
identity matrix
copy DJ Foreman 2012 72
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
73
Views
Orthographic view direction is perpendicular to projection
plane Matrix mode
Modelview Projection View
copy DJ Foreman 2012
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
74
The RenderMan Interface Bytestream
The RenderMan Interface is a standard interface between modeling programs and rendering programs capable of producing photorealistic quality images
httpwwwfaqsorgfaqsgraphicsrenderman-faq
httpwwwfaqsorgfaqsgraphicsrenderman-faqixzz0ggQ8591Y
RIB programs the target device (like PostScript does for printers)
copy DJ Foreman 2012
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
75
RIB files
The specification from Pixar httpsrendermanpixarcomproductsrispe
crispec_pdfRISpec3_2pdf
Course notes from SIGGRAPH conferences Siggraph 90 course 18 - The RenderMan
Interface amp Shading Language Siggraph 92 course 21 - Writing
RenderMan Shaders
copy DJ Foreman 2012
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
Image Manipulation
Photographic images Gimp Photoshop etc
Modeling OpenGL ndash for creating and lighting a model modeling programs
Rendering converting a 3D (possibly wireframe) model to
a 2D image then lighting and (possibly) shading it
Ray tracing another way of lighting amp shading an image very CPU intensive
copy DJ Foreman 2012 76
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
77
Ray tracing Method for calculating the path of waves
particles through a system Two distinct forms
Ray tracing (physics) which is used for analyzing optical and other systems
Ray tracing (graphics) Used to generate a 3D image
Trace the path of light through pixels in an image plane Simulate the effects of striking virtual objects
Simulate reflection refraction scattering and chromatic (color) aberration
Compute intensive Ray-casting + follow reflections refractions amp
shadows
copy DJ Foreman 2012
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
78
Light terms
Absorption ndash unreflected+refracted light
Reflection ndash change of direction at an interface eg airsolid Angle = angle of incidence
Refraction ndash change of direction through media eg airwater Angle depends on refractive indices of both
media
copy DJ Foreman 2012
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
79
Ray tracing methodology Rays start at viewpoint go through viewplane
Eliminates unused rays from source Rays that do not strike objects Rays that are blocked by objects closer to viewer
Alternative (from light source) is photon mapping Each ray tested for intersections with objects Rays can be computed in parallel Algorithms
Estimate the incoming light at the intersection Examine the material properties of the object Combine above to calculate final color of the pixel
copy DJ Foreman 2012
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
80
Basic Ray tracing algorithm - 1 Initiate a ray (from viewpoint)
shade_pixel (original ray) This is a recursive call
(see next slide)
Apply ldquofinalrdquo shade to pixel Flat shading ndash assumes mono-colored object Cook-Torrance ndash uses physical attributes of
object Lambertian shading (cosine method)
Point brightness based on normal vector at point and vector from point to light source
httpwwwcsuncedu~rademachxroads-RTRTarticlehtml
copy DJ Foreman 2012
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
81
Ray tracing algorithm - 2
Shade_pixel (ray) Find 1st intersection C1= color of intersection point If reflective
Create reflection ray C2=Shade_pixel (new ray)
If transparent Create refraction ray C3=Shade_pixel (new ray)
Return combined color value (C1 C2 C3)
copy DJ Foreman 2012
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
82
Final coloring
shade = light_vector bull normal_vector if ( shade lt 0 )
shade = 0 point_color = object_color
( ambient_coefficient + diffuse_coefficient shade )
copy DJ Foreman 2012
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
83
Breeze DesignerAyam - lab Create a model of NCC1701 in BD
Notes The ldquofrontrdquo view is actually the back view Perspective view is not accurate
Save your model (it will be a CAD file) Use the ldquoprint screenrdquo key (copy to clipboard) Open RIB with Notepad amp print it Paste into Word amp print it Export your model as a RIB file Open RIB file (use ldquoopenrdquo not ldquoimportrdquo) in
Ayam Click on ldquoRunrdquo Use print screen again
copy DJ Foreman 2012
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
84
GimpGnu Image Manipulation Program
copy DJ Foreman 2012
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
Operational concepts
3 windows Image ndash contains the actual results Tools ndash operators such as clone paint etc Layers ndash portions of the final image
Avoids need for changes to original Stackable Allows separation of collections of changes Allows complex changes to be applied
independently of other changes
copy DJ Foreman 2012 85
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
Gimp - lab
Load a digital landscape or cityscape photo Load a picture of yourself Select the image portion that is just yourself Extract the image of yourself ndash save it as a
JPG Insert it (inverted 90-180 deg) into the lsquoscape Save the new image as a new JPG file Send all 3 files (lsquoscape you merge) to the TA Grading
Image properly selected (20 pts) Image rotated (20 pts) Image properly overlayed (60 pts)
copy DJ Foreman 2012 86
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
Video games vs Movies
Elements to manipulate Content Perspective Lightingshading Sequence Coloring
Video game images Dynamic
Movie images Pre-determined
copy DJ Foreman 2012 87
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
Alice
Developed at CMU
Language-free programming
environment
Point amp click usage
Rapid prototyping
copy DJ Foreman 2012 88
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
89
Alice environment
List of ldquoworldrdquo objects Insert into methods Modify characteristics (length width etc)
List of world details Properties Methods (user created) Functions (built-in methods)
World window (results) Method builder
copy DJ Foreman 2012
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
90
Alice - Programming
Programmer Selects ldquoverbsrdquo from list
For While Etc
Applies values for properties Statement structure pre-defined
Blocks pre-defined Function calls for ifhellip thenhellip else
No need to learn a ldquolanguagerdquocopy DJ Foreman 2012
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
91
Alice Lab
Open any of the Alice worlds Apply the following rules
1 Display 3 articulated characters (A B and C)
2 A flips onto its head and rotates slowly (5 times)
3 B waves both of its arms slowly (5 times)4 C walks around A and B while (2 and 3
happen)5 C changes direction and walks around A
and B once (while 2 and 3 repeat)6 Stop
Timing size shape position are your choice
copy DJ Foreman 2012
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
STEGANOGRAPHY and LSB analysis
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
93
What is it
Hiding a message in plain sight Inside another message Original is called the ldquocoverrdquo
Presence of message is not obvious Can be done with text or graphics
copy DJ Foreman 2012
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
94
How is it done
Many algorithms for hiding a message Embedding is easier than detection Extraction is straight-forward if
you know there is a hidden message you know the algorithm used
Can be very complex to detect amp extract
copy DJ Foreman 2012
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
95
LSB embedding ndash using MatlabFreemat
Concept Change the least significant bit of a set of
bytes 1rarr0 and 0rarr1 For all bytes
Repeat for multiple sets of bytes Example with grayscale images
One pixel is 8 bits Change the last bit Changes that pixelrsquos shade VERY slightly Repeat for every pixel
copy DJ Foreman 2012
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
96
LSB embedding - algorithmEmbedding function Emb (using Matlab or Freelab syntax)
c = imread(lsquomy_decoy_imagebmprsquo) Grayscale cover image lsquobrsquo is a vector of m bits (secret message)
k = 1 k is an index into the message bitsfor i = 1 height for j = 1 width LSB = mod(c[i j] 2) if kgtm | LSB = b[k] m is msg length in BITS s[i j] = c[i j] else
s[i j] = c[i j] + b[k] ndash LSB end k = k + 1 endend
imwrite(s lsquostego_imagebmprsquo lsquobmprsquo) write new image ldquosrdquo to
diskcopy DJ Foreman 2012
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
97
imread
The return value ldquocrdquo is an array containing the image data If the file contains a grayscale image ldquocrdquo is
an M-by-N array If the file contains a truecolor image ldquocrdquo is
an M-by-N-by-3 array The class of ldquocrdquo depends on the bits-per-
sample of the image data rounded to the next byte boundary Eg imread returns 24-bit color data as an
array of uint8 data because the sample size for each color component is 8 bits
copy DJ Foreman 2012
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
98
LSB extraction - algorithm
Extraction function Ext (Matlab syntax)s = imread(lsquostego_imagebmprsquo) Grayscale stego imagek = 1for i = 1 height for j = 1 width if k m how do I set the value of m b[k] = mod(s[i j] 2) k = k + 1 end endend
b is the extracted secret message as a bit string
copy DJ Foreman 2012
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
99
Properties of LSB flipping
LSBflip(x) = x + 1 ndash 2(x mod 2)
FlipLSB(x) is idempotent eg LSBflip(LSBflip(x)) = x for all x
LSB flipping induces a permutation on 0 hellip 255
0 1 2 3 4 5 hellip 254 255
LSB flipping is ldquoasymmetricalrdquo (eg 3 may change to 2 but never
to 4)
| LSB(x) ndash x | = 1 for all x (embedding distortion is 1 per pixel)
copy DJ Foreman 2012
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
100
Augmented Reality
Addition of computer-generated imagery to a real-world view
Differs from ldquomediated realityrdquo Real-world modified (+ or -)
Real-time Adds interactivity to real world view
ldquoComputer visionrdquo Systems that extract info from images
Medicalphotovideo scanners ldquoObject recognitionrdquo
copy DJ Foreman 2012
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
101
AR usage
Robotics Machining Assembly
Human interaction Auditory amp visual cueing Machine control
copy DJ Foreman 2012
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
102
AR developments
Heads-up displays Visual data projected onto a transparent screen
Windshield cockpit etc Used in auto and military ground- amp air-craft
Head-mounted displays Device with glasses goggles etc Also used in VR environments
Incorporated technology GPS Optical amp wireless sensors Gyroscopes Accelerometers
copy DJ Foreman 2012
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
103
Adobe Flash
(Originally Macromedia Flash) Vector graphics Uses
Video in web pages (swf) Movies (swf) Video games (swf) Standalone video (flv or exe)
Used with a flash player Telephones and other web-enabled devices
copy DJ Foreman 2012
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
104
Flash - 2
Support Adobe Flash CSn Adobe Integrated Runtime
Installs (silently) with Adobe Reader
Usage in a web pageltobject data=movieswfrdquo
type=applicationx-shockwave-flash width=500rdquo height=500gt ltparam name=movie value=movieswf gt
ltobjectgt
copy DJ Foreman 2012
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
105
Virtual Reality Modeling Language VRML
Text file format URLs may be associated with graphical
components May have JS or Java attached Called ldquoworldsrdquo File extension =wrl Many modeling programs can output VRML Supported by OpenVRML Succeeded by X3D
copy DJ Foreman 2012
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
106
X3D
Extensible 3D Graphics ISO standard XML-based file format Successor to VRML Applications which can parse X3D files
Project Wonderland Blender Freewrl
copy DJ Foreman 2012
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
107
X3d-Samplex3d
copy DJ Foreman 2012
ltxml version=10 encoding=UTF-8gtltDOCTYPE X3D PUBLIC ISOWeb3DDTD X3D 32EN
httpwwwweb3dorgspecificationsx3d-32dtdgtltX3D profile=Interchangeldquo version=32 xmlnsxsd=httpwwww3org2001XMLSchema-instance xsdnoNamespaceSchemaLocation= httpwwwweb3dorgspecificationsx3d-32xsd gtltScenegt lt-- viewable using freewrl --gt ltShapegt ltIndexedFaceSet coordIndex=0 1 2gt ltCoordinate point=0 0 0 1 0 0 05 1 0gt ltIndexedFaceSetgt ltShapegtltScenegtltX3Dgt
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
DIGITAL AUDIO
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
109
Terms
Level ndash power or volume dB ndash decibel - signal strength ratio
0 dB is the human hearing threshold Perception is proportional to log10 values of
level Breathing = 10dB A whisper = 30dB Lawnmower = 85dB Snowmobile = 105dB Jet engine (running) = 130 dB Rock concert = 110 ndash 140 dB
Hearing damage starts at 85 dB Pain starts at 130dB
copy DJ Foreman 2012
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
110
Basic editing
Effects Filter ndash specific freq Equalize - volume Modulate (speed reverb) Time amp pitch ndash stretch change pitch Noise reduction ndash breathing background
crackles Delay amp echo ndash room effects Clip - cut segments or limit
volumefrequency Copymovepaste - segments
copy DJ Foreman 2012
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
111
Multi-track editing-1
copy DJ Foreman 2012
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
112
Multi-track editing-2
Editing leftright channels of stereo signal
Comparison (eg forensics) Merging Appending Re-arranging
copy DJ Foreman 2012
MP3 MID
113
Music creation
Midi ndash Musical Instrument Digital Interface Specifies musical instructions
Instrument to play Notes durations volumes Binary file much smaller than MP3 Two ways
1 From instrument Instrument connects to PC PC software creates filesheet music
2 From written instruction Use a GUI to define notes positions
copy DJ Foreman 2012
114
copy DJ Foreman 2012
DIGITAL VIDEO
115
TV Formats
Digital (ATSC) Over the Air (OTA) ndash full HD (1080P) Scrambled cable ndash full HD (1080P) standard definition (SD=480i) QAM ndash NOT scrambled ndash 720P to 1080P
Cable-box NOT required Includes ATSC and converted NTSC Limited availabilty
Analog (NTSC) Unscrambled cable (limited availability) VCR output Only in SD
copy DJ Foreman 2012
116
Recording
Digital Cameras SD card files - AVI format DVD ndashMPG-2 or -4 Tape ndash DV format requires real-time transfer
Digital TV tuners Receives pre-encoded (OTAcable) MPG-2 to
PC Eg SiliconDust HD Homerun
Analog TV tuners Receives NTSC QAM amp cableVCR analog Does hardware encoding to MPG-2
Eg Hauppauge WinTV HVR 2250 also does OTA digital
copy DJ Foreman 2012
117
Remote Playback Hardware
Boxee - pre-recorded formats + internet WDTV Live ndash pre-recorded formats + internet Sage TV ndash pre-recorded formats + Tuner scheduler
Supports only SageTV streaming receiver Bought by Google removed from market
Roku - pre-recorded formats + Apple HLS (HTTP Live TV streaming)
Xbox ndash WMV only PS3 - MPG-2 only Wii ndash flash only using Opera browser
Software Tversity
httpwwwmaximumpccomarticlestreamingpage=01 WMC ndashWindows 7 (gt=Home Premium)+ Xbox SageTV Hauppauge WinTV
copy DJ Foreman 2012
118
copy DJ Foreman 2012
DV EDITING
119
Terms Asset list - Collection of clips frames amp stills Preview
Editable version of show Allows playback control + editing
Timeline - Time-marks abovebelow path of frames
Footage ndash DV segments measured in time units Effects - Re-size crop tone color super-
imposing Transitions
Straight cut (no transition) Fade inout dissolve cross-dissolve etc
copy DJ Foreman 2012
120
DV Editing -1
Import clips frames stills Trim footage (recurring action)
Insertremove segments Re-arrange segments Does not alter original file
Linear edit Simple re-arranging insertion removal
Non-linear edit (next slide) Note next slide compare ALL edits to
original
copy DJ Foreman 2012
121
Non-linear editing original
Ripple (middle expands 1)
Rolling (middle expands 1) (following In changed)
Slip (middle InOut change)
Slide (Middle moves left 2) (Prev Out next In
change)
DV Editing - 2
copy DJ Foreman 2012
a cb d 4 5 76 C FED G
a cb d 4 5 76 C FED G8
a cb d 4 5 76 FED G8
a cb d 1 2 3 C FED G4
a b 4 5 76 A DCB E F G
122
DV Editing -3
Ripple ndash changes overall length Following edits maintain overall length
Rolling ndash shortenslengthens following clip by amount of insertiondeletion
Slip ndash only the edited clips In and Out points change
Slide ndashchanges prev Out next In
copy DJ Foreman 2012
123
DV formats VCD
VHS quality Plays anywhere 7480 minutes recording for 650700MB CDs
SVCD 3560 minutes Many standalone DVD players all CDDVD-ROM
CVD = lower resolution less noise than SVCD DVD
2 hrs of good quality video (437 GB)+ DolbyDTS audio
Blu-ray High quality video
copy DJ Foreman 2012
124
DVD types Singledouble-sided 437GB 875GB
DVDForum DVD ndash R non-rewritable DVD ndash RW re-writable
DVD+RW Alliance DVD + R non-rewritable DVD + RW re-writeable
DVD plusmn R DL Dual layer 159GB
DVD-RAM NOT player compatible asymp HDD
+ and ndash differences are old hw designscopy DJ Foreman 2012
125
DVD types -2
Blu-ray 25 GB per layer up to 2 layers
AVCHD (Panasonic SONY) For Hi-DEF camcorders
HD-DVD (MS Toshiba NEC Sanyo) discontinued in 2008
VHS (everyone) vs Betamax (SONY)
copy DJ Foreman 2012
- CS338
- Overview -1
- Topics (general order of coverage)
- Software we might use (all free)
- Image formats
- Bitmap formats
- Vector formats
- Compression
- Codecs
- Codecs - 2
- Codec References
- Considerations
- Lossless Encoding
- Methodology
- Huffman lab ndash more details
- Sample of prefix coding amp compression
- Compression-Model types
- Standard Compressed-File Content
- Example 1 Header file for AU files
- Audio Encoding Enumerations
- Example 2 ZIP files (PkWare)
- More info on encoding
- Recording Bit-Rate
- Lab assignment ndash 1 ndash part 1
- Lab assignment 1 ndash part 2
- Working with Images
- Containers
- Container file format (MP4QT)
- Pixels
- Image creation
- Photography (film)
- FYI ndash a little chemistry
- Resolution
- Photography (digital)
- Some Comparisons
- Working Definitions - 1
- Working Definitions - 2
- Renderman Interface Bytestream (RIB) copy Pixar
- Context
- OpenGL 20 Fixed Function Pipeline
- The Vertex Processor
- OpenGL 20 Vertex Processor Pipeline
- DirectX vs OpenGL
- The Model-view Matrix
- Graphic fragment
- Fragment Processing
- Lab 2- basic modeling
- Lab 2 grading
- Some Examples for lab 2
- OpenGL
- Window Control
- Programming in OpenGL
- OpenGL Program Structure
- Views
- Image Rotation
- Program structure
- GlutReshapeFunc
- GL_Load_Identity
- gluLookAt
- Matrix Modes
- Keyboard-related functions
- Basic Functions in ldquomainrdquo
- Skeleton of program ndash slide 1
- Skeleton of program ndash slide 2
- Skeleton of program ndash slide 3
- Initialize the program ndash slide 4
- Create a colored line ndash slide 5
- Skeleton of program ndash slide 6
- Basic Display Function
- openGL 31
- The Vertex Shader
- Drawing
- Views (2)
- The RenderMan Interface Bytestream
- RIB files
- Image Manipulation
- Ray tracing
- Light terms
- Ray tracing methodology
- Basic Ray tracing algorithm - 1
- Ray tracing algorithm - 2
- Final coloring
- Breeze DesignerAyam - lab
- Gimp Gnu Image Manipulation Program
- Operational concepts
- Gimp - lab
- Video games vs Movies
- Alice
- Alice environment
- Alice - Programming
- Alice Lab
- Steganography and LSB analysis
- What is it
- How is it done
- LSB embedding ndash using MatlabFreemat
- LSB embedding - algorithm
- imread
- LSB extraction - algorithm
- Properties of LSB flipping
- Augmented Reality
- AR usage
- AR developments
- Adobe Flash
- Flash - 2
- Virtual Reality Modeling Language
- X3D
- X3d-Samplex3d
- Digital audio
- Terms
- Basic editing
- Multi-track editing-1
- Multi-track editing-2
- Music creation
- Digital Video
- TV Formats
- Recording
- Remote Playback
- DV Editing
- Terms
- DV Editing -1
- DV Editing - 2
- DV Editing -3
- DV formats
- DVD types
- DVD types -2
-
114
copy DJ Foreman 2012
DIGITAL VIDEO
115
TV Formats
Digital (ATSC) Over the Air (OTA) ndash full HD (1080P) Scrambled cable ndash full HD (1080P) standard definition (SD=480i) QAM ndash NOT scrambled ndash 720P to 1080P
Cable-box NOT required Includes ATSC and converted NTSC Limited availabilty
Analog (NTSC) Unscrambled cable (limited availability) VCR output Only in SD
copy DJ Foreman 2012
116
Recording
Digital Cameras SD card files - AVI format DVD ndashMPG-2 or -4 Tape ndash DV format requires real-time transfer
Digital TV tuners Receives pre-encoded (OTAcable) MPG-2 to
PC Eg SiliconDust HD Homerun
Analog TV tuners Receives NTSC QAM amp cableVCR analog Does hardware encoding to MPG-2
Eg Hauppauge WinTV HVR 2250 also does OTA digital
copy DJ Foreman 2012
117
Remote Playback Hardware
Boxee - pre-recorded formats + internet WDTV Live ndash pre-recorded formats + internet Sage TV ndash pre-recorded formats + Tuner scheduler
Supports only SageTV streaming receiver Bought by Google removed from market
Roku - pre-recorded formats + Apple HLS (HTTP Live TV streaming)
Xbox ndash WMV only PS3 - MPG-2 only Wii ndash flash only using Opera browser
Software Tversity
httpwwwmaximumpccomarticlestreamingpage=01 WMC ndashWindows 7 (gt=Home Premium)+ Xbox SageTV Hauppauge WinTV
copy DJ Foreman 2012
118
copy DJ Foreman 2012
DV EDITING
119
Terms Asset list - Collection of clips frames amp stills Preview
Editable version of show Allows playback control + editing
Timeline - Time-marks abovebelow path of frames
Footage ndash DV segments measured in time units Effects - Re-size crop tone color super-
imposing Transitions
Straight cut (no transition) Fade inout dissolve cross-dissolve etc
copy DJ Foreman 2012
120
DV Editing -1
Import clips frames stills Trim footage (recurring action)
Insertremove segments Re-arrange segments Does not alter original file
Linear edit Simple re-arranging insertion removal
Non-linear edit (next slide) Note next slide compare ALL edits to
original
copy DJ Foreman 2012
121
Non-linear editing original
Ripple (middle expands 1)
Rolling (middle expands 1) (following In changed)
Slip (middle InOut change)
Slide (Middle moves left 2) (Prev Out next In
change)
DV Editing - 2
copy DJ Foreman 2012
a cb d 4 5 76 C FED G
a cb d 4 5 76 C FED G8
a cb d 4 5 76 FED G8
a cb d 1 2 3 C FED G4
a b 4 5 76 A DCB E F G
122
DV Editing -3
Ripple ndash changes overall length Following edits maintain overall length
Rolling ndash shortenslengthens following clip by amount of insertiondeletion
Slip ndash only the edited clips In and Out points change
Slide ndashchanges prev Out next In
copy DJ Foreman 2012
123
DV formats VCD
VHS quality Plays anywhere 7480 minutes recording for 650700MB CDs
SVCD 3560 minutes Many standalone DVD players all CDDVD-ROM
CVD = lower resolution less noise than SVCD DVD
2 hrs of good quality video (437 GB)+ DolbyDTS audio
Blu-ray High quality video
copy DJ Foreman 2012
124
DVD types Singledouble-sided 437GB 875GB
DVDForum DVD ndash R non-rewritable DVD ndash RW re-writable
DVD+RW Alliance DVD + R non-rewritable DVD + RW re-writeable
DVD plusmn R DL Dual layer 159GB
DVD-RAM NOT player compatible asymp HDD
+ and ndash differences are old hw designscopy DJ Foreman 2012
125
DVD types -2
Blu-ray 25 GB per layer up to 2 layers
AVCHD (Panasonic SONY) For Hi-DEF camcorders
HD-DVD (MS Toshiba NEC Sanyo) discontinued in 2008
VHS (everyone) vs Betamax (SONY)
copy DJ Foreman 2012
- CS338
- Overview -1
- Topics (general order of coverage)
- Software we might use (all free)
- Image formats
- Bitmap formats
- Vector formats
- Compression
- Codecs
- Codecs - 2
- Codec References
- Considerations
- Lossless Encoding
- Methodology
- Huffman lab ndash more details
- Sample of prefix coding amp compression
- Compression-Model types
- Standard Compressed-File Content
- Example 1 Header file for AU files
- Audio Encoding Enumerations
- Example 2 ZIP files (PkWare)
- More info on encoding
- Recording Bit-Rate
- Lab assignment ndash 1 ndash part 1
- Lab assignment 1 ndash part 2
- Working with Images
- Containers
- Container file format (MP4QT)
- Pixels
- Image creation
- Photography (film)
- FYI ndash a little chemistry
- Resolution
- Photography (digital)
- Some Comparisons
- Working Definitions - 1
- Working Definitions - 2
- Renderman Interface Bytestream (RIB) copy Pixar
- Context
- OpenGL 20 Fixed Function Pipeline
- The Vertex Processor
- OpenGL 20 Vertex Processor Pipeline
- DirectX vs OpenGL
- The Model-view Matrix
- Graphic fragment
- Fragment Processing
- Lab 2- basic modeling
- Lab 2 grading
- Some Examples for lab 2
- OpenGL
- Window Control
- Programming in OpenGL
- OpenGL Program Structure
- Views
- Image Rotation
- Program structure
- GlutReshapeFunc
- GL_Load_Identity
- gluLookAt
- Matrix Modes
- Keyboard-related functions
- Basic Functions in ldquomainrdquo
- Skeleton of program ndash slide 1
- Skeleton of program ndash slide 2
- Skeleton of program ndash slide 3
- Initialize the program ndash slide 4
- Create a colored line ndash slide 5
- Skeleton of program ndash slide 6
- Basic Display Function
- openGL 31
- The Vertex Shader
- Drawing
- Views (2)
- The RenderMan Interface Bytestream
- RIB files
- Image Manipulation
- Ray tracing
- Light terms
- Ray tracing methodology
- Basic Ray tracing algorithm - 1
- Ray tracing algorithm - 2
- Final coloring
- Breeze DesignerAyam - lab
- Gimp Gnu Image Manipulation Program
- Operational concepts
- Gimp - lab
- Video games vs Movies
- Alice
- Alice environment
- Alice - Programming
- Alice Lab
- Steganography and LSB analysis
- What is it
- How is it done
- LSB embedding ndash using MatlabFreemat
- LSB embedding - algorithm
- imread
- LSB extraction - algorithm
- Properties of LSB flipping
- Augmented Reality
- AR usage
- AR developments
- Adobe Flash
- Flash - 2
- Virtual Reality Modeling Language
- X3D
- X3d-Samplex3d
- Digital audio
- Terms
- Basic editing
- Multi-track editing-1
- Multi-track editing-2
- Music creation
- Digital Video
- TV Formats
- Recording
- Remote Playback
- DV Editing
- Terms
- DV Editing -1
- DV Editing - 2
- DV Editing -3
- DV formats
- DVD types
- DVD types -2
-
115
TV Formats
Digital (ATSC) Over the Air (OTA) ndash full HD (1080P) Scrambled cable ndash full HD (1080P) standard definition (SD=480i) QAM ndash NOT scrambled ndash 720P to 1080P
Cable-box NOT required Includes ATSC and converted NTSC Limited availabilty
Analog (NTSC) Unscrambled cable (limited availability) VCR output Only in SD
copy DJ Foreman 2012
116
Recording
Digital Cameras SD card files - AVI format DVD ndashMPG-2 or -4 Tape ndash DV format requires real-time transfer
Digital TV tuners Receives pre-encoded (OTAcable) MPG-2 to
PC Eg SiliconDust HD Homerun
Analog TV tuners Receives NTSC QAM amp cableVCR analog Does hardware encoding to MPG-2
Eg Hauppauge WinTV HVR 2250 also does OTA digital
copy DJ Foreman 2012
117
Remote Playback Hardware
Boxee - pre-recorded formats + internet WDTV Live ndash pre-recorded formats + internet Sage TV ndash pre-recorded formats + Tuner scheduler
Supports only SageTV streaming receiver Bought by Google removed from market
Roku - pre-recorded formats + Apple HLS (HTTP Live TV streaming)
Xbox ndash WMV only PS3 - MPG-2 only Wii ndash flash only using Opera browser
Software Tversity
httpwwwmaximumpccomarticlestreamingpage=01 WMC ndashWindows 7 (gt=Home Premium)+ Xbox SageTV Hauppauge WinTV
copy DJ Foreman 2012
118
copy DJ Foreman 2012
DV EDITING
119
Terms Asset list - Collection of clips frames amp stills Preview
Editable version of show Allows playback control + editing
Timeline - Time-marks abovebelow path of frames
Footage ndash DV segments measured in time units Effects - Re-size crop tone color super-
imposing Transitions
Straight cut (no transition) Fade inout dissolve cross-dissolve etc
copy DJ Foreman 2012
120
DV Editing -1
Import clips frames stills Trim footage (recurring action)
Insertremove segments Re-arrange segments Does not alter original file
Linear edit Simple re-arranging insertion removal
Non-linear edit (next slide) Note next slide compare ALL edits to
original
copy DJ Foreman 2012
121
Non-linear editing original
Ripple (middle expands 1)
Rolling (middle expands 1) (following In changed)
Slip (middle InOut change)
Slide (Middle moves left 2) (Prev Out next In
change)
DV Editing - 2
copy DJ Foreman 2012
a cb d 4 5 76 C FED G
a cb d 4 5 76 C FED G8
a cb d 4 5 76 FED G8
a cb d 1 2 3 C FED G4
a b 4 5 76 A DCB E F G
122
DV Editing -3
Ripple ndash changes overall length Following edits maintain overall length
Rolling ndash shortenslengthens following clip by amount of insertiondeletion
Slip ndash only the edited clips In and Out points change
Slide ndashchanges prev Out next In
copy DJ Foreman 2012
123
DV formats VCD
VHS quality Plays anywhere 7480 minutes recording for 650700MB CDs
SVCD 3560 minutes Many standalone DVD players all CDDVD-ROM
CVD = lower resolution less noise than SVCD DVD
2 hrs of good quality video (437 GB)+ DolbyDTS audio
Blu-ray High quality video
copy DJ Foreman 2012
124
DVD types Singledouble-sided 437GB 875GB
DVDForum DVD ndash R non-rewritable DVD ndash RW re-writable
DVD+RW Alliance DVD + R non-rewritable DVD + RW re-writeable
DVD plusmn R DL Dual layer 159GB
DVD-RAM NOT player compatible asymp HDD
+ and ndash differences are old hw designscopy DJ Foreman 2012
125
DVD types -2
Blu-ray 25 GB per layer up to 2 layers
AVCHD (Panasonic SONY) For Hi-DEF camcorders
HD-DVD (MS Toshiba NEC Sanyo) discontinued in 2008
VHS (everyone) vs Betamax (SONY)
copy DJ Foreman 2012
- CS338
- Overview -1
- Topics (general order of coverage)
- Software we might use (all free)
- Image formats
- Bitmap formats
- Vector formats
- Compression
- Codecs
- Codecs - 2
- Codec References
- Considerations
- Lossless Encoding
- Methodology
- Huffman lab ndash more details
- Sample of prefix coding amp compression
- Compression-Model types
- Standard Compressed-File Content
- Example 1 Header file for AU files
- Audio Encoding Enumerations
- Example 2 ZIP files (PkWare)
- More info on encoding
- Recording Bit-Rate
- Lab assignment ndash 1 ndash part 1
- Lab assignment 1 ndash part 2
- Working with Images
- Containers
- Container file format (MP4QT)
- Pixels
- Image creation
- Photography (film)
- FYI ndash a little chemistry
- Resolution
- Photography (digital)
- Some Comparisons
- Working Definitions - 1
- Working Definitions - 2
- Renderman Interface Bytestream (RIB) copy Pixar
- Context
- OpenGL 20 Fixed Function Pipeline
- The Vertex Processor
- OpenGL 20 Vertex Processor Pipeline
- DirectX vs OpenGL
- The Model-view Matrix
- Graphic fragment
- Fragment Processing
- Lab 2- basic modeling
- Lab 2 grading
- Some Examples for lab 2
- OpenGL
- Window Control
- Programming in OpenGL
- OpenGL Program Structure
- Views
- Image Rotation
- Program structure
- GlutReshapeFunc
- GL_Load_Identity
- gluLookAt
- Matrix Modes
- Keyboard-related functions
- Basic Functions in ldquomainrdquo
- Skeleton of program ndash slide 1
- Skeleton of program ndash slide 2
- Skeleton of program ndash slide 3
- Initialize the program ndash slide 4
- Create a colored line ndash slide 5
- Skeleton of program ndash slide 6
- Basic Display Function
- openGL 31
- The Vertex Shader
- Drawing
- Views (2)
- The RenderMan Interface Bytestream
- RIB files
- Image Manipulation
- Ray tracing
- Light terms
- Ray tracing methodology
- Basic Ray tracing algorithm - 1
- Ray tracing algorithm - 2
- Final coloring
- Breeze DesignerAyam - lab
- Gimp Gnu Image Manipulation Program
- Operational concepts
- Gimp - lab
- Video games vs Movies
- Alice
- Alice environment
- Alice - Programming
- Alice Lab
- Steganography and LSB analysis
- What is it
- How is it done
- LSB embedding ndash using MatlabFreemat
- LSB embedding - algorithm
- imread
- LSB extraction - algorithm
- Properties of LSB flipping
- Augmented Reality
- AR usage
- AR developments
- Adobe Flash
- Flash - 2
- Virtual Reality Modeling Language
- X3D
- X3d-Samplex3d
- Digital audio
- Terms
- Basic editing
- Multi-track editing-1
- Multi-track editing-2
- Music creation
- Digital Video
- TV Formats
- Recording
- Remote Playback
- DV Editing
- Terms
- DV Editing -1
- DV Editing - 2
- DV Editing -3
- DV formats
- DVD types
- DVD types -2
-
116
Recording
Digital Cameras SD card files - AVI format DVD ndashMPG-2 or -4 Tape ndash DV format requires real-time transfer
Digital TV tuners Receives pre-encoded (OTAcable) MPG-2 to
PC Eg SiliconDust HD Homerun
Analog TV tuners Receives NTSC QAM amp cableVCR analog Does hardware encoding to MPG-2
Eg Hauppauge WinTV HVR 2250 also does OTA digital
copy DJ Foreman 2012
117
Remote Playback Hardware
Boxee - pre-recorded formats + internet WDTV Live ndash pre-recorded formats + internet Sage TV ndash pre-recorded formats + Tuner scheduler
Supports only SageTV streaming receiver Bought by Google removed from market
Roku - pre-recorded formats + Apple HLS (HTTP Live TV streaming)
Xbox ndash WMV only PS3 - MPG-2 only Wii ndash flash only using Opera browser
Software Tversity
httpwwwmaximumpccomarticlestreamingpage=01 WMC ndashWindows 7 (gt=Home Premium)+ Xbox SageTV Hauppauge WinTV
copy DJ Foreman 2012
118
copy DJ Foreman 2012
DV EDITING
119
Terms Asset list - Collection of clips frames amp stills Preview
Editable version of show Allows playback control + editing
Timeline - Time-marks abovebelow path of frames
Footage ndash DV segments measured in time units Effects - Re-size crop tone color super-
imposing Transitions
Straight cut (no transition) Fade inout dissolve cross-dissolve etc
copy DJ Foreman 2012
120
DV Editing -1
Import clips frames stills Trim footage (recurring action)
Insertremove segments Re-arrange segments Does not alter original file
Linear edit Simple re-arranging insertion removal
Non-linear edit (next slide) Note next slide compare ALL edits to
original
copy DJ Foreman 2012
121
Non-linear editing original
Ripple (middle expands 1)
Rolling (middle expands 1) (following In changed)
Slip (middle InOut change)
Slide (Middle moves left 2) (Prev Out next In
change)
DV Editing - 2
copy DJ Foreman 2012
a cb d 4 5 76 C FED G
a cb d 4 5 76 C FED G8
a cb d 4 5 76 FED G8
a cb d 1 2 3 C FED G4
a b 4 5 76 A DCB E F G
122
DV Editing -3
Ripple ndash changes overall length Following edits maintain overall length
Rolling ndash shortenslengthens following clip by amount of insertiondeletion
Slip ndash only the edited clips In and Out points change
Slide ndashchanges prev Out next In
copy DJ Foreman 2012
123
DV formats VCD
VHS quality Plays anywhere 7480 minutes recording for 650700MB CDs
SVCD 3560 minutes Many standalone DVD players all CDDVD-ROM
CVD = lower resolution less noise than SVCD DVD
2 hrs of good quality video (437 GB)+ DolbyDTS audio
Blu-ray High quality video
copy DJ Foreman 2012
124
DVD types Singledouble-sided 437GB 875GB
DVDForum DVD ndash R non-rewritable DVD ndash RW re-writable
DVD+RW Alliance DVD + R non-rewritable DVD + RW re-writeable
DVD plusmn R DL Dual layer 159GB
DVD-RAM NOT player compatible asymp HDD
+ and ndash differences are old hw designscopy DJ Foreman 2012
125
DVD types -2
Blu-ray 25 GB per layer up to 2 layers
AVCHD (Panasonic SONY) For Hi-DEF camcorders
HD-DVD (MS Toshiba NEC Sanyo) discontinued in 2008
VHS (everyone) vs Betamax (SONY)
copy DJ Foreman 2012
- CS338
- Overview -1
- Topics (general order of coverage)
- Software we might use (all free)
- Image formats
- Bitmap formats
- Vector formats
- Compression
- Codecs
- Codecs - 2
- Codec References
- Considerations
- Lossless Encoding
- Methodology
- Huffman lab ndash more details
- Sample of prefix coding amp compression
- Compression-Model types
- Standard Compressed-File Content
- Example 1 Header file for AU files
- Audio Encoding Enumerations
- Example 2 ZIP files (PkWare)
- More info on encoding
- Recording Bit-Rate
- Lab assignment ndash 1 ndash part 1
- Lab assignment 1 ndash part 2
- Working with Images
- Containers
- Container file format (MP4QT)
- Pixels
- Image creation
- Photography (film)
- FYI ndash a little chemistry
- Resolution
- Photography (digital)
- Some Comparisons
- Working Definitions - 1
- Working Definitions - 2
- Renderman Interface Bytestream (RIB) copy Pixar
- Context
- OpenGL 20 Fixed Function Pipeline
- The Vertex Processor
- OpenGL 20 Vertex Processor Pipeline
- DirectX vs OpenGL
- The Model-view Matrix
- Graphic fragment
- Fragment Processing
- Lab 2- basic modeling
- Lab 2 grading
- Some Examples for lab 2
- OpenGL
- Window Control
- Programming in OpenGL
- OpenGL Program Structure
- Views
- Image Rotation
- Program structure
- GlutReshapeFunc
- GL_Load_Identity
- gluLookAt
- Matrix Modes
- Keyboard-related functions
- Basic Functions in ldquomainrdquo
- Skeleton of program ndash slide 1
- Skeleton of program ndash slide 2
- Skeleton of program ndash slide 3
- Initialize the program ndash slide 4
- Create a colored line ndash slide 5
- Skeleton of program ndash slide 6
- Basic Display Function
- openGL 31
- The Vertex Shader
- Drawing
- Views (2)
- The RenderMan Interface Bytestream
- RIB files
- Image Manipulation
- Ray tracing
- Light terms
- Ray tracing methodology
- Basic Ray tracing algorithm - 1
- Ray tracing algorithm - 2
- Final coloring
- Breeze DesignerAyam - lab
- Gimp Gnu Image Manipulation Program
- Operational concepts
- Gimp - lab
- Video games vs Movies
- Alice
- Alice environment
- Alice - Programming
- Alice Lab
- Steganography and LSB analysis
- What is it
- How is it done
- LSB embedding ndash using MatlabFreemat
- LSB embedding - algorithm
- imread
- LSB extraction - algorithm
- Properties of LSB flipping
- Augmented Reality
- AR usage
- AR developments
- Adobe Flash
- Flash - 2
- Virtual Reality Modeling Language
- X3D
- X3d-Samplex3d
- Digital audio
- Terms
- Basic editing
- Multi-track editing-1
- Multi-track editing-2
- Music creation
- Digital Video
- TV Formats
- Recording
- Remote Playback
- DV Editing
- Terms
- DV Editing -1
- DV Editing - 2
- DV Editing -3
- DV formats
- DVD types
- DVD types -2
-
117
Remote Playback Hardware
Boxee - pre-recorded formats + internet WDTV Live ndash pre-recorded formats + internet Sage TV ndash pre-recorded formats + Tuner scheduler
Supports only SageTV streaming receiver Bought by Google removed from market
Roku - pre-recorded formats + Apple HLS (HTTP Live TV streaming)
Xbox ndash WMV only PS3 - MPG-2 only Wii ndash flash only using Opera browser
Software Tversity
httpwwwmaximumpccomarticlestreamingpage=01 WMC ndashWindows 7 (gt=Home Premium)+ Xbox SageTV Hauppauge WinTV
copy DJ Foreman 2012
118
copy DJ Foreman 2012
DV EDITING
119
Terms Asset list - Collection of clips frames amp stills Preview
Editable version of show Allows playback control + editing
Timeline - Time-marks abovebelow path of frames
Footage ndash DV segments measured in time units Effects - Re-size crop tone color super-
imposing Transitions
Straight cut (no transition) Fade inout dissolve cross-dissolve etc
copy DJ Foreman 2012
120
DV Editing -1
Import clips frames stills Trim footage (recurring action)
Insertremove segments Re-arrange segments Does not alter original file
Linear edit Simple re-arranging insertion removal
Non-linear edit (next slide) Note next slide compare ALL edits to
original
copy DJ Foreman 2012
121
Non-linear editing original
Ripple (middle expands 1)
Rolling (middle expands 1) (following In changed)
Slip (middle InOut change)
Slide (Middle moves left 2) (Prev Out next In
change)
DV Editing - 2
copy DJ Foreman 2012
a cb d 4 5 76 C FED G
a cb d 4 5 76 C FED G8
a cb d 4 5 76 FED G8
a cb d 1 2 3 C FED G4
a b 4 5 76 A DCB E F G
122
DV Editing -3
Ripple ndash changes overall length Following edits maintain overall length
Rolling ndash shortenslengthens following clip by amount of insertiondeletion
Slip ndash only the edited clips In and Out points change
Slide ndashchanges prev Out next In
copy DJ Foreman 2012
123
DV formats VCD
VHS quality Plays anywhere 7480 minutes recording for 650700MB CDs
SVCD 3560 minutes Many standalone DVD players all CDDVD-ROM
CVD = lower resolution less noise than SVCD DVD
2 hrs of good quality video (437 GB)+ DolbyDTS audio
Blu-ray High quality video
copy DJ Foreman 2012
124
DVD types Singledouble-sided 437GB 875GB
DVDForum DVD ndash R non-rewritable DVD ndash RW re-writable
DVD+RW Alliance DVD + R non-rewritable DVD + RW re-writeable
DVD plusmn R DL Dual layer 159GB
DVD-RAM NOT player compatible asymp HDD
+ and ndash differences are old hw designscopy DJ Foreman 2012
125
DVD types -2
Blu-ray 25 GB per layer up to 2 layers
AVCHD (Panasonic SONY) For Hi-DEF camcorders
HD-DVD (MS Toshiba NEC Sanyo) discontinued in 2008
VHS (everyone) vs Betamax (SONY)
copy DJ Foreman 2012
- CS338
- Overview -1
- Topics (general order of coverage)
- Software we might use (all free)
- Image formats
- Bitmap formats
- Vector formats
- Compression
- Codecs
- Codecs - 2
- Codec References
- Considerations
- Lossless Encoding
- Methodology
- Huffman lab ndash more details
- Sample of prefix coding amp compression
- Compression-Model types
- Standard Compressed-File Content
- Example 1 Header file for AU files
- Audio Encoding Enumerations
- Example 2 ZIP files (PkWare)
- More info on encoding
- Recording Bit-Rate
- Lab assignment ndash 1 ndash part 1
- Lab assignment 1 ndash part 2
- Working with Images
- Containers
- Container file format (MP4QT)
- Pixels
- Image creation
- Photography (film)
- FYI ndash a little chemistry
- Resolution
- Photography (digital)
- Some Comparisons
- Working Definitions - 1
- Working Definitions - 2
- Renderman Interface Bytestream (RIB) copy Pixar
- Context
- OpenGL 20 Fixed Function Pipeline
- The Vertex Processor
- OpenGL 20 Vertex Processor Pipeline
- DirectX vs OpenGL
- The Model-view Matrix
- Graphic fragment
- Fragment Processing
- Lab 2- basic modeling
- Lab 2 grading
- Some Examples for lab 2
- OpenGL
- Window Control
- Programming in OpenGL
- OpenGL Program Structure
- Views
- Image Rotation
- Program structure
- GlutReshapeFunc
- GL_Load_Identity
- gluLookAt
- Matrix Modes
- Keyboard-related functions
- Basic Functions in ldquomainrdquo
- Skeleton of program ndash slide 1
- Skeleton of program ndash slide 2
- Skeleton of program ndash slide 3
- Initialize the program ndash slide 4
- Create a colored line ndash slide 5
- Skeleton of program ndash slide 6
- Basic Display Function
- openGL 31
- The Vertex Shader
- Drawing
- Views (2)
- The RenderMan Interface Bytestream
- RIB files
- Image Manipulation
- Ray tracing
- Light terms
- Ray tracing methodology
- Basic Ray tracing algorithm - 1
- Ray tracing algorithm - 2
- Final coloring
- Breeze DesignerAyam - lab
- Gimp Gnu Image Manipulation Program
- Operational concepts
- Gimp - lab
- Video games vs Movies
- Alice
- Alice environment
- Alice - Programming
- Alice Lab
- Steganography and LSB analysis
- What is it
- How is it done
- LSB embedding ndash using MatlabFreemat
- LSB embedding - algorithm
- imread
- LSB extraction - algorithm
- Properties of LSB flipping
- Augmented Reality
- AR usage
- AR developments
- Adobe Flash
- Flash - 2
- Virtual Reality Modeling Language
- X3D
- X3d-Samplex3d
- Digital audio
- Terms
- Basic editing
- Multi-track editing-1
- Multi-track editing-2
- Music creation
- Digital Video
- TV Formats
- Recording
- Remote Playback
- DV Editing
- Terms
- DV Editing -1
- DV Editing - 2
- DV Editing -3
- DV formats
- DVD types
- DVD types -2
-
118
copy DJ Foreman 2012
DV EDITING
119
Terms Asset list - Collection of clips frames amp stills Preview
Editable version of show Allows playback control + editing
Timeline - Time-marks abovebelow path of frames
Footage ndash DV segments measured in time units Effects - Re-size crop tone color super-
imposing Transitions
Straight cut (no transition) Fade inout dissolve cross-dissolve etc
copy DJ Foreman 2012
120
DV Editing -1
Import clips frames stills Trim footage (recurring action)
Insertremove segments Re-arrange segments Does not alter original file
Linear edit Simple re-arranging insertion removal
Non-linear edit (next slide) Note next slide compare ALL edits to
original
copy DJ Foreman 2012
121
Non-linear editing original
Ripple (middle expands 1)
Rolling (middle expands 1) (following In changed)
Slip (middle InOut change)
Slide (Middle moves left 2) (Prev Out next In
change)
DV Editing - 2
copy DJ Foreman 2012
a cb d 4 5 76 C FED G
a cb d 4 5 76 C FED G8
a cb d 4 5 76 FED G8
a cb d 1 2 3 C FED G4
a b 4 5 76 A DCB E F G
122
DV Editing -3
Ripple ndash changes overall length Following edits maintain overall length
Rolling ndash shortenslengthens following clip by amount of insertiondeletion
Slip ndash only the edited clips In and Out points change
Slide ndashchanges prev Out next In
copy DJ Foreman 2012
123
DV formats VCD
VHS quality Plays anywhere 7480 minutes recording for 650700MB CDs
SVCD 3560 minutes Many standalone DVD players all CDDVD-ROM
CVD = lower resolution less noise than SVCD DVD
2 hrs of good quality video (437 GB)+ DolbyDTS audio
Blu-ray High quality video
copy DJ Foreman 2012
124
DVD types Singledouble-sided 437GB 875GB
DVDForum DVD ndash R non-rewritable DVD ndash RW re-writable
DVD+RW Alliance DVD + R non-rewritable DVD + RW re-writeable
DVD plusmn R DL Dual layer 159GB
DVD-RAM NOT player compatible asymp HDD
+ and ndash differences are old hw designscopy DJ Foreman 2012
125
DVD types -2
Blu-ray 25 GB per layer up to 2 layers
AVCHD (Panasonic SONY) For Hi-DEF camcorders
HD-DVD (MS Toshiba NEC Sanyo) discontinued in 2008
VHS (everyone) vs Betamax (SONY)
copy DJ Foreman 2012
- CS338
- Overview -1
- Topics (general order of coverage)
- Software we might use (all free)
- Image formats
- Bitmap formats
- Vector formats
- Compression
- Codecs
- Codecs - 2
- Codec References
- Considerations
- Lossless Encoding
- Methodology
- Huffman lab ndash more details
- Sample of prefix coding amp compression
- Compression-Model types
- Standard Compressed-File Content
- Example 1 Header file for AU files
- Audio Encoding Enumerations
- Example 2 ZIP files (PkWare)
- More info on encoding
- Recording Bit-Rate
- Lab assignment ndash 1 ndash part 1
- Lab assignment 1 ndash part 2
- Working with Images
- Containers
- Container file format (MP4QT)
- Pixels
- Image creation
- Photography (film)
- FYI ndash a little chemistry
- Resolution
- Photography (digital)
- Some Comparisons
- Working Definitions - 1
- Working Definitions - 2
- Renderman Interface Bytestream (RIB) copy Pixar
- Context
- OpenGL 20 Fixed Function Pipeline
- The Vertex Processor
- OpenGL 20 Vertex Processor Pipeline
- DirectX vs OpenGL
- The Model-view Matrix
- Graphic fragment
- Fragment Processing
- Lab 2- basic modeling
- Lab 2 grading
- Some Examples for lab 2
- OpenGL
- Window Control
- Programming in OpenGL
- OpenGL Program Structure
- Views
- Image Rotation
- Program structure
- GlutReshapeFunc
- GL_Load_Identity
- gluLookAt
- Matrix Modes
- Keyboard-related functions
- Basic Functions in ldquomainrdquo
- Skeleton of program ndash slide 1
- Skeleton of program ndash slide 2
- Skeleton of program ndash slide 3
- Initialize the program ndash slide 4
- Create a colored line ndash slide 5
- Skeleton of program ndash slide 6
- Basic Display Function
- openGL 31
- The Vertex Shader
- Drawing
- Views (2)
- The RenderMan Interface Bytestream
- RIB files
- Image Manipulation
- Ray tracing
- Light terms
- Ray tracing methodology
- Basic Ray tracing algorithm - 1
- Ray tracing algorithm - 2
- Final coloring
- Breeze DesignerAyam - lab
- Gimp Gnu Image Manipulation Program
- Operational concepts
- Gimp - lab
- Video games vs Movies
- Alice
- Alice environment
- Alice - Programming
- Alice Lab
- Steganography and LSB analysis
- What is it
- How is it done
- LSB embedding ndash using MatlabFreemat
- LSB embedding - algorithm
- imread
- LSB extraction - algorithm
- Properties of LSB flipping
- Augmented Reality
- AR usage
- AR developments
- Adobe Flash
- Flash - 2
- Virtual Reality Modeling Language
- X3D
- X3d-Samplex3d
- Digital audio
- Terms
- Basic editing
- Multi-track editing-1
- Multi-track editing-2
- Music creation
- Digital Video
- TV Formats
- Recording
- Remote Playback
- DV Editing
- Terms
- DV Editing -1
- DV Editing - 2
- DV Editing -3
- DV formats
- DVD types
- DVD types -2
-
119
Terms Asset list - Collection of clips frames amp stills Preview
Editable version of show Allows playback control + editing
Timeline - Time-marks abovebelow path of frames
Footage ndash DV segments measured in time units Effects - Re-size crop tone color super-
imposing Transitions
Straight cut (no transition) Fade inout dissolve cross-dissolve etc
copy DJ Foreman 2012
120
DV Editing -1
Import clips frames stills Trim footage (recurring action)
Insertremove segments Re-arrange segments Does not alter original file
Linear edit Simple re-arranging insertion removal
Non-linear edit (next slide) Note next slide compare ALL edits to
original
copy DJ Foreman 2012
121
Non-linear editing original
Ripple (middle expands 1)
Rolling (middle expands 1) (following In changed)
Slip (middle InOut change)
Slide (Middle moves left 2) (Prev Out next In
change)
DV Editing - 2
copy DJ Foreman 2012
a cb d 4 5 76 C FED G
a cb d 4 5 76 C FED G8
a cb d 4 5 76 FED G8
a cb d 1 2 3 C FED G4
a b 4 5 76 A DCB E F G
122
DV Editing -3
Ripple ndash changes overall length Following edits maintain overall length
Rolling ndash shortenslengthens following clip by amount of insertiondeletion
Slip ndash only the edited clips In and Out points change
Slide ndashchanges prev Out next In
copy DJ Foreman 2012
123
DV formats VCD
VHS quality Plays anywhere 7480 minutes recording for 650700MB CDs
SVCD 3560 minutes Many standalone DVD players all CDDVD-ROM
CVD = lower resolution less noise than SVCD DVD
2 hrs of good quality video (437 GB)+ DolbyDTS audio
Blu-ray High quality video
copy DJ Foreman 2012
124
DVD types Singledouble-sided 437GB 875GB
DVDForum DVD ndash R non-rewritable DVD ndash RW re-writable
DVD+RW Alliance DVD + R non-rewritable DVD + RW re-writeable
DVD plusmn R DL Dual layer 159GB
DVD-RAM NOT player compatible asymp HDD
+ and ndash differences are old hw designscopy DJ Foreman 2012
125
DVD types -2
Blu-ray 25 GB per layer up to 2 layers
AVCHD (Panasonic SONY) For Hi-DEF camcorders
HD-DVD (MS Toshiba NEC Sanyo) discontinued in 2008
VHS (everyone) vs Betamax (SONY)
copy DJ Foreman 2012
- CS338
- Overview -1
- Topics (general order of coverage)
- Software we might use (all free)
- Image formats
- Bitmap formats
- Vector formats
- Compression
- Codecs
- Codecs - 2
- Codec References
- Considerations
- Lossless Encoding
- Methodology
- Huffman lab ndash more details
- Sample of prefix coding amp compression
- Compression-Model types
- Standard Compressed-File Content
- Example 1 Header file for AU files
- Audio Encoding Enumerations
- Example 2 ZIP files (PkWare)
- More info on encoding
- Recording Bit-Rate
- Lab assignment ndash 1 ndash part 1
- Lab assignment 1 ndash part 2
- Working with Images
- Containers
- Container file format (MP4QT)
- Pixels
- Image creation
- Photography (film)
- FYI ndash a little chemistry
- Resolution
- Photography (digital)
- Some Comparisons
- Working Definitions - 1
- Working Definitions - 2
- Renderman Interface Bytestream (RIB) copy Pixar
- Context
- OpenGL 20 Fixed Function Pipeline
- The Vertex Processor
- OpenGL 20 Vertex Processor Pipeline
- DirectX vs OpenGL
- The Model-view Matrix
- Graphic fragment
- Fragment Processing
- Lab 2- basic modeling
- Lab 2 grading
- Some Examples for lab 2
- OpenGL
- Window Control
- Programming in OpenGL
- OpenGL Program Structure
- Views
- Image Rotation
- Program structure
- GlutReshapeFunc
- GL_Load_Identity
- gluLookAt
- Matrix Modes
- Keyboard-related functions
- Basic Functions in ldquomainrdquo
- Skeleton of program ndash slide 1
- Skeleton of program ndash slide 2
- Skeleton of program ndash slide 3
- Initialize the program ndash slide 4
- Create a colored line ndash slide 5
- Skeleton of program ndash slide 6
- Basic Display Function
- openGL 31
- The Vertex Shader
- Drawing
- Views (2)
- The RenderMan Interface Bytestream
- RIB files
- Image Manipulation
- Ray tracing
- Light terms
- Ray tracing methodology
- Basic Ray tracing algorithm - 1
- Ray tracing algorithm - 2
- Final coloring
- Breeze DesignerAyam - lab
- Gimp Gnu Image Manipulation Program
- Operational concepts
- Gimp - lab
- Video games vs Movies
- Alice
- Alice environment
- Alice - Programming
- Alice Lab
- Steganography and LSB analysis
- What is it
- How is it done
- LSB embedding ndash using MatlabFreemat
- LSB embedding - algorithm
- imread
- LSB extraction - algorithm
- Properties of LSB flipping
- Augmented Reality
- AR usage
- AR developments
- Adobe Flash
- Flash - 2
- Virtual Reality Modeling Language
- X3D
- X3d-Samplex3d
- Digital audio
- Terms
- Basic editing
- Multi-track editing-1
- Multi-track editing-2
- Music creation
- Digital Video
- TV Formats
- Recording
- Remote Playback
- DV Editing
- Terms
- DV Editing -1
- DV Editing - 2
- DV Editing -3
- DV formats
- DVD types
- DVD types -2
-
120
DV Editing -1
Import clips frames stills Trim footage (recurring action)
Insertremove segments Re-arrange segments Does not alter original file
Linear edit Simple re-arranging insertion removal
Non-linear edit (next slide) Note next slide compare ALL edits to
original
copy DJ Foreman 2012
121
Non-linear editing original
Ripple (middle expands 1)
Rolling (middle expands 1) (following In changed)
Slip (middle InOut change)
Slide (Middle moves left 2) (Prev Out next In
change)
DV Editing - 2
copy DJ Foreman 2012
a cb d 4 5 76 C FED G
a cb d 4 5 76 C FED G8
a cb d 4 5 76 FED G8
a cb d 1 2 3 C FED G4
a b 4 5 76 A DCB E F G
122
DV Editing -3
Ripple ndash changes overall length Following edits maintain overall length
Rolling ndash shortenslengthens following clip by amount of insertiondeletion
Slip ndash only the edited clips In and Out points change
Slide ndashchanges prev Out next In
copy DJ Foreman 2012
123
DV formats VCD
VHS quality Plays anywhere 7480 minutes recording for 650700MB CDs
SVCD 3560 minutes Many standalone DVD players all CDDVD-ROM
CVD = lower resolution less noise than SVCD DVD
2 hrs of good quality video (437 GB)+ DolbyDTS audio
Blu-ray High quality video
copy DJ Foreman 2012
124
DVD types Singledouble-sided 437GB 875GB
DVDForum DVD ndash R non-rewritable DVD ndash RW re-writable
DVD+RW Alliance DVD + R non-rewritable DVD + RW re-writeable
DVD plusmn R DL Dual layer 159GB
DVD-RAM NOT player compatible asymp HDD
+ and ndash differences are old hw designscopy DJ Foreman 2012
125
DVD types -2
Blu-ray 25 GB per layer up to 2 layers
AVCHD (Panasonic SONY) For Hi-DEF camcorders
HD-DVD (MS Toshiba NEC Sanyo) discontinued in 2008
VHS (everyone) vs Betamax (SONY)
copy DJ Foreman 2012
- CS338
- Overview -1
- Topics (general order of coverage)
- Software we might use (all free)
- Image formats
- Bitmap formats
- Vector formats
- Compression
- Codecs
- Codecs - 2
- Codec References
- Considerations
- Lossless Encoding
- Methodology
- Huffman lab ndash more details
- Sample of prefix coding amp compression
- Compression-Model types
- Standard Compressed-File Content
- Example 1 Header file for AU files
- Audio Encoding Enumerations
- Example 2 ZIP files (PkWare)
- More info on encoding
- Recording Bit-Rate
- Lab assignment ndash 1 ndash part 1
- Lab assignment 1 ndash part 2
- Working with Images
- Containers
- Container file format (MP4QT)
- Pixels
- Image creation
- Photography (film)
- FYI ndash a little chemistry
- Resolution
- Photography (digital)
- Some Comparisons
- Working Definitions - 1
- Working Definitions - 2
- Renderman Interface Bytestream (RIB) copy Pixar
- Context
- OpenGL 20 Fixed Function Pipeline
- The Vertex Processor
- OpenGL 20 Vertex Processor Pipeline
- DirectX vs OpenGL
- The Model-view Matrix
- Graphic fragment
- Fragment Processing
- Lab 2- basic modeling
- Lab 2 grading
- Some Examples for lab 2
- OpenGL
- Window Control
- Programming in OpenGL
- OpenGL Program Structure
- Views
- Image Rotation
- Program structure
- GlutReshapeFunc
- GL_Load_Identity
- gluLookAt
- Matrix Modes
- Keyboard-related functions
- Basic Functions in ldquomainrdquo
- Skeleton of program ndash slide 1
- Skeleton of program ndash slide 2
- Skeleton of program ndash slide 3
- Initialize the program ndash slide 4
- Create a colored line ndash slide 5
- Skeleton of program ndash slide 6
- Basic Display Function
- openGL 31
- The Vertex Shader
- Drawing
- Views (2)
- The RenderMan Interface Bytestream
- RIB files
- Image Manipulation
- Ray tracing
- Light terms
- Ray tracing methodology
- Basic Ray tracing algorithm - 1
- Ray tracing algorithm - 2
- Final coloring
- Breeze DesignerAyam - lab
- Gimp Gnu Image Manipulation Program
- Operational concepts
- Gimp - lab
- Video games vs Movies
- Alice
- Alice environment
- Alice - Programming
- Alice Lab
- Steganography and LSB analysis
- What is it
- How is it done
- LSB embedding ndash using MatlabFreemat
- LSB embedding - algorithm
- imread
- LSB extraction - algorithm
- Properties of LSB flipping
- Augmented Reality
- AR usage
- AR developments
- Adobe Flash
- Flash - 2
- Virtual Reality Modeling Language
- X3D
- X3d-Samplex3d
- Digital audio
- Terms
- Basic editing
- Multi-track editing-1
- Multi-track editing-2
- Music creation
- Digital Video
- TV Formats
- Recording
- Remote Playback
- DV Editing
- Terms
- DV Editing -1
- DV Editing - 2
- DV Editing -3
- DV formats
- DVD types
- DVD types -2
-
121
Non-linear editing original
Ripple (middle expands 1)
Rolling (middle expands 1) (following In changed)
Slip (middle InOut change)
Slide (Middle moves left 2) (Prev Out next In
change)
DV Editing - 2
copy DJ Foreman 2012
a cb d 4 5 76 C FED G
a cb d 4 5 76 C FED G8
a cb d 4 5 76 FED G8
a cb d 1 2 3 C FED G4
a b 4 5 76 A DCB E F G
122
DV Editing -3
Ripple ndash changes overall length Following edits maintain overall length
Rolling ndash shortenslengthens following clip by amount of insertiondeletion
Slip ndash only the edited clips In and Out points change
Slide ndashchanges prev Out next In
copy DJ Foreman 2012
123
DV formats VCD
VHS quality Plays anywhere 7480 minutes recording for 650700MB CDs
SVCD 3560 minutes Many standalone DVD players all CDDVD-ROM
CVD = lower resolution less noise than SVCD DVD
2 hrs of good quality video (437 GB)+ DolbyDTS audio
Blu-ray High quality video
copy DJ Foreman 2012
124
DVD types Singledouble-sided 437GB 875GB
DVDForum DVD ndash R non-rewritable DVD ndash RW re-writable
DVD+RW Alliance DVD + R non-rewritable DVD + RW re-writeable
DVD plusmn R DL Dual layer 159GB
DVD-RAM NOT player compatible asymp HDD
+ and ndash differences are old hw designscopy DJ Foreman 2012
125
DVD types -2
Blu-ray 25 GB per layer up to 2 layers
AVCHD (Panasonic SONY) For Hi-DEF camcorders
HD-DVD (MS Toshiba NEC Sanyo) discontinued in 2008
VHS (everyone) vs Betamax (SONY)
copy DJ Foreman 2012
- CS338
- Overview -1
- Topics (general order of coverage)
- Software we might use (all free)
- Image formats
- Bitmap formats
- Vector formats
- Compression
- Codecs
- Codecs - 2
- Codec References
- Considerations
- Lossless Encoding
- Methodology
- Huffman lab ndash more details
- Sample of prefix coding amp compression
- Compression-Model types
- Standard Compressed-File Content
- Example 1 Header file for AU files
- Audio Encoding Enumerations
- Example 2 ZIP files (PkWare)
- More info on encoding
- Recording Bit-Rate
- Lab assignment ndash 1 ndash part 1
- Lab assignment 1 ndash part 2
- Working with Images
- Containers
- Container file format (MP4QT)
- Pixels
- Image creation
- Photography (film)
- FYI ndash a little chemistry
- Resolution
- Photography (digital)
- Some Comparisons
- Working Definitions - 1
- Working Definitions - 2
- Renderman Interface Bytestream (RIB) copy Pixar
- Context
- OpenGL 20 Fixed Function Pipeline
- The Vertex Processor
- OpenGL 20 Vertex Processor Pipeline
- DirectX vs OpenGL
- The Model-view Matrix
- Graphic fragment
- Fragment Processing
- Lab 2- basic modeling
- Lab 2 grading
- Some Examples for lab 2
- OpenGL
- Window Control
- Programming in OpenGL
- OpenGL Program Structure
- Views
- Image Rotation
- Program structure
- GlutReshapeFunc
- GL_Load_Identity
- gluLookAt
- Matrix Modes
- Keyboard-related functions
- Basic Functions in ldquomainrdquo
- Skeleton of program ndash slide 1
- Skeleton of program ndash slide 2
- Skeleton of program ndash slide 3
- Initialize the program ndash slide 4
- Create a colored line ndash slide 5
- Skeleton of program ndash slide 6
- Basic Display Function
- openGL 31
- The Vertex Shader
- Drawing
- Views (2)
- The RenderMan Interface Bytestream
- RIB files
- Image Manipulation
- Ray tracing
- Light terms
- Ray tracing methodology
- Basic Ray tracing algorithm - 1
- Ray tracing algorithm - 2
- Final coloring
- Breeze DesignerAyam - lab
- Gimp Gnu Image Manipulation Program
- Operational concepts
- Gimp - lab
- Video games vs Movies
- Alice
- Alice environment
- Alice - Programming
- Alice Lab
- Steganography and LSB analysis
- What is it
- How is it done
- LSB embedding ndash using MatlabFreemat
- LSB embedding - algorithm
- imread
- LSB extraction - algorithm
- Properties of LSB flipping
- Augmented Reality
- AR usage
- AR developments
- Adobe Flash
- Flash - 2
- Virtual Reality Modeling Language
- X3D
- X3d-Samplex3d
- Digital audio
- Terms
- Basic editing
- Multi-track editing-1
- Multi-track editing-2
- Music creation
- Digital Video
- TV Formats
- Recording
- Remote Playback
- DV Editing
- Terms
- DV Editing -1
- DV Editing - 2
- DV Editing -3
- DV formats
- DVD types
- DVD types -2
-
122
DV Editing -3
Ripple ndash changes overall length Following edits maintain overall length
Rolling ndash shortenslengthens following clip by amount of insertiondeletion
Slip ndash only the edited clips In and Out points change
Slide ndashchanges prev Out next In
copy DJ Foreman 2012
123
DV formats VCD
VHS quality Plays anywhere 7480 minutes recording for 650700MB CDs
SVCD 3560 minutes Many standalone DVD players all CDDVD-ROM
CVD = lower resolution less noise than SVCD DVD
2 hrs of good quality video (437 GB)+ DolbyDTS audio
Blu-ray High quality video
copy DJ Foreman 2012
124
DVD types Singledouble-sided 437GB 875GB
DVDForum DVD ndash R non-rewritable DVD ndash RW re-writable
DVD+RW Alliance DVD + R non-rewritable DVD + RW re-writeable
DVD plusmn R DL Dual layer 159GB
DVD-RAM NOT player compatible asymp HDD
+ and ndash differences are old hw designscopy DJ Foreman 2012
125
DVD types -2
Blu-ray 25 GB per layer up to 2 layers
AVCHD (Panasonic SONY) For Hi-DEF camcorders
HD-DVD (MS Toshiba NEC Sanyo) discontinued in 2008
VHS (everyone) vs Betamax (SONY)
copy DJ Foreman 2012
- CS338
- Overview -1
- Topics (general order of coverage)
- Software we might use (all free)
- Image formats
- Bitmap formats
- Vector formats
- Compression
- Codecs
- Codecs - 2
- Codec References
- Considerations
- Lossless Encoding
- Methodology
- Huffman lab ndash more details
- Sample of prefix coding amp compression
- Compression-Model types
- Standard Compressed-File Content
- Example 1 Header file for AU files
- Audio Encoding Enumerations
- Example 2 ZIP files (PkWare)
- More info on encoding
- Recording Bit-Rate
- Lab assignment ndash 1 ndash part 1
- Lab assignment 1 ndash part 2
- Working with Images
- Containers
- Container file format (MP4QT)
- Pixels
- Image creation
- Photography (film)
- FYI ndash a little chemistry
- Resolution
- Photography (digital)
- Some Comparisons
- Working Definitions - 1
- Working Definitions - 2
- Renderman Interface Bytestream (RIB) copy Pixar
- Context
- OpenGL 20 Fixed Function Pipeline
- The Vertex Processor
- OpenGL 20 Vertex Processor Pipeline
- DirectX vs OpenGL
- The Model-view Matrix
- Graphic fragment
- Fragment Processing
- Lab 2- basic modeling
- Lab 2 grading
- Some Examples for lab 2
- OpenGL
- Window Control
- Programming in OpenGL
- OpenGL Program Structure
- Views
- Image Rotation
- Program structure
- GlutReshapeFunc
- GL_Load_Identity
- gluLookAt
- Matrix Modes
- Keyboard-related functions
- Basic Functions in ldquomainrdquo
- Skeleton of program ndash slide 1
- Skeleton of program ndash slide 2
- Skeleton of program ndash slide 3
- Initialize the program ndash slide 4
- Create a colored line ndash slide 5
- Skeleton of program ndash slide 6
- Basic Display Function
- openGL 31
- The Vertex Shader
- Drawing
- Views (2)
- The RenderMan Interface Bytestream
- RIB files
- Image Manipulation
- Ray tracing
- Light terms
- Ray tracing methodology
- Basic Ray tracing algorithm - 1
- Ray tracing algorithm - 2
- Final coloring
- Breeze DesignerAyam - lab
- Gimp Gnu Image Manipulation Program
- Operational concepts
- Gimp - lab
- Video games vs Movies
- Alice
- Alice environment
- Alice - Programming
- Alice Lab
- Steganography and LSB analysis
- What is it
- How is it done
- LSB embedding ndash using MatlabFreemat
- LSB embedding - algorithm
- imread
- LSB extraction - algorithm
- Properties of LSB flipping
- Augmented Reality
- AR usage
- AR developments
- Adobe Flash
- Flash - 2
- Virtual Reality Modeling Language
- X3D
- X3d-Samplex3d
- Digital audio
- Terms
- Basic editing
- Multi-track editing-1
- Multi-track editing-2
- Music creation
- Digital Video
- TV Formats
- Recording
- Remote Playback
- DV Editing
- Terms
- DV Editing -1
- DV Editing - 2
- DV Editing -3
- DV formats
- DVD types
- DVD types -2
-
123
DV formats VCD
VHS quality Plays anywhere 7480 minutes recording for 650700MB CDs
SVCD 3560 minutes Many standalone DVD players all CDDVD-ROM
CVD = lower resolution less noise than SVCD DVD
2 hrs of good quality video (437 GB)+ DolbyDTS audio
Blu-ray High quality video
copy DJ Foreman 2012
124
DVD types Singledouble-sided 437GB 875GB
DVDForum DVD ndash R non-rewritable DVD ndash RW re-writable
DVD+RW Alliance DVD + R non-rewritable DVD + RW re-writeable
DVD plusmn R DL Dual layer 159GB
DVD-RAM NOT player compatible asymp HDD
+ and ndash differences are old hw designscopy DJ Foreman 2012
125
DVD types -2
Blu-ray 25 GB per layer up to 2 layers
AVCHD (Panasonic SONY) For Hi-DEF camcorders
HD-DVD (MS Toshiba NEC Sanyo) discontinued in 2008
VHS (everyone) vs Betamax (SONY)
copy DJ Foreman 2012
- CS338
- Overview -1
- Topics (general order of coverage)
- Software we might use (all free)
- Image formats
- Bitmap formats
- Vector formats
- Compression
- Codecs
- Codecs - 2
- Codec References
- Considerations
- Lossless Encoding
- Methodology
- Huffman lab ndash more details
- Sample of prefix coding amp compression
- Compression-Model types
- Standard Compressed-File Content
- Example 1 Header file for AU files
- Audio Encoding Enumerations
- Example 2 ZIP files (PkWare)
- More info on encoding
- Recording Bit-Rate
- Lab assignment ndash 1 ndash part 1
- Lab assignment 1 ndash part 2
- Working with Images
- Containers
- Container file format (MP4QT)
- Pixels
- Image creation
- Photography (film)
- FYI ndash a little chemistry
- Resolution
- Photography (digital)
- Some Comparisons
- Working Definitions - 1
- Working Definitions - 2
- Renderman Interface Bytestream (RIB) copy Pixar
- Context
- OpenGL 20 Fixed Function Pipeline
- The Vertex Processor
- OpenGL 20 Vertex Processor Pipeline
- DirectX vs OpenGL
- The Model-view Matrix
- Graphic fragment
- Fragment Processing
- Lab 2- basic modeling
- Lab 2 grading
- Some Examples for lab 2
- OpenGL
- Window Control
- Programming in OpenGL
- OpenGL Program Structure
- Views
- Image Rotation
- Program structure
- GlutReshapeFunc
- GL_Load_Identity
- gluLookAt
- Matrix Modes
- Keyboard-related functions
- Basic Functions in ldquomainrdquo
- Skeleton of program ndash slide 1
- Skeleton of program ndash slide 2
- Skeleton of program ndash slide 3
- Initialize the program ndash slide 4
- Create a colored line ndash slide 5
- Skeleton of program ndash slide 6
- Basic Display Function
- openGL 31
- The Vertex Shader
- Drawing
- Views (2)
- The RenderMan Interface Bytestream
- RIB files
- Image Manipulation
- Ray tracing
- Light terms
- Ray tracing methodology
- Basic Ray tracing algorithm - 1
- Ray tracing algorithm - 2
- Final coloring
- Breeze DesignerAyam - lab
- Gimp Gnu Image Manipulation Program
- Operational concepts
- Gimp - lab
- Video games vs Movies
- Alice
- Alice environment
- Alice - Programming
- Alice Lab
- Steganography and LSB analysis
- What is it
- How is it done
- LSB embedding ndash using MatlabFreemat
- LSB embedding - algorithm
- imread
- LSB extraction - algorithm
- Properties of LSB flipping
- Augmented Reality
- AR usage
- AR developments
- Adobe Flash
- Flash - 2
- Virtual Reality Modeling Language
- X3D
- X3d-Samplex3d
- Digital audio
- Terms
- Basic editing
- Multi-track editing-1
- Multi-track editing-2
- Music creation
- Digital Video
- TV Formats
- Recording
- Remote Playback
- DV Editing
- Terms
- DV Editing -1
- DV Editing - 2
- DV Editing -3
- DV formats
- DVD types
- DVD types -2
-
124
DVD types Singledouble-sided 437GB 875GB
DVDForum DVD ndash R non-rewritable DVD ndash RW re-writable
DVD+RW Alliance DVD + R non-rewritable DVD + RW re-writeable
DVD plusmn R DL Dual layer 159GB
DVD-RAM NOT player compatible asymp HDD
+ and ndash differences are old hw designscopy DJ Foreman 2012
125
DVD types -2
Blu-ray 25 GB per layer up to 2 layers
AVCHD (Panasonic SONY) For Hi-DEF camcorders
HD-DVD (MS Toshiba NEC Sanyo) discontinued in 2008
VHS (everyone) vs Betamax (SONY)
copy DJ Foreman 2012
- CS338
- Overview -1
- Topics (general order of coverage)
- Software we might use (all free)
- Image formats
- Bitmap formats
- Vector formats
- Compression
- Codecs
- Codecs - 2
- Codec References
- Considerations
- Lossless Encoding
- Methodology
- Huffman lab ndash more details
- Sample of prefix coding amp compression
- Compression-Model types
- Standard Compressed-File Content
- Example 1 Header file for AU files
- Audio Encoding Enumerations
- Example 2 ZIP files (PkWare)
- More info on encoding
- Recording Bit-Rate
- Lab assignment ndash 1 ndash part 1
- Lab assignment 1 ndash part 2
- Working with Images
- Containers
- Container file format (MP4QT)
- Pixels
- Image creation
- Photography (film)
- FYI ndash a little chemistry
- Resolution
- Photography (digital)
- Some Comparisons
- Working Definitions - 1
- Working Definitions - 2
- Renderman Interface Bytestream (RIB) copy Pixar
- Context
- OpenGL 20 Fixed Function Pipeline
- The Vertex Processor
- OpenGL 20 Vertex Processor Pipeline
- DirectX vs OpenGL
- The Model-view Matrix
- Graphic fragment
- Fragment Processing
- Lab 2- basic modeling
- Lab 2 grading
- Some Examples for lab 2
- OpenGL
- Window Control
- Programming in OpenGL
- OpenGL Program Structure
- Views
- Image Rotation
- Program structure
- GlutReshapeFunc
- GL_Load_Identity
- gluLookAt
- Matrix Modes
- Keyboard-related functions
- Basic Functions in ldquomainrdquo
- Skeleton of program ndash slide 1
- Skeleton of program ndash slide 2
- Skeleton of program ndash slide 3
- Initialize the program ndash slide 4
- Create a colored line ndash slide 5
- Skeleton of program ndash slide 6
- Basic Display Function
- openGL 31
- The Vertex Shader
- Drawing
- Views (2)
- The RenderMan Interface Bytestream
- RIB files
- Image Manipulation
- Ray tracing
- Light terms
- Ray tracing methodology
- Basic Ray tracing algorithm - 1
- Ray tracing algorithm - 2
- Final coloring
- Breeze DesignerAyam - lab
- Gimp Gnu Image Manipulation Program
- Operational concepts
- Gimp - lab
- Video games vs Movies
- Alice
- Alice environment
- Alice - Programming
- Alice Lab
- Steganography and LSB analysis
- What is it
- How is it done
- LSB embedding ndash using MatlabFreemat
- LSB embedding - algorithm
- imread
- LSB extraction - algorithm
- Properties of LSB flipping
- Augmented Reality
- AR usage
- AR developments
- Adobe Flash
- Flash - 2
- Virtual Reality Modeling Language
- X3D
- X3d-Samplex3d
- Digital audio
- Terms
- Basic editing
- Multi-track editing-1
- Multi-track editing-2
- Music creation
- Digital Video
- TV Formats
- Recording
- Remote Playback
- DV Editing
- Terms
- DV Editing -1
- DV Editing - 2
- DV Editing -3
- DV formats
- DVD types
- DVD types -2
-
125
DVD types -2
Blu-ray 25 GB per layer up to 2 layers
AVCHD (Panasonic SONY) For Hi-DEF camcorders
HD-DVD (MS Toshiba NEC Sanyo) discontinued in 2008
VHS (everyone) vs Betamax (SONY)
copy DJ Foreman 2012
- CS338
- Overview -1
- Topics (general order of coverage)
- Software we might use (all free)
- Image formats
- Bitmap formats
- Vector formats
- Compression
- Codecs
- Codecs - 2
- Codec References
- Considerations
- Lossless Encoding
- Methodology
- Huffman lab ndash more details
- Sample of prefix coding amp compression
- Compression-Model types
- Standard Compressed-File Content
- Example 1 Header file for AU files
- Audio Encoding Enumerations
- Example 2 ZIP files (PkWare)
- More info on encoding
- Recording Bit-Rate
- Lab assignment ndash 1 ndash part 1
- Lab assignment 1 ndash part 2
- Working with Images
- Containers
- Container file format (MP4QT)
- Pixels
- Image creation
- Photography (film)
- FYI ndash a little chemistry
- Resolution
- Photography (digital)
- Some Comparisons
- Working Definitions - 1
- Working Definitions - 2
- Renderman Interface Bytestream (RIB) copy Pixar
- Context
- OpenGL 20 Fixed Function Pipeline
- The Vertex Processor
- OpenGL 20 Vertex Processor Pipeline
- DirectX vs OpenGL
- The Model-view Matrix
- Graphic fragment
- Fragment Processing
- Lab 2- basic modeling
- Lab 2 grading
- Some Examples for lab 2
- OpenGL
- Window Control
- Programming in OpenGL
- OpenGL Program Structure
- Views
- Image Rotation
- Program structure
- GlutReshapeFunc
- GL_Load_Identity
- gluLookAt
- Matrix Modes
- Keyboard-related functions
- Basic Functions in ldquomainrdquo
- Skeleton of program ndash slide 1
- Skeleton of program ndash slide 2
- Skeleton of program ndash slide 3
- Initialize the program ndash slide 4
- Create a colored line ndash slide 5
- Skeleton of program ndash slide 6
- Basic Display Function
- openGL 31
- The Vertex Shader
- Drawing
- Views (2)
- The RenderMan Interface Bytestream
- RIB files
- Image Manipulation
- Ray tracing
- Light terms
- Ray tracing methodology
- Basic Ray tracing algorithm - 1
- Ray tracing algorithm - 2
- Final coloring
- Breeze DesignerAyam - lab
- Gimp Gnu Image Manipulation Program
- Operational concepts
- Gimp - lab
- Video games vs Movies
- Alice
- Alice environment
- Alice - Programming
- Alice Lab
- Steganography and LSB analysis
- What is it
- How is it done
- LSB embedding ndash using MatlabFreemat
- LSB embedding - algorithm
- imread
- LSB extraction - algorithm
- Properties of LSB flipping
- Augmented Reality
- AR usage
- AR developments
- Adobe Flash
- Flash - 2
- Virtual Reality Modeling Language
- X3D
- X3d-Samplex3d
- Digital audio
- Terms
- Basic editing
- Multi-track editing-1
- Multi-track editing-2
- Music creation
- Digital Video
- TV Formats
- Recording
- Remote Playback
- DV Editing
- Terms
- DV Editing -1
- DV Editing - 2
- DV Editing -3
- DV formats
- DVD types
- DVD types -2
-