CS 352: Computer Graphics Input and Interaction. Interactive Computer GraphicsChapter 3 - 2 What...
-
Upload
horace-atkinson -
Category
Documents
-
view
219 -
download
0
Transcript of CS 352: Computer Graphics Input and Interaction. Interactive Computer GraphicsChapter 3 - 2 What...
CS 352: Computer Graphics
Input and
Interaction
Interactive Computer GraphicsChapter 3 - 2
What good is Computer Graphics? JS1k canvas examples: 1 2 3 4 5 Games, visual demos…of what value? Is
there a Christian perspective? Communication of information to the user
Data visualization, simulation, GUIEven a word processor is an "interactive graphics
program" Communication between users
"Collaborative environments" are hot (multi-player games?)
Social networking is transforming the world…
Interaction is an essential component
Interactive Computer GraphicsChapter 3 - 3
Interaction Much of the fun and utility of graphics
is in interaction: accepting user input that affects the display
Paint programs, modeling, games, word processors, spreadsheets, powerpoint … User initiates input events such as clicking
on a menu or drawing a circle Computer response by changing graphical
display
Interactive Computer GraphicsChapter 3 - 4
Interactive Computer GraphicsChapter 3 - 5
Projects 3 Projects 3 will be a paint program
we'll learn 2-D graphics, interaction, event-loop programming, double-buffering simple animation, basic image processing
Features: Pen, line, and rectangle tools Color, pen size selectors Image processing (sharpen, blur, detect edges) Open, save images Toolbar and menu for controlling application
How to set up an application UI? How to make a menu? How to make a color picker? How to make a toolbar? How to make toolbar buttons pop in
and out?
Interactive Computer GraphicsChapter 3 - 6
How to paint? How do I make a colored line follow the
mouse or fingertip?
Interactive prog: Input devices Event handlers Event loop
Interactive Computer GraphicsChapter 3 - 7
Interactive Computer GraphicsChapter 3 - 8
Input devices Interaction requires handling input
devices Physical: mouse, fingertip, keyboard,
joystick, digitizer, accelerometer, head tracker…
Logical: Text Locator Valuator (slider) Stroke Color picker
How to read?
Interactive Computer GraphicsChapter 3 - 9
Input devices Sample mode
Program reads the current state of input device (frequently)
Event mode Each click or motion generates an event
that goes on a queue Program can process events from queue
HTML?
Event-loop programming Events: button press, slider value
change, menu selection, mouse move, keypress
Event loop: Poll (or wait) for events Process events, update state Update display (ideally: wait for rest of frame time to
elapse)
Interactive Computer GraphicsChapter 3 - 10
State-driven Typically, the event loop is driven by a
big state table e.g. depressing the paintbrush tool releases
other tools, puts me in "paint mode" Good libraries will handle some of the
bookkeeping for you You still typically have to handle some
state
Interactive Computer GraphicsChapter 3 - 11
Interactive Computer GraphicsChapter 3 - 12
State diagram
Event loop in HTML/JS HTML/JS provides event queue, support
for many basic events (mousedown, mouseup mouseover, mousemove, keypress, key release, value change, button click, etc.)
You are on your own for higher-level events, e.g. clicking on a toolbar tool
It's also possible to set a function to run every 15 ms, sample input devices
Interactive Computer GraphicsChapter 3 - 13
Events for painting? mousedown:
go into paint mode store mouse position draw initial dot
mouseup: exit paint mode
mousemove: if in paint mode
draw a line from old mouse position to current set old mouse position to current position
Interactive Computer GraphicsChapter 3 - 14
Event handling with jQuery Binding events to functions
$(cpaint.canvas).bind('mousedown', cpaint.drawStart)
Processing eventscpaint.drawStart = function(ev) { var x, y; x = ev.pageX - $(cpaint.canvas).offset().left; y = ev.pageY - $(cpaint.canvas).offset().top; ev.preventDefault(); cpaint.paintmode = true; cpaint.oldX = x; cpaint.oldY = y;
Interactive Computer GraphicsChapter 3 - 15
Looking neat and spiffy How to avoid crinkles in your paint
strokes?
Interactive Computer GraphicsChapter 3 - 16
Looking neat and spiffy How to avoid crinkles in your paint
strokes? Draw connected paths
Or just use line caps
Interactive Computer GraphicsChapter 3 - 17
Menus There are many jQuery menu plug-ins…
<ul id="mainmenu"> <li>File <ul> <li id="menuNew">New</li> <li id="menuOpen">Open</li> <li id="menuSave">Save</li>
-------------------------------------
$('#menuNew').bind('click', cpaint.clear); $('#menuOpen').bind('click',cpaint.open); $('#menuSave').bind('click',cpaint.save);
Interactive Computer GraphicsChapter 3 - 18
Toolbar
How to make a toolbar?
How should buttons behave?
State diagram?
Interactive Computer GraphicsChapter 3 - 19
Interactive Computer GraphicsChapter 3 - 20
Buttons Widgets may have several states State should be evident in visual feedback E.g. how exactly does a button work? States? State transition diagram
Most buttons: six states, with six different appearances
neutral neutral-highlighted neutral-depressed selected selected-highlighted selected-depressed
Events: mousedown, mouseup, enter, exit Transitions: what happens in each state under each
event?
Interactive Computer GraphicsChapter 3 - 21
Button state diagram (buttons selectable, not unselectable)
S
SHNH
N
NHD SHD
Press
Move
N: neutralH: highlighted (usually mouse over)
S: selectedD: mouse down
Interactive Computer GraphicsChapter 3 - 22
Other button considerations Could also consider
Tooltips Whether button merely clicks and releases
or can be selected Whether button can be unselected
(e.g. B/I/U vs. Left, Center, Right) Want consistent appearance, behavior
over whole program – or whole computer
Really need a library implementation and a strict set of UI guidelines…
4-State toolbar buttons: CSS.toolbarCell { background-color:#ddd; width:20pt; height:20pt; border: solid #eee 2px; box-shadow: 2px 2px 2px #666; }
#markerButton { background: url(img/paintbrush.png) no-repeat center center; }
.selected { box-shadow: inset 2px 2px 2px #666; }
.toolbarCell:hover { border:solid #555 2px; }
Interactive Computer GraphicsChapter 3 - 23
4-State toolbar buttons: JS$('#markerButton').bind('click', {tool:"marker"},
cpaint.selectTool);
-------------------
cpaint.selectTool = function(ev) { cpaint.tool = ev.data.tool; // get tool name
$('.toolbarCell').each(function(index) { // unselect $(this).removeClass('selected'); // others });
var tool = '#' + cpaint.tool + 'Button'; // get ID $(tool).addClass('selected'); // select }
Interactive Computer GraphicsChapter 3 - 24
Paintbrush size selector How?
Interactive Computer GraphicsChapter 3 - 25
Color picker Google "jQuery color picker"…
<input type=“color”> works in some browsers
Interactive Computer GraphicsChapter 3 - 26
How draw a line? What kind of feedback is normal?
Rubber Banding
Events and actions?
Interactive Computer GraphicsChapter 3 - 27
Save & restore Create your own off-screen canvas Copy it back on each mouse movement Events and actions?
Interactive Computer GraphicsChapter 3 - 28
Save & Restore event handling Mousedown
enter line mode remember mouse position as startx, starty save screen draw initial dot
Mousemove paste saved screen draw line from startx, starty to current
mouse pos Mouseup
exit line mode
Interactive Computer GraphicsChapter 3 - 29
Analysis Drawbacks?
slow – eats memory bandwidth for breakfast
copy smallest possible rectangle? Only points from line?
possible flickering use double buffering?
Interactive Computer GraphicsChapter 3 - 30
Interactive Computer GraphicsChapter 3 - 31
Double buffering Have two frame buffers, "front" and
"back" Draw into back buffer At vertical retrace time, swap buffers
This is a fundamental graphics technique…
…not built into canvas… …though, some drawing aggregation
seems to happen automatically, behind the scenes;
…not usually necessary in canvas, but browser dependent
Fake Double Buffering in Canvas Create off-screen canvas
canvasBuffer = document.createElement('canvas');canvasBuffer.width = canvas.width;canvasBuffer.height= canvas.height;canvasBufferContext= canvasBuffer.getContext('2d');
Draw into off-screen canvascanvasBufferContext.[drawSomeStuff…]
Copy onto display canvascontext.drawImage(canvasBuffer, 0, 0);
Interactive Computer GraphicsChapter 3 - 32
How to erase? Draw background color…
Interactive Computer GraphicsChapter 3 - 33
Save, Load? Save
Copy pixels to an image so users can right-click, save-as
Load What are the security risks? Can only load files from the same server Can use a file chooser if it's a local app Security policies are not entirely in place Consider chrome option
--allow-file-access-from-files
Interactive Computer GraphicsChapter 3 - 34
Resizing How would you allow the user to shrink,
expand image? What would happen to image
resolution?
Interactive Computer GraphicsChapter 3 - 35
Interactive Computer GraphicsChapter 3 - 36
Store drawing commands Realistically, to redraw effectively, you
have to save all drawing commandspen color redfill color blueline width 5circle 5 10 12textfont Times 12 text 10 10 "hello world"
Replay commands to redraw scene Could store commands in a file (e.g. Mac PICT
file) For some kinds of drawings, files are smaller
and more accurate than bitmaps
Interactive Computer GraphicsChapter 3 - 37
Other toolkits HTML offers basic UI capabilities in a
cross-platform context Writing your own UI extensions is
tedious jQuery plugins extend capabilities Ideally, all UI elements ought to be built
in In the real world, they come in
platform-specific 'toolkits' or 'windowing libraries'
E.g. MFC, QT, OpenLook, Cocoa …
Image processing Image processing examples:
blur, sharpen detect edges enhance contrast noise reduction posterize fisheye redeye reduction find faces …
Interactive Computer GraphicsChapter 3 - 38
How to fade an image? Could average pixel colors with white Or just decrease alpha…
Interactive Computer GraphicsChapter 3 - 39
How to blur an image?
Interactive Computer GraphicsChapter 3 - 40
How to blur an image? Blurring is an averaging process Convolution: apply a convolution kernel, e.g.
Center kernel on pixel of interest Multiply each kernel value by color value
underneath Add results Gaussian smoothing
Interactive Computer GraphicsChapter 3 - 41
1/9 1/9 1/9
1/9 1/9 1/9
1/9 1/9 1/9
Gaussian smoothing
Interactive Computer GraphicsChapter 3 - 42
Interactive Computer GraphicsChapter 3 - 43
How to sharpen an image? It's a differentiation process What's Unsharp Masking?
Interactive Computer GraphicsChapter 3 - 44
How to sharpen an image? It's a differentiation process What's Unsharp Masking?
Subtract blurred version from original Add back to original
Interactive Computer GraphicsChapter 3 - 45
Sharpening Subtract neighbors Like subtracting a blurred version of the
image Unsharp Masking E.g. convolve with a kernel like one of
these:
Interactive Computer GraphicsChapter 3 - 46
Edge detection?
Wikipedia
Interactive Computer GraphicsChapter 3 - 47
Sobel Edge Operators Look for horizontal and vertical
variation
Could do this at different sales or resolutions
Note: results maybe positive or negative numbers; must normalize
Interactive Computer GraphicsChapter 3 - 48
1 2 1
0 0 0
-1 -2 -1
-1 0 1
-2 0 2
-1 0 1
Image Compression What if you created say six different
lower-resolution images and only stored the difference at each resolution?
Note: most of the data is in the highest-frequency component
An early image compression technique
Interactive Computer GraphicsChapter 3 - 49
Fourier Transform
Interactive Computer GraphicsChapter 3 - 50
The alg that changed imagery…
Discrete cosinetransform (DCT)
(Wikipedia)
Interactive Computer GraphicsChapter 3 - 51
JPEG image compression DCT to convert to frequency domain Perceptual modeling Coefficient quantization Entropy coding
Interactive Computer GraphicsChapter 3 - 52
Research in image processing and computer vision How would you clear up a blurry
photograph?
How would you recognize a face in a photograph?
Interactive Computer GraphicsChapter 3 - 53
Face recognition One approach: statistically derive a set of
“standardized face ingredients” from many examples, resulting in “eigenfaces”
Represent each face as linear combinationof eigenfaces
Interactive Computer GraphicsChapter 3 - 54
Summary Event loop programming Interaction and event handling in HTML State diagrams Painting Rubber banding Double buffering Basic image processing and
convolutions "Photoshop Nano"
Interactive Computer GraphicsChapter 3 - 55