encoding for the imaginationDouglas Goodwin03/04/2013

Monday, March 4, 13encoding for the imagination, doug goodwin, march fourth, 2013

encoding strategy: Parrhasius’ grapesZeuxis’ grapes fooled natureParrhasius’ grapes fooled Zeuxis!

Monday, March 4, 13According to the Naturalis Historia of Pliny the Elder, Zeuxis and his contemporary Parrhasius (of Ephesus and later Athens) staged a contest to determine the greater artist. When Zeuxis unveiled his painting of grapes, they appeared so real birds flew down to peck at them. But when Zeuxis asked Parrhasius to pull aside the curtain from his painting, the curtain itself turned out to be a painted illusion. Parrhasius won, and Zeuxis admitted, 'I have deceived the birds, but Parrhasius has deceived Zeuxis.'

overview

1. how does digital video represent an image?

2. how can we tap into the system that lies between stimuli and sensation?

3. how does video take advantage of imagination and memory?

4. the grapes will soon be forgotten. will we starve?

5. Lossless #3 & #4

6. q&a

Monday, March 4, 13overview

how does digital video encode an image?1. cull what cannot be seen (ex: everything below a certain

luminance value)

2. divide the x/y frame into a matrix of macroblocks

3. analyze temporal difference between current frame and the next frame for each block (motion analysis)

4. if temporal difference < n then retain the current block

5. split the x/y frame luminance and chrominance

6. throw out a lot of chrominance (4:2:2)

7. reorder the frames for efficient storage

8. transform the frame into waveforms and then digitize them.

Monday, March 4, 13

MPEG Compression & how it works in IPTV and other digital media delivery systems.http://youtu.be/P7abyWT4dss

we call this “lossy” compression because so much of the image is discarded in the encoding process.

Monday, March 4, 13overview

are you missing that image data?

Monday, March 4, 13overview

what can you offer to replace it?

Monday, March 4, 13overview

the curtain.

Monday, March 4, 13overview

what do I mean by that?

we may be enticed to forget about the high-resolution image. how?

1. there may be images that suggest the image without showing it,

2. images that please us more than the grapes,

3. and/or images that go around the problem altogether.

Monday, March 4, 13overview

representing the invisible

Monday, March 4, 13

Monday, March 4, 13

here is an example. This image contains a symbol within it. once seen it is not easily forgotten. and more to our point: once seen it does not need to be shown again. ready?

Monday, March 4, 13

once seen, it can never be unseen

Monday, March 4, 13

do I ever need to show you the arrow again?

what is this? how do you know?

Monday, March 4, 13

how do you interpret this image? After you find an anchor point your eyes track to one eye, then to the other, then to the mouth, back to the eye, then up to the forehead, along the hairline to the side of the cheek, down to the chin, etc.

your eyes travel in a predictable pattern when you look at faces.

what could you do with that?

Monday, March 4, 13

you have done this before. Your saccades have travelled down this road. That’s how you know it’s a face.

discarding details

Monday, March 4, 13

Aoccdrnig to rscheearch at Cmabrigde Uinervtisy, it deosn't mttaer in waht oredr the ltteers in a wrod are, the olny iprmoatnt tihng is taht the frist and lsat ltteer be at the rghit pclae. The rset can be a toatl mses and you can sitll raed it wouthit a porbelm. Tihs is bcuseae the huamn mnid deos not raed ervey lteter by istlef, but the wrod as a wlohe.

Monday, March 4, 13

can you read this? did you need to think about it?could a machine? where is the processing that makes this possible?does it suggest ways you might hack the perceptual pipeline?

system hacks

Monday, March 4, 13

Monday, March 4, 13

say the color of each word as fast as possible. would a computer have trouble with this?

Monday, March 4, 13

this image is -not- moving

placing the viewer with representational style

Monday, March 4, 13

Monday, March 4, 13

example from scott mccloudalso: mcluhan’s hot vs. cold media

The bedroom window—the one nearest the hallway— opens outward. The upper part of A .. ,'s body is framed within it. She says "Hello" in the playful tone of someone who has slept well and awakened in a good mood; or of someone who prefers not to show what she is thinking about—if anything—and always flashes the same smile, on principle; the same smile, which can be interpreted as derision just as well as affection, or the total absence of any feeling whatever.

Besides, she has not awakened just now. It is obvious she has already taken her shower. She is still wearing her dressing gown, but her lips are freshly made up—the lipstick color the same as their natural color, a trifle deeper, and her carefully brushed hair gleams in the light from the window when she turns her head, shifting the soft, heavy curls whose black mass falls over the white silk of her shoulder.

She goes to the heavy chest against the rear partition. She opens the top drawer to take out a small object and turns back toward the light. On the log bridge the crouching native has disappeared. There is no one visible around the house. No cutting crew is working in this sector, for the moment.

Monday, March 4, 13

robbe-grillet, jealousy“objective” language

This room was too big, the ceiling was too high, the doors were too tall, and the white carpet that went from wall to wall looked like a fresh fall of snow at Lake Arrowhead. There were full-length mirrors and crystal doodads all over the place. The ivory furniture had chromium on it, and the enormous ivory drapes lay tumbled on the white carpet a yard from the windows. The white made the ivory look dirty and the ivory made the white look bled out. The windows stared towards the darkening foothills. It was going to rain soon. There was pressure in the air already.

I sat down on the edge of a deep soft chair and looked at Mrs. Regan. She was worth a stare. She was trouble. She was stretched out on a modernistic chaise-longue with her slippers off, so I stared at her legs in the sheerest silk stockings. They seemed to be arranged to stare at. They were visible to the knee and one of them well beyond. The knees were dimpled, not bony and sharp. The calves were beautiful, the ankles long and slim and with enough melodic line for a tone poem. She was tall and rangy and strong-looking. Her head was against an ivory satin cushion. Her hair was black and wiry and parted in the middle and she had the hot black eyes of the portrait in the hall. She had a good mouth and a good chin. There was a sulky droop to her lips and the lower lip was full.

She had a drink. She took a swallow from it and gave me a cool level stare over the rim of the glass.

Monday, March 4, 13

raymond chandler, the big sleep

Monday, March 4, 13

Other systems: Tineye imaginationwhat does the web “see” in an image?

Monday, March 4, 13

Monday, March 4, 13

Monday, March 4, 13

lossless serieswith rebecca baron2008-09

Monday, March 4, 13what happens when the keyframes are taken away in mpeg2?

artifact 12010

Monday, March 4, 13what would it be like to have a very large temporal buffer?

thanks!

doug goodwindgoodwin@gmail.com

Monday, March 4, 13