Image Processing and Image Filters in Java
-
Upload
dudenick101 -
Category
Documents
-
view
237 -
download
0
Transcript of Image Processing and Image Filters in Java
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 1/48
John F. McGowan, Ph.D.E-Mail: [email protected] 2/16/98 1
Image Processing and Image
Filters in Java
by John F. McGowan, Ph.D.
Desktop Video Expert Center NASA Ames Research Center
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 2/48
John F. McGowan, Ph.D.E-Mail: [email protected] 2/16/98 2
Introduction
n Java is too slow for production image processing, but may offer advantages for
prototyping and demonstration of image processing algorithms.
n Algorithm developers and scientists want
a tool that allows them to concentrate on their work (math, science), not on the tool (C, Mathematica, Java).
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 3/48
John F. McGowan, Ph.D.E-Mail: [email protected] 2/16/98 3
Image Filters in Java
n Standard Java Wrapper for Image Processing Operations.
n java.awt.image Package • ImageFilter and RGBImage Filter classes
• user derived filter classes
n In JDK 1.0 n Extended slightly in JDK 1.1
n Extended heavily in JDK 1.2 Beta
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 4/48
John F. McGowan, Ph.D.E-Mail: [email protected] 2/16/98 4
Image Filters and NASA
n Astronomical Images
• Planets
• Sun and Stars
n Satellite Images
n Aerial Photographs
n Images from Simulations • Computational Fluid Dynamics
• Solar Models
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 5/48
John F. McGowan, Ph.D.E-Mail: [email protected] 2/16/98 5
Image Filters and NASA
n Edge Detection and Enhancement
n Image Segmentation
n Image Enhancement
n Image Analysis
• e.g. Fourier Transforms
n Computer Vision
n Special Effects
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 6/48
John F. McGowan, Ph.D.E-Mail: [email protected] 2/16/98 6
Outline of Talk
n Desktop Video Expert Center
n Algorithm Development
n Image Filters and Image Processing in Java
• Java Demo
• How Java Images Work n Conclusions
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 7/48
John F. McGowan, Ph.D.E-Mail: [email protected] 2/16/98 7
Desktop Video Expert Center
n Advanced Applications (R and D)
n Installation and Support of
Videoconferencing Systems
n NASA Space Shuttle MBONE (Multicast Backbone) Broadcasts over Internet
n Evaluations of COTS Desktop Video Products
n http://zeus.arc.nasa.gov/
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 8/48
John F. McGowan, Ph.D.E-Mail: [email protected] 2/16/98 8
DVEC Advanced Applications
n Research and Evaluate Future Desktop Video and Networked Video
Technologies.n Develop New Technologies as
Appropriate
n Software and Algorithm Development n http://zeus.arc.nasa.gov/adv_apps.html
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 9/48
John F. McGowan, Ph.D.E-Mail: [email protected] 2/16/98 9
Some Current Projects
n DCTune: Perceptual Optimization of JPEG Images (with Vision Science and
Technology Group)n Studies of Wavelet, Fractal, and Other
Leading Edge Video Coding
Technologies • Smooth full-motion video over Internet?
• Smooth full-motion video over telephone?
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 10/48
John F. McGowan, Ph.D.E-Mail: [email protected] 2/16/98 10
DCTune
n Patented NASA algorithm developed by Andrew B. Watson, Al Ahumada, and
others with the Vision Science and Technology Group.
n Mathematica Prototype
n Converted to compiled C language binary executable
n http://vision.arc.nasa.gov/dctune1.1.html
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 11/48
John F. McGowan, Ph.D.E-Mail: [email protected] 2/16/98 11
Who am I?
n Converted DCTune from Mathematica Prototype to portable ANSI C language
version.n Developed commercial MPEG-1 and
MPEG-2 audio and video playback
(decoder) software written in C for PC/Windows, Power Macintosh, and Unix platforms.
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 12/48
John F. McGowan, Ph.D.E-Mail: [email protected] 2/16/98 12
Who am I?
n Image,Video, and Audio Compression Algorithms
n Ph.D. in Physics, University of Illinois at Urbana-Champaign
• Maximum Likelihood Fitting Methods
• Monte Carlo Simulations • Pattern Recognition
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 13/48
John F. McGowan, Ph.D.E-Mail: [email protected] 2/16/98 13
Developing Algorithms
n Cumbersome process
n Symbolic Manipulation Programs such
as Mathematica, MATLAB, and MAPLE n C/C++ Prototypes
n Pros and Cons to Both Methods
n Looking for Better Tools to Develop,Prototype and Demonstrate Algorithms
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 14/48
John F. McGowan, Ph.D.E-Mail: [email protected] 2/16/98 14
Traditional Algorithm
Development
n Proof of Concept in Software on Supercomputer, Mainframe or Unix
Workstation n Make Presentation or Publish a Paper
n Present Simple Demo
• Grayscale Images • Very Slow
n Fund Further Development
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 15/48
John F. McGowan, Ph.D.E-Mail: [email protected] 2/16/98 15
Traditional Algorithm
Development
n Design and Fabricate VLSI Chip Implementing the Algorithm
n Develop Better Software and wait for CPU’s to get Fast Enough
• Optimize the Software for Speed
• Add Color Support • Support Arbitrary Picture Dimensions
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 16/48
John F. McGowan, Ph.D.E-Mail: [email protected] 2/16/98 16
The New World
n CPU Speeds Heading Above 500 MHz
n Algorithms Can Run on Desktop PC
n Very Fast Transition from Prototype to Product
n Argues for developing in C/C++ in
PC/Windows environment.n What can Java do?
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 17/48
John F. McGowan, Ph.D.E-Mail: [email protected] 2/16/98 17
Mathematica Advantages
n Symbolic manipulation
n Interpreted, Interactive
n High Level Mathematics
• matrix multiplication m = a . b
• special math functions
• much more n Built-In Graphical Rendering
• Show[graphics]
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 18/48
John F. McGowan, Ph.D.E-Mail: [email protected] 2/16/98 18
Mathematica Disadvantages
n Slow
n Proprietary (Wolfram Research)
n Expensive ($2000)
n Requires Much Memory (300 MB)
n Arcane Syntax
• e.g., MapThread[f, {{a,b,c},{ap,bp,cp}}]
n Learning Curve for C or other procedural language programmers
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 19/48
John F. McGowan, Ph.D.E-Mail: [email protected] 2/16/98 19
Other Symbolic Packages
n MATLAB
n MAPLE
n REDUCE
n Probably similar advantages and disadvantages to Mathematica.
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 20/48
John F. McGowan, Ph.D.E-Mail: [email protected] 2/16/98 20
Many Symbolic Language
Prototypes
n Hard to reach mass audience.
• For example, must own Mathematica
n Usually must convert to C, assembler, or an ASIC for a product.
n Usually must embed C version in
platform OS or API • Video for Windows codec
• QuickTime Component
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 21/48
John F. McGowan, Ph.D.E-Mail: [email protected] 2/16/98 21
C/C++ Advantages
n Fast/Compiled
n Executables are Free
n Unix GNU Compiler is Free
n Mac and PC Commercial Compilers
n De Facto Standard
• Lingua Franca of Programmers
• ANSI Standard
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 22/48
John F. McGowan, Ph.D.E-Mail: [email protected] 2/16/98 22
C/C++ Disadvantages
n No Built-In Graphics
• Microsoft Windows (PC)
• Mac Toolbox (MacOS)• X Windows (Unix)
• Many Graphic File Formats (e.g. PPM)
n
No High Level Mathematics n No symbolic manipulation
n Porting Problems
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 23/48
John F. McGowan, Ph.D.E-Mail: [email protected] 2/16/98 23
ANSI C Porting Problems
n Byte-Order Dependent Code
n Gaps in Standard such as definition of
bit shift by a negative number: a >> -1n Bugs in Compilers. Video Compression
Algorithms Stress Compilers.
n Complex Multimedia Algorithms Don’t Always Port Easily
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 24/48
John F. McGowan, Ph.D.E-Mail: [email protected] 2/16/98 24
Many C/C++ Prototypes
n MPEG Software Simulation Group
n Berkeley MPEG Tools
n Wavelet Image Construction Kit
n Telenor H.263 Code
n Read/write sequences of still
images (PPM, TGAetc.)
n Often Grayscale Only
n Much work to convert to graphical applications.
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 25/48
John F. McGowan, Ph.D.E-Mail: [email protected] 2/16/98 25
Possible Java Advantages
n Immediate Demonstration on Web
• Administrators and Funding Agencies
• Colleagues • Potential Customers and Licensors
n Built-In Graphics Rendering
n Image and ImageFilter Classes n Portable
n Free
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 26/48
John F. McGowan, Ph.D.E-Mail: [email protected] 2/16/98 26
Possible Java Disadvantages
n Slow (Interpreted Byte-code)
n No High Level Mathematics
n No symbolic manipulation
n Algorithm Must be Ported from Java to C to Create Product
n Not Portable n Not Free
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 27/48
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 28/48
John F. McGowan, Ph.D.E-Mail: [email protected] 2/16/98 28
Motivation for Demo
Algorithms
n Study edge enhancement and edge detection algorithms for possible
incorporation in a compression algorithm.
n Edge detection solves many problems: character recognition, flaw detection in silicon wafers, remote surveillance, and many other applications.
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 29/48
John F. McGowan, Ph.D.E-Mail: [email protected] 2/16/98 29
Why Edge Detection for
Compression?
n Leading lossy video compression algorithms such as Discrete Cosine
Transform and wavelets throw away or distort critical details at edges.
n Need to preserve sharp edges and lines for low bitrate image coding to look “natural” to a human viewer!
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 30/48
John F. McGowan, Ph.D.E-Mail: [email protected] 2/16/98 30
Edges and Block DCT
JPEG Encoder in Paint Shop Pro
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 31/48
John F. McGowan, Ph.D.
E-Mail: [email protected] 2/16/98 31
Edges and Wavelets
Wavelet Image Construction Kit (Geoff Davis)
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 32/48
John F. McGowan, Ph.D.
E-Mail: [email protected] 2/16/98 32
Demo Java Filter Applet
n Java Filter Applet (using JDK 1.1.4)
n http://zeus.arc.nasa.gov/sword.html
n The Filter Applet works differently under different Web browsers and operating systems. :-(
n Fails completely for a few browsers and operating system combinations. :-(
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 33/48
John F. McGowan, Ph.D.
E-Mail: [email protected] 2/16/98 33
Images in Java
n Poorly documented
n java.awt Package
• Image class
n java.awt.image Package
• ImageProducer, ImageConsumer, etc.
n Here is my educated guess how it works
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 34/48
John F. McGowan, Ph.D.
E-Mail: [email protected] 2/16/98 34
Images in Java
n Image class
n ImageProducer interface
n ImageConsumer interface
n ImageFilter class (implements ImageConsumer interface)
n FilteredImageSource class (implements ImageProducer interface)
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 35/48
John F. McGowan, Ph.D.
E-Mail: [email protected] 2/16/98 35
Java Graphics and Images
Applet::paint(Graphics g) { g.drawImage(image,...) }
image = createImage(FilteredImageSource)
Applet Image FilteredImageSource
Java Virtual Machine decides when to update data?
(ImageProducer)
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 36/48
John F. McGowan, Ph.D.
E-Mail: [email protected] 2/16/98 36
Java Image Processing
Graphics FilteredImageSource
ImageFilter
ImageProducer
Applet.getImage().getSource()
Java
Java Virtual Machine decides when to update data?
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 37/48
John F. McGowan, Ph.D.
E-Mail: [email protected] 2/16/98 37
Image Class
n Abstract class represents a displayable image in a platform-independent way.
n java.awt Package (NOT java.awt.image)n Don’t use constructor to create an Image
• Applet.getImage() or createImage(producer)
n Image.getSource() returns the ImageProducer that produces the image data.
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 38/48
John F. McGowan, Ph.D.
E-Mail: [email protected] 2/16/98 38
ImageProducer Interface
n Defines the methods that any class that produces image data must define to
enable communication with ImageConsumer classes.
n FilteredImageSource (java.awt.image) is an ImageProducer
n java.awt.image Package
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 39/48
John F. McGowan, Ph.D.
E-Mail: [email protected] 2/16/98 39
ImageProducer Interface
n SOURCE of the image data
n not the IMAGE data
n file on hard disk
n URL of image (JPEG or GIF) on network
n a filter that processes an image
• FilteredImageSource
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 40/48
John F. McGowan, Ph.D.
E-Mail: [email protected] 2/16/98 40
ImageConsumer Interface
n Interface defines the methods necessary for a class that consumes image data
to communicate with a class that produces image data, an ImageProducer.
n ImageFilters in Java implement the ImageConsumer Interface
n java.awt.image Package
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 41/48
John F. McGowan, Ph.D.
E-Mail: [email protected] 2/16/98 41
ImageFilter Class
n java.awt.image Package
n Implement the ImageConsumer interface
n Represent a filter that performs an operation on an image
n Filter method run when the ImageFilter
retrieves new data from an ImageProducer? (JVM?)
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 42/48
John F. McGowan, Ph.D.
E-Mail: [email protected] 2/16/98 42
FilteredImageSource Class
n Implements the ImageProducer Interface
n Constructor
FilteredImageSource(ImageProducer orig, ImageFilter imgf)
n Image.getSource() retrieves the
ImageProducer associated with a Java Image
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 43/48
John F. McGowan, Ph.D.
E-Mail: [email protected] 2/16/98 43
FilteredImageSource Class
n Represents an object that takes data from another ImageProducer and
applies a filter, represented by an ImageFilter class, and passes the filtered image on to another java object.
n Is itself an ImageProducer
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 44/48
John F. McGowan, Ph.D.
E-Mail: [email protected] 2/16/98 44
Image Events?
n ImageProducers send “events” to ImageConsumers requesting image
data.n Images, ImageConsumers,
ImageProducers maintain local buffers with copies of the image.
n Different Java Virtual Machines decide to send image events at different times.
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 45/48
John F. McGowan, Ph.D.
E-Mail: [email protected] 2/16/98 45
Image Events?
n Standard does not define or products incorrectly implement the handshaking
between ImageProducers, Consumers,and other graphics components.
n Updating of data in different objects occurs at different times with different Java Virtual Machines.
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 46/48
John F. McGowan, Ph.D.
E-Mail: [email protected] 2/16/98 46
Conclusions
n ImageFilter Class is a Great Idea.
n Java GUI easy to code.
n Slow, but slow is acceptable for a prototype.
n Portability is NOT THERE YET!
• Biggest problem with Java ImageFilters.• Limits DEMONSTRATION of prototypes.
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 47/48
John F. McGowan, Ph.D.
E-Mail: [email protected] 2/16/98 47
Suggestions for Sun
n Improve Portability
• standardize image updating?
n More Speed (Interpreted Java)n Java to Machine Code Compiler
n High Level Math Classes
• Matrices and Matrix Operations • Special Functions (e.g. Error Function)
8/8/2019 Image Processing and Image Filters in Java
http://slidepdf.com/reader/full/image-processing-and-image-filters-in-java 48/48
John F. McGowan, Ph.D.
E-Mail: [email protected] 2/16/98 48
Where to Get This Talk
n http://zeus.arc.nasa.gov/sword.html
• The Java Demo
n http://zeus.arc.nasa.gov/jugfeb18.pdf • Adobe PDF Format Version
n http://zeus.arc.nasa.gov/jugfeb18.ppt
• Microsoft Windows Power Point Version n http://zeus.arc.nasa.gov/
• Desktop Video Expert Center Web Site