Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and...
-
Upload
ginger-riley -
Category
Documents
-
view
216 -
download
0
Transcript of Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and...
![Page 1: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/1.jpg)
Automation and Quality in Image Digital Libraries with Annotations
Edward Fox, Uma Murthy and Ricardo Torres
Florence, Italy17 February 2007
![Page 2: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/2.jpg)
2
Outline
• Acknowledgements• Digital Libraries• Scenarios, Requirements• Superimposed Information• Content Based Information Retrieval• CBISC, SIERRA• Theory, Quality• References• Summary
![Page 3: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/3.jpg)
3
Acknowledgements: Students
• Pavel Calado, Yuxin Chen, Fernando Das Neves, Shahrooz Feizabadi, Robert France, Marcos Gonçalves, Doug Gorton, Nithiwat Kampanya, Rohit Kelapure, S.H. Kim, Neill Kipp, Aaron Krowne, Bing Liu, Ming Luo, Roberto Marchesini, Paul Mather, Sudarshan Murthy, Uma Murthy, Sanghee Oh, Ananth Raghavan, Unni. Ravindranathan, Ryan Richardson, Rao Shen, Ohm Sornil, Hussein Suleman, Ricardo da Silva Torres, Srinivas Vemuri, Wensi Xi, Seungwon Yang, Baoping Zhang, Qinwei Zhu, …
![Page 4: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/4.jpg)
4
Acknowledgements: Faculty, Staff
• Lillian Cassel, Lois Delcambre, Debra Dudley, Roger Ehrich, Joanne Eustis, Weiguo Fan, James Flanagan, C. Lee Giles, Sandy Grant, Eric Hallerman, Eberhard Hilf, John Impagliazzo, Filip Jagodzinski, Douglas Knight, Deborah Knox, Alberto Laender, David Maier, Gail McMillan, Claudia Medeiros, Manuel Perez-Quinones, Jeff Pomerantz, Naren Ramakrishnan, Layne Watson, Barbara Wildemuth, …
![Page 5: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/5.jpg)
5
Other Collaborators (Selected)
• Brazil: FUA, UFMG, UNICAMP• Case Western Reserve University• Emory, Notre Dame, Oregon State• Germany: Univ. Oldenburg• Mexico: UDLA (Puebla), Monterrey• College of NJ, Hofstra, Penn State, Villanova• Portland State University• University of Arizona, University of Florida,
Univ. of Illinois, University of Virginia• VTLS (slides on digital repositories, NDLTD)
![Page 6: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/6.jpg)
Acknowledgements: Support
ACM, Adobe, AOL, CAPES, CNI, CNPq, CONACyT, DFG, FAEPEX, FAPESP, IBM, IMLS, Microsoft, NASA, NDLTD, NLM, NSF (IIS-9986089, 0080748, 0086227, 0307867, 0325579, 0532825, 0535057, 0535060; ITR-0325579; DUE-0121679, 0121741, 0136690, 0333531, 0333601, 0435059), OCLC, SOLINET, SUN, SURA, UNESCO, US Dept. Ed. (FIPSE), VTLS, …
![Page 7: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/7.jpg)
7
Outline
• Acknowledgements
• Digital Libraries• Scenarios, Requirements• Superimposed Information• Content Based Information Retrieval• CBISC, SIERRA• Theory, Quality• References• Summary
![Page 8: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/8.jpg)
Digital Libraries --- Objectives
• World Lit.: 24hr / 7day / from desktop• Integrated “super” information systems: 5S:
Table of related areas and their coverage• Ubiquitous, Higher Quality, Lower Cost • Education, Knowledge Sharing, Discovery• Disintermediation -> Collaboration • Universities Reclaim Property• Interactive Courseware, Student Works• Scalable, Sustainable, Usable, Useful
![Page 9: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/9.jpg)
9
D ig ita l L ib ra r y C o n te n t
A rtic le s ,R e p o rts,
B o o ks
T e xtD o cum e n ts
S p ee ch ,M u s ic
V id eoA u d io
(A e ria l)P h o tos
G e og rap h icIn fo rm ation
M o d e lsS im u la tio ns
S o ftw a re ,P ro g ra m s
G e no m eH u m a n,a n im a l,
p la n t
B ioIn fo rm ation
2 D , 3 D ,V R ,C A T
Im ag es a ndG ra p h ics
C o nte n tT yp e s
![Page 10: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/10.jpg)
10
Alliteration
• 5S– Societies
• Users• Collaboration, Web 2.0
– Scenarios• Workflow, Stories• Services, Components
– Spaces: GIS– Structures: DBMS– Streams: DSMS
• 3C– Content
• Content Management Systems
– Context• Link Structure• NLP• Mental models
– Criticism, commentary• Annotation, Talmud• Cataloging, indexing• Abstracting• Summarizing• Secondary literature
![Page 11: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/11.jpg)
11
2-a: Collection development/selection policies2-b: Digitization
3-a: Text resources3-b: Multimedia3-c (8-b): File formats, transformation, migration
4-a: Metadata, cataloging, metadata markup, metadata harvesting4-b: Ontologies, classification, categorization4-c: Vocabulary control, thesauri, terminologies
4-d: Subject description4-e: Information architecture (e.g., hypertext, hypermedia)4-f: Object description and organization for a specific domain
5-a: Architecture overviews/models5-b: Applications5-c: Identifiers, handles, DOI, PURL
6-a: Info needs, relevance, evaluation6-b: Search strategy, info seeking behavior, user modeling
8-a: Repositories, archives, storage8-b (3-c): File formats, transformation, migration
9-a: Project management9-b: DL case studies9-c: DL evaluation9-d: Usability assessment, user studies
9-e: Bibliometrics, Webometrics9-f: Legal issues (e.g., copyright)9-g: Cost/economic issues9-h: Social issues
10-a: Future of DLs10-b: Education for digital librarians
Digital Objects3
Collection Development
2
Overview1
Architecture (agents, mediators)
5
CORE TOPICS
DL education and research
10
7-a: Search engines, IR, indexing methods7-b: Reference services7-c: Recommender systems
5-d: Protocols5-e: Interoperability5-f: Security
2-c: Harvesting2-d: Document and e-publishing/presentation markup
6-c: Sharing, networking, interchange (e.g., social)6-d: Interaction design, info summarization and visualization, usability assessment
User Behavior/ Interactions
6
7-d: Routing, community filtering7-e: Web publishing (e.g., wiki, rss, Moodle, etc.)Services7
8-c: Sustainability
Management and Evaluation
9
Archiving and Preservation
Integrity8
1-a (10-c): Conceptual frameworks, theories
10-c (1-a): Conceptual framework, theories10-d: DL research initiatives
Info/ Knowledge Organization
4
![Page 12: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/12.jpg)
12
Outline
• Acknowledgements• Digital Libraries
• Scenarios, Requirements• Superimposed Information• Content Based Information Retrieval• CBISC, SIERRA• Theory, Quality• References• Summary
![Page 13: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/13.jpg)
13
Consider this scenario
1. Ingrid is a graduate student in the Fisheries department doing research on freshwater fish
2. In a field visit, she finds a unique-looking fish, and wants to know more.
3. She wants to search for related information based on others’ observa-tions, in the dept. DB. Also, she wants to enter new infor-mation about the fish into the DB.
Source: http://umd.edu/ Source: http://umd.edu/
![Page 14: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/14.jpg)
14
EKEY: The electronic key for identifying freshwater fishes
![Page 15: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/15.jpg)
15
• Next, Ingrid works on an assignment to gain familiarity with the capabilities of a new Biodiversity Information System. She is required to make the system help her with her complex integrated information need:
• “Retrieve fish descriptions of all fish whose shape is similar to that shown in the figure below, which belong to genus “Notropis”, which have “large eyes” and “dorsal stripe”, and have been observed within the catchments of the “Tennessee” river.”
![Page 16: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/16.jpg)
16
Here is another scenario …• An archeologist wants to write
commentaries on artifacts discovered in the field
– Manually annotate images (and parts)
– Search for images (and parts), and annotations
– Automatically annotate/tag similar images (and parts)
– Share annotations and images
• Using an Archeology digital library in his study, he wants to be able to:
Sources: http://www.dorsetforyou.com, http://www.archaeology.org
Source: http://www.bewegende-plaatjes.net
![Page 17: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/17.jpg)
17
Functionality required
• Digital Library (DL) users need, but get little assistance, regarding tasks:– Selecting and Annotating images and parts of
images• Preserve original context of information• Manual and automated annotation
– Content-based image retrieval of images and parts of images (+ GIS + metadata + text …), machine learning of proper set of descriptors
– Sharing selections and annotations
![Page 18: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/18.jpg)
18
New Microsft Research grant
• Virginia Tech and UNICAMP (Brazil)
• Fisheries & Wildlife, Computer Science
• Tablet PCs:
Content-Based Image Retrieval
Superimposed Information
+
![Page 19: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/19.jpg)
19
Outline
• Acknowledgements• Digital Libraries• Scenarios, Requirements
• Superimposed Information• Content Based Information Retrieval• CBISC, SIERRA• Theory, Quality• References• Summary
![Page 20: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/20.jpg)
20
Superimposed information (SI)
• New interpretation of existing information– New content, new structures
• Focuses on – Information at sub-document granularity– Information from heterogeneous sources
(multimedia content)– Working with information in situ
![Page 21: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/21.jpg)
21
Origin of SI
• This basic need had been addressed in diverse ways, with varying degrees of success, for many years:– concordances, annotations, comments
– bookmarks, concept maps, digital annotations, …
• The term “SI” was coined in 1999 by researchers, currently collaborating with us, now at Portland State University– Lois Delcambre
– David Maier
![Page 22: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/22.jpg)
22
Layers in an SI system
Superimposed
Layer
Base Layer
Information Source1
Information Source2
Information Sourcen
…
marks
* Source: ICDE04 presentation by Murthy, et. al
![Page 23: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/23.jpg)
23
Benefits
• Specificity of reference• Flexibility
– Identifying interesting (parts of) objects– Making connections between selections– Managing collections of selections
• References sub-document information– Preservation of context– Facilitates easy sharing of information
![Page 24: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/24.jpg)
24
Superimposed Applications
SIMPEL: A SuperImposed Multimedia Presentation Editor and pLayer
0 5 10 15 20
A
C
B
Enhanced CMapTools
![Page 25: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/25.jpg)
25
Combining CBIR and SI
• Associate images and parts of images, with related information such as annotations, hyperlinks, metadata records, etc.
• Perform CBIR on images and parts of images that have been annotated
• Combine text- (on annotations and other associated text information) and content-based (image content) search for more effective retrieval of images and parts of images
![Page 26: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/26.jpg)
26
Outline• Acknowledgements
• Digital Libraries
• Scenarios, Requirements
• Superimposed Information
• Content Based Information Retrieval• CBISC, SIERRA
• Theory, Quality
• References
• Summary
![Page 27: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/27.jpg)
27
Content-Based Image Retrieval (CBIR)
• Retrieve images similar to a user-defined specification or pattern (e.g., shape sketch, image example)
• Goal: To support image retrieval based on content properties (e.g., shape, color or texture), usually encoded into feature vectors
![Page 28: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/28.jpg)
28
Textual information retrieval
Query on Google using Sunset and Rio de Janeiro
Query result
![Page 29: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/29.jpg)
29
Content BasedInformationRetrieval
![Page 30: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/30.jpg)
30
Effective Image Description + Feature Extraction
Feature Vector[0.98, 0.91, 0.73, ……]
R
B
G
B
![Page 31: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/31.jpg)
Image descriptors
• Image Descriptor
![Page 32: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/32.jpg)
32
Example: Histogram
Image
Corresponding histogram
• Frequency count of each individual color
• Most commonly used color feature representation
Source: Andrade, D.
![Page 33: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/33.jpg)
33
Texture Descriptors
![Page 34: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/34.jpg)
34
Contour Saliences
![Page 35: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/35.jpg)
35
Contour Segment Saliences
![Page 36: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/36.jpg)
36
Multiscale Fractal Dimension
• Complex geometric shapes
• Defined by simple algorithms
• Non integer dimension
• Invariant under scaling
![Page 37: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/37.jpg)
37
Multiscale Fractal Dimension (Experiments)
![Page 38: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/38.jpg)
38
• Introduced by Punam et al. in
2003.
• For a pixel p, it is the largest
ellipse centered at p within
the same homogeneous
region.
• It extracts local structure
information (thickness,
orientation, and anisotropy).
Tensor Scale Descriptor
![Page 39: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/39.jpg)
39
0° 180°90°
Tensor Scale Image
![Page 40: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/40.jpg)
40
Tensor Scale Image
![Page 41: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/41.jpg)
41
Tensor Scale Descriptor
![Page 42: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/42.jpg)
42
Tensor Scale Descriptor
![Page 43: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/43.jpg)
43
![Page 44: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/44.jpg)
44
A typical CBIR systemInterface
Query Specification Visualization
Image Database
Ranking
Similarity ComputationQuery-processing
Module
Query Pattern Similar Images
Feature VectorExtraction
FeatureVectors
Images
Data Insertion
![Page 45: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/45.jpg)
45
Outline
• Acknowledgements• Digital Libraries• Scenarios, Requirements• Superimposed Information• Content Based Information Retrieval
• CBISC, SIERRA• Theory, Quality• References• Summary
![Page 46: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/46.jpg)
46
CBISC
• An OAI-compliant component that supports queries on image collections using content-based image retrieval
• May be customized to support different image collections
![Page 47: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/47.jpg)
47
CBISC in ETANA
![Page 48: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/48.jpg)
48
CBISC Descriptor Training
![Page 49: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/49.jpg)
49
System’s Architecture
Mediator
InterfaceInterface
Data Insertion ModuleData Insertion Module Query Processing ModuleQuery Processing Module
GISDBMS
Geo. DBMetadataImage DB
Databases
![Page 50: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/50.jpg)
50
Content-Based ImageSearch Component
(CBISC)
OAI
EcoCollection Metadata
Taxonomic Trees
Metadata-Based Search Component
(ESSEX)
Geographic Data
Search Component
(GDSC)Web Feature Server(WFS)
GeoCollection MetadataMaps
ImageCollection Image
MetadataImage
DescriptorsImages
Image Collection
InterfaceQuery
Specification Visualization
Query Mediator
AnalysisMerging
Execution
BIS Manager
HTTP Request(ListDescriptors)
HTTP Request(GetImages)
HTTP Request(keywords)
HTTP Request(GetCapabilities)
HTTP Request(GetFeatureType)
HTTP Request(GetFeature)
![Page 51: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/51.jpg)
51
CBISC Configuration Tool
![Page 52: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/52.jpg)
52
![Page 53: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/53.jpg)
53
Integrated support for SI applications in Biomedical Information Systems
![Page 54: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/54.jpg)
54
SIERRA
• A tool that allows users to select parts of images and associate them with text annotations.
• Performs information retrieval as annotations and associated marks in two ways, either for:– images or marks similar (in content) to a
specified image or mark– annotations containing specified query terms
![Page 55: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/55.jpg)
55
Annotating an image
![Page 56: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/56.jpg)
56
Searching over annotations
![Page 57: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/57.jpg)
57
Searching over images/sub-images
![Page 58: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/58.jpg)
58
DL services
and tools
drive quality
Formal frameworks
![Page 59: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/59.jpg)
59
Outline
• Acknowledgements• Digital Libraries• Scenarios, Requirements• Superimposed Information• Content Based Information Retrieval• CBISC, SIERRA
• Theory, Quality• References• Summary
![Page 60: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/60.jpg)
60
The 5S framework
• A DL framework that defines constructs that lead to the definition of a minimal digital library
• Then, an archaeological DL• Then, a practical DL• Then, DL handling superimposed
information ...• Plus, theory based Quality Models and
Digital Librarian’s Quality Toolkit
![Page 61: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/61.jpg)
61
The 5 S’s
Ss Examples Objectives
Streams Text; video; audio; image Describes properties of the DL content such as encoding and language for textual material or particular forms of multimedia data
Structures Collection; catalog; hypertext; document; metadata
Specifies organizational aspects of the DL content
Spaces Measure; measurable, topological, vector, probabilistic
Defines logical and presentational views of several DL components
Scenarios Searching, browsing, recommending
Details the behavior of DL services
Societies Service managers, learners, teachers, etc.
Defines managers, responsible for running DL services; actors, that use those services; and relationships among them
![Page 62: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/62.jpg)
62
Browsing Collaborating Customizing Filtering Providing access Recommending Requesting Searching Visualizing
Annotating Classifying Clustering Evaluating Extracting Indexing
Measuring Publicizing
Rating Reviewing (peer)
Surveying Translating
(language)
Conserving Converting
Copying/Replicating Emulating Renewing
Translating (format)
Acquiring Cataloging
Crawling (focused) Describing Digitizing
Federating Harvesting Purchasing Submitting
Preservational Creational
Add Value
Repository-Building
Information Satisfaction
Services
Infrastructure Services
![Page 63: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/63.jpg)
63
5S
structures (d.10)streams (d.9) spaces (d.18) scenarios (d.21) societies (d. 24)
structural metadataspecification(d.25)
descriptive metadataspecification(d.26)
repository(d. 33)
collection (d. 31)
(d.34)indexingservice
structured stream (d.29)
digitalobject (d.30)
metadata catalog (d.32)
browsingservice
(d.37)
searchingservice (d.35)
digital library(minimal) (d. 38)
services (d.22)
sequence (d. 3)
graph (d. 6)function (d. 2)
measurable(d.12), measure(d.13), probability (d.14), vector (d.15), topological (d.16) spaces
event (d.10)state (d. 18)
hypertext(d.36)
sequence (d. 3)
transmission(d.23)
relation (d. 1) language (d.5)
grammar (d. 7)
tuple (d. 4)*
5S and DL formal definitions and compositions (April 2004 TOIS)
![Page 64: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/64.jpg)
64
Digital Object
RepositoryCollection Minimal DL
Metadata Catalog
Descriptive Metadata
Specification
A Minimal DL in the 5S Framework
Structural Metadata
Specification
Streams Structures Spaces Scenarios Societies
indexing
browsing searching
services
hypertext
Structured Stream
![Page 65: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/65.jpg)
65
Streams Structures Spaces Scenarios Societies
indexing
browsing searching
services
hypertext
Structured Stream
Descriptive Metadata
specification
SpaTemOrg
StraDia
Arch Descriptive Metadata specification
ArchDO
ArchObj
ArchColl
Arch Metadata catalog
ArchDColl ArchDR Minimal ArchDL
A Minimal ArchDL in the 5S Framework
![Page 66: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/66.jpg)
66
Formalizing CBIR services in DLs
![Page 67: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/67.jpg)
67
Information model
![Page 68: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/68.jpg)
68
Tools/Applications
5S MetaModel
5SGraphDL
Expert
DL Designer
5SL DL
Model
5SLGen
Practitioner
Researcher
TailoredDL
Teacher
componentpool
ODLSearch,ODLBrowse,ODLRate,ODLReview,
…….
Logging ModuleXMLLog
![Page 69: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/69.jpg)
5SQual:
A Quality Assessment
Tool for Digital Libraries
![Page 70: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/70.jpg)
70
Digital Objects
Metadata
Services
• Completeness
• Conformance
• Accessibility
• Similarity
• Significance
• Timeliness
• Efficiency
• Reliability
Numeric
Indicators
5SQual - Dimensions
![Page 71: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/71.jpg)
71
5SQual Archi-texture
![Page 72: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/72.jpg)
Evaluations – XML Report
![Page 73: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/73.jpg)
Evaluations – Charts
![Page 74: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/74.jpg)
Evaluations – Charts
![Page 75: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/75.jpg)
75
Outline
• Acknowledgements• Digital Libraries• Scenarios, Requirements• Superimposed Information• Content Based Information Retrieval• CBISC, SIERRA• Theory, Quality
• References• Summary
![Page 76: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/76.jpg)
76
References (selected)
• Uma Murthy, Ricardo da Silva Torres, Edward A. Fox: SIERRA - A Superimposed Application for Enhanced Image Description and Retrieval. ECDL 2006: 540-543
• Uma Murthy, Ricardo da Silva Torres, Edward A. Fox: Integrated Support for Superimposed Applications in Biomedical Information Systems, Virginia Tech, 2006 (for the National Library of Medicine), http://si.dlib.vt.edu/publications/NLMWhitePaperSI2.pdf .
• M. A. Gonçalves. Streams, Structures, Spaces, Scenarios, and Societies: A Formal Framework for Digital Libraries and Its Applications: Defining a Quality Model for Digital Libraries (Chapter 8) – PHD thesis, Virginia Tech CS Dept., Blacksburg, VA, 2004. http://scholar.lib.vt.edu/theses/available/etd_12052004_135923/
• M. A. Gonçalves, B. L. Moreira, E. A. Fox, L. T. Watson. What is a good digital library? - defining a quality model for digital libraries. To appear in Information Processing and Management, 2007.
• http://fox.cs.vt.edu/cv.htm
![Page 77: Automation and Quality in Image Digital Libraries with Annotations Edward Fox, Uma Murthy and Ricardo Torres Florence, Italy 17 February 2007.](https://reader030.fdocuments.in/reader030/viewer/2022032723/56649f535503460f94c788a4/html5/thumbnails/77.jpg)
77
Summary
• Acknowledgements• Digital Libraries• Scenarios, Requirements• Superimposed Information• Content Based Information Retrieval• CBISC, SIERRA• Theory, Quality• References• Summary