Supplement 145 Whole Slide Imaging – background and design decisions
-
Upload
dieter-herring -
Category
Documents
-
view
21 -
download
0
description
Transcript of Supplement 145 Whole Slide Imaging – background and design decisions
Supplement 145 Whole Slide Imaging –background and design decisions
Harry Solomon GE Healthcare
2 Sup 145 WSIHarry Solomon
DICOM BASICS
3 Sup 145 WSIHarry Solomon
DICOM Image Information Object Definition
PatientModule
PatientStudy
Module
GeneralSeriesModule
GeneralStudy
Module
Frame ofReference
Module
GeneralEquipment
Module
GeneralImage
Module
ImagePlane
Module
Contrast/Bolus
Module
ImagePixel
Module
ModalityImage
Module
Multi-frame
Module…
VOILUT
Module
SOPCommonModule
PatientInformation
StudyInformation
SeriesInformation
Image(Instance)
Information
Patient NamePatient ID
Patient SexPatient Birthdate
Study Unique IDAccession Number
Study Date/TimeStudy Description
Referring MD
Rows/ColumnsBits per PixelPhotometric
DICOM CompositeInformation Model
Hierarchy
Dwight Simon
4 Sup 145 WSIHarry Solomon
Data Element Encoding
0020000Dhex UI 1.2.840.1.113709.9.0.0.5743.14575602.126hex
Study InstanceUnique Identifier
(0020,000D)
Instance UID encoded as “dotted decimal”
Data Set
Data Element
Tag Value Length Value Field
optional field - dependent on negotiated Transfer Syntax
ValueRepresen-
tation
Data Elem. Data Elem.Data Elem. Data Elem.
order of transmission
Attributes are the logical concepts associated with an information entity
Data elements are how attributes are encoded in an information object
Similar to TIFF
5 Sup 145 WSIHarry Solomon
(0008,0005) CS 30 ISO 2022 IR 13\ISO 2022 IR 87
(0008,0008) CS 22 ORIGINAL\PRIMARY\AXIAL
(0008,0016) UI 26 1.2.840.10008.5.1.4.1.1.2
(0008,0018) UI 58 1.2.392.200036.9116.2.6.1.48.1211393615.1211874194.564494
(0008,0020) DA 8 20080527
(0008,0021) DA 8 20080527
(0008,0022) DA 8 20080527
(0008,0023) DA 8 20080527
(0008,0030) TM 10 163836.000
(0008,0031) TM 10 164306.390
(0008,0032) TM 10 164039.850
(0008,0033) TM 10 164040.397
(0008,0050) SH 8 5162581
(0008,0060) CS 2 CT
(0008,0070) LO 8 TOSHIBA
…
(7FE0,0010) OW 524288 00 00 00 00 ff ff ff ff 00 0f 4c 4a 49 46 00 01...
Part of a DICOM object
Tags in increasing numeric orderValue length always an even numberAttributes related to modules and
information model levels all jumbled up
PatientInformation
StudyInformation
SeriesInformation
Image(Instance)
Information
6 Sup 145 WSIHarry Solomon
Attributes
Logical concepts in the description of an Information Entity
May have 0, 1 or many Values• 0 (empty) means the creating application doesn’t know the value
of the attribute, e.g. Accession Number (0008,0050) • Multi-value example: Specific Character Set (0008,0005) • Value Multiplicity (VM) specified in Part 6 (possibly further
constrained in Part 3)
Attribute value will be a complex data structure for a Sequence attribute
7 Sup 145 WSIHarry Solomon
Sequence Attributes and Items
Sequence attribute has a “value” of a structure of subsidiary attributes• Sequence Attribute name typically includes word “Sequence”• Subsidiary attributes specified in Part 3 with > character
Each instantiated set of attributes is a Sequence Item
Number of allowed Items specified in Part 3
For editorial convenience the attributes of a Sequence are often documented in a separate Table as a Macro• Include ‘x Macro’ Table m-n• Facilitates reuse of structure in other sequence attributes
8 Sup 145 WSIHarry Solomon
Example: Scheduled Protocol Code Sequence attribute
9 Sup 145 WSIHarry Solomon
Scheduled Protocol Code Sequence attributeScheduled Protocol Code Sequence (0040,0008)
>Code Value (0008,0100)
>Coding Scheme Version (0008,0103)
>Coding Scheme Designator (0008,0102)
>Code Meaning (0008,0104)
>Protocol Context Sequence (0040,0440)
>>Value Type (0040,A040)
>>Concept Name Code Sequence (0040,A043)
>>>Code Value (0008,0100)
>>>Coding Scheme Designator (0008,0102)
>>>Coding Scheme Version (0008,0103)
>>>Code Meaning (0008,0104)
>>DateTime (0040,A120)
>>Person Name (0040,A123)
>>Text Value (0040,A160)
>>Concept Code Sequence (0040,A168)
>>>Code Value (0008,0100)
…
>>Content Item Modifier Sequence (0040,0441)
>>>Value Type (0040,A040)
>>>Concept Name Code Sequence (0040,A043)
>>>>Code Value (0008,0100)
>>>>Coding Scheme Designator (0008,0102)
…
Sequence attribute encoding
Sequence Items are the “values” of Sequence attributes• Structure placed in the Data Element Value Field
Item structure is a “nested data set” of attributes• Attributes in each Item in tag order• Item “wrapped” using special data elements specified in Part 5
Sequence attributes and wrappers may have an “undefined length” flag• Length of Sequence or Item terminated by explicit Delimiter data elements
Sequence Data Element
Tag Value Length Value Field
ValueRepresen-tation SQ
ItemIntroducer
Attribute1
Attribute2
ItemDelimiter
ItemIntroducer
Attribute1
Attribute2
ItemDelimiter
SequenceDelimiter
May be “undefined length”
Specifies length of Item, or may say “undefined length”
Required if “undefined length” Item
Required if “undefined length” Sequence Attribute
11 Sup 145 WSIHarry Solomon
Image Compression
Pixel data can be monochrome, color (RGB or YCbCr), or palette color (monochrome colorized through LUT)• No definitions yet for hyperspectral, but it has been discussed
Pixel data can be ‘native DICOM’ (with color by-plane or by-pixel)
Pixel data can be compressed using standard compression schemes, and compressed stream put in pixel data element• JPEG, JPEG-LS, JPEG2000 (each lossy or lossless)• MPEG2• Run-Length Encoding (Packbits)
Private compression schemes can also be used
Compressed Image Encoding
Uses structure similar to Sequence attribute• Allows “undefined length” attribute – eliminates 232 byte limitation• 1st Item is ‘Basic Offset Table’ - pointers to individual frames of a multi-frame
image (optional)
JPEG and JPEG2000Part1 encode each frame of a multi-frame image in a separate Encapsulated Stream Fragment
JPEG2000Part2 (multicomponent) allows arbitrary mapping of frames to stream fragments to allow component collections (inter-frame compression)
Pixel Data Element
Tag Value Length Value Field
ValueRepresen-tation OB
ItemIntroducer
Basic OffsetTable
SequenceDelimiter
May be “undefined length
Specifies length of Basic Offset Table
Required if “undefined length” Pixel Data Attribute
(7FE0,0010)
ItemIntroducer
Encapsulated StreamFragment 1
ItemIntroducer
Encapsulated StreamFragment 2
Specifies length of Stream Fragment
13 Sup 145 WSIHarry Solomon
MULTIFRAME IMAGES
14 Sup 145 WSIHarry Solomon
Enhanced Multi-frame paradigm
Basic concept used for all new multiframe IODs• MR (Image and Spectroscopy), CT, XA, US, PET
Multi-frame object to support 1000+ image studies• Dynamic image header supports functional or acquisition
attributes changing during scan• Dimensions allow multiple views of data
File size flexibility through concatenations
15 Sup 145 WSIHarry Solomon
Single-frame to MultiFrame
N Objects, N Headers
N Frames, One Header
Pixel dataDimension dataPer-frame headerFixed Header
16 Sup 145 WSIHarry Solomon
Functional Groups and the Per-Frame Header
Other attributes
Per-frame Functional Groups Sequence
Shared Functional Groups Sequence
> Functional Group A
> Functional Group B
> Functional Group M
…..
Other attributes
Functional Groups with attributes constant for all
frames
Item 1 (Frame 1) Sequence of
Functional Groups for each individual frame
Pixel Data
Frame 1
Frame 2
Frame n
…..
> Functional Group C
> Functional Group B
> Functional Group M
…..
Item n (Frame n)
> Functional Group C
> Functional Group B
> Functional Group M
Item 2 (Frame 2)
> Functional Group C
…..
…..
…..
…..
> Functional Group K
17 Sup 145 WSIHarry Solomon
Table C.7.6.16-8 CARDIAC SYNCHRONIZATION MACRO ATTRIBUTES
Attribute Name Tag Type Attribute Description
Cardiac Synchronization Sequence (0018,9118) 1
>Nominal Percentage of Cardiac Phase (0020,9241) 1C
>Nominal Cardiac Trigger Delay Time (0020,9153) 1
>Actual Cardiac Trigger Delay Time (0020,9252) 1C
>Intervals Acquired (0018,1083) 3
>Intervals Rejected (0018,1084) 3
>Heart Rate (0018,1088) 3
>R – R Interval Time Nominal (0020,9251) 1C
>Low R-R Value (0018,1081) 3
>High R-R Value (0018,1082) 3
Functional Groups
Collection of set of closely related attributes • A “mini Module”
Structured as a sequence of (usually 1) item under a main Sequence attribute
Invoked as a ‘Macro’ in either Shared Functional Groups Sequence or Per-Frame Functional Groups Sequence
Keeps items together in encoding under the main Sequence attribute
18 Sup 145 WSIHarry Solomon
position
time
b-value
cardiac phase
volume
orientation
time
echo
Dimensions – properties that may change
19 Sup 145 WSIHarry Solomon
Phase (Time) Position Index
65
43
21
65
43
21
1 2 3
65
43
21
Physical Location(Stack) Index
Frame number 1-6 Frame number 7-12Frame number 13-18
Slice Order for phase 1
Phase order for slice 2
Image frames can be sorted/displayed independent of encoded frame order
Multi-phase / Multi-slice
20 Sup 145 WSIHarry Solomon
Concatenations
What is a concatenation?• set of image objects• in the same series• with the same dimension indexes• uniquely identified with a Concatenation UID (0020,9161) • “contained” image objects must have the same Instance Number
Why?• file system limits – e.g., 600 MB CD-R• pseudo real-time transfer of a stream of images
– workstation needs to post process images in near real time to figure out when the scan is to be terminated
21 Sup 145 WSIHarry Solomon
An object may be split up into two or more SOP Instances, using the same concatenation UID
Legend:
Pixel data (not on scale)
Dimension data (not on scale)
Per-frame header
Fixed Header
Concatenations
22 Sup 145 WSIHarry Solomon
IMAGE RETRIEVAL
23 Sup 145 WSIHarry Solomon
DICOM Query/Retrieve
Allows a system to query another system for a list of available images (query)
Also allows a system to request another system to send images (retrieve)
Query Request
Retrieve Request
Image(s) Send
Store Response(s)
Retrieve Response
Query Match(es)PACS
Query/RetrieveSCP
Query/RetrieveSCU
Workstation
24 Sup 145 WSIHarry Solomon
Hierarchical Query
DICOM query is not a full SQL-type feature• Limited attributes, no Join capability• Directed toward production imaging department requirements
Hierarchical data structure• (Patient), Study, Series, Image levels
– Patient attributes typically subsumed in Study level
• Query at any level requires specification of unique entity at each higher level
25 Sup 145 WSIHarry Solomon
Typical Hierarchical Query
PACS
Query/RetrieveSCP
Query/RetrieveSCU
WorkstationLevel: STUDYPatient ID: D73001Date: 20090521-20090524
Study ID: 09-35541Study UID: 1.2.789.45.63Patient ID: D73001Date: 20090521
Study ID: 09-35602Study UID: 1.2.789.87.11Patient ID: D73001Date: 20090522
Study ID: 09-35819Study UID: 1.2.789.154.3Patient ID: D73001Date: 20090524
Level: SERIESStudy UID: 1.2.789.87.11
Study UID: 1.2.789.87.11Series Num: 2Series UID: 1.2.405.31.2Modality: CT
Study UID: 1.2.789.87.11Series Num: 1Series UID: 1.2.405.31.1Modality: CT
Level: IMAGEStudy UID: 1.2.789.87.11Series UID: 1.2.405.31.1
Study UID: 1.2.789.87.11Series UID: 1.2.405.31.1Image UID: 1.2.405.31.1.99.1
Study UID: 1.2.789.87.11Series UID: 1.2.405.31.1Image UID: 1.2.405.31.1.99.2
Study UID: 1.2.789.87.11Series UID: 1.2.405.31.1Image UID: 1.2.405.31.1.99.3
26 Sup 145 WSIHarry Solomon
Classical Hierarchical Retrieve
Retrieve can be at any hierarchical level• (Patient), Study, Series, Image• Retrieve at any level requires unique ID of entity at each higher level
Object transfer can be on separate Association (C-MOVE) or on same Association (C-GET)• C-MOVE object transfer can be directed to third party
Examples:• Retrieve all objects under Study UID 1.2.789.87.11• Retrieve all objects under Study UID 1.2.789.87.11 / Series UID
1.2.405.31.1• Retrieve single object Study UID 1.2.789.87.11 / Series UID
1.2.405.31.1 / Instance UID 1.2.405.31.1.99.1
Retrieved objects sent and confirmed as wholes
27 Sup 145 WSIHarry Solomon
Interactive JPIP Retrieve
Image Store SCU and SCP can negotiate a JPEG 2000 Interactive Protocol (JPIP) Transfer Syntax• Image header (i.e., entire object minus pixel data) transferred and
confirmed as usual• Pixel data replaced by URL to JPIP service for this image
Limitations• Pixel data must be in JPEG 2000 format• Storage Commitment not allowed• Duration of availability of JPIP not specified or guaranteed
Capabilities• Retrieve subset of image (ROI)• Retrieve at a lower resolution (e.g., for quick navigation)
28 Sup 145 WSIHarry Solomon
Frame-based retrieve
Retrieve subset of frames from a multi-frame image• Selected frames of a volumetric stack (ROI)• Decimated volume (e.g., every 10th slice)• Single dimension of a multi-dimensional image• Time snippet of motion image (video)
SCU & SCP negotiate “Instance Root Retrieve” SOP Class
SCU specifies selected frames or time interval
SCP creates new multi-frame image with derivation attributes• Frame Derivation Module and Contributing Equipment Sequence• Correct subset of Functional Group Sequence Items
New in
2009
Supplement
119
29 Sup 145 WSIHarry Solomon
VOCABULARY AND STRUCTURED REPORTING
30 Sup 145 WSIHarry Solomon
Vocabulary-intensive messaging
There’s a lot of things we want to say about imaging that cannot be pre-defined in fixed DICOM attributes
•E.g., specimen processing
How do we define message attributes to handle what we need to say?
31 Sup 145 WSIHarry Solomon
Name-value pairs
< BodyPartExamined “ABDOMENPELVIS” />
<el> <name “BodyPartExamined” /><value “ABDOMENPELVIS” />
</el>
<el><name code=00180015 system=DICOM meaning=“Body
Part Examined” /> <value code=R-FAB57 system=SNOMED
meaning=“Abdomen and pelvis” /></el>
00180015 = Body Part Examined
00180015 ABDOMENPELVIS
Why would we want to do this?
32 Sup 145 WSIHarry Solomon
External coded/concept terminologies
Flexibility and extensibility
Leverage externally defined/maintained concepts
Semantic rigor through referenced dictionary/ ontology
General structure – higher layer of abstraction
Allows generalized messaging applications
Shared vocabulary across disparate systems
33 Sup 145 WSIHarry Solomon
SNOMED
Systematized Nomenclature of Medicine
Most comprehensive clinical healthcare terminology• 375,000 concepts; 900,000 relationships between concepts• Multi-hierarchically organized
Primary external vocabulary system for DICOM• Anatomy• Procedures (including radiographic views and methods)• Clinical findings
Originally developed by the College of American Pathologists, now managed by an international consortium of governmental agencies (IHTSDO)
34 Sup 145 WSIHarry Solomon
LOINC
Logical Observation Identifier Names and Codes
Standard coding system for laboratory and clinical observations• Hosted by Regenstrief Institute • Supported by National Library of Medicine
Particularly focused on names of laboratory and clinical tests• 50,000 codes; over 275,000 relationships
Major external code system for DICOM and HL7
Code SequencesTable 8.8-1 Common Attribute Set for Code Sequence Attributes
(Invoked as “Code Sequence Macro”) Attribute Name Tag Type Attribute Description
Code Value (0008,0100) 1 See Section 8.1.
Coding Scheme Designator
(0008,0102) 1 See Section 8.2.
Coding Scheme Version
(0008,0103) 1C See Section 8.2. Required if the value of Coding Scheme Designator (0008,0102) is not sufficient to identify the Code Value (0008,0100) unambiguously.
Code Meaning (0008,0104) 1 See Section 8.3.
DICOMPart 3
“Triplet coding” : code value, scheme, meaning(version seldom used)
Context Groups (Value Sets)
DICOMPart 3
DICOMPart 16
Content Items
Generic Name:Value pair using external coding for Name concept
Encoded as Item in Sequence attributes:• Acquisition Context Sequence (in image IODs)• Protocol Context Sequence (in Modality Worklist)• Content Sequence (in Structured Reporting IODs)• Specimen Preparation Step Sequence (in Specimen Module)
Content Item
Concept NameSequence
(0040,A043)
Value Type
(0040,A040)
DateTime Value (0040,A120)
Text Value (0040,A160)
Code(0008,0100)
Scheme(0008,0102)
Meaning(0008,0104)
Concept ValueSequence
(0040,A168)
Code(0008,0100)
Scheme(0008,0102)
Meaning(0008,0104)
Person Name Value (0040,A123)
Referenced SOP Sequence
(0008,1199)
Numeric Value
(0040,A30A)
Measurement Units Sequence
(0040,08EA)
Code(0008,0100)
Scheme(0008,0102)
Meaning(0008,0104)
SOP Class UID(0008,0050)
SOP Instance UID(0008,0055)
UID Value (0040,A124)
Templates
Structure for Content Items - like Modules are a structure for Attributes
Specified in DICOM Part 16
Coding Scheme
Code Value Code Meaning HL7 v3 ActClass equivalent
SRT P3-02000 Specimen collection SPECCOLLECT
SRT P3-05013 Specimen receiving CONTREG
SRT P3-4000A Sampling of tissue specimen PROC
SRT P3-00003 Staining SPCTRT
SRT P3-05000 Specimen processing SPCTRT
DCM 111729 Specimen storage STORE
39 Sup 145 WSIHarry Solomon
ANNOTATION AND SEGMENTATION
40 Sup 145 WSIHarry Solomon
DICOM annotation principles
Annotations are conveyed in information objects separate from the original image
Annotations may be created at a time much later than the image acquisition, and in a completely different environment
Multiple annotation objects can reference the same image
Selection of an annotation object for display implicitly invokes display of the referenced image
41 Sup 145 WSIHarry Solomon
Annotation types
Presentation States
Structured Reporting
Segmentation
42 Sup 145 WSIHarry Solomon
Presentation State
Softcopy Presentation States define how referenced image(s) will be displayed• Transforms to device independent grayscale/color space (LUTs)• Selection of display area (ROI) of the image• Image rotate or flip• Graphical and textual annotations, overlays, shutters
Grayscale, color, and pseudo-color SPSs
Blending SPS overlays a pseudo-color image on a grayscale image • E.g., for PET/CT• Blending on grayscale originals (currently no standard for blending of
color originals)
43 Sup 145 WSIHarry Solomon
Structured Reporting
Presentation State annotations are for human reading, not interoperable for automated applications• No controlled and coded vocabulary, no structural semantics
(relationships between annotations)
SR important for (semi-)automated imaging analysis and review processes
44 Sup 145 WSIHarry Solomon
Key Image Note
SR-type object that provides a classification and a textual comment for a referenced object• Formally known as “Key Object Selection”, but commonly
denoted “Key Image Note” after IHE use case and profile
Classifications typically identify intended subsequent use of referenced objects• “For Referring Provider”, “For Research”, “For Report
Attachment”• “Rejected for Quality Reasons”, “Signed Complete Study
Content”
45 Sup 145 WSIHarry Solomon
Segmentation
Derived image object• Uses enhanced multi-frame mechanism
Multiple segments per object• Each segment linked to a categorization• Pixels show presence of category at pixel location• Binary (1-bit/pixel) or fractional (probability or occupancy)
Segmentation object is typically in same Frame of Reference as source image
Segments can be displayed as overlays on source image
46 Sup 145 WSIHarry Solomon
Segmentation Example
Segment Sequence
Other attributes
Item 1 (Segment 1)
Pixel Data
Frame 1
Frame 2
Frame 4
Item 2 (Segment 2)
Segment Number = 1
Segment Name = White Matter
Segment Category = Brain
Segment Type = White Matter
Segment Number = 2
Segment Name = Grey Matter
Segment Category = Brain
Segment Type = Grey Matter
Per-frame Functional Groups Sequence
Item 1 (Frame 1)
Segment Identification Sequence
Referenced Segment Number = 2
Segment Identification Sequence
Item 1
Item 2 (Frame 2)
Referenced Segment Number = 3
Segment Identification Sequence
Item 1
Item 3 (Frame 3)
Frame 3
Shared Functional Groups Sequence
Item 1 (Shared – all frames)
Derivation Image Sequence
Item 3 (Segment 3)
Segment Number = 3
Segment Name = Lesion Part 1
Segment Category = Brain
Segment Type = Lesion
Item 4 (Segment 4) Segment Number = 4
Segment Name = Lesion Part 2
Segment Category = Brain
Segment Type = Lesion
Referenced Segment Number = 4
Segment Identification Sequence
Item 1
Item 4 (Frame 4)
Referenced Segment Number = 1
Item 1
Item 1
Reference to source image (external object)
White Matter Gray Matter, CSFand Partial Volume
Lesion – Part 1 Lesion – Part 2
Binary Segmentation Results
Segment 12
3 4
Segment 2
Segment 3Segment 4
47 Sup 145 WSIHarry Solomon
PATHOLOGY IN DICOM – SPECIMEN AND WORKFLOW
48 Sup 145 WSIHarry Solomon
What’s NOT in Sup145
All the modules already standardized• Patient, Study, Series, Equipment, General Image• Multi-Frame Functional Groups and Dimensions• Sup122 Specimen Module
Explicit description of workflow• Use of Modality Worklist, Modality Performed Procedure Step, Image Availability Notification, etc.
49 Sup 145 WSIHarry Solomon
Sup 122 Specimen Identification
Support for pathology lab workflow, specimen-based imaging• Gross specimens, blocks, vials, slides• Image-guided biopsy samples
Specimen Module at image level of hierarchy• Identification, processing history• May be used with current Visible Light image object definitions
Update to Modality Worklist to convey Specimen Module• Enables automated slide scanning devices to fully populate header
Update to Modality Performed Procedure Step to identify imaged specimen• Allows LIS/APLIS to track images for specimens
50 Sup 145 WSIHarry Solomon
SpecimenPhysical object
ContainerBox, Block, Slide, etc.
Is child of
n
PreparationStep
Collect, Sample, Stain, Process
Image
Isacquired
on
1
Series
1
Contains
n
1
CreatesEquipment
Modality
1 n
Study
Contains
Patient
1
Has
n
1
n
Issource
of
1
Contains
Contains1 n
n
Has1 n
1
Has
1ComponentBase, Coverslip
n
Specimen Imaging Information Model
Basic DICOM Information Model
Disambiguates specimen and container
Container is target of image
Container may have more than one specimen
Specimens have a physical derivation (preparation) from parent specimens
When more than one specimen in an imaged container, each specimen is distinguished (e.g., by position or color-coding)
Preparation Step
0-n Preparation Steps per Specimen
Each Preparation Step described by 1-n structured Content Items (name:value pairs)
• Acquisition Context plus structuring into steps
DICOM Template 8001 Specimen Preparation
Coding Scheme
Code Value Code Meaning HL7 v3 ActClass equivalent
SRT P3-02000 Specimen collection SPECCOLLECT
SRT P3-05013 Specimen receiving CONTREG
SRT P3-4000A Sampling of tissue specimen PROC
SRT P3-00003 Staining SPCTRT
SRT P3-05000 Specimen processing SPCTRT
DCM 111729 Specimen storage STORE
Preparation steps example
53 Sup 145 WSIHarry Solomon
Managed Workflow Concepts (IHE)
PROCEDURE STEP : The smallest unit of managed workin the workflow
Scheduled Procedure Step: ‘A unit of work to do’Performed Procedure Step: ‘A unit of work done’
ORDER : A request for departmental
service
REQUESTED PROCEDURE : Unit of work resulting in one Report
with associated codified, billable acts
54 Sup 145 WSIHarry Solomon
Simple Workflow
One Order – One Procedure – One Study – One Report
AcquisitionModality
ORDERA request for DepartmentalService
Imaging Department
Set of Codifiable,
Billable, ActsOne or more series of images
PerformedProcedure
StepRequestedProcedure
ScheduledProcedure
Step
Report
DICOMModality Worklist
Charles Parisot - IHE
AcquisitionModality
AcquisitionModality
Multiple Modality Steps
ORDERA request for DepartmentalService
Imaging Department
Set of Codifiable,
Billable, ActsOne or more series of images
PerformedProcedure
Step P1
ScheduledProcedure
Step B
RequestedProcedure
ScheduledProcedure
Step A
Report
One or more series of images
PerformedProcedure
Step P2
DICOMModality Worklist
DICOMModality Worklist
56 Sup 145 WSIHarry Solomon
Anatomic Pathology Imaging Workflow
Slide preparationSlide preparation history data
Whole Slide Scanner
Modality Worklist Queryby slide barcode
Images w/slide prep history
Imaging task completion w/list of imagesand specimen IDs
Workstation
Images
Interpretation Worklistby accession
Gross specimen
accessioning
Images
Surgical or biopsy
procedure
Images – X-ray, U/S, optical, etc.
Specimen accessioning data
Imaging task w/slide preparation history data
Pathology order
57 Sup 145 WSIHarry Solomon
SUP145 WHOLE SLIDE IMAGING PROPOSAL
58 Sup 145 WSIHarry Solomon
Sup145 multi-frame tiling concept
In 1 or more DICOM Series
Thumbnail Image
Intermediate Image Tiles
BaselineImage Tiles
Multi-frame image(single object)
Multi-frame image(single object)may include multipleZ-planes, color planes
Single frame image
Use multi-frame image objects (not object per tile)
59 Sup 145 WSIHarry Solomon
Alternate approach (not in draft!)
BaselineImage
JPIP low-res view of baseline image
Multi-frame (Z-planes, colors) image(single object)
Remove 64k2 image matrix restrictionCan leverage JPEG2000 Part2 multi-component compressionUse JPEG Interactive Protocol capabilities
JPIP medium-res view of baseline image
60 Sup 145 WSIHarry Solomon
Total Pixel Matrix
Total pixel matrix origin at top left hand corner of imaged volume
Frame (tile) rows and columns align with total pixel matrix rows and columns
Frames limited to 216 columns and rows
Total pixel matrix limited to 232 columns and rows
Total Pixel Matrix Origin
Columns →Rows
↓
Frame Pixel Matrix Origin
61 Sup 145 WSIHarry Solomon
Z-planes
Z-planes are identified as nominal physical height of image focal plane above reference surface (μm)
Z-plane information is used for relative spatial positioning of image planes, and nominal inter-plane distance
An image plane may track variable specimen thickness / surface contour, but only one Z-value usedSpecimen↑
Z
Cover slip
↑Z
Slide substrate (glass)
62 Sup 145 WSIHarry Solomon
Z planes track curved surface
Z plane 1, Z plane 2, Z plane 3, Z plane 4
Viktor Sebestyén Varga – 3DHISTECH Ltd.
63 Sup 145 WSIHarry Solomon
Organization of tiles into objects
All valid:
Single Multi-frame image
Multi-frame image per Z-plane
Multi-frame image per spatial region
64 Sup 145 WSIHarry Solomon
Sparse tiling
Only selected tiles encoded
Full image matrix might be encoded at lower resolution
Multi-frame hi-res image
Multi-frame med-res image
65 Sup 145 WSIHarry Solomon
Localizer Image Flavor
Thumbnail image (single frame) plus multi-resolution navigation links
Each tile of other resolution images has its corresponding area identified in thumbnail
Full description of target tiles• Object UID and frame #• Resolution• Z-plane
Multiple target frames can overlap• Different resolution, Z-plane, color, etc.
Presentation and any interactive behavior is not defined in standard
66 Sup 145 WSIHarry Solomon
Label Image Flavor
Purpose is to capture slide label• Any specimen captured is irrelevant
Image IOD includes Slide Label Module• Barcode (if deciphered)• Label Text (if deciphered)
Burned In Annotation (0028,0301) might be “NO” if the label includes only a specimen identifier and not patient identifying data
Labe
l
67 Sup 145 WSIHarry Solomon
C.8.12.2 Slide Coordinates
Used in VL Slide-Coordinates Microscopic Image IOD• Single frame image, typically from
microscope-mounted camera
Used to localize center of VL SCM Image
DICOM Frame of Reference associated with slide corner origin
Reproducibility not guaranteed across different mountings of slide, even on same equipment
Label
X →
↑Y
Specimen
Specimen
Slide Coordinates Origin
↑Z
Cover slip
Slide substrate (glass)
VL SCM Image area
68 Sup 145 WSIHarry Solomon
C.7.4.1 Frame Of Reference Module
When a Frame of Reference is identified, it is not important how the [imaging target] is positioned relative to the imaging equipment or where the origin of the Frame Of Reference is located. It is important that the position of the [imaging target] and the origin are constant in relationship to a specific Frame Of Reference
The Position Reference Indicator may or may not coincide with the origin of the fixed frame of reference related to the Frame of Reference UID. The Position Reference Indicator shall be used only for annotation purposes and is not intended to be used as a mathematical spatial reference.
69 Sup 145 WSIHarry Solomon
WSI Image Pixel Matrix
Image Matrix not necessarily aligned to slide edge, nor to Slide-Coordinates
Image Matrix origin (top left hand corner) located relative to Slide-Coordinates Frame of Reference origin (X,Y in mm)
Direction of rows and columns given as cosines in Slide-Coordinates Frame of Reference
Each tile (frame) TLHC located relative to Image Matrix origin (column, row)
Each tile center located relative to Slide-Coordinates origin (X,Y in mm)
Columns →Rows
↓
Specimen
Labe
l
Slide-Coordinates Origin
↑Z
Cover slip
Slide substrate (glass)
X →
↑ Y
Image Matrix Origin
70 Sup 145 WSIHarry Solomon
Optical paths
Each combination of light source, lenses, illumination method, detected wavelengths, etc. used in an acquisition is an optical path• Each path described in an Item of the Optical Path Sequence
Examples:• Full spectrum light, transmission, RGB color sensors• uV light, excitation, blue monochrome sensor
Each frame may specify a different optical path• Allows different colors in a single object, including hyperspectral (n
monochrome planes)• Identified in Optical Path Functional Group by reference to Optical Path
Sequence Item
71 Sup 145 WSIHarry Solomon
Optical paths
What parameters are needed?• To be added in Part 16 Context Groups
Is a “macro image” simply a selected optical path?
Illumination:Color(s)IntensityType (laser)
Filters:Color(s)Polarization
Lens: Illumination Method:TransmissionReflectionScatterExcitation
Lens: Filters:Color(s)Polarization
Sensor:Color(s)
72 Sup 145 WSIHarry Solomon
WSI Functional Groups
Standard• Pixel Measures (pixel spacing, layer thickness) – shared• Frame Content (datetime, dimensional location) – per-frame• Referenced Image, Derivation (if required for individual
frames)
WSI Specific• Plane Position (relative to total matrix and to SCM FoR)• Optical Path• Specimen Reference (if multiple specimens on slide are
automatically distinguishable)
73 Sup 145 WSIHarry Solomon
Pixel Measures functional group
Redefinition
74 Sup 145 WSIHarry Solomon
Plane Position (Slide) functional group
Attribute Name Tag Type Attribute DescriptionPlane Position (Slide) Sequence
(gggg,nn1A) 1 Describes position of frame in the Total Pixel Matrix and in the Slide Coordinate System Frame of Reference. Only a single Item may be present in this Sequence.
>Position In Image Pixel Matrix
(gggg,nn1F) 1 The coordinate of the top left pixel of the frame in the Total Pixel Matrix (see C.8.12.X3.1.1), given as column\row. Column is the horizontal position and row is the vertical position. The coordinate of the top left pixel of the Total Pixel Matrix is 1\1.
>Image Center Point Coordinates Sequence
(0040,071A) 1 Identifies the coordinates of the center point of this frame in the Slide Coordinate System Frame of Reference. Only a single Item shall be permitted in this sequence. See Section C.8.12.2.1.1 for further explanation.
Note: This attribute allows simplified transformation of a single frame of a multi-frame VL WSI SOP Instance into an instance of the VL Slide Coordinates Microscopy SOP Class.
>>X Offset in Slide Coordinate System
(0040,072A) 1 The X offset in millimeters from the Origin of the Slide Coordinate System. See Figure C.8-16.
>>Y Offset in Slide Coordinate System
(0040,073A) 1 The Y offset in millimeters from the Origin of the Slide Coordinate System. See Figure C.8-16.
>Z Offset in Slide Coordinate System
(0040,074A) 1 The Z offset in microns from the Origin of the Slide Coordinate System, nominally the surface of the glass slide substrate. See Figure C.8-17
Note: Required even if only a single focal plane was acquired.
Do we really need this sequence introducer (consistency w/ C.8.12.2)? Could just specify X and Y as center point coordinates
Do we need to separate to two attributes to support independent dimensions?
75 Sup 145 WSIHarry Solomon
Dimensions
Based on attributes in functional groups (i.e., values that change on a per-frame basis)
Typical dimensions for WSI:• Total Matrix Column Origin• Total Matrix Row Origin• Z-Plane• Optical Path (color/polarization)
Attributes used for Dimensions specified in Multi-frame Dimension Module
Each frame specifies its dimensional indexes in Frame Content functional group
• Index values (ordinals) mapped to dimensional attribute values• E.g., with (Column, Row) dimensions, and 40962 pixel frames, frame with index
value (2,3) would have origin column\row values of 8193\12289
76 Sup 145 WSIHarry Solomon
Annotations of WSI - Segmentations
Segmentations can be created frame-by-frame / pixel-by-pixel against selected frames of original image• Reference through Derivation Image Functional Group• 1-bit/source-pixel, or 8-bits/source-pixel
Segmentations can be created against arbitrary areas within a specified Frame of Reference• Requires Plane Position and Plane Orientation Functional Groups – may
not be usable with slide coordinates Frame of Reference
Display of segmentation can implicitly invoke a non-standardized overlay or blending with source image• Segmentation can specify its preferred color
77 Sup 145 WSIHarry Solomon
Annotations of WSI - Presentation States
Color Presentation State supports annotation of a source image
Displayed Area Selection allows up to 231-1 rows/columns, currently relative to frame-based rows/columns• Proposed enhancement (with new attribute and new enumerated value)
to allow Displayed Area Selection and annotation location relative to WSI total matrix, rather than to frame
• Implicitly applies to all dimensions (Z-planes, colors), only constrained by explicit frame numbers; should there be a general mechanism to limit by dimension (as is done for segments)?
Placement of annotations limited to 24-bit precision (IEEE 754 32-bit float)• May be image relative or selected display area relative• Allows sub-pixel resolution up to 8M rows/columns for image relative
78 Sup 145 WSIHarry Solomon
Presentation State - Displayed Area Selection
Attribute Name Tag Type Attribute DescriptionDisplayed Area Selection Sequence
(0070,005A) 1 A sequence of Items each of which describes the displayed area selection for a group of images or frames. Sufficient Items shall be present to describe every image and frame listed in the Presentation State Relationship Module.
One or more Items shall be present.>Referenced Image Sequence
(0008,1140) 1C
>>Include ‘Image SOP Instance Reference Macro’ Table 10-3>Pixel Origin Interpretation
(gggg,bb01) 1C For a referenced multi-frame image, specifies whether the Displayed Area Top Left Hand Corner (0070,0052)
and Displayed Area Bottom RIght Hand Corner (0070,0053) are to be interpreted relative to the individual frame pixel origins, or relative to the Total Pixel Matrix origin (see C.8.12.X3.1.1).
Required if the Referenced Image Sequence (0008,1140) >Referenced SOP Class UID (0008,1150) value is 1.2.840.10008.5.1.4.1.1.xxx (VL Whole Slide Microscopy Image). May be present otherwise.
Enumerated Values:
FRAME
VOLUME
If not present, TLHC and BRHC are defined relative to the frame pixel origins.
>Displayed Area Top Left Hand Corner
(0070,0052) 1
>Displayed Area Bottom Right Hand Corner
(0070,0053) 1
…
79 Sup 145 WSIHarry Solomon
Presentation State - Graphic Annotation Units
Attribute Name Tag Type Attribute DescriptionGraphic Annotation Sequence
(0070,0001) 1 A sequence of Items each of which represents a group of annotations composed of graphics or text or both.
One or more Items shall be present.
…>>Bounding Box Annotation Units
(0070,0003) 1C Units of measure for the axes of the text bounding box.
Defines whether or not the annotation is Image or Displayed Area relative. Both dimensions shall have the same units.
Enumerated Values:
PIXEL = Image relative position specified with sub-pixel resolution such that the origin at the Top Left Hand Corner (TLHC) of the TLHC pixel is 0.0\0.0, the Bottom Right Hand Corner (BRHC) of the TLHC pixel is 1.0\1.0, and the BRHC of the BRHC pixel is Columns\Rows (see figure C.10.5-1). The values must be within the range 0\0 to Columns\Rows.
DISPLAY = Fraction of Specified Displayed Area where 0.0\0.0 is the TLHC and 1.0\1.0 is the BRHC. The values must be within the range 0.0 to 1.0.
MATRIX = Image relative position specified with sub-pixel resolution such that the origin at the Top Left Hand Corner (TLHC) of the TLHC pixel of the Total Pixel Matrix is 0.0\0.0, the Bottom Right Hand Corner (BRHC) of the TLHC pixel is 1.0\1.0, and the BRHC of the BRHC pixel of the Total Pixel Matrix is Total Pixel Matrix Columns\Total Pixel Matrix Rows (see C.8.12.X3.1.3). The values must be within the range 0\0 to Total Pixel Matrix Columns\Total Pixel Matrix Rows. This value is valid only if the Referenced Image Sequence (0008,1140) >Referenced SOP Class UID (0008,1150) value is 1.2.840.10008.5.1.4.1.1.xxx (VL Whole Slide Microscopy Image).
Required if Bounding Box Top Left Hand Corner (0070,0010) or Bounding Box Bottom Right Hand Corner (0070,0011) is present.
80 Sup 145 WSIHarry Solomon
Modality Worklist
Scheduled Specimen Sequence added to MWL in Sup122• Allows query by Container ID (slide barcode)• Allows return from SCP of complete Specimen Module (slide processing
history to be used for imaging set up and/or inclusion in WSI header
Other parameters can be passed in Protocol Context Sequence • Template specification for Content Items
Proposed Protocol Optical Paths Sequence • Parallel to Protocol Context Sequence• General VL attributes