Department of Biomedical Informatics University of Pittsburgh School of Medicine [email protected]...

TIES Honest Broker TrainingLearn how to access and use TIES

Department of Biomedical InformaticsUniversity of Pittsburgh School of [email protected]

Elizabeth Legowski

http://ties.upmc.comIntroduction to TIES

What is TIES?

• TIES stands for Text Information Extraction System

• Deployed UPMC-approved system for retrieving data and identifying and ordering tissue

• Currently contains 4 million de-identified surgical pathology reports from 2003-present across all UPMC hospitals– Updated monthly and automatically from clinical

systems


Some key features

• De-identification• Access from anywhere• Concept-based queries• Hierarchical indexing• Negation• Temporal queries• Honest broker support• Role-based access


TIES User Roles


TIES Uses for HSTB

• Search for reports– Facilitate researchers

• Filling tissue orders– Assigned as honest broker to protocol– Set item status, order status


TIES Website

http://ties.upmc.com• Access the TIES Application• Submit account requests• How to videos and manuals• Get status updates• Helpdesk contact information

http://ties.upmc.comLogging In

Minimum Requirements

• Windows / MAC / Linux

• Java 1.5+– Download latest version at

http://www.java.com

http://www.java.com/


Link to the TIES Client

The TIES client is served as a Java Webstart application

To access TIES:1. Access the website: http://ties.upmc.com

2. Click the ‘Launch TIES’ button:

3. Click the ‘Run’ button on the Java Webstart window. It will take a short time to download the TIES application to your computer. Fortunately, this only happens the first time you are accessing TIES. After Webstart has finished downloading, you will be asked to pick a data network to access.

http://ties.upmc.com/


Logging into TIES

1. Select ‘Test Data Network’ from the drop down box and then click the ‘OK’ button

The Test Data Network contains dummy data (not real data) and is used for training purposes.


Logging into TIES

2. Enter your username and password and click the ‘Login’ button.

Logging in takes about 1 minute.


Logging into TIES

For TIES users, there are three roles to choose from: Honest Broker, Researcher and Preliminary User.

3. Select ‘Preliminary User’ as the role from the drop down box, and the appropriate study. Click on ‘OK’ to finish the logging in process.


Logging into TIES

4. Read USAGE AGREEMENT and click ‘Accept’ if you agree to the 3 terms provided in the box.

http://ties.upmc.comPreliminary User Perspective

TIES DesktopCurrent role and distribution protocol for this session

Tool bar provides quick access

Tab bar is used to switch between open tabs


Preliminary User Perspective

Query Builder pane Dashboard View

Select report type and

sections to search

Switch query views

‘Start Search’ and ‘Start Over’ buttons

Type search terms here

Chart optionsSwitch

between query and results

‘Advanced Dashboard’ link



Query Builder pane

Text search not available in ‘Preliminary User’ perspective

At least 1 of the search terms must be present

Advanced Dashboard

Search term must be

present in the negated

form

All of the search terms

must be present




Diagram View

Currently selected

filter

Query Builder Pane

Filters palette for adding and deleting filters. Filters can also be added by right clicking in the diagram.


http://ties.upmc.comDashboard Query Builder

Simple Dashboard• The simple dashboard query works very much like a Google™ search bar. It is

designed for very basic queries like: “Search for reports with adenocarcinoma”• TIES selects the best matching NCI metathesaurus concept for each search

term entered, and returns the reports that contain the concept. If there are multiple search terms, the returned reports contain all of the matching concepts.

Enter search terms here NOTE: Race is NOT

consistently captured on UPMC System


• If a word is misspelled, it will appear with red wavy lines underneath, like this:

• To correct the spelling, right click on the word and choose the correctly spelled word from the drop downs.

• Preliminary Users are not able to create Temporal Queries at this time.

• Print Screen function can capture results which can be placed in grants, reports and preliminary research documents.

• All Dashboard queries (Basic or Advanced) will translate into Diagrams when you switch to the Diagram View. However, not all Diagram queries can be shown in the Dashboard View.

Simple Dashboard Notes


Concept Search vs. Text Search• Treating search terms as Concepts generally returns more

accurate results.• Concept searching returns reports containing the search term,

as well as reports containing synonyms of the search term.– For example, when searching for reports containing ‘Bronchiolitis

Obliterans Organizing Pneumonia’, Text search will only return reports containing exactly that string, whereas Concept search will also return reports containing BOOP, COP, and any other synonyms of BOOP.

• In the Advanced Dashboard, search terms entered in the NOT box (bottom empty search box) should be absent from the report text when doing a Text search. When performing a Concept search, the search term will need to be presented in the negated form within the report.– For example, if you enter ‘Adenocarcinoma’ in the NOT box for a Text

search, it will return all reports that do not contain the word ‘Adenocarcinoma’. If you do a Concept search, only reports containing phrases such as ‘No evidence of adenocarcinoma’ or ‘Adenocarcinoma not found’ will be returned.


Simple Dashboard ExercisesQueries with a single word search termProblem: Search for pathology reports with adenocarcinoma.

STEP 1: Type ‘adenocarcinoma’ in the search box. The text will turn blue once it matches to a concept.

STEP 2: Select the ‘Pathology’ checkbox under Report Types to limit your search to pathology reports.

STEP 3: Click on ‘Start Search’ button.

STEP 3

STEP 2

STEP 1

http://ties.upmc.comBrowsing Results

Browsing the results

Can change sub- &

category

Can change type of chart displayed

Matches categorized by ‘Event Year’

10 Matches from 20044 Matches from 2011

Bar Chart View

Chart Legend

Number of matches

returned in ‘any

section’

Hit ‘Update Chart’ button if any

changes are made to chart type or

categories


Simple Dashboard Exercises

Changes to query resultsProblem: Change chart type to a pie chart showing age distribution.

STEP 1: Click on ‘Pie’ Chart Type.

STEP 2: Click on ‘Age’ in Categorize By drop down menu.

STEP 3: Click on ‘Update Chart’ button.

STEP 1

STEP 2

STEP 3



Highest number of matches in 20-29

age range

Color coding for age range legend

Pie Chart ViewEach age

range shown



Queries with a multi-word search termProblem: Search for reports with atypical ductal hyperplasia.

STEP 1: Since “atypical ductal hyperplasia” is one concept, we type these words in the ‘All of these Concepts’ field. Type ‘atypical ductal hyperplasia’ in the search box.


STEP 2

STEP 1



Queries with multiple conceptsProblem: Search for reports with biopsies indicating atypical ductal hyperplasia.

STEP 1: Type ‘atypical ductal hyperplasia’ in the search box, a comma ‘,’, and then type ‘biopsy’.


STEP 2

STEP 1


Simple Dashboard ExercisesQueries with demographicsProblem: Create a bar chart of 30-40 year old females with atypical ductal hyperplasia categorized by event year.

STEP 1: Type ‘atypical ductal hyperplasia’ in the search box.

STEP 2: Under Report Types, select the Pathology checkbox.

STEP 3: Under Age, select ‘ranging between’; type ‘30’ in the Min box; type ‘40’ in the Max box.

STEP 4: Select ‘Female Gender’.

STEP 5: Choose ‘Bar Chart’.

STEP 6: Categorize by ‘Event Year’.

STEP 7: Click on ‘Start Search’ button. STEP 5

ST

EP

4

STEP 1

STEP 3

STEP 6

STEP 2STEP 3

http://ties.upmc.comResearcher Perspective

Switching RolesClick on ‘Switch Role’ button and

change to ‘Researcher’ perspective


Researcher Perspective

Read USAGE AGREEMENT and click ‘Accept’ if you agree to the 4 terms provided in the box.


Researcher vs. Preliminary User

Can now set “Number of Results”

Note expanded toolbar

http://ties.upmc.comAdvanced Dashboard Query Builder

Switching to Advanced Dashboard

For more advanced queries, switch to the “Advanced Dashboard” by clicking on the blue link under “More options?”

To switch to the ‘Advanced Dashboard’ – click on this blue link


Advanced Dashboard

To switch back to the ‘Basic Dashboard’ -

click on this blue link

The Advanced Dashboard is always synced with the Simple Dashboard, so if you had a query already entered in the Simple Dashboard, it will be copied over to the Advanced Dashboard.

Same as ‘Basic Dashboard’ box

Additional search boxes let you specify boolean OR and NOT logic

Specify demographic criteria for patient here

Select ‘Concept’ to map the search terms to NCI metathesaurus concepts and search with those concepts.

Select ‘text’ to treat the search terms as text strings.


Advanced Dashboard Exercises

Queries with ORProblem: Search for reports with nevus or melanoma.

STEP 1: Type ‘nevus’ in the OR search box, followed by a comma ‘,’ and ‘melanoma’.

STEP 2: Specify the ‘No. of Results:’ to ‘100’.


STEP 1 STEP 3

STEP 2



Report results in text format

Report details

Choose how report text is visually displayed

Patient details

Results in tree

format.

Reports grouped

by patient.

Click on any report in the tree to view.

Can save reports in separate

case sets. Later,

case sets can be

viewed or submitted as orders for tissue.

‘Change’ returns to original query

http://ties.upmc.comQA Options

QA Options

• Two QA options are available in the File menu:– Request Review

Use for flagging problematic reports (ex: report doesn’t fit search criteria, concepts are not colored correctly in the annotations)

– Quarantine Use for reports that are not completely de-

identified. Once quarantined, reports are not available for

searches until un-quarantined by a system administrator.

http://ties.upmc.comDiagram Query Builder

Diagram Query Builder

Diagram button

Initial diagram with a single search term box

Filters palette for adding and deleting filters. Filters can also be added by right clicking in the diagram.


Diagram Components

• A query diagram consists of boxes connected by arrows.

• A box in the diagram view is used to represent a filter constraint on the query. The arrows indicate the order in which the various constraints are checked on any report.

• The placeholders, ‘All Reports’ and ‘Filtered Reports’, are special boxes that represent the start and end of the diagram.

• The results of the query are all the reports that make it to the ‘Filtered Reports’ Placeholder.

• Forks in the path indicate boolean OR conditions.• Filters on the same path are logically ANDed together.


Filter Box Types

• Report Type

• Search Term

• NOT Search Term

• Event Year

• Gender

• Age


Interacting with the diagram

• Right clicking on any part of the diagram will display a context sensitive menu– Add/remove filter boxes, edit filter properties, or add/remove

temporal sub-queries– Menu options differ based on what was right clicked.

• All elements in the diagram can be moved around by selecting them and dragging the mouse pointer.

• You can resize any filter box by dragging any of the 8 small squares displayed around it when selected.

• Filter boxes (except the ‘All Reports’ and ‘Filtered Reports’ placeholders) can be deleted by:

– Selecting it and pressing the ‘Delete’ key on the keyboard OR– Right clicking on the filter box and selecting ‘Delete Filter’

• Can also use the Filters palette to add/remove filter boxes


Diagram Modeler Exercises

Query requiring Boolean ANDProblem: Search for reports with biopsies indicating atypical ductal hyperplasia.

STEP 1: Right-click on the green ‘Search Term’ box. A dropdown menu will appear. Choose Edit filter properties. Alternatively, double click the green ‘Search Term’ box.

STEP 2: Type ‘atypical ductal hyperplasia’ in the ‘add concept here’ textbox

STEP 3: Click the + button, or press Enter.

STEP 4: Select the term(s) that best fit(s) your concept. You can select multiple concepts if you feel they are both good matches. When multiple concepts are added to a single box, they are ORed together.

STEP 5: Click the ‘Done’ button.

continued on next slide…

STEP 1

STEP 2

ST

EP

3

STEP 4

STEP 5



STEP 6: Step 5 will send you automatically back to the original ‘Search Term Filter Properties’ popup window. Click the ‘Save’ button.

To add a filter box for biopsy we will use the Filters palette on the left-hand side of the screen:

STEP 7: Click the button in the Filters palette.

STEP 8: Similar to how we added a concept for ‘atypical ductal hyperplasia’ we need to add a concept for biopsy. Repeat steps 2-6, this time for ‘biopsy’ .

Your final diagram should look like this:

STEP 6


Diagram Modeler ExercisesQuery with Boolean ORProblem: Search for reports with either lobular carcinoma with microcalcification in the breast or primary carcinoma of the liver with cirrhosis.

STEP 1: We will reuse the empty search term box that is added when a new query is created. Modify its properties to add ‘breast’ as a concept.

STEP 2: To add ‘liver’, with an OR relationship to ‘breast’ we will select the ‘breast’ box as the reference box and right click. From the dropdown menu, choose ‘Add ORed Filter’ and click ‘OK’ for ‘Search Term’ filter type.

STEP 3: Type ‘liver’ in the ‘add concept here’ textbox.

STEP 4: Click the + button, or press Enter.

STEP 5: Select the term(s) that best fit(s) your concept.

STEP 6: Click the Select Concepts ‘Done’ button.

STEP 7: Click the ‘Save’ button.


STEP 1

STEP 2

STEP 3

ST

EP

4

STEP 6

STEP 7

STEP 5


Diagram Modeler ExercisesSTEP 8: Now we need to add ‘lobular carcinoma' and ‘microcalcification’ with a boolean AND relationship to the breast box, and add ‘primary carcinoma' and ‘cirrhosis’ to the liver box. Using steps learned before, add these search terms.

Your final diagram should look like this:


Temporal QueryProblem: Search for patients with a benign biopsy followed by a report indicating carcinoma within 5 years.

STEP 1: For a temporal query, we create the first part of the query just as we did in earlier exercises. Your diagram should look like the one shown here.




STEP 2: Now we add another query to represent the second part of the query. Right click on any clear area within the box enclosing the query to popup the query menu, and select ‘Add Temporally related sub-query’ from the menu. Alternatively, click the Patient Event button in the Filters palette. Your diagram should now look like this:




STEP 3: Edit the search term box in the second query to specify a ‘carcinoma’ concept. Your diagram should now look like this:




STEP 4: The last thing is specifying the temporal constraint. Double click on the arrow connecting the sub-queries to specify it, or right click the arrow and select ‘Modify Temporal filter properties’. Then select the ‘equal/less/greater” radio button and specify less than 5 years using the widgets. Press ‘OK’ to finish.





Now, your diagram should look like this:

The Patient Events can be as complex as you want and there can be more temporal events following Patient Event 2.



Diagram Exercise:

Find patients who had no evidence of thyrotroph adenoma less than 3 years prior to being diagnosed with Brain Hemangioma



Solution:

http://ties.upmc.comSaving & Loading Queries

Saving Queries

You can save queries in TIES for later retrieval by using the ‘Save Query’ button or ‘Save Query’ and ‘Save Query As’ options in the file menu. The ‘Save Query As’ dialog box is shown here:

Queries already saved by this user

Name and description for the query

http://ties.upmc.comSaving & Loading Queries

Opening QueriesYou can open queries previously saved by you, or other members of your distribution protocol. Go to File and choose ‘Open Query’. The ‘Open Query’ dialog box is shown here:

View queries created by others using the dropdown menu here

http://ties.upmc.comHonest Broker Perspective

Honest Broker Perspective: Querying

Can search using identifiers

Note expanded toolbar

http://ties.upmc.comHonest Broker Perspective

Honest Broker Perspective: Results

Name, MRN, and SSN shown in Patient Details

Accession number shown in Report Details.

http://ties.upmc.comTissue Orders

Case Sets and Orders

• Researchers can group reports into case sets.• Case sets can be used to group interesting cases

together and can be viewed and modified by all researchers in the distribution protocol.

• Case sets can be submitted as orders for tissue/data.

• Tissue ordering in TIES is a multi-step process involving communication with the honest broker to confirm the order.


Tissue Ordering

Step 1: Create a case set

Researcher

Path reports, additional info

My Case Sets Tab


Tissue Ordering

Step 2: Submit the order

Researcher

Submit Add to queue

Honest Broker


Tissue Ordering

Step 3: Verify the order

Honest Broker

1. Examine IRB approval

2. Verify tissue availability

3. Update status of each order item. Status options are

Not Found, Retrieved, Unusable, Reviewed, Processed, Distributed


Order Item Status

Order Item status can be any of the following. It can be set only by the Honest Broker.

• Not Found – The requested tissue for this order item was not found in the tissue bank.

• Unusable – The tissue was found, but was not usable. For example, too little tissue was left on the block.

• Retrieved – The requested tissue was retrieved successfully from the Tissue Bank and is usable.

• Reviewed – The tissue has been reviewed by the Tissue Utilization Committee and approved for distribution.

• Processed – The tissue has been prepared for distribution.• Distributed – The tissue was picked up/shipped to the

researcher.


Tissue Ordering

Step 4: Request confirmation from the researcher

Researcher

Set order status to retrieved or

incomplete

Add to awaiting confirmation queue

Honest Broker


Tissue Ordering

Step 5: Confirm the order

Researcher

Confirm orderAdd to confirmed

queue

Honest Broker


Tissue Ordering

Step 6: Fulfill the order

Honest Broker

1. Cut tissue/slides

2. Ship tissue to PI/hold tissue for collection by PI

3. Update order status to Distributed


Order StatusOrder status can be any of the following. They are displayed in the general order of occurrence• Pending – The order has been submitted by the researcher and is in the honest broker’s

queue.• Reviewing – The order is being reviewed by the Honest Broker. When an order status is

switched to this, it is displayed in the ‘In Process’ group in the Manage Orders Tab.• Incomplete – Not all the requested tissue was available/usable. Awaiting confirmation

from researcher for further processing of order. When an order status is switched to this, it is displayed in the ‘Awaiting Confirmation’ group in the Manage Orders Tab.

• Retrieved – All the tissue requested was successfully retrieved from the tissue bank. Awaiting confirmation from researcher for further processing of order. When an order status is switched to this, it is displayed in the ‘Awaiting Confirmation’ group in the Manage Orders Tab.

• Confirmed – The researcher confirms the order to be processed. • Processed – The tissue/slides have been prepared for distribution.

When an order is in the following states, it is displayed in the “Completed Orders” group in the Manage Orders tab.• Distributed – The tissue was picked up/shipped to the researcher.• Cancelled – The order was cancelled by the researcher.


Creating Case Sets

Researchers create case sets by clicking the + button under Case Sets in the Results view.

Reports can be added to existing case sets by dragging reports to the desired case set, or using the Add To button and choosing the appropriate case set.


My Case Sets TabSubmit or delete an order here

Case set name

All orders are classified into Case Sets, Submitted, or Completed orders

6 reports added as order items. Click on an order item to load its view in the right pane.

Order View


Case Set Item View for Researcher

Area to specify ordering instructions specific to this case

De-identified pathology report for this case


Submitting Case SetsResearchers can submit a case set by selecting it in the ‘My Case Sets’ tab and clicking on the ‘Submit’ button in the ‘Available Tasks’ section.

TIES will add the order to the honest broker’s queue. It will also move the order to the ‘Submitted Orders’ group for the researcher.


Honest Broker Functions

• Honest brokers have access to identified data for reports when viewing them as part of a search result or a submitted order.

• Honest brokers have a ‘Manage Orders’ tab that displays all orders that have been submitted to them.

• They can modify these orders, and update status for the orders and individual order items here.


Manage Orders Tab

A new order with 2 order items. Clicking on any order item opens its detailed view in the right pane.

Order information includes name and address of order creator and IRB information

Orders are classified into these categories.


Order Item View for Honest Brokers

Ordering instructions are specified by the researcher before the order is submitted.

Select the status of the order item and click ‘Save’ to change the status.

Information for each order item includes, the identified report information such as Accession no., and identified patient information such as name, SSN, MRN etc.


Modifying Orders as Honest Brokers

• Honest Brokers can add order items to pending orders that are present in their queue.

• Searching as an Honest Broker is similar to the Researcher’s search perspective. Honest Brokers can search with identifiers, and are shown additional identified information in addition to the de-identified information when viewing a report in the search results.

http://ties.upmc.com

TIES Team

Rebecca Crowley Jacobson (Principal Investigator)[email protected]

Girish Chavan (Product Manager)[email protected]

Kevin Mitchell (Senior Developer)[email protected]

Eugene Tseytlin (Senior Developer)[email protected]

Elizabeth Legowski (User Support)[email protected]

Department of Biomedical Informatics University of Pittsburgh School of Medicine [email protected]...

Documents

Transcript of Department of Biomedical Informatics University of Pittsburgh School of Medicine [email protected]...