II-SDV 2016 Deep SEARCH 9

© 2016 Deep SEARCH 9 GmbH

De

ep S

EAR

CH

9

1

Deep SEARCH 9Deep SEARCH 9

Klaus Kater

Founder and Managing Partner ofDeep SEARCH 9 GmbH

Let me show you, how Deep SEARCH 9 can help you to reach out to sources on the web in

a way that can gather new intelligence.

My name is


De

ep S

EAR

CH

9

2

We at Deep SEARCH 9

…scrape it and crunch itto finally squeeze theintelligence out of it.

are the people who have the means to...

…combine this with other data toget the bigger picture…

…get you the most relevant information fromthe web…


De

ep S

EAR

CH

9

3

Being Independent

❸ Free choice of sources:• Must recent information, • Must original information,• Comparision of sources,• …

❷ Clever combination of methods and tools allows us to develop new sources for intelligence:

❶ Setup and operation of searchand web analytics based solutionsthat protect from informationpaternalism and savekeep strategicinterest

1 + 1 > 2

Let‘s setup a tracker forclinical trials based on the

European Clinical Trial Register


De

ep S

EAR

CH

9

4

News TrackerThe website of the

European Clinical Trial Register


De

ep S

EAR

CH

9

5

News Tracker

We want an easy to navigate overview, which companiesare actively executing clinical trials and for which medical

conditions. And that on a daily base.


De

ep S

EAR

CH

9

6

Like the Deep SEARCH 9 News Tracker

News Tracker


De

ep S

EAR

CH

9

7

Let‘s configure one together.Step by step…

News Tracker


De

ep S

EAR

CH

9

8

Setup Step by Step

1. Can we get to the data on a daily base?

• A manual search shows us howthis site works

https://www.clinicaltrialsregister.eu/

ctr-search/rest/download/full?query=

dateFrom=%DATE%&dateTo=%DATE%

&mode=current_page

Returns all filings in the period as text file.

This file contains full details on each clinical trial selectedmulti-state trials have been downloaded full information formember states/countries involved in the trial are included

SummaryEudraCT Number: 2015-005111-32Sponsor's Protocol Code Number: ABO-MELI-15National Competent Authority: Italy - Italian Medicines Agency Clinical Trial Type: EEA CTATrial Status: OngoingDate on which this record was first entered in the EudraCT database: 2016-04-04Link: https://www.clinicaltrialsregister.eu/ctr-search/trial/2015-005111-32/IT/

A. Protocol InformationA.1 Member State Concerned: Italy - Italian Medicines AgencyA.2 EudraCT number: 2015-005111-32A.3 Full title of the trial: Multicenter, Prospective, Comparative, RandomizedControlled Clinical investigation on the performance of Promelaxin® micro-enemas versus oral administration of Macrogol 4000, in the treatment ofchronic constipation in infants aged between 6 and 24 months.A.3 Full title of the trial (it): Indagine Clinica Multicentrica, Prospettica, Comparativa, Randomizzata, Controllata sulla prestazione di Microclismi a Base di Promelaxin® versus Macrogol 4000 per Via Orale nel Trattamento della StipsiCronica Funzionale in Lattanti di Età Compresa tra i 6 e i 24 Mesi.A.3.1 Title of the trial for lay people, in easily understood, i.e. non-technical, language: Multicenter, Prospective, Comparative, Randomized ControlledClinical investigation on the performance of Promelaxin® micro-enemas versus oral administration of Macrogol 4000, in the treatment of chronic constipationin infants aged between 6 and 24 months.A.3.1 Title of the trial for lay people, in easily understood, i.e. non-technical, language (it): Indagine Clinica Multicentrica, Prospettica, Comparativa, Randomizzata, Controllata sulla prestazione di Microclismi a Base di Promelaxin® versus Macrogol 4000 per Via Orale nel Trattamento della StipsiCronica Funzionale in Lattanti di Età Compresa tra i 6 e i 24 Mesi.A.3.2 Name or abbreviated title of the trial where available: Not availableA.3.2 Name or abbreviated title of the trial where available (it): Non Disponibile

…

This is all we need to startthe configuration!

Part 1: Problem Analysis


De

ep S

EAR

CH

9

9

Setup Step by Step

Video embedded

Deep SEARCH 9 Development Environment


De

ep S

EAR

CH

9

10

2. We need to calculate the deep web URL on a daily base3. Crawl the filings of the day4. Extract the data…

Setup Step by Step

(?s)(?i)Summary.*?(?:Trial Status:\s*(.*?)Date.*?)?Link:\s*([^\s]*).*?

A.3\s*Full title of the trial:\s*(.*?)A.3.*?B.1.1.*?Name of Sponsor:\s*(.*?)

B.1.*?Country:\s*(.*?)B.3.*?E.1.1.*?Medical condition\(s\) being investigated:\s*(.*?)

E.1.*?Medical condition in easily understood language:\s*(.*?)E.1(?:.*?

E.1.2.*?Term:\s*(.*?)E\.)?.*?E.2.*?Main objective of the trial:\s*(.*?)E.2.*?Secondary

objectives of the trial:\s*(.*?)E.2.

Building a Regular Expression to scrape the data…

…looks complicated, but the most difficult part is to decide which data to extract!

Part II: Configuration


De

ep S

EAR

CH

9

11

5. Add some nice icons based on the status(ongoing , closed , unknown )

6. Add the retrieval date (not contained in the data)7. Add all retrieved records to our archive8. Define a News Tracker Viewer

Setup Step by Step

Still configuring…

… done! Did only take a few hours.

Now the News Tracker is fully functional and delivers up-to-date information


De

ep S

EAR

CH

9

12

5. Add some nice icons based on the status(ongoing , closed , unknown )

6. Add the retrieval date (not contained in the data)7. Add all retrieved records to our archive8. Define a News Tracker Viewer

Setup Step by Step

9. Define a Faceted Viewer10. Add some Thesaurus based Query Term Expansion11. Deploy in a Web or Intranet Application12. …

Still configuring…

Now the News Tracker is fully functional and delivers up-to-date information


De

ep S

EAR

CH

9

13

News Tracker

Visit us outside and we will showYou what else can be done with theinformation available on the Web.

II-SDV 2016 Deep SEARCH 9

Internet

Transcript of II-SDV 2016 Deep SEARCH 9