HOBBIT Survey results
-
Upload
holistic-benchmarking-of-big-linked-data -
Category
Engineering
-
view
166 -
download
0
Transcript of HOBBIT Survey results
![Page 1: HOBBIT Survey results](https://reader036.fdocuments.in/reader036/viewer/2022092622/5879f1c91a28ab70298b4da3/html5/thumbnails/1.jpg)
HOBBITSurvey results
Ruben Taelman
Horizon 2020GA No 688227
01/12/2016–30/11/2018
ESWC 2016Crete, GreeceJune 22, 2016
Ruben Taelman (iMinds) HOBBITSurvey results June 22, 2016 1 / 1
![Page 2: HOBBIT Survey results](https://reader036.fdocuments.in/reader036/viewer/2022092622/5879f1c91a28ab70298b4da3/html5/thumbnails/2.jpg)
Goal
In what areas are organizations active?What do people expect from benchmarks?How are benchmarks being used?
Ruben Taelman (iMinds) HOBBITSurvey results June 22, 2016 2 / 1
![Page 3: HOBBIT Survey results](https://reader036.fdocuments.in/reader036/viewer/2022092622/5879f1c91a28ab70298b4da3/html5/thumbnails/3.jpg)
Participant profiles
50 participants
Profile CountSolution provider 39Technology user 39Scientific community 40
Ruben Taelman (iMinds) HOBBITSurvey results June 22, 2016 3 / 1
![Page 4: HOBBIT Survey results](https://reader036.fdocuments.in/reader036/viewer/2022092622/5879f1c91a28ab70298b4da3/html5/thumbnails/4.jpg)
Linked Data areas
Evenly distributed:Storage / QueryingInterlinkingClassification / EnrichmentDiscoveryExtractionReasoning
Solution providers mainly active in storage and querying.
Ruben Taelman (iMinds) HOBBITSurvey results June 22, 2016 4 / 1
![Page 5: HOBBIT Survey results](https://reader036.fdocuments.in/reader036/viewer/2022092622/5879f1c91a28ab70298b4da3/html5/thumbnails/5.jpg)
Linked Data areas
Other areas:ReportingVisualization
Ruben Taelman (iMinds) HOBBITSurvey results June 22, 2016 5 / 1
![Page 6: HOBBIT Survey results](https://reader036.fdocuments.in/reader036/viewer/2022092622/5879f1c91a28ab70298b4da3/html5/thumbnails/6.jpg)
Key performance indicators
CorrectnessAccuracy (precision, recall, F-measure, mean reciprocal rank)Runtime / SpeedScalabilityMemory usageCPU usage
Ruben Taelman (iMinds) HOBBITSurvey results June 22, 2016 6 / 1
![Page 7: HOBBIT Survey results](https://reader036.fdocuments.in/reader036/viewer/2022092622/5879f1c91a28ab70298b4da3/html5/thumbnails/7.jpg)
Benchmarks
48% already evaluates software using a benchmark78% of benchmarking done for storage and querying, much less for otherdomains
Ruben Taelman (iMinds) HOBBITSurvey results June 22, 2016 7 / 1
![Page 8: HOBBIT Survey results](https://reader036.fdocuments.in/reader036/viewer/2022092622/5879f1c91a28ab70298b4da3/html5/thumbnails/8.jpg)
Datasets
Types of datasets:22% synthetic data46% real data32% both
66% is or can be made public!
Ruben Taelman (iMinds) HOBBITSurvey results June 22, 2016 8 / 1
![Page 9: HOBBIT Survey results](https://reader036.fdocuments.in/reader036/viewer/2022092622/5879f1c91a28ab70298b4da3/html5/thumbnails/9.jpg)
Dataset sizes
Innumber of triples.
Ruben Taelman (iMinds) HOBBITSurvey results June 22, 2016 9 / 1
![Page 10: HOBBIT Survey results](https://reader036.fdocuments.in/reader036/viewer/2022092622/5879f1c91a28ab70298b4da3/html5/thumbnails/10.jpg)
Survey still open!
http://goo.gl/forms/1iRIoG4Xpb
Ruben Taelman (iMinds) HOBBITSurvey results June 22, 2016 10 / 1
![Page 11: HOBBIT Survey results](https://reader036.fdocuments.in/reader036/viewer/2022092622/5879f1c91a28ab70298b4da3/html5/thumbnails/11.jpg)