Data Enhancing the RSC Archive

Colin Batchelor, Ken Karapetyan, Alexey Pshenichov, Dave Sharpe, Jon Steele, Valery

Tkachenko and Antony WilliamsACS New Orleans April 2013

Overview

• The big picture• Where we’ve been• Statistics as well as semantics• New directions in experimental data• Where we’re going

The big picture

We have journal articles going back to 1841 and the aim is to extract:•Every small molecule we can (graphics and text)•Reactions•Spectra•Data in tablesand classify every paper in a way that makes sense to the reader.

Background

• RSC Publishing moved to an all-XML workflow at the turn of the millennium.

• We digitized the backfile (to 1841) in 2005.• We launched Project Prospect in 2007.• We acquired ChemSpider in 2009.

RSC Advances

New high-volume journal covering all of chemistry launched in 2011.

Need a sensible way of navigating all this.

http://www.rsc.org/advanceshttp://www.rsc.org/RSCAdvancesSubjects

Strategy

• Use topic modelling: latent Dirichlet allocation (LDA) and Gibbs sampling to determine a set of “true” topics

Thomas L. Griffiths and Mark Steyvers, “Finding scientific topics”, Proc. Natl. Acad. Sci. USA, 2004, 101, 5228–5235.

• Publishing expertise gives us 12 broad subjects that will be intuitive to users

• Merge first set to form second• Tweak

Classify that classification

Generated 128 topics based on 2009 and 2010’s articles (> 20000 papers).

Generated Wordle images (www.wordle.net) of the topics for internal staff.

Classify that classification: results

7 topics (75, 57, 65, 67, 82, 113, 123) were rejected for being nonsense.

1 topic (127) was rejected for being too general.120 topics were classified under the 12 headings

and given names.

Examples…

Examples1: “kinetics” → Physical2: “coordination complexes” → Inorganic3: “general materials” → Materials4: “misc. organic” → Organic 5: “bacteria” → Biological + Food and health6: “theoretical” → Physical7: “cells” → Bio8: “water and solution chemistry” → Physical9: “gels” → Materials10: “inorganic material properties” → Physical + Inorganic + Materials11: “general organic” → Organic12: “coordination chemistry” → Inorganic13: “photochemistry” → Inorganic + Materials + Energy

“Very useful!”

“… will make it easier for readers to identify papers which might be interesting to them.”

“Superb!”

What now?

Shortly rolling out the subject classification to other general journals:•Chemical Communications•Chemical Science•Journal of Materials Chemistry A, B and C•New Journal of Chemistry

Beyond Prospect: further steps in text-mining

Migration to Oscar 4https://bitbucket.org/wwmm/oscar4/wiki/HomeMultiple name to structure engines

OPSIN, ACD/Labs, LexichemACD/Labs DictionaryBetter disambiguationParallelization with HadoopStructure validation and standardization (see later)Reaction extraction from text (see later)

On an experimental run with names from Organic and Biomolecular Chemistry

Is any structure returned at all by a given n2s engine?

Lexichem = a (2798)ACD = b (3049)OPSIN = c (3309)

Structure disagreements

Out of 2588 names where at least one of the engines differed or didn’t return a result:

A = ACD(1538 in total)B = Lexichem(1301 in total)C = OPSIN(2097 in total)

Iterations

With the Hadoop cluster, we can mine thousands of articles a night.

We’re initially iterating over the material back to 2000, for which we have native XML. Then it’s a case of going back and testing out the OCRed material.

http://cv.beta.rsc-us.org/

This is the beta site for•Extracting chemical structures from ChemDraw files•Most importantly: structure validation and standardization

We will be using this for all of the extracted structures.

Reaction extraction from text

We have had some preliminary experience of this with Daniel Lowe (NextMove, formerly Cambridge)’s ChemicalTagger work.

To go to ChemSpider Reactions:http://csr.dev.rsc-us.org/

Experimental data

We’ve already seen the possibilities for extracting data from organic experimental sections, but what about other sorts of data?

Given chemical structures and extracted data we may be able to start building models and making them available.

New directions in experimental data (1)

We are working with William Brouwer (Penn State) to extract data from graphs.

Obviously this is faute de mieux and we’d rather have the original data, but we’re giving a flavour of what might be possible.

Recent Work

Digitized Spectrum

Comparison of Spectra

And now on ChemSpider…

New directions in experimental data (2)

Dye solar cell data is every bit as systematic as organic experimental sections.

Human curation of results

Previously: built into partly-manual annotation workflow.

Currently: macro-scale, iterative.

Coming: Challenger

DERA

• DERA will unveil from our archive– Chemicals– Reactions– Figures– Spectra/Analytical Data– Property Data

– And yes….it will need curation and filtering!