Publication patterns in HEP computing · − Non-pertinent entries in the data sample ! Omission...

Maria Grazia Pia, INFN Genova 1

Publication patterns in HEP computing

M. G. Pia1, T. Basaglia2, Z. W. Bell3, P. V. Dressendorfer4

1INFN Genova, Genova, Italy 2CERN, Geneva, Switzerland 3ORNL, Oak Ridge, TN, USA 4IEEE, Piscataway, NJ, USA

CHEP 2012, NYC


Analysis topics ! General tools − Geant4 − ROOT

! HEP experiments − LEP

§  ALEPH, DELPHI, L3, OPAL

− BaBar − LHC

§  ALICE, ATLAS, CMS, LHCb, TOTEM

! Grid computing − LCG

! What they publish ! How much ! Where ! Citations ! Technology vs physics ! Software vs hardware ! Software/DAQ-trigger


Data sources ! Thomson-Reuters: ISI Web of Knowledge

− CERN subscription: since 1970, conference database not included −  Search by keywords, collaboration name

! Journal web sites −  IEEE TNS − NIM, Comp. Phys. Comm. (Elsevier) −  JINST (IOP/SISSA) ➤ Full-text searches

! CERN databases − CERN Document System − Greybook

! Years: 1982-2011 (LEP), 1992-2011 (BaBar, LHC) − Reproducible sample


Data sample ! Contamination

− Non-pertinent entries in the data sample ! Omission

− Pertinent papers are not included in the data sample ➩ Cross-checks

− WoS/CDS, WoS/publishers’ web sites ! WoS inconsistencies and errors

− Total number of citations includes Conference database − Proceedings papers: false classifications and omissions ➩ Manually corrected whenever possible

! Automated analysis (whenever possible)

! Manual evaluation: abstracts and full-text papers − Some degree of subjectivity


S. Agostinelli et al. Geant4: a simulation toolkit NIM A, vol. 506, no. 3, pp. 250-303, 2003

J. Allison et al. Geant4 Developments and Applications IEEE Trans. Nucl. Sci., vol. 53, no. 1, pp. 270-278, 2006

2934 citations (14 May 2012)

2026 citations excluding proceedings

Most cited CERN publication in WoS (excluding Rev. Part. Properties)



Many papers cite the NIM paper, but they omit citing the TNS one, even though both are indicated in http://cern.ch/geant4 Many papers that use Geant4 do not cite either reference

Citation analysis: until 2011 (reproducibility)


0

100

200

300

400

500

2003 2004 2005 2006 2007 2008 2009 2010 2011

Cita

tions

Year

Geant4 NIM Geant4 TNS

0 100 200 300 400

Radiat. Prot. Dosim. J. Korean Phys. Soc.

Radiat. Meas. Appl. Radiat. Isot.

J. Phys. G JHEP

Astrop. Phys. EPJC JINST NIM B

Phys. Lett. B Phys. Rev. C

Phys. Med. Biol. Med. Phys.

Phys. Rev. Lett. TNS

Phys. Rev. D NIM A

Citations

Geant4 NIM: Citing Journals

30% Physics

75% citations (plot)

0 50 100 150 200

ISOLDE ALICE

JET EFDA BES III N TOF

MiniBooNE LUNA

CDF HARP LHCb CMS

ATLAS BaBar

Citations

G4 NIM: Citing Collaborations

LHC HEP Other

16% citations (plot) 19% citations from collaborations

Born from LHC experimental requirements Multidisciplinary sources of citations


R. Brun and F. Rademakers ROOT - An object oriented data analysis framework NIM A, vol. 389, no. 1-2, pp. 81-86, 1997

I. Antcheva et al. ROOT - A C++ framework for petabyte data storage, statistical analysis and visualization Comp. Phys Comm., vol. 180, no. 12, pp. 2499-2512, 2009





AIHENP Workshop proceedings paper

Citation analysis: until 2011 (reproducibility)


0

10

20

30

40

50

60

1997

19

98

1999

20

00

2001

20

02

2003

20

04

2005

20

06

2007

20

08

2009

20

10

2011

Cita

tions

Year

ROOT Proc. ROOT CPC

0 20 40 60 80 100 120

NIM A TNS

Comp. Phys. Comm. Phys. Rev. C Phys. Rev. D

JINST Phys. Med. Biol.

EPJC Med. Phys.

JHEP NIM B

Lect. Notes Comp. Astropart. Phys.

Citations

ROOT Proc.: Citing Journals

0 1 2 3

AUGER BELLE

D0 GLAST

H1 HADES

JET-EFDA KIMS

PHOBOS R3B

RISING BABAR ALICE ATLAS

T2K N TOF

D0 CLAS

CDF CMS

Citations

75% citations

8% of all citations from collaborations Geant4 % ROOT %

Technology 30.3 49.6 Physics 29.9 18.2 BioMedical 13.9 6.0

Field of citing journals


HEP experiments

LEP •  ALEPH •  DELPHI •  L3 •  OPAL

BaBar LHC

•  ALICE •  ATLAS •  CMS •  LHCb •  TOTEM

LEP: 1989 BaBar: 1999 LHC: 2008

Start of run

Collaboration members

Experiment

Mem

bers

0

500

1000

1500

2000

2500

3000

3500

ALEPH

DELPHI L3

OPALBabar

ALICEATLASCMSLHCb

TOTEM


Time distribution LEP: 1989 BaBar: 1999 LHC: 2008

Run start

Publication year Rescaled w.r.t. year of start run

1985 1990 1995 2000 2005 20100

50

100

150

200

Publication year

Year

Num

ber o

f pub

licat

ions

AllLEPBaBarLHC

−20 −10 0 10 200

50

100

150

Publications vs operation year

Year

Num

ber o

f pub

licat

ions

LEPBaBarLHC


Time distribution LEP: 1989 BaBar: 1999 LHC: 2008

Run start

Same as previous slide, rescaled by the number of experiment members

−20 −10 0 10 200.00

0.02

0.04

0.06

0.08

0.10

0.12Publications/member vs. operation year

Year

Num

ber

of public

ations

LEPBaBarLHC

1985 1990 1995 2000 2005 20100.00

0.02

0.04

0.06

0.08

0.10

0.12Publications/member vs. year

Year

Num

ber

of public

ations

LEPBaBarLHC


Publications Publications

Experiment

Num

ber o

f pub

licat

ions

0

100

200

300

400

500

600

ALEPH

DELPHI L3

OPAL

Babar

ALICEAT

LAS

CMSLH

Cb

TOTEM

generalphysicshardwareDAQ−triggersoftware

Technological publications

Experiment

Num

ber o

f pub

licat

ions

0.0

0.2

0.4

0.6

0.8

1.0

ALEPH

DELPHI L3

OPAL

Babar

ALICEAT

LAS

CMSLH

Cb

TOTEM

Share of hardware, software and DAQ-trigger

publications


Physics publications

LEP experiments completed their life-cycle LHC experiments: at an early stage of their physics production

●

●

●

●

●

●

●

●

●●0

100

200

300

400

500

Physics publications

Experiment

Num

ber o

f pub

licat

ions

ALEPH

DELPHI L3

OPAL

Babar

ALICE

ATLA

SCMS

LHCb

TOTEM

●●

●

●

●

● ● ● ●●

0.0

0.2

0.4

0.6

0.8

1.0

Physics publications/member

Experiment

Num

ber o

f pub

licat

ions

/mem

bers

ALEPH

DELPHI L3

OPAL

Babar

ALICE

ATLA

SCMS

LHCb

TOTEM



Roughly constant trends, once the number of publications is normalized to the number of collaborators

0

50

100

150

200

250


Experiment

Num

ber o

f pub

licat

ions

ALEPH

DELPHI L3

OPAL

Babar

ALICE

ATLA

SCMS

LHCb

TOTEM

● ●

● ●

● ●

●

●

●

●

● Software.DAQ−triggerHardware

Technological publications/member

Experiment

Num

ber o

f pub

licat

ions

/mem

bers0.00

0.02

0.04

0.06

0.08

0.10

0.12

0.14

0.16

0.18

● ●

●●

●

●● ●

●●

● Software.DAQ−triggerHardware

ALEPH

DELPHI L3

OPAL

Babar

ALICE

ATLA

SCMS

LHCb

TOTEM


Software vs. hardware

Hardware publications: approximately 4 times more than software DAQ-trigger publications: approximately 1.3 times more than software

●

●

●

●

●

●

●

●

●

●

Hardware/software publications

Experiment

Ratio

0

2

4

6

8

10

ALEPH

DELPHI L3

OPAL

Babar

ALICE

ATLASCMSLHCb

TOTEM

●●

●

●

●

●

●

●

●

●0

2

4

6

8

10

DAQ−trigger/software publications

Experiment

Ratio

0

2

4

6

8

10

ALEPH

DELPHI L3

OPAL

Babar

ALICE

ATLASCMSLHCb

TOTEM


Journals hardware

DAQ-trigger

software

Journals

Journal

Num

ber

of public

ations

0

100

200

300

400

500

600

700

EPJC

JHEP

Nuc

l. Phy

s. B

Phys. L

ett.

B

Phys. R

ev. D

Phys. R

ev. L

ett.

Z. Phy

s. C

New

J.P

hys.

EPL

Astro

p.

Phys. R

ep.

CPC

JINST

NIM

A

NIM

BTN

S

EPJC

JHEP

Nucl. Phys. B

Phys. Lett. B

Phys. Rev. D

Phys. Rev. Lett.

Z. Phys. C

New J. Phys

EPL

Astrop. Phys.

Phys. Rep.

CPC

JINST

NIM A

NIM B

IEEE TNS

TNS

NIMA

JINST


Journals, LHC publications

Journal

Nu

mb

er

of

pu

blic

atio

ns

0

100

200

300

400

EPJC

JHEP

Nuc

l. Phy

s. B

Phys. L

ett.

B

Phys. R

ev. D

Phys. R

ev. L

ett.

Z. Phy

s. C

New

J.P

hys.

EPL

Astro

p.

Phys. R

ep.

CPC

JINST

NIM

A

NIM

BTN

S

EPJC

JHEP

Nucl. Phys. B

Phys. Lett. B

Phys. Rev. D

Phys. Rev. Lett.

Z. Phys. C

New J. Phys

EPL

Astrop. Phys.

Phys. Rep.

CPC

JINST

NIM A

NIM B

IEEE TNS

Journals: LEP and LHC

Still dominated by technological publications

LHC LEP

Dominated by physics publications

Journals, LEP publications

Journal

Nu

mb

er

of

pu

blic

atio

ns

0

100

200

300

400

500

600

700

EPJC

JHEP

Nuc

l. Phy

s. B

Phys. L

ett.

B

Phys. R

ev. D

Phys. R

ev. L

ett.

Z. Phy

s. C

New

J.P

hys.

EPL

Astro

p.

Phys. R

ep.

CPC

JINST

NIM

A

NIM

BTN

S

EPJC

JHEP

Nucl. Phys. B

Phys. Lett. B

Phys. Rev. D

Phys. Rev. Lett.

Z. Phys. C

New J. Phys

EPL

Astrop. Phys.

Phys. Rep.

CPC

JINST

NIM A

NIM B

IEEE TNS


Journals: pre- and post-2000

! IEEE TNS is the most popular journal for HEP technological publications in recent years

Journals, publications 1982−1999

Journal

Nu

mb

er

of

pu

blic

atio

ns

0

100

200

300

400

EPJC

JHEP

Nuc

l. Phy

s. B

Phys. L

ett.

B

Phys. R

ev. D

Phys. R

ev. L

ett.

Z. Phy

s. C

New

J.P

hys.

EPL

Astro

p.

Phys. R

ep.

CPC

JINST

NIM

A

NIM

BTN

S

EPJC

JHEP

Nucl. Phys. B

Phys. Lett. B

Phys. Rev. D

Phys. Rev. Lett.

Z. Phys. C

New J. Phys

EPL

Astrop. Phys.

Phys. Rep.

CPC

JINST

NIM A

NIM B

IEEE TNS

Journals, publications since 2000

Journal

Nu

mb

er

of

pu

blic

atio

ns

0

100

200

300

400

EPJC

JHEP

Nuc

l. Phy

s. B

Phys. L

ett.

B

Phys. R

ev. D

Phys. R

ev. L

ett.

Z. Phy

s. C

New

J.P

hys.

EPL

Astro

p.

Phys. R

ep.

CPC

JINST

NIM

A

NIM

BTN

S

EPJC

JHEP

Nucl. Phys. B

Phys. Lett. B

Phys. Rev. D

Phys. Rev. Lett.

Z. Phys. C

New J. Phys

EPL

Astrop. Phys.

Phys. Rep.

CPC

JINST

NIM A

NIM B

IEEE TNS


Citations The most cited papers are often the general reference papers about the detector published by each experiment

Citations of the most cited paper ALEPH: 340 DELPHI: 309 L3: 509 OPAL: 473 BaBar: 859 ALICE: 116 CMS: 129 LHCb: 101 TOTEM: 35 ATLAS: ATLAS pixel detector electronics and sensors: 185

Physics

CitationsN

um

be

r o

f p

ublic

atio

ns

0 20 40 60 80 1000

10

20

30

40

50

60

70

Hardware

Citations

Nu

mb

er

of

pu

blic

atio

ns

0 20 40 60 80 1000

20

40

60

80

100

120

140

DAQ−trigger

Citations

Nu

mb

er

of

pu

blic

atio

ns

0 20 40 60 80 1000

10

20

30

40

50

60

70

Software

CitationsN

um

be

r o

f p

ublic

atio

ns

0 20 40 60 80 1000

10

20

30

40

0 citations: 4% 0 citations: 17%

0 citations: 27% 0 citations: 25%


More references

more citations

References Physics papers cite

more references than technological

papers

Bibliographical entries in software papers are often

web sites

Physics

ReferencesN

um

be

r o

f p

ublic

atio

ns

0 20 40 60 80 1000

10

20

30

40

50

Hardware

References

Nu

mb

er

of

pu

blic

atio

ns

0 20 40 60 80 1000

10

20

30

40

50

60

DAQ−Trigger

References

Nu

mb

er

of

pu

blic

atio

ns

0 20 40 60 80 1000

5

10

15

20

Software

ReferencesN

um

be

r o

f p

ublic

atio

ns

0 20 40 60 80 1000

2

4

6

8

10

12

14


Pages ! The number of

pages of a paper depends on the format of the journal −  1 pageTNS ≈ 2.5 pagesJINST

! Different journal formats in the same category

! Evolutions of the format of some journals (e.g. NIM)

Physics

PagesN

umbe

r of p

ublic

atio

ns

0 10 20 30 40 500

50

100

150

200

Hardware

Pages

Num

ber o

f pub

licat

ions

0 10 20 30 40 500

20

40

60

80

100

DAQ−Trigger

Pages

Num

ber o

f pub

licat

ions

0 10 20 30 40 500

10

20

30

40

50

Software

PagesN

umbe

r of p

ublic

atio

ns

0 10 20 30 40 500

5

10

15

20


Sources of citations to physics papers

0 5 10 15 20 25

Nucl. Phys. A Phys. Atom. Nucl. Mod. Phys. Lett. A

Phys. Rep. NIM A

J. Phys. G Acta Phys. Pol. B

Int. J. Mod. Phys. A Z. Phys. C

JHEP Phys. Rev. Lett.

Nucl. Phys. B Proc. Suppl. Nucl. Phys. B

EPJC Phys. Lett. B Phys. Rev. D

Citations (%)

DELPHI ALEPH

0 5 10 15 20 25 30

Ann. Rev. Nucl. Part. Sci. J. Cosm. Astrop. Phys.

JINST New J. Phys.

Progr. Theor. Phys. Suppl. Int. J. Mod. Phys. A

J. Phys. G Nucl. Mod. Phys. Lett. A

Nucl. Phys. A Phys. Rev. C

Acta Phys. Pol. B Phys. Rev. Lett.

EPJC Phys. Lett. B

JHEP Phys. Rev. D

Citations (%)

CMS ATLAS

LHC LEP

Samples in plots account for >90% of citations

Citations to HEP physics papers mostly come from journals specialized in HEP and a few related fields (astroparticle and nuclear physics)


Sources of citations to technological papers

0 10 20 30 40 50 60

Int. J. Mod. Phys A

Comp.Phys. Comm.

Phys. Lett. B

Nucl. Phys. B Proc. Suppl.

JHEP

Phys. Rev. D

EPJC

JINST

TNS

NIM A

Citations (%)

CMS ATLAS

0 10 20 30 40

Rep. Prog. Phys. Rev. Mod. Phys.

Ann. Rev. Nucl. Part. Sci. Phys. Rep.

Int. J. Mod. Phys. A Acta Phys. Pol. B

JHEP Phys. Rev. D

Nucle. Phys. B Comp. Phys. Comm.

Z. Phys. C Nucl. Phys. B Proc. Suppl.

TNS Phys. Lett. B

EPJC NIM A

Citations (%)

DELPHI ALEPH

Citations from HEP physics and technology journals

LHC LEP


2008-2011 More refined analysis of technological papers published since start of LHC run

0

5

10

15

20

25

ATLAS CMS LHCb ALICE TOTEM LHC

Num

ber o

f pap

ers

TNS 2008-2011

Hardware Software DAQ-trigger

0

5

10

15

20

25

30

35

40

45

50


Num

ber o

f pap

ers

NIM 2008-2011

Hardware Software


0

5

10

15

20

25

30

35

40


Num

ber o

f sel

f-cita

tions


0

5

10

15

20

25

30

35

40


Num

ber o

f out

side

cita

tions


TNS TNS

NIM A NIM A

Citations 2008-2011 Self-citations Outside citations

0

10

20

30

40

50

60

70

80


Num

ber o

f sel

f-cita

tions

Hardware Software

0

10

20

30

40

50

60

70

80


Num

ber o

f out

side

cita

tions

Hardware Software


LCG – LHC Computing Grid Sakamoto, H Data grid deployment for high energy physics in Japan CPC 2007 Shiers, J The Worldwide LHC Computing Grid (worldwide LCG) CPC 2007 Belov, S et al. LCG MCDB - a knowledgebase of Monte-Carlo simulated events CPC 2008 Yin, Fet al. Grid resource management policies for load-balancing and energy-saving

by vacation queuing theory CPC 2009

Malawski, M et al. Invocation of operations from script-based Grid applications Fut. Gen. Comp. Syst.

2010

Huedo, E et al. A modular meta-scheduling architecture for interfacing with pre-WS and WS Grid resource management services

Fut. Gen. Comp. Syst.

2007

Agarwal, A et al. GridX1: A Canadian computational grid Fut. Gen. Comp. Syst.

2007

Chytracek, R et al. POOL development status and production experience TNS 2005 Hatlo, M et al. Developments of mathematical software libraries for the LHC experiments TNS 2005 Pfeiffer, A et al. The LCG PI project: Using interfaces for physics data analysis TNS 2005 Munro, C et al. Measurement of the LCG2 and gLite File Catalogue's performance TNS 2006 Li, H Realistic Workload Modeling and Its Performance Impacts in Large-Scale

eScience Grids IEEE Trans. Par. Distr. Syst.

2010

Andreeva, J et al. High-Energy Physics on the Grid: the ATLAS and CMS Experience J. Grid Comp. 2008 Munoz, VM et al. A Decentralized Deployment Strategy and Performance Evaluation of

LCG File Catalog Service J. Grid Comp. 2011

Hou, S et al. PacCAF: a Grid Portal in Pacific Asia for the CDF Experiment J. Grid Comp. 2009 Kim, BK et al. A Composition of Monitoring Services for the LHC Computing Grid J. Grid Comp. 2009

WoS


LCG

2004 2006 2008 2010 20120

1

2

3

4

Publication years

Year

Nu

mb

er

of

pu

blic

atio

ns

Journals

Journal

Nu

mb

er

of

pu

blic

atio

ns

0

1

2

3

4

5

CPC

Fut. G

en.

TNS

Tran

s. P

ar.

J Grid

Com

p

0 5 10 15 20 25 30 350

1

2

3

4

Citations

Citations

Nu

mb

er

of

pu

blic

atio

ns

0 10 20 30 40 500

1

2

3

4

References

References

Nu

mb

er

of

pu

blic

atio

ns

Small sample of publications Hard to perform any statistical analysis


Conclusions ! Software is largely underrepresented in

HEP scholarly literature w.r.t. hardware ! Publication patterns appear similar in

the LEP and LHC era ! Citation patterns are different for

publications by HEP experiments and about general software tools

Publish! …and don’t forget to cite

Publication patterns in HEP computing · − Non-pertinent entries in the data sample ! Omission...

Documents

Transcript of Publication patterns in HEP computing · − Non-pertinent entries in the data sample ! Omission...