Middlesex University Research RepositoryAn open access repository of

Middlesex University research

http://eprints.mdx.ac.uk

Harzing, Anne-Wil ORCID: https://orcid.org/0000-0003-1509-3003 (2016) Microsoft Academic(Search): a Phoenix arisen from the ashes? Scientometrics, 108 (3) . pp. 1637-1647. ISSN

0138-9130 [Article] (doi:10.1007/s11192-016-2026-y)

Final accepted version (with author’s formatting)

This version is available at: http://eprints.mdx.ac.uk/19984/

Copyright:

Middlesex University Research Repository makes the University’s research available electronically.

Copyright and moral rights to this work are retained by the author and/or other copyright ownersunless otherwise stated. The work is supplied on the understanding that any use for commercial gainis strictly forbidden. A copy may be downloaded for personal, non-commercial, research or studywithout prior permission and without charge.

Works, including theses and research projects, may not be reproduced in any format or medium, orextensive quotations taken from them, or their content changed in any way, without first obtainingpermission in writing from the copyright holder(s). They may not be sold or exploited commercially inany format or medium without the prior written permission of the copyright holder(s).

Full bibliographic details must be given when referring to, or quoting from full items including theauthor’s name, the title of the work, publication details where relevant (place, publisher, date), pag-ination, and for theses or dissertations the awarding institution, the degree type awarded, and thedate of the award.

If you believe that any material held in the repository infringes copyright law, please contact theRepository Team at Middlesex University via the following email address:

eprints@mdx.ac.uk

The item will be removed from the repository while any claim is being investigated.

See also repository copyright: re-use policy: http://eprints.mdx.ac.uk/policies.html#copy

http://eprints.mdx.ac.ukhttp://eprints.mdx.ac.uk/19984/mailto:eprints@mdx.ac.ukhttp://eprints.mdx.ac.uk/policies.html#copy

Microsoft Academic (Search): a Phoenix arisen from the ashes? Anne-Wil Harzing Version June 2016 Accepted for Scientometrics Copyright © 2016, Anne-Wil Harzing All rights reserved. Prof. Anne-Wil Harzing Middlesex University The Burroughs, Hendon London NW4 4BT Email: anne@harzing.com Web: www.harzing.com

1

MicrosoftAcademic(Search):aPhoenixarisenfromtheashes?

ANNE-WILHARZINGMiddlesexUniversity

TheBurroughs,Hendon,LondonNW44BTEmail:anne@harzing.comWeb:www.harzing.com

AbstractIncomparisontothemanydozensofarticlesreviewingandcomparing(coverageof)theWebofScience,Scopus,andGoogleScholar,thebibliometricresearchcommunityhaspaidverylittleattentiontoMicrosoftAcademicSearch(MAS).Animportantreasonforthebibliometriccommunity’slackofenthusiasmmighthavebeenthatMAScoveragewasfairlylimited,andthatalmostnonewcoveragehadbeenaddedsince2012.Recently,however,Microsoftintroducedanewservice–MicrosoftAcademic–builtoncontentthatsearchengineBingcrawlsfromtheweb.ThisarticleassessesMicrosoftAcademiccoveragethroughadetailedcomparisonofthepublicationandcitationrecordofasingleacademicforeachthefourmaincitationdatabases:GoogleScholar,MicrosoftAcademic,theWebofScience,andScopus.Overall,thisfirstsmall-scalecasestudysuggeststhatthenewincarnationofMicrosoftAcademicpresentsuswithanexcellentalternativeforcitationanalysis.Ifourfindingscanbeconfirmedbylarger-scalestudies,MicrosoftAcademicmightwellturnouttocombinetheadvantagesofbroadercoverage,asdisplayedbyGoogleScholar,withtheadvantageofamorestructuredapproachtodatapresentation,typicalofScopusandtheWebofScience.Ifso,thenewMicrosoftAcademicservicewouldtrulybeaPhoenixarisenfromtheashes.

2

MicrosoftAcademic(Search):aPhoenixarisenfromtheashes?

IntroIncomparisontothemanydozensofarticlesreviewingandcomparing(coverageof)theWebofScience,Scopus,andGoogleScholar(forthelatestseee.g.Delgado-López-Cózar&Repiso-Caballero,2013,Wildgaard,2015,Harzing&Alakangas,2016),thebibliometricresearchcommunityhaspaidverylittleattentiontoMicrosoftAcademicSearch.AGoogleScholarsearchforjournalarticleswithMicrosoftAcademicSearch(MAS)inthetitleprovidesonly5results.ThesamesearchforGoogleScholar,theWebofScience,orScopusprovidesmanyhundredsofjournalarticlesforeachdatabase.Thisisquitesurprisinggiventhatin2014NaturereporterRichardNoorden,afrequentcommentatoronbibliometricdevelopments,wrote:“Afewyearsago,MicrosoftAcademicSearch(MAS)wasvyingwithGoogleScholartobetheweb’spre-eminentfreescholarlysearchengine.Bothproductsindexedtensofmillionsofscholarlydocuments,trackedtheircitations,andmadeprofilepagesforacademics.[…]Thestagewassetforbibliometricbattle.” Jacso(2011)wasthefirsttowriteaboutMAS,providingareviewofthemajorcontentandsoftwarefeaturesanditsshortcomings.Hisverdictwas:“thisfreebibliometricserviceisaprojectofgreatinteresttothoseinterestedinmetrics-basedresearchperformanceevaluation”(Jacso,2011:983].Surprisingly,afullthreeyearspassedwithoutanyarticlesdealingwithMASuntilOrtegapublishedtwoarticlesin2014.Thefirstcompared771authorprofilesbetweenGoogleScholarCitationsandMAS(Ortega&Aguilo,2014)andconcludedthatGoogleScholarreportedmorepublicationsandcitations.Thesecond(Ortega,2014)usedMAStostudyco-authornetworksandhighlyrecommendedMASforcollaborationstudies,providedproblemswithduplicateprofilesandinfrequentupdatingcouldberesolved. SowhyhasthebibliometriccommunityalmostcompletelyignoredMAS?Oneofthereasonsmighthavebeenthatitsnativeinterfacewasnotverysuitableforcitationanalysis.However,thesameistrueforGoogleScholarandbibliometricresearchershaveturnedenmassetoPublishorPerish(2007)todobibliometricresearchwithGoogleScholar.PublishorPerishhasincludedasearchoptionforMASsince2013,whichwasusedbyHaley(2014)tocompareEconomics&FinancejournalsinGoogleScholarandMAS.HaleyfoundcitationslevelstobesubstantiallyhigherinGoogleScholarthaninMAS,withthemeanh-indexroughlytwiceashigh.Rankcorrelations,however,werefoundtobeveryhigh. The–todate–lastarticlepublishedonMASmightexplainthebibliometriccommunity’slackofenthusiasm.Orduña-Malea,Martín-Martín,Ayllon,&DelgadoLopez-Cozar(2014)publishedacomprehensiveanalysisofMAScoverageandshowed–asmanyusershadnodoubtnoticedthroughincidentalsearches–thatalmostnonewcoveragehadbeenaddedsince2012.However,fastforwardtwoyearsandMicrosoftAcademicSearchhasarisenfromtheasheswithanewservice–MicrosoftAcademic–builtoncontentthatsearchengineBingcrawlsfromtheweb,includingpublisherwebsites,universityrepositories,researcher,anddepartmentalwebpages.Citationcountsarethesumsofthereferencelinksbetweenthepapers.

3

However,thebigquestionthatwillburnonbibliometricians’mindsis:Isitscoverageanybetterthanitspreviousincarnation?Thisarticleprovidesafirstattempttoanswerthisquestionthroughacomparisonofthepublicationandcitationrecordofasingleacademicforeachofthefourmaincitationdatabases:GoogleScholar,MicrosoftAcademic,theWebofScience,andScopus.

AnindividualacademicrecordasacasestudyInordertoassessthecoverageofthenewMicrosoftAcademicincomparisontoGoogleScholar,Scopus,andtheWebofScience,Iconductedadetailedanalysisofmyownpublicationrecord.AlthoughthisisobviouslyalimitedtestofMicrosoftAcademicasanewdatasourceforcitationanalysis,thereareseveralreasonswhyIthinkmyownpublicationrecordpresentsanappropriatetest.

First,itincludesenoughpublications–varyingfrom47intheWebofScienceto124inGoogleScholar–toavoididiosyncraticresults.Inaddition,withover10,000GoogleScholarcitationsandrelativelyfewpublicationswithoutcitations(generally2016publicationsandconferencepapers),citationlevelsarealsohighenoughtoavoididiosyncraticresults.

Second,covering22years(1995-2016),itincludesbotholderandyoungerpublications,includingsomepapersonlyavailableinonlinefirst.ThisshouldallowustoassesstowhatextentMicrosoftAcademiccoversolderpublicationsaswellasveryrecentones.

Third,itincludesawidevarietyofpublications.Lookingatthe124GoogleScholarpublications,47areinjournalsthatcouldbeconsideredtobemainstreamintheirfield,22areinsecondaryjournals,20arebookchapters,15areconferencepapers,12arewhitepaperspublishedonlyonmywebsite,3arebooks,and3non-refereedpublications(2newsletterarticles,1companyreport).Thetwofinalonesareajournalrankingavailableonlyonmywebsite(TheJournalQualityList)andasoftwareprogram(PublishorPerish).ThisvarietyshouldallowustoassesstheextenttowhichMicrosoftAcademiccoversnon-traditionalpublications.

Fourth,virtuallyallofmyacademicpublicationsareincludedinGoogleScholar,includingallofmyjournalarticles,booksandbookchapters,aswellas12ofmy14whitepapers.TheonlytwowhitepapersthatarenotlistedinGoogleScholarrelatetoteaching(“Writingcourseworkassignments”and“Howtoaddressyourteacher”).NotallofmyconferencepapersarelistedinGoogleScholar,butthisisonlynatural,asmanyofthemneverappearedonline.Hence,myGoogleScholarpublicationrecordprovidesanexcellentbaselineforourcomparisonacrossdatabases.

DatacollectionAlldatawerecollectedonthe16thofMay2016.SearchesforGoogleScholarandMicrosoftAcademicwereconductedwithPublishorPerish(2007).PublishorPerishisusedprimarilyinconjunctionwithGoogleScholar,buthasrecentlyimplementedexperimentalMicrosoftAcademicsupport.Italsooffersextensivedataimportfacilities,providingtheabilitytoimportamongstothersScopusandWebofSciencedata.SearchesforScopusandtheWebofSciencewerethusconductedintheirnativeinterfaces,exportedandsubsequentlyimportedintoPublishorPerishtoallowforcalculationofthevariouscitationmetrics.ResultsforallfourdatabasesweresubsequentlyexportedtoExcel,allowingforone-on-onematchingofpublicationsandcomparisonofcitationscounts.

4

Onlypublicationswithsubstantiveacademiccontentwereincludedinourcomparison.Thismeansthatweexcludedbookreviews,errataandcorrigendaforallfourdatabases.Straypublications,i.e.publicationsreferringtothesamemasterrecordwithslightlydifferentbibliographicdetailsweremergedintotheirmasterrecordforbothGoogleScholarandMicrosoftAcademic.Obviousparsingerrors,suchaslistsofreviewers,werealsoexcluded,aswerepublicationsbyotherauthorsinmyeditedtextbook.TherewerefarmorestraypublicationsandparsingerrorsforGoogleScholarthanforMicrosoftAcademic.

MicrosoftAcademiconlydisplayedtwoclearparsingerrors.Inbothcasesauthorsofonepublicationwerecombinedwithpublicationdetailsofanother.Inaddition,therewereabouttenincongruousstraypublicationscreatedbypickinguppre-publicationversionswithadifferenttitleorpublicationsfromtwodifferentsources;noneofthesehadanycitations.AspecialcategoryofstraypublicationsinMicrosoftAcademicconcernedfivearticleswherecitationsweresplitbetweenaversionwiththemaintitleonly,andaversionwithboththemaintitleandasub-title.Inaddition,thereweretwoarticleswherecitationsweresplitbetweentwoversionsofthedocument,becausetheseparatorbetweenmainandsub-titlewasprocessedindifferentways.Forinstanceaquestionmarkwasvariouslyreplacedby|[quest]|andasingleletterq.

Finally,wediscoveredoneotherproblemwithMicrosoftAcademicthatwouldneedtobefixedbeforeanymetricsbasedontheyearofpublicationcouldbeused:thefactthatthedatabaseindicatedthewrongpublicationyearforsomepapers,eventhoughthecorrectjournalvolumewaslisted.IncorrectyearallocationsarebynomeansuncommoninGoogleScholareither.Infact,sevenofmy124publicationswereallocatedthewrongpublicationyearinGoogleScholar,twobecauseofinexplicableparsingerrors(thesourcedocumentdisplayedthecorrectyear)andfivebecauseGoogleScholarusedapre-publicationversionasitsmasterrecord.However,astheseincorrectyearallocationswereonlyyearoneyear“out”,thisisnotgenerallyamajorproblem.

However,incorrectyearallocationsweremorefrequentinMicrosoftAcademic:nolessthaneighteenoutofmy89publicationshadthewrongpublicationyear.Outofthese,onewasaninexplicableparsingerrorofafairlyobscurebookchapterandoneoccurredasMicrosoftAcademicuseda2012reprintinaRomanianjournalasthesourcefora2008whitepaper.JustlikeGoogleScholar,MicrosoftAcademicincorrectyearallocationsalsooccurredbecauseofusingthepre-publicationoronlinefirstversionasasourcerecord(sevenoccurrencesintotal).Inallthesecasesthepublicationyearwasonlyoneyearout,whichisunlikelytocausemajorproblems.Amoredisturbingproblemwasthefactthatnineofmypublicationscarriedthewrongyearinspiteofreferringtoasourcedocumentwiththecorrectyear.Inthiscase,thepublicationyearwasoften“wayout”(tenyearsormoreinthreecases).AllninerecordsconcernedjournalspublishedbyeitherEmeraldorTaylor&Francis,withthefiveEmeraldrecordsallbeingallocateda2013publicationyear(withactualpublicationyearsvaryingbetween2001and2012).Hence,itwouldappearthatthereisaparsingproblemwiththesetwospecificpublishers’websites,whichwillhopefullyberesolvedsoon.

ResultsFigures1and2visuallydisplaythecomparativecoverageofthefourdatabaseswithregardtopublicationsandcitations.Forbothcases,wewillfirstdiscusstheoverlapincoverageacrossthedatabasesandthenlookatthepublicationsandcitationsuniquetoeachofthefourdatabases.AsourinterestinthisarticleisinMicrosoftAcademiccoverage,wedonotprovideacomparisonbetweenGoogleScholarontheonehandand

5

theWebofScienceandScopusontheotherhand,orbetweentheWebofScienceandScopus.Therearemanypublicationsthathavealreadydonesointhepast,includingmostrecentlyHarzing&Alakangas(2016).Figure1:Comparingpublicationcoverageacrossfourdata-bases

Publications:overlapbetweenthefourdatabasesAsindicatedabove,Ihave124uniquepublicationsinGoogleScholar.Ofthese,89werealsopresentinMicrosoftAcademicSearch;thisincludedall69ofmyjournalpublications;allthreebooks,sevenofthefifteenconferencepapers,sevenofthetwentybookchapters,oneofthewhitepapersandbothofthenewsletterarticles. Ofthe89publicationslistedinMicrosoftAcademic,only46werelistedintheWebofScience.Allofthesewerejournalarticles.Thisincluded40ofthe47publicationsinmainstreamjournals,butonly6ofthe22publicationsinsecondaryjournals. Ofthe89publicationslistedinMicrosoftAcademic,only59werelistedinScopus.Allbutthreeofthesewerejournalarticles.Thisincluded44ofthe47publicationsinmainstreamjournals(includingtwoin-pressarticles)and12ofthe22publicationsinsecondaryjournals.ScopusalsocoveredtwoconferencepaperspublishedintheAcademyofManagement’sbestpapersproceedingsandonebookchapterintheseriesProgressinInternationalBusinessResearch(Emeraldpublishers),ayearlyresearchannualwithselectedpaperspresentedattheEuropeanInternationalBusinessAcademyconference.

GoogleScholar WebofScienceMicrosoftAcademic

Scopus

89

59

46

B1:0

B2:43

B3:30

A1:35 A2:1

A3:2

6

ConclusionIncomparisontotheWebofScienceandScopus,MicrosoftAcademiccoversafarlargernumberofpublicationsthatarelistedinGoogleScholarand–importantly–coversalljournalpublicationsandbooksthatarealsocoveredinGoogleScholar.ThissuggeststhatMicrosoftAcademichasexcellentcoverageofwhatareusuallyconsideredtobethemostimportantacademicoutputs:journalarticlesandbooks.

Publications:uniquecoverageinthefourdatabases

MicrosoftAcademiccomparedwithGoogleScholarTherearenopublicationscoveredinMicrosoftAcademicthatarenotcoveredinGoogleScholar(B1=0).GoogleScholarincluded35publicationsthatwerenotincludedinMicrosoftAcademic(A1=35).Asindicatedabove,MicrosoftAcademicincludedalljournalsarticlesandbooksinourcasestudy.Hencethe35publicationsuniquetoGoogleScholarwerebookchapters(13),whitepapers(11),conferencepapers(8),aweb-basedjournalranking(theJournalQualityList),asoftwareproduct(PublishorPerish),andacompanyreport. Fornearlyhalfofthesepublications(17publications),GoogleScholarrecordsareofthe“[citation]”type,indicatingthatalthoughGoogleScholarfoundcitationstothesepublications,itwasnotabletofindtheoriginalpublication.Elevenpublicationswerefoundontheauthor’spersonalacademicwebsite,threeonGoogleBooks,threeinonlineconferenceproceedings,andoneonthewebsiteofEmeraldpublishing. AsMicrosoftAcademicdidfindsevenofthebookchapters,sevenoftheconferencepapersandoneofthewhitepapers,wetriedtoestablishwhethertheydifferedinanywayfromtheonesthatwereonlylistedinGoogleScholar.ThiswaseasyforthesolewhitepaperasMicrosoftAcademicactuallyfoundareprintofthiswhitepaperinaRomanianjournal.Ofthesevenbookchapters,fourweresourcedfrompre-publicationversionsattheauthor’swebsite,onefromResearchgate,onefromaninstitutionalrepository,andonedidn’thaveasourceitem.Ofthesevenconferencepapers,fourweresourcedfromtheAcademyofManagementproceedings,onefromapre-publicationversionattheauthor’swebsiteandtwofromuniversityrepositories.Itisunclearwhysomebookchaptersandconferencepapersavailableaspre-publicationontheauthor’swebsiteweresourcedbyMicrosoftAcademicandotherswerenot.

MicrosoftAcademiccomparedwithWebofScienceIntotal,thereare43publicationscoveredinMicrosoftAcademicthatarenotcoveredinISI(B2=43).MicrosoftAcademiccoveredtwentynon-journalpublications(books,bookchapters,conferencepapers,whitepapers,andnewsletterarticles)thatwerenotincludedintheWebofScience.

However,sevenofthearticlespublishedinmainstreamjournalsincludedinMicrosoftAcademicwerenotincludedintheWebofScienceeither.Forthreeofthose,thiswascausedbythefactthatthepublicationswereeitheravailableonlyinonlinefirst(two)orwererecentlypublished,butnotyetenteredintotheWebofSciencedatabase.TheremainingfourjournalarticlesuniquetoMicrosoftAcademicconcernedpublicationsin1995,1996,1997and2003injournalsthatwerenotISIlistedatthetime,butareincludedintheWebofSciencenow.

Ofthetwenty-twopublicationsinsecondaryjournalsthatarecoveredinbothGoogleScholarandMicrosoftAcademicSearch,sixteenwerenotlistedintheWebofScienceatthetimethepublicationsappeared.Thesepublicationsrepresenteleven

7

differentjournalsandallbutoneofthepublicationsoccurredbetween2001and2008.Oftheseelevenjournals,allbutoneiarenowincludedintheWebofScience.

Incontrast,thereisonlyonepublicationlistedinISIthatisnotlistedinMicrosoftAcademic(A2=1).Thisconcernsabookchapterinaneditedbook,publishedbyRoutledgein2011.

MicrosoftAcademiccomparedwithScopusIntotalthereare30publicationscoveredinMicrosoftAcademicthatarenotcoveredinScopus(B3=30).ThecomparisonbetweenMicrosoftAcademicandScopusfornon-journalpublicationsissimilarinnaturetothatbetweenMicrosoftAcademicandtheWebofScienceinthatMicrosoftAcademicincludedseventeennon-journalpublicationsthatScopusdidnotcover.

ThethreeuniquepublicationsinmainstreamjournalsinMicrosoftAcademicincludedtwoarticlespublishedin1995and1996beforetheoriginalstartofScopuscoveragein1996ii.Afinalpublicationin2003waspublishedinajournalthatwasnotlistedinScopusuntil2005.

Ofthe22publicationsinsecondaryjournalsthatarecoveredinbothGoogleScholarandMicrosoftAcademicSearch,tenwerenotlistedinScopusatthetimethepublicationsappeared.Thesepublicationsrepresenteightdifferentjournalsandallbutoneofthepublicationsoccurredbetween2001and2008.AlleightjournalsarenowincludedinScopus,withScopusadoptionnearlyalwaysoccurringonlyoneortwoyearsaftertherelevantpublicationswerepublished.

Incontrast,thereareonlytwopublicationslistedinScopusthatarenotlistedinMicrosoftAcademic(A3=2).ThisconcernsthesamebookchapteraslistedintheWebofScience,plusanotherbookchapterinaresearchannualAdvancesinInternationalManagement,publishedbyEmeraldpublishersin2003.

ConclusionMicrosoftAcademicperformsverywellinourcomparisonofuniquecoverageinthefourdatabases.Ontheonehand,itdoesnotdisplayanyuniquecoveragevis-à-visGoogleScholar,whereasGoogleScholarhas35additionalpublicationsnotcoveredbyMicrosoftAcademic.Ontheotherhand,itdoesdisplayasubstantialuniquecoveragevis-à-visboththeWebofScience(43publications)andScopus(30publications).UniquecoveragefortheWebofScienceandScopusvis-à-visMicrosoftAcademicisminiscule:onebookchapterfortheWebofScienceandtwobookchaptersforScopus.

Inadditiontomanynon-journalpublications,theuniquecoverageforMicrosoftAcademicincludes23journalarticleswhencomparedtotheWebofScienceand13uniquearticleswhencomparedtoScopus.ItmustbeacknowledgedthatallbutoneoftherelevantjournalsarenowcoveredinboththeWebofScienceandScopus,thusindicatingthattheywerebynomeansobscurejournals.Hence,forveryrecentjournalpublicationstheremightbelittle,ifany,differencebetweenthecoverageofGoogleScholar,MicrosoftAcademic,theWebofScienceandScopus.Thisisoflittlesolace,however,foracademicswith(aninterestin)publicationsthatstretchbackintime.InthosesituationsonlyGoogleScholarandMicrosoftAcademicwillprovidesufficientcoverage.

Citations:overlapbetweenthefourdatabasesFigure2providesavisualillustrationofboththeoverlapandtheuniquecoverageofthefourdatabasesintermsofthecitationsassociatedwiththerelevantpublications.For

8

those89publicationsthatoverlapbetweenMicrosoftAcademicandGoogleScholar,GoogleScholarhasmorethan2.5timesasmanycitationsasMicrosoftAcademic.

PartofthereasonforthisisthatMicrosoftAcademiccitationcountsfornon-journalpublicationsinparticularwerequitemodest.With97citations,onlytheManagementtheMultinationalsbookhadasubstantivenumberofcitations,althoughthiswasstillconsiderablylowerthaninGoogleScholar(433citations).However,forthetwootherbooks,thecomparisonwithGoogleScholarwasevenmoreunfavourable:20vs.203citationsforThePublishorPerishBookand14vs.392citationsfortheInternationalHRMtextbook.Mostoftheseventeenconferencepapers,bookchapters,andnon-refereedpublicationshadeitherzerooronecitationinMicrosoftAcademic.InfactthetotalnumberofcitationsfortheseseventeenpublicationsinMicrosoftAcademicwasonly26.GoogleScholar’scitationslevelfortheseseventeenpublicationswasnotveryhigheither,butat187wasstillseventimesashigh.Figure2:Comparingcitationcoverageacrossfourdata-bases

Whencomparingcitationsforthe46publicationsthatarelistedinboth

MicrosoftAcademicandtheWebofScience,wefindthatMicrosoftAcademichasapproximately20%highercitationslevelsoverall.Thisdoesn’tmeanthateveryindividualpublicationshowsthesamepattern.Morethanonethirdofthepublications(17outof46)hasatleast20%morecitationsinMicrosoftAcademic,goingupto94%and170%fortwospecificjournalarticles.Anotherthirdofthepublications(16outof46)hasbetween3%and19%morecitationsorcitationlevelsequaltotheWebof

Scopus

MAS:3424

GS:9099

MAS:2826Scopus:2861

MAS:2212

WoS:1844

B1:0B2:1210

B3:596

A1:1310 A2:0

A3:85

GoogleScholarWebofScience

MicrosoftAcademic

WebofScience

9

Science.ThirteenarticleshadfewercitationsinMicrosoftAcademicthanintheWebofScience,butthedifferenceinallcaseswasmarginal,1-3citationsforelevenarticlesand4or5fortheremainingtwo. Whencomparingcitationsforthe59publicationsthatarelistedinbothMicrosoftAcademicandScopus,wefindthatoverallcitationlevelsareverysimilarindeed,withcitationsinMicrosoftAcademicbeinglessthan1%lowerthaninScopus.Thisisreflectedinthearticle-by-articlecomparisonwhereroughlyhalfofthearticleshadmorecitationsinMicrosoftAcademicandhalfhadmorecitationsinScopus.Absolutedifferences,however,werefairlysmall;onlyeightarticlesdifferedbymorethan10citationseitherway,andmorethanhalfofthearticlesdifferedby3citationsatmost.

ConclusionMicrosoftAcademicperformsverywellintermsofcitationcountsforarticlesthatoverlapwithotherdatabases.ItoutperformstheWebofSciencefornearlyallarticlesandisanequaltoScopus.OnlyGoogleScholarstilloutperformsMicrosoftAcademicinthisrespect.

Citations:uniquecoverageinthefourdatabasesInadditiontocomparingcitationsforarticlesthatcanbematchedacrossdatabases,itisimportanttoassesstowhatextentuniquearticlesineachdatabasecontributetotheoverallcitationcount.

MicrosoftAcademiccomparedwithGoogleScholarAstherearenopublicationsuniquetoMicrosoftAcademic,therearenouniquecitationsforMicrosoftAcademicwhencomparedtoGoogleScholar(B1=0).Thereare,however,35uniquepublicationsinGoogleScholarthathaveaccumulated1310citationsintotal(A1=1310).MostofthesecitationscamefromPublishorPerish(521citations)andtwobookchapterspublishedinresearchannuals(189and101)thatwerenotcoveredMicrosoftAcademic.FourfurtherpublicationsuniquetoGoogleScholarwithsignificantcitationlevelsweretheJournalQualitylist(79),threechaptersoninternationalassignmentsinthreedifferenteditionsofmyInternationalHumanResourceManagementbook(67,51and48citations)andaconferencepapercomparingGoogleScholarwiththeWebofScience(46citations).Hence,84%oftheuniquecitationsinGoogleScholarcamefromlessthanaquarteroftheuniquepublications.

MicrosoftAcademiccomparedwithWebofScienceThereare43uniquepublicationsinMicrosoftAcademicwhencomparedtotheWebofScience,whichhaveaccumulated1210uniquecitations(B2=1210).Mostoftheseuniquecitationscamefromjournalpublications,includingfourfairlyhighlycitedpublications(63-207citations)insecondaryjournals.Morethanathird–generallyeitherconferencepapersorveryrecentlypublishedjournalarticles–ofthe43uniquepublicationshadeithernoorjust1citation.Hence,threequartersoftheuniquecitationsinMicrosoftAcademiccamefromjust16%oftheuniquepublications.

TheonlyuniquepublicationlistedinISI(abookchapterinaneditedbook)didn’thaveasinglecitation.HencetherearenouniquecitationsinWebofSciencewhencomparedtoMicrosoftAcademic.

10

MicrosoftAcademiccomparedwithScopusThereare30uniquepublicationsinMicrosoftAcademicwhencomparedtoScopus,whichhaveaccumulated596uniquecitations(B3=596).Mostoftheseuniquecitationscamefromjournalpublications,includingfourfairlyhighlycitedpublications(37-71citations)insecondaryjournals.Athirdofthe30uniquepublications–generallyeitherconferencepapersorbookchapters–hadnocitations.Hence,morethanthreequartersoftheuniquecitationsinMicrosoftAcademiccamefromlessthanaquarteroftheuniquepublications.

OnlyoneofthetwouniquepublicationslistedinScopus(abookchapterinaresearchannual)hadcitations.Asthisbookchapterwasfairlyhighlycited(A3=85citations),incontrasttotheWebofScience,Scopusdidhaveanon-negligiblenumberofuniquecitationswhencomparedtoMicrosoftAcademic.

ConclusionMicrosoftAcademicperformsverywellinourcomparisonofuniquecitationsinthefourdatabases.Ontheonehand,itdoesnotdisplayanyuniquecitationsvis-à-visGoogleScholar,whereasGoogleScholarhas1310additionalcitationsnotcoveredbyMicrosoftAcademic.Ontheotherhand,itdoesdisplayasubstantialnumberofuniquecitationsvis-à-visboththeWebofScience(1210citations)andScopus(596citations).UniquecitationsfortheWebofScienceandScopusareeithernon-existent(WebofScience)orrelativelymodest(Scopus).

MostoftheuniquecitationsinMicrosoftAcademicrelatetojournalarticlesanditmustbeacknowledgedthatuniquecitationsareconcentratedinafairlysmallnumberofuniquepublications.However,theconclusionthatMicrosoftAcademicperformswellincomparisontotheWebofScienceandScopusincitationcoverageaswellaspublicationcoverageisinescapable.

ConclusionOurdetailedcomparedofcoverageacrossfourdatabasesshowedthatMicrosoftAcademicsignificantlyoutperformstheWebofScienceintermsofbothpublicationandcitationcoverage.MicrosoftAcademiccanalsobeconsideredtobeatleastanequaltoScopusonbothcounts.OnlyGoogleScholaroutperformsMicrosoftAcademicintermsofbothpublicationsandcitations.

ThebiggestdifferencebetweenGoogleScholarandMicrosoftAcademicliesintwoareas.First,GoogleScholarincludescoverageofnon-standardresearchoutputs,suchasthePublishorPerishsoftware,thusprovidingadditionalcitationsforuniquepublications.Second,GoogleScholarhasmorecitationsforalloftheoverlappingpublications,andsubstantiallymoreinsomecases. WedidfindthattheadditionaljournalcoverageofbothGoogleScholarandMicrosoftAcademicconcernedjournalsthatcurrentlyareincludedinboththeWebofScienceandScopus.Thusdifferencesbetweendatabasesmightbecomesmallerovertime.However,forthoseinterestedinacross-sectionofyoungerandolderpublications,bothGoogleScholarandMicrosoftAcademicappeartobeabetterchoicethantheWebofScienceorScopus. Sowhatdoesthismeanforanindividualacademic?Acomparisonofmyh-indexacrossdatabasesshowsittobemorethantwiceashighinGoogleScholar(46)thanintheWebofScience(22).MicrosoftAcademic(30)andScopus(27)providevaluesinbetweenthesetwoextremes.IntermsofthehIa–anindividualannualizedh-index(seeHarzing,Alakangas&Adams,2014),differencesaresmallerasbothScopusandtheWeb

11

ofSciencemisscoverageofarangeofolderarticles,thusreducingthenumberofyearssincemyfirstpublication.Asaresult,thevaluesofthehIaforScopus(1.11),MicrosoftAcademic(1.10)andtheWebofScience(1.06)areveryclosetogether.At1.81,thehIainGoogleissubstantiallyhigher. Overall,thisfirstsmall-scalecasestudysuggestthat–providedsometeethingproblemswithregardtopublicationduplicatesandwrongyearallocationscanberesolved–thenewincarnationofMicrosoftAcademicpresentsuswithanexcellentalternativeforcitationanalysis,especiallyifcoverageforbooksandnon-traditionalresearchoutputscouldbefurtherimproved.Ifourfindingscanbeconfirmedbylarger-scalestudies,MicrosoftAcademicmightwellturnouttocombinetheadvantagesofbroadercoverage,asdisplayedbyGoogleScholar,withtheadvantageofamorestructuredapproachtodatapresentationtypicalofScopusandtheWebofScience.Ifso,thenewMicrosoftAcademicservicewouldtrulybeaPhoenixarisenfromtheashes.

ReferencesButler,D.(2011).Computinggiantslaunchfreesciencemetrics.Nature,476(7358),18-18.Chrysomallis,M.(2014,December8).Scopuscontinuestoaddpre-1996citations[Weblogpost].http://blog.scopus.com/posts/scopus-continues-to-add-pre-1996-citations,accessed21May2016.

Delgado-López-Cózar,E.,&Repiso-Caballero,R.(2013).Elimpactodelasrevistasdecomunicación:comparandoGoogleScholarMetrics,WebofScienceyScopus.Comunicar:RevistaCientíficadeComunicaciónyEducación,21(41),45-52.Haley,M.R.(2014).RankingtopeconomicsandfinancejournalsusingMicrosoftacademicsearchversusGooglescholar:howdoesthenewpublishorperishoptioncompare?.JournaloftheAssociationforInformationScienceandTechnology,65(5),1079-1084.

Harzing,A.W.;Alakangas,S.(2016)GoogleScholar,ScopusandtheWebofScience:Alongitudinalandcross-disciplinarycomparison,Scientometrics,106(2):787-804.

Harzing,A.W.;Alakangas,S.;Adams,D.(2014)hIa:Anindividualannualh-indextoaccommodatedisciplinaryandcareerlengthdifferences,Scientometrics,99(3):811-821.

Jacsó,P.(2011).TheprosandconsofMicrosoftAcademicSearchfromabibliometricperspective.OnlineInformationReview,35(6),983-997.

Orduña-Malea,E.,Martín-Martín,A.,M.Ayllon,J.,&DelgadoLopez-Cozar,E.(2014).Thesilentfadingofanacademicsearchengine:thecaseofMicrosoftAcademicSearch.OnlineInformationReview,38(7),936-953.

Ortega,J.L.(2014).Influenceofco-authorshipnetworksintheresearchimpact:EgonetworkanalysesfromMicrosoftAcademicSearch.JournalofInformetrics,8(3),728-737.

Ortega,J.L.,&Aguillo,I.F.(2014).MicrosoftacademicsearchandGooglescholarcitations:Comparativeanalysisofauthorprofiles.JournaloftheAssociationforInformationScienceandTechnology,65(6),1149-1156.

VanNoorden,R.(2014)ThedeclineandfallofMicrosoftAcademicSearch[Weblogpost].http://blogs.nature.com/news/2014/05/the-decline-and-fall-of-microsoft-academic-search.html,accessed21May2016.

Wildgaard,L.(2015).Acomparisonof17author-levelbibliometricindicatorsforresearchersinAstronomy,EnvironmentalScience,PhilosophyandPublicHealthinWebofScienceandGoogleScholar.Scientometrics,1-34.

iInterestingly,thisjournalhasbeenlistedinScopussinceitsfirstissuein2001.iiThisislikelytochangeasScopushasrecentlymadeafirmcommitmenttofurtherexpanditscoverageofpre-1996publicationsandcitations(Chrysomallis2014).