Building composite indices – methodology and quality issues A. Saltelli

download Building composite indices – methodology and quality issues A. Saltelli

If you can't read please download the document

TAGS:

description

Building composite indices – methodology and quality issues A. Saltelli European Commission, Joint Research Centre of Ispra, Italy [email protected] Indicators and Assessment Workshop EEA & USEPA/OEI joint meeting, September 27th – 28 th , 2004, Brussels. - PowerPoint PPT Presentation

Transcript of Building composite indices – methodology and quality issues A. Saltelli

  • Building composite indices methodology and quality issues

    A. SaltelliEuropean Commission, Joint Research Centre of Ispra, Italy [email protected]

    Indicators and Assessment Workshop EEA & USEPA/OEI joint meeting, September 27th 28th, 2004, Brussels

  • Prepared with Michaela Saisana and Stefano Tarantola based on:

    Saisana M., Saltelli A., Tarantola S. (2005) Uncertainty and Sensitivity analysis techniques as tools for the quality assessment of composite indicators, J. R. Stat. Soc. A, 168, Part 2, pp.1-17.

    Forthcoming Joint OECD JRC handbook on good practices in composite indictors building

  • http://farmweb.jrc.cec.eu.int/ci/

  • Outline

    CI controversy Composite Indicators as models Wackernagels critique of ESI Putting the critique into practice: the TAI example Conclusions

  • CI controversy

    EU structural indicators:

    scoreboard versus index

  • Report from the Commission to the Spring European Council 2004, Annex 1 Relative PerformanceRelative Improvement in Performance (av. since 1999)

  • Assessing policies: Green Country policy on a good path; Yellow Country policy on a bad path (expert judgment)

  • The Commission also provided the Secretary General with an aggregated analysis table

  • Aggregated analysis of relative performance Good - Countries on a good path Number of (good cells - bad cells) 3Poor - Countries on a bad path Number of (bad cells - good cells) 3Average All others

  • Source: Financial Times Thursday January 22 2004Enter the FT analysts Source: Spring Report, European Commission 2004

  • Categorisation (star rating[*]) in three groupsLEADERS UK, NL SE, DK, AT,LUMIDDLE OF THE ROAD DE, FI, IE, BE, FRLAGGARDS IT, GR, ES, PTdone by FT and based likely on same synoptic performance and improvement tables in the Spring Report, 2004, Annex 1 (yellow-green boxes) [*] Like in the UK NHS hospital rating

  • Can league tables be avoided? Or are they an ingredient of an overall analysis and presentational strategy:

    Long list of 107Short List of 14 Synoptic tables League tables

  • Literature Review of Frameworks for Macro-indicators, Andrew Sharpe, 2004, Centre for the Study of Living Standards, Ottawa, CAN.

  • Reviews on methodologies and practices on composite indicators : State-of-the-art Report on Current Methodologies and Practices for Composite Indicator Development (2002) Michaela Saisana & Stefano Tarantola, European Commission, Joint Research CentreComposite indicators of country performance: a critical assessment (2003) Michael Freudenberg, OECD.Literature Review of Frameworks for Macro-indicators (2004), Andrew Sharpe, Centre for the Study of Living Standards, Ottawa, CAN. Measuring performance: An examination of composite performance indicators (2004) Rowena Jacobs, Peter Smith, Maria Goddard, Centre for Health Economics, University of York, UK.Methodological Issues Encountered in the Construction of Indices of Economic and Social Well-being (2003) Andrew Sharpe Julia Salzman Methodological Choices Encountered in the Construction of Composite Indices of Economic and Social Well-Being, Julia Salzman , (2004) Center for the Study of Living Standards , Ottawa, CAN. http://farmweb.jrc.cec.eu.int/ci/

  • Pros & Cons (Saisana and Tarantola, 2002)

    Pros Composite indicators can be used to summarise complex or multi-dimensional issues, in view of supporting decision-makers. Composite indicators provide the big picture. They can be easier to interpret than trying to find a trend in many separate indicators. They facilitate the task of ranking countries on complex issues. Composite indicators can help attracting public interest by providing a summary figure with which to compare the performance across countries and their progress over time. Composite indicators could help to reduce the size of a list of indicators or to include more information within the existing size limit.

  • Pros & Cons (EC, 2002) Cons Composite indicators may send misleading, non-robust policy messages if they are poorly constructed or misinterpreted. Sensitivity analysis can be used to test composite indicators for robustness. The simple big picture results which composite indicators show may invite politicians to draw simplistic policy conclusions. Composite indicators should be used in combination with the sub-indicators to draw sophisticated policy conclusions. The construction of composite indicators involves stages where judgement has to be made: the selection of sub-indicators, choice of model, weighting indicators and treatment of missing values etc. These judgements should be transparent and based on sound statistical principles. There could be more scope for disagreement among Member States about composite indicators than on individual indicators. The selection of sub-indicators and weights could be the target of political challenge The composite indicators increase the quantity of data needed because data are required for all the sub-indicators and for a statistically significant analysis.

  • Pros & Cons (JRSS paper) [] it is hard to imagine that debate on the use of composite indicators will ever be settled [] official statisticians may tend to resent composite indicators, whereby a lot of work in data collection and editing is wasted or hidden behind a single number of dubious significance. On the other hand, the temptation of stakeholders and practitioners to summarise complex and sometime elusive processes (e.g. sustainability, single market policy, etc.) into a single figure to benchmark country performance for policy consumption seems likewise irresistible.

  • Composite indicators as models and the critique of models

  • Indicators as models and the critique of modelsThe nature of models, after Rosen

    N

    Natural system

    F

    Formal system

    Encoding

    Decoding

    Entailment

    Entailment

  • The critique of modelsAfter Rosen, 1991, World (the natural system) and Model (the formal system) are internally entailed - driven by a causal structure. [Efficient, material, final for world formal for model] Nothing entails with one another World and Model; the association is hence the result of a craftsmanship.

  • Wackernagels critique of ESI

  • Environmental sustainability Index, figure from The Economist, Green and growing, The Economist, Jan 25th 2001,

    Produced on behalf of the World Economic Forum (WEF), and presented to the annual Davos summit this year.The critique of indicators

  • Mathis Wackernagel, mental father of the Ecological Footprint and thus an authoritative source in the Sustainable Development expert community, concludes an argumented critique of the study done presented at Davos by noting: The critique of indicators: Robustness

  • "Overall, the report would gain from a more extensive peer review and a sensitivity analysis. The lacking sensitivity analysis undermines the confidence in the results since small changes in the index architecture or the weighting could dramatically alter the ranking of the nations.

    The critique of indicators: Robustness

  • Putting the critique into practice: the TAI example

  • How to tackle it: the TAI example We have tried to address the issue of robustness taking as example the UN Technology Achievement index (2001), Human Development Report series[An example on the ESI 2005 itself will be ready before the end of the year draft on 2002 data available]

  • The TAI example Technology achievement index: 4 dimensions, 8 indicators Creation of technologyII) Diffusion of recent innovationsIII) Diffusion of old innovationsIV) Human skills

  • How to tackle it: the TA example Tools: Uncertainty Analysis (UA) and Sensitivity Analysis (SA). UA focuses on how uncertainty in the input factors propagates through the structure of the composite indicator and affects the composite indicator values. SA studies how much each individual source of uncertainty contributes to the output variance.

  • Uncertainty analysis = Mapping assumptions onto inferencesSensitivity analysis = The reverse processSimplified diagram - fixed model

  • Answers being sought from Tai example: Does the use of one normalisation method and one set of weights in the development of the composite indicator (e.g. original TAI) provide a biased picture of the countries performance?To what extent do the uncertain input factors (normalisation methods, weighting schemes and weights) affect the countries ranks with respect to the original TAI?

  • Set up of the analysis - Weighting schemes Two participatory methods:

    Budget allocation (experts allocate a finite number of points among the set of indicators)

    or

    Analytic hierarchy process (expert compare the indicators pairwise, and express numerically the relative importance on a 1 [indifference]-9 [much more important] scale. Possibility of inconsistencies.

  • Propagation of UncertaintyInputsModelOutputValues of Technology Achievement Index for different countries : indicators : weights1. selection of sub-indicators 2. data selection 3. data editing 4. data normalisation 5. weighting scheme 6. weights values 7. composite indicator formula Steps in building a composite indicatorxiwie.g. in its simplest form:

  • Uncertainty analysis Results for The values of the composite indicator are displayed in the form of confidence boundsBlue original TAIRed median of Monte Carlo TAI

    Chart1

    76.0368837.064017.74377274.4

    72.0296286.0373447.18693673.3

    70.2887198.97404111.53417666.6

    69.9714115.5804896.55146970.3

    67.6055189.5440059.66979969.8

    62.50160110.0035949.47704158.5

    61.0635775.9324487.02934660.6

    60.0729239.99828411.97382558.7

    59.5116857.3990457.2325663

    59.109056.4345869.37389158.9

    59.0977557.2538748.37454158.3

    56.335475.0314467.38578556.6

    55.0345529.85648112.86678357.9

    55.0305684.5075278.0081555.3

    53.5809525.2887738.20024254.4

    52.9336596.1572948.14216253.5

    52.6027898.25077311.7519354.8

    50.9107915.2727848.80014351.4

    49.2560676.589048.74466748.1

    47.1673996.0574048.76773547.1

    46.739188.4330389.17764246.4

    46.1492687.4379199.382346.5

    45.2737146.9001759.33693844.7

    45.0891516.0441758.74461545.8

    44.6919347.52102510.37892443.7

    44.620525.95939610.09435745.5

    41.9344485.9226959.92409741.1

    41.429675.5098388.6766741.9

    40.8555566.1945169.98112540.7

    39.6744018.8914549.18846738.9

    39.6282285.0289338.75069339.1

    39.4790535.8742599.92180738.1

    38.8501939.5104349.79696739.6

    38.0419515.4040710.06390237.1

    36.4588716.56275811.17676338.6

    36.3917246.20111410.4275835.7

    36.2881336.5730238.66393935.8

    34.5304186.3479628.66277133.7

    33.8904094.06442910.3296534.3

    33.0246324.6410129.07057830

    32.9965875.5566911.04973132.1

    31.8042315.5732259.48420834

    31.0601624.763498.78203729.9

    30.927465.64479410.99256632.8

    30.452982.9461918.17170627.7

    29.7668754.5646448.93473231.1

    29.0194614.59805610.45466827.1

    27.7332872.5820599.73438627.4

    26.676993.15579310.33381725.3

    26.5420281.8274639.33160325.3

    26.5036991.8636449.18331125.5

    25.8801483.79550110.16411926

    25.0857142.8072719.64803624.4

    24.7979584.61613110.46886725.4

    24.6031072.85830810.61079824

    24.4165052.01742610.22126723.6

    23.7571413.2706899.64610422.1

    23.6122632.1896688.49838921.1

    23.1420185.50288711.02918226.1

    22.9230942.6588869.76323722

    22.4438271.798419.60261120.8

    22.2835053.73518810.40102220.3

    22.195272.5647558.82766720.1

    20.5367072.8952778.91575118.5

    18.2091313.1825278.475222316.7

    17.7198355.4858297.06210115.8

    15.6202343.8776078.361697413.9

    15.1947694.4277837.526810612.9

    9.89393947.17378964.77491838

    9.79072166.93326345.39955478.1

    8.4488877.543084.9203397.1

    8.21712968.42190844.25836676.6

    Composite Indicator

    data

    Diffusion of recentWeights

    innovations

    TechnologycreationHigh- andDiffusionof oldHumanskillsDiffusionof oldBudget allocation0.1100.1070.1090.1810.0980.0630.1480.184

    PatentsReceipts ofmedium-innovationsGross tertiaryinnovationsAnalytic Hierarchy Process0.0850.1020.0720.2090.0450.0630.1780.246

    Technologygranted toroyalties andInternettechnologyTelephonesElectricityMean yearsscienceTelephonesElectricity

    achievementresidentslicense feeshostsexports(mainline andconsumptionof schoolingschoolingenrolmentconsumption

    index(per million(US$ per 1,000(per 1,000(as % of totalcellular, per(kilowatt-hours(age 15 andratioLOGLOG

    (TAI)people)people)people)goods exports)1,000 people)per capita)above)(%)Indices calculated based on goalpostsIndices calculated based on mean and std deviation (and LOG for Telephones and Electricity)

    value1998 a1999 b20001999199919982000199597 cPatentsRoyaltiesInternetTech exportsTelephonesElectricitySchoolingUniversityTAIPatentsRoyaltiesInternetTech exportsTelephonesElectricitySchoolingUniversityTAI

    1Finland0.744187125.6200.250.71,20314,1291027.43.084.150.1880.4610.8610.6271.0001.0000.8211.0000.7450.3902.0903.2850.8991.1551.4861.0983.0971.688

    2United States0.733289130179.166.299311,8321213.93.004.070.2910.4770.7710.8191.0001.0001.0000.5050.7330.9022.1852.8721.5741.0261.3651.8860.9321.593

    3Sweden0.703271156.6125.859.71,24713,95511.415.33.104.140.2730.5740.5410.7391.0001.0000.9460.5570.7040.8122.7621.8291.2911.1791.4781.6491.1571.520

    4Japan0.69899464.64980.81,0077,3229.5103.003.861.0000.2370.2111.0001.0001.0000.7770.3630.6994.4410.7660.3272.2101.0351.0380.9020.3071.378

    5Korea, Rep. of0.6667799.84.866.79384,49710.823.22.973.650.7840.0360.0210.8251.0000.9240.8930.8460.6663.362-0.424-0.5381.5960.9870.7061.4132.4241.191

    6Netherlands0.63189151.213650.91,0425,9089.49.53.023.770.1900.5550.5850.6301.0000.9710.7680.3440.6300.4002.6452.0290.9081.0580.8920.8620.2261.128

    7United Kingdom0.6068213457.461.91,0375,3279.414.93.023.730.0820.4920.2470.7661.0000.9530.7680.5420.606-0.1372.2720.4911.3871.0550.8220.8621.0920.980

    8Canada0.5893138.610848.788115,07111.614.22.944.180.0310.1420.4650.6030.9971.0000.9640.5160.590-0.3930.2011.4810.8120.9451.5301.7280.9800.911

    9Australia0.5877518.2125.916.28628,71710.925.32.943.940.0750.0670.5420.2000.9931.0000.9020.9230.588-0.172-0.2421.831-0.6030.9301.1571.4532.7610.889

    10Singapore0.585825.572.374.99016,7717.124.22.953.830.0080.0940.3110.9271.0000.9950.5630.8830.598-0.508-0.0830.7831.9530.9600.985-0.0432.5840.829

    11Germany0.58323536.841.264.28745,68110.214.42.943.750.2360.1350.1770.7950.9960.9650.8390.5240.5830.6310.1620.1741.4870.9390.8651.1771.0120.806

    12Norway0.57910320.2193.6191,32924,60711.911.23.124.390.1040.0740.8330.2351.0001.0000.9910.4070.581-0.031-0.1983.156-0.4811.2221.8641.8460.4990.985

    13Ireland0.566106110.348.653.69244,7609.412.32.973.680.1070.4050.2090.6631.0000.9340.7680.4470.567-0.0161.7570.3191.0250.9770.7450.8620.6750.793

    14Belgium0.5537273.958.947.68177,2499.313.62.913.860.0720.2710.2530.5890.9861.0000.7590.4950.553-0.1870.9670.5200.7640.8941.0320.8230.8840.712

    15New Zealand0.54810313146.715.47208,21511.713.12.863.910.1040.0480.6310.1910.9671.0000.9730.4760.549-0.031-0.3542.238-0.6380.8091.1171.7680.8040.714

    16Austria0.54416514.884.250.39876,1758.413.62.993.790.1660.0540.3620.6231.0000.9790.6790.4950.5450.280-0.3151.0150.8811.0220.9220.4690.8840.645

    17France0.53520533.636.458.99436,2877.912.62.973.800.2060.1230.1570.7291.0000.9820.6340.4580.5360.4810.0930.0801.2560.9910.9350.2720.7240.604

    18Israel0.5147443.643.2459185,4759.6112.963.740.0740.1600.1860.5571.0000.9580.7860.3990.515-0.1770.3100.2130.6510.9730.8400.9410.4670.527

    19Spain0.481428.62153.47304,1957.315.62.863.620.0420.0320.0900.6610.9690.9120.5800.5680.482-0.338-0.450-0.2211.0160.8180.6590.0361.2050.341

    20Italy0.471139.830.4519914,4317.2133.003.650.0130.0360.1310.6311.0000.9210.5710.4730.472-0.483-0.424-0.0370.9121.0240.696-0.0040.7880.309

    21Czech Republic0.465284.22551.75604,7489.58.22.753.680.0280.0150.1080.6400.9300.9330.7770.2970.466-0.408-0.545-0.1430.9420.6390.7430.9020.0180.269

    22Hungary0.464266.221.663.55332,8889.17.72.733.460.0260.0230.0930.7860.9230.8470.7410.2780.465-0.418-0.502-0.2091.4560.6060.4050.744-0.0620.252

    23Slovenia0.458105420.349.56875,0967.110.62.843.710.1060.0150.0870.6130.9600.9460.5630.3850.459-0.021-0.550-0.2350.8470.7770.791-0.0430.4030.246

    24Hong Kong, China (SAR)0.455633.633.61,2125,2449.49.83.083.720.0060.0000.1450.4161.0000.9510.7680.3550.455-0.518-0.6370.0260.1541.1600.8110.8620.2740.267

    25Slovakia0.447242.710.248.74783,8999.39.52.683.590.0240.0100.0440.6030.9070.8990.7590.3440.449-0.428-0.578-0.4320.8120.5320.6090.8230.2260.196

    26Greece0.43716.417.98393,7398.717.22.923.570.0000.0000.0710.2220.9900.8920.7050.6260.438-0.548-0.637-0.311-0.5290.9120.5800.5871.4610.189

    27Portugal0.41962.717.740.78923,3965.9122.953.530.0060.0100.0760.5040.9990.8750.4550.4360.420-0.518-0.578-0.2860.4630.9530.515-0.5160.6270.083

    28Bulgaria0.411233.7303973,1669.510.32.603.500.0230.0000.0160.3710.8800.8630.7770.3740.413-0.433-0.637-0.559-0.0020.4070.4670.9020.3550.062

    29Poland0.407300.611.436.23652,4589.86.62.563.390.0300.0020.0490.4480.8670.8190.8040.2380.407-0.398-0.624-0.4090.2680.3500.2951.020-0.2390.033

    30Malaysia0.3962.467.43402,5546.83.32.533.410.0000.0000.0100.8340.8570.8260.5360.1170.398-0.548-0.637-0.5851.6260.3020.321-0.161-0.768-0.056

    31Croatia0.39196.741.74312,4636.310.62.633.390.0090.0000.0290.5160.8920.8190.4910.3850.393-0.503-0.637-0.5010.5070.4620.296-0.3580.403-0.041

    32Mexico0.38910.49.266.31921,5137.252.283.180.0010.0010.0400.8210.7730.7350.5710.1790.390-0.543-0.628-0.4521.578-0.083-0.036-0.004-0.495-0.083

    33Cyprus0.38616.9237353,4689.242.873.540.0000.0000.0730.2850.9700.8790.7500.1430.388-0.548-0.637-0.301-0.3070.8230.5290.783-0.656-0.039

    34Argentina0.38180.58.7193221,8918.8122.513.280.0080.0020.0370.2350.8490.7730.7140.4360.382-0.508-0.626-0.462-0.4810.2660.1160.6260.627-0.055

    35Romania0.371710.22.725.32271,6269.57.22.363.210.0710.0010.0120.3130.7970.7470.7770.2600.372-0.192-0.632-0.579-0.2070.0300.0130.902-0.143-0.101

    36Costa Rica0.3580.34.152.62391,4506.15.72.383.160.0000.0010.0180.6510.8050.7270.4730.2050.360-0.548-0.630-0.5520.9820.064-0.065-0.437-0.383-0.196

    37Chile0.3576.66.26.13582,0827.613.22.553.320.0000.0240.0270.0750.8640.7900.6070.4800.358-0.548-0.493-0.510-1.0430.3370.1820.1540.820-0.138

    38Uruguay0.343219.613.33661,7887.67.32.563.250.0020.0000.0840.1650.8680.7640.6070.2640.344-0.538-0.637-0.248-0.7300.3520.0780.154-0.127-0.212

    39South Africa0.341.78.430.22703,8326.13.42.433.580.0000.0060.0360.3740.8230.8960.4730.1210.341-0.548-0.600-0.4670.0060.1470.597-0.437-0.752-0.257

    40Thailand0.33710.31.648.91241,3456.54.62.093.130.0010.0010.0070.6050.7080.7140.5090.1650.339-0.543-0.630-0.6000.821-0.379-0.116-0.279-0.560-0.286

    41Trinidad and Tobago0.3287.714.22463,4787.83.32.393.540.0000.0000.0330.1760.8090.8790.6250.1170.330-0.548-0.637-0.481-0.6900.0840.5310.232-0.768-0.285

    42Panama0.3211.95.12511,2118.68.52.403.080.0000.0000.0080.0630.8120.6960.6960.3080.323-0.548-0.637-0.595-1.0870.097-0.1880.5470.066-0.293

    43Brazil0.31120.87.232.92381,7934.93.42.383.250.0020.0030.0310.4070.8040.7640.3660.1210.312-0.538-0.619-0.4910.1240.0610.080-0.909-0.752-0.381

    44Philippines0.30.10.432.8774518.25.21.892.650.0000.0000.0020.4060.6380.5250.6610.1870.302-0.548-0.634-0.6240.120-0.700-0.8600.390-0.463-0.415

    45China0.29910.10.1391207466.43.22.082.870.0010.0000.0000.4830.7040.6120.5000.1140.302-0.543-0.634-0.6300.389-0.401-0.518-0.319-0.784-0.430

    46Bolivia0.27710.20.3261134095.67.72.052.610.0010.0010.0010.3220.6950.5080.4290.2780.279-0.543-0.632-0.626-0.177-0.441-0.927-0.634-0.062-0.505

    47Colombia0.27410.21.913.72368665.35.22.372.940.0010.0010.0080.1700.8030.6380.4020.1870.276-0.543-0.632-0.595-0.7120.056-0.416-0.752-0.463-0.507

    48Peru0.2710.20.72.91076427.67.52.032.810.0000.0010.0030.0360.6870.5860.6070.2710.274-0.548-0.632-0.618-1.182-0.478-0.6200.154-0.094-0.502

    49Jamaica0.2612.40.41.52552,2525.31.62.413.350.0000.0090.0020.0190.8140.8040.4020.0550.263-0.548-0.584-0.624-1.2430.1080.235-0.752-1.041-0.556

    50Iran, Islamic Rep. of0.26121331,3435.36.52.123.130.0010.0000.0000.0250.7190.7140.4020.2340.262-0.543-0.637-0.632-1.222-0.331-0.117-0.752-0.255-0.561

    51Tunisia0.2551.119.79682453.81.982.920.0000.0040.0000.2440.6710.6290.3750.1360.257-0.548-0.613-0.632-0.451-0.551-0.450-0.870-0.688-0.600

    52Paraguay0.25435.30.521377566.22.22.142.880.0000.1290.0020.0250.7230.6140.4820.0770.257-0.5480.130-0.622-1.222-0.311-0.509-0.397-0.945-0.553

    53Ecuador0.2530.33.21226256.462.092.800.0000.0000.0010.0400.7060.5810.5000.2160.256-0.548-0.637-0.626-1.169-0.390-0.638-0.319-0.335-0.583

    54El Salvador0.2530.20.319.21385595.23.62.142.750.0000.0010.0010.2380.7240.5620.3930.1280.256-0.548-0.632-0.626-0.473-0.306-0.714-0.791-0.720-0.601

    55Dominican Republic0.2441.75.71486274.95.72.172.800.0000.0000.0070.0710.7350.5820.3660.2050.246-0.548-0.637-0.599-1.061-0.259-0.636-0.909-0.383-0.629

    56Syrian Arab Republic0.241.21028385.84.62.012.920.0000.0000.0000.0150.6800.6320.4460.1650.242-0.548-0.637-0.632-1.256-0.510-0.438-0.555-0.560-0.642

    57Egypt0.2360.70.18.8778615.52.91.892.940.0000.0030.0000.1090.6380.6370.4200.1030.239-0.548-0.621-0.630-0.926-0.700-0.420-0.673-0.832-0.669

    58Algeria0.2211545635.461.732.750.0000.0000.0000.0120.5860.5630.4110.2160.224-0.548-0.637-0.632-1.265-0.940-0.709-0.712-0.335-0.722

    59Zimbabwe0.220.512368965.41.61.562.950.0000.0000.0020.1490.5270.6440.4110.0550.224-0.548-0.637-0.622-0.786-1.213-0.393-0.712-1.041-0.744

    60Indonesia0.2110.217.94032053.11.602.510.0000.0000.0010.2220.5420.4650.3750.1100.214-0.548-0.637-0.628-0.529-1.142-1.094-0.870-0.800-0.781

    61Honduras0.2088.2574464.831.762.650.0000.0000.0000.1010.5940.5230.3570.1060.210-0.548-0.637-0.632-0.952-0.903-0.868-0.949-0.816-0.788

    62Sri Lanka0.2030.25.2492446.91.41.692.390.0000.0000.0010.0640.5720.4180.5450.0480.206-0.548-0.637-0.628-1.082-1.005-1.279-0.122-1.073-0.797

    63India0.20110.116.6283845.11.71.452.580.0010.0000.0000.2050.4900.4970.3840.0590.205-0.543-0.637-0.630-0.586-1.383-0.970-0.830-1.025-0.825

    64Nicaragua0.1850.43.6392814.63.81.592.450.0000.0000.0020.0450.5380.4420.3390.1360.188-0.548-0.637-0.624-1.152-1.159-1.183-1.027-0.688-0.877

    65Pakistan0.1670.17.9243373.91.41.382.530.0000.0000.0000.0980.4670.4740.2770.0480.171-0.548-0.637-0.630-0.965-1.487-1.059-1.303-1.073-0.963

    66Senegal0.1580.228.5271112.60.51.432.050.0000.0000.0010.3530.4840.2810.1610.0150.162-0.548-0.637-0.628-0.068-1.408-1.816-1.815-1.217-1.017

    67Ghana0.1394.1122893.90.41.082.460.0000.0000.0000.0510.3650.4470.2770.0110.144-0.548-0.637-0.632-1.130-1.955-1.164-1.303-1.233-1.075

    68Kenya0.1290.27.2111294.20.31.042.110.0000.0000.0010.0890.3520.3070.3040.0070.133-0.548-0.637-0.628-0.995-2.014-1.713-1.185-1.249-1.121

    69Nepal0.0810.11.912472.40.71.081.670.0000.0000.0000.0240.3650.1320.1430.0220.086-0.548-0.637-0.630-1.226-1.955-2.401-1.893-1.185-1.309

    70Tanzania, U. Rep. of0.086.76542.70.20.781.730.0000.0000.0000.0830.2630.1560.1700.0040.085-0.548-0.637-0.632-1.017-2.423-2.306-1.775-1.265-1.325

    71Sudan0.0710.49472.10.70.951.670.0000.0000.0000.0050.3230.1320.1160.0220.075-0.548-0.637-0.632-1.291-2.149-2.401-2.011-1.185-1.357

    72Mozambique0.06612.25541.10.20.701.730.0000.0000.0000.1510.2370.1560.0270.0040.072-0.548-0.637-0.632-0.777-2.546-2.306-2.405-1.265-1.390

    goalposts994272.6232.480.89016,9691227.4

    00001220.80.1

    STATISTICS of the raw data

    min1.000.100.100.405.0047.001.100.20

    mean109.2529.3332.3030.06443.283496.297.218.09

    max994.00156.60200.2080.801329.0024607.0012.0027.40

    std199.2146.0751.1222.97402.374274.912.546.23

    skewness3.361.762.000.360.592.51-0.121.04

    CorrelationsPatentsRoyaltiesInternetExportsTelephonesElectricitySchoolingEnrollment

    Patents0.2940.1900.4990.4200.2520.3860.323

    Royalties0.6460.4200.6540.5000.5070.415

    Internet0.3330.7410.8880.6690.623

    Exports0.6640.4500.5390.547

    Telephones0.8150.7720.800

    Electricity0.7220.641

    Schooling0.711

    data

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    weights

    PatentsRoyaltiesInternetExportsTelephonesElectricitySchoolingEnrolment

    Riccardo0.0250.0560.0560.4450.0520.0150.1380.213weights from AHP

    Marco0.0380.0450.0290.1790.0230.0230.2740.389

    Francis0.0450.0400.3570.0700.0190.0120.1760.280

    Andrea0.0240.0410.1330.2250.0620.0310.2420.242

    Gabriele0.0760.2020.0920.2790.0540.0560.1270.115

    Ioannis0.0240.0540.0660.1200.0850.1160.2450.290

    Francesca0.0940.0240.0650.1710.0410.0190.3530.234

    Stefano0.0650.2650.0380.1280.0270.0170.2290.229

    Michaela0.0800.0650.0670.1210.0360.0200.2490.362

    Roman0.1230.0280.0330.4400.0200.0130.0830.260

    Angelo0.1760.0990.1030.2270.0330.0530.0470.262

    Matina

    Lorenzo0.1140.1530.0130.1540.0130.1410.0940.317

    Furio0.0900.0490.0110.0300.0180.4180.1380.246

    Per0.1160.2880.0260.3020.0300.0280.0550.155

    Martin0.0780.1100.0510.1820.0660.0880.1650.259

    Silvio0.2290.1350.0590.1760.0390.0390.2520.072

    Jochen0.1090.1030.0290.1170.0300.0140.3010.297

    Spyros

    Thelksi

    AthinaKarv.0.0290.0810.0700.3960.1650.0290.0260.204

    Riccardo0.050.050.050.20.10.050.20.3weights from Budget Allocation

    Marco0.150.150.10.150.10.050.20.1

    Francis0.200.10.2000.20.3

    Andrea0.070.070.10.150.130.060.230.19

    Gabriele0.10.150.150.30.050.050.20

    Ioannis0.10.050.20.10.150.050.150.2

    Francesca0.20.20.050.10.10.150.150.05

    Stefano0.10.050.150.20.10.10.150.15

    Michaela0.10.10.150.150.10.050.20.15

    Roman0.20.050.050.30.0500.050.3

    Angelo0.120.150.120.150.10.120.080.16

    Matina0.050.050.20.10.050.050.250.25

    Lorenzo0.10.050.10.250.050.150.10.2

    Furio0.050.050.050.10.20.050.20.3

    Per0.10.30.020.330.030.020.050.15

    Martin0.10.150.10.20.150.10.10.1

    Silvio0.150.130.110.140.10.10.150.12

    Jochen0.10.150.10.20.100.150.2

    Spyros0.120.220.090.090.090.090.10.2

    Thelksi0.050.020.20.150.150.030.150.25

    AthinaKarv.0.10.10.10.250.150.050.050.2

    10th perc.0.03640.03720.02860.10000.02010.01290.05000.1000

    90th perc.0.18070.20560.16000.30760.15000.12420.25040.3000

    Equal0.1250.1250.1250.1250.1250.1250.1250.125

    Correlations (tai_weights.sta)

    Marked correlations are significant at p < .05000

    N=39 (Casewise deletion of missing data)

    PATENTSROYALTINTERNETTEXPORTSTELEPHONELECTRICSCHOOLUNIV_ST

    PATENTS0.228-0.177-0.029-0.2190.042-0.212-0.349

    ROYALT-0.3010.071-0.090-0.002-0.324-0.516

    INTERNET-0.2620.197-0.1860.071-0.131

    TEXPORTS-0.101-0.391-0.471-0.153

    TELEPHON-0.010-0.170-0.242

    ELECTRIC-0.158-0.125

    SCHOOL0.204

    UNIV_ST

    0.230.180.030.220.040.210.35

    0.300.070.090.000.320.52

    0.260.200.190.070.13

    0.100.390.470.15

    0.010.170.24

    0.160.13

    0.20

    0.0361

    The weights are not included, because of the very high inconsistency.

    did not reply on time

    weights

    000000000000000000000000

    000000000000000000000000

    000000000000000000000000

    000000000000000000000000

    000000000000000000000000

    000000000000000000000000

    000000000000000000000000

    000000000000000000000000

    Robustness_Sobol_TAI-order

    000000000000000000000000000000000000000000000

    000000000000000000000000000000000000000000000

    000000000000000000000000000000000000000000000

    000000000000000000000000000000000000000000000

    000000000000000000000000000000000000000000000

    000000000000000000000000000000000000000000000

    000000000000000000000000000000000000000000000

    000000000000000000000000000000000000000000000

    BA

    AHP

    10th and 90th percentiles

    Weight value

    Robustness_Sobol_median_order

    0000000000000000000000000000000000000000000000

    0000000000000000000000000000000000000000000000

    0000000000000000000000000000000000000000000000

    0000000000000000000000000000000000000000000000

    0000000000000000000000000000000000000000000000

    0000000000000000000000000000000000000000000000

    0000000000000000000000000000000000000000000000

    0000000000000000000000000000000000000000000000

    BA

    AHP

    10th and 90th percentiles

    Weight

    Robustness_Sobol_ranks

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    Patents

    Robustness_Sobol_ranks2

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    Royalties

    Worthiness

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    Internet

    Sobol_indices

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    Exports

    Netherlands-Singapore

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    Telephones

    Hist-NL-SG

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    Electricity

    plot-NL-SG-1

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    Schooling

    plot-NL-SG-2

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    Enrolment

    plot-NL-SG-3

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    Patents

    WHO-OECD

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    Royalties

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    Internet

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    Exports

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    Telephones

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    Electricity

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    Schooling

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    0

    Enrolment

    median5th prc.95th prc.Original value

    Finland76.047.7447.06468.2983.1074.4

    United States72.037.1876.03764.8478.0773.3

    Sweden69.976.5515.58063.4275.5570.3

    Japan67.619.6709.54457.9477.1569.8

    Korea, Rep. of70.2911.5348.97458.7579.2666.6

    Netherlands59.517.2337.39952.2866.9163

    United Kingdom61.067.0295.93254.0367.0060.6

    Canada59.119.3746.43549.7465.5458.9

    Australia60.0711.9749.99848.1070.0758.7

    Singapore62.509.47710.00453.0272.5158.5

    Germany59.108.3757.25450.7266.3558.3

    Norway55.0312.8679.85642.1764.8957.9

    Ireland56.347.3865.03148.9561.3756.6

    Belgium55.038.0084.50847.0259.5455.3

    New Zealand52.6011.7528.25140.8560.8554.8

    Austria53.588.2005.28945.3858.8754.4

    France52.938.1426.15744.7959.0953.5

    Israel50.918.8005.27342.1156.1851.4

    Spain49.268.7456.58940.5155.8548.1

    Italy47.178.7686.05738.4053.2247.1

    Czech Republic46.159.3827.43836.7753.5946.5

    Hungary46.749.1788.43337.5655.1746.4

    Slovenia45.098.7456.04436.3451.1345.8

    Hong Kong, China (SAR)44.6210.0945.95934.5350.5845.5

    Slovakia45.279.3376.90035.9452.1744.7

    Greece44.6910.3797.52134.3152.2143.7

    Portugal41.438.6775.51032.7546.9441.9

    Bulgaria41.939.9245.92332.0147.8641.1

    Poland40.869.9816.19530.8747.0540.7

    Malaysia38.859.7979.51029.0548.3639.6

    Croatia39.638.7515.02930.8844.6639.1

    Mexico39.679.1888.89130.4948.5738.9

    Cyprus36.4611.1776.56325.2843.0238.6

    Argentina39.489.9225.87429.5645.3538.1

    Romania38.0410.0645.40427.9843.4537.1

    Costa Rica36.298.6646.57327.6242.8635.8

    Chile36.3910.4286.20125.9642.5935.7

    Uruguay33.8910.3304.06423.5637.9534.3

    South Africa31.809.4845.57322.3237.3834

    Thailand34.538.6636.34825.8740.8833.7

    Trinidad and Tobago30.9310.9935.64519.9336.5732.8

    Panama33.0011.0505.55721.9538.5532.1

    Brazil29.778.9354.56520.8334.3331.1

    Philippines33.029.0714.64123.9537.6730

    China31.068.7824.76322.2835.8229.9

    Bolivia30.458.1722.94622.2833.4027.7

    Colombia27.739.7342.58218.0030.3227.4

    Peru29.0210.4554.59818.5633.6227.1

    Jamaica23.1411.0295.50312.1128.6426.1

    Iran, Islamic Rep. of25.8810.1643.79615.7229.6826

    Tunisia26.509.1831.86417.3228.3725.5

    Paraguay24.8010.4694.61614.3329.4125.4

    Ecuador26.6810.3343.15616.3429.8325.3

    El Salvador26.549.3321.82717.2128.3725.3

    Dominican Republic25.099.6482.80715.4427.8924.4

    Syrian Arab Republic24.6010.6112.85813.9927.4624

    Egypt24.4210.2212.01714.2026.4323.6

    Algeria23.769.6463.27114.1127.0322.1

    Zimbabwe22.929.7632.65913.1625.5822

    Indonesia23.618.4982.19015.1125.8021.1

    Honduras22.449.6031.79812.8424.2420.8

    Sri Lanka22.2810.4013.73511.8826.0220.3

    India22.208.8282.56513.3724.7620.1

    Nicaragua20.548.9162.89511.6223.4318.5

    Pakistan18.218.4753.1839.7321.3916.7

    Senegal17.727.0625.48610.6623.2115.8

    Ghana15.628.3623.8787.2619.5013.9

    Kenya15.197.5274.4287.6719.6212.9

    Nepal9.795.4006.9334.3916.728.1

    Tanzania, U. Rep. of9.894.7757.1745.1217.078

    Sudan8.454.9207.5433.5315.997.1

    Mozambique8.224.2588.4223.9616.646.6

    &R&F\&A

    Uncertainty is estimated considering uncertainty in weights and normalisation method (two triggers), 11256 runs in Sobol sampling

    Countries ranked according to the original TAI

    7.064017.743772

    6.0373447.186936

    5.5804896.551469

    9.5440059.669799

    8.97404111.534176

    7.3990457.23256

    5.9324487.029346

    6.4345869.373891

    9.99828411.973825

    10.0035949.477041

    7.2538748.374541

    9.85648112.866783

    5.0314467.385785

    4.5075278.00815

    8.25077311.75193

    5.2887738.200242

    6.1572948.142162

    5.2727848.800143

    6.589048.744667

    6.0574048.767735

    7.4379199.3823

    8.4330389.177642

    6.0441758.744615

    5.95939610.094357

    6.9001759.336938

    7.52102510.378924

    5.5098388.67667

    5.9226959.924097

    6.1945169.981125

    9.5104349.796967

    5.0289338.750693

    8.8914549.188467

    6.56275811.176763

    5.8742599.921807

    5.4040710.063902

    6.5730238.663939

    6.20111410.42758

    4.06442910.32965

    5.5732259.484208

    6.3479628.662771

    5.64479410.992566

    5.5566911.049731

    4.5646448.934732

    4.6410129.070578

    4.763498.782037

    2.9461918.171706

    2.5820599.734386

    4.59805610.454668

    5.50288711.029182

    3.79550110.164119

    1.8636449.183311

    4.61613110.468867

    3.15579310.333817

    1.8274639.331603

    2.8072719.648036

    2.85830810.610798

    2.01742610.221267

    3.2706899.646104

    2.6588869.763237

    2.1896688.498389

    1.798419.602611

    3.73518810.401022

    2.5647558.827667

    2.8952778.915751

    3.1825278.4752223

    5.4858297.062101

    3.8776078.3616974

    4.4277837.5268106

    6.93326345.3995547

    7.17378964.7749183

    7.543084.920339

    8.42190844.2583667

    Composite indicator

    median5th prc.95th prc.Original value

    Finland76.047.7447.06468.2983.1074.4

    United States72.037.1876.03764.8478.0773.3

    Korea, Rep. of70.2911.5348.97458.7579.2666.6

    Sweden69.976.5515.58063.4275.5570.3

    Japan67.619.6709.54457.9477.1569.8

    Singapore62.509.47710.00453.0272.5158.5

    United Kingdom61.067.0295.93254.0367.0060.6

    Australia60.0711.9749.99848.1070.0758.7

    Netherlands59.517.2337.39952.2866.9163

    Canada59.119.3746.43549.7465.5458.9

    Germany59.108.3757.25450.7266.3558.3

    Ireland56.347.3865.03148.9561.3756.6

    Norway55.0312.8679.85642.1764.8957.9

    Belgium55.038.0084.50847.0259.5455.3

    Austria53.588.2005.28945.3858.8754.4

    France52.938.1426.15744.7959.0953.5

    New Zealand52.6011.7528.25140.8560.8554.8

    Israel50.918.8005.27342.1156.1851.4

    Spain49.268.7456.58940.5155.8548.1

    Italy47.178.7686.05738.4053.2247.1

    Hungary46.749.1788.43337.5655.1746.4

    Czech Republic46.159.3827.43836.7753.5946.5

    Slovakia45.279.3376.90035.9452.1744.7

    Slovenia45.098.7456.04436.3451.1345.8

    Greece44.6910.3797.52134.3152.2143.7

    Hong Kong, China (SAR)44.6210.0945.95934.5350.5845.5

    Bulgaria41.939.9245.92332.0147.8641.1

    Portugal41.438.6775.51032.7546.9441.9

    Poland40.869.9816.19530.8747.0540.7

    Mexico39.679.1888.89130.4948.5738.9

    Croatia39.638.7515.02930.8844.6639.1

    Argentina39.489.9225.87429.5645.3538.1

    Malaysia38.859.7979.51029.0548.3639.6

    Romania38.0410.0645.40427.9843.4537.1

    Cyprus36.4611.1776.56325.2843.0238.6

    Chile36.3910.4286.20125.9642.5935.7

    Costa Rica36.298.6646.57327.6242.8635.8

    Thailand34.538.6636.34825.8740.8833.7

    Uruguay33.8910.3304.06423.5637.9534.3

    Philippines33.029.0714.64123.9537.6730

    Panama33.0011.0505.55721.9538.5532.1

    South Africa31.809.4845.57322.3237.3834

    China31.068.7824.76322.2835.8229.9

    Trinidad and Tobago30.9310.9935.64519.9336.5732.8

    Bolivia30.458.1722.94622.2833.4027.7

    Brazil29.778.9354.56520.8334.3331.1

    Peru29.0210.4554.59818.5633.6227.1

    Colombia27.739.7342.58218.0030.3227.4

    Ecuador26.6810.3343.15616.3429.8325.3

    El Salvador26.549.3321.82717.2128.3725.3

    Tunisia26.509.1831.86417.3228.3725.5

    Iran, Islamic Rep. of25.8810.1643.79615.7229.6826

    Dominican Republic25.099.6482.80715.4427.8924.4

    Paraguay24.8010.4694.61614.3329.4125.4

    Syrian Arab Republic24.6010.6112.85813.9927.4624

    Egypt24.4210.2212.01714.2026.4323.6

    Algeria23.769.6463.27114.1127.0322.1

    Indonesia23.618.4982.19015.1125.8021.1

    Jamaica23.1411.0295.50312.1128.6426.1

    Zimbabwe22.929.7632.65913.1625.5822

    Honduras22.449.6031.79812.8424.2420.8

    Sri Lanka22.2810.4013.73511.8826.0220.3

    India22.208.8282.56513.3724.7620.1

    Nicaragua20.548.9162.89511.6223.4318.5

    Pakistan18.218.4753.1839.7321.3916.7

    Senegal17.727.0625.48610.6623.2115.8

    Ghana15.628.3623.8787.2619.5013.9

    Kenya15.197.5274.4287.6719.6212.9

    Tanzania, U. Rep. of9.894.7757.1745.1217.078

    Nepal9.795.4006.9334.3916.728.1

    Sudan8.454.9207.5433.5315.997.1

    Mozambique8.224.2588.4223.9616.646.6

    &R&F\&A

    Uncertainty is estimated considering uncertainty in weights and normalisation method (two triggers), 11256 runs in Sobol sampling

    Countries ranked according to the median value

    7.064017.743772

    6.0373447.186936

    8.97404111.534176

    5.5804896.551469

    9.5440059.669799

    10.0035949.477041

    5.9324487.029346

    9.99828411.973825

    7.3990457.23256

    6.4345869.373891

    7.2538748.374541

    5.0314467.385785

    9.85648112.866783

    4.5075278.00815

    5.2887738.200242

    6.1572948.142162

    8.25077311.75193

    5.2727848.800143

    6.589048.744667

    6.0574048.767735

    8.4330389.177642

    7.4379199.3823

    6.9001759.336938

    6.0441758.744615

    7.52102510.378924

    5.95939610.094357

    5.9226959.924097

    5.5098388.67667

    6.1945169.981125

    8.8914549.188467

    5.0289338.750693

    5.8742599.921807

    9.5104349.796967

    5.4040710.063902

    6.56275811.176763

    6.20111410.42758

    6.5730238.663939

    6.3479628.662771

    4.06442910.32965

    4.6410129.070578

    5.5566911.049731

    5.5732259.484208

    4.763498.782037

    5.64479410.992566

    2.9461918.171706

    4.5646448.934732

    4.59805610.454668

    2.5820599.734386

    3.15579310.333817

    1.8274639.331603

    1.8636449.183311

    3.79550110.164119

    2.8072719.648036

    4.61613110.468867

    2.85830810.610798

    2.01742610.221267

    3.2706899.646104

    2.1896688.498389

    5.50288711.029182

    2.6588869.763237

    1.798419.602611

    3.73518810.401022

    2.5647558.827667

    2.8952778.915751

    3.1825278.4752223

    5.4858297.062101

    3.8776078.3616974

    4.4277837.5268106

    7.17378964.7749183

    6.93326345.3995547

    7.543084.920339

    8.42190844.2583667

    Composite indicator

    7.064017.743772

    6.0373447.186936

    8.97404111.534176

    5.5804896.551469

    9.5440059.669799

    10.0035949.477041

    5.9324487.029346

    9.99828411.973825

    7.3990457.23256

    6.4345869.373891

    7.2538748.374541

    5.0314467.385785

    9.85648112.866783

    4.5075278.00815

    Composite indicatorr

    7.064017.743772

    6.0373447.186936

    8.97404111.534176

    5.5804896.551469

    9.5440059.669799

    10.0035949.477041

    5.9324487.029346

    9.99828411.973825

    7.3990457.23256

    6.4345869.373891

    7.2538748.374541

    5.0314467.385785

    9.85648112.866783

    4.5075278.00815

    5.2887738.200242

    6.1572948.142162

    8.25077311.75193

    5.2727848.800143

    6.589048.744667

    6.0574048.767735

    8.4330389.177642

    7.4379199.3823

    6.9001759.336938

    6.0441758.744615

    7.52102510.378924

    5.95939610.094357

    5.9226959.924097

    5.5098388.67667

    6.1945169.981125

    8.8914549.188467

    5.0289338.750693

    5.8742599.921807

    9.5104349.796967

    5.4040710.063902

    6.56275811.176763

    6.20111410.42758

    6.5730238.663939

    6.3479628.662771

    4.06442910.32965

    4.6410129.070578

    5.5566911.049731

    5.5732259.484208

    4.763498.782037

    5.64479410.992566

    2.9461918.171706

    4.5646448.934732

    4.59805610.454668

    2.5820599.734386

    3.15579310.333817

    1.8274639.331603

    1.8636449.183311

    3.79550110.164119

    2.8072719.648036

    4.61613110.468867

    2.85830810.610798

    2.01742610.221267

    3.2706899.646104

    2.1896688.498389

    5.50288711.029182

    2.6588869.763237

    1.798419.602611

    3.73518810.401022

    2.5647558.827667

    2.8952778.915751

    3.1825278.4752223

    5.4858297.062101

    3.8776078.3616974

    4.4277837.5268106

    7.17378964.7749183

    6.93326345.3995547

    7.543084.920339

    8.42190844.2583667

    Composite Indicator

    most frequent5th prc.95th prc.Original value

    Finland10314

    United States21315

    Sweden31326

    Japan54419

    Korea, Rep. of21618

    Netherlands1163514

    United Kingdom725512

    Canada1032712

    Australia6211417

    Singapore637313

    Germany924713

    Norway1488622

    Ireland1234916

    Belgium13131216

    New Zealand17791026

    Austria16411217

    France17611118

    Israel18211619

    Spain19331622

    Italy20241824

    Czech Republic22331925

    Hungary21551626

    Slovenia24332127

    Hong Kong, China (SAR)25432128

    Slovakia24322126

    Greece26761932

    Portugal27262533

    Bulgaria27152632

    Poland28152733

    Malaysia351132438

    Croatia32432835

    Mexico32942336

    Cyprus36543140

    Argentina29172836

    Romania35513036

    Costa Rica36643040

    Chile36673043

    Uruguay38253643

    South Africa42443846

    Thailand37353442

    Trinidad and Tobago46614047

    Panama40563546

    Brazil47624149

    Philippines41443745

    China44533947

    Bolivia46514147

    Colombia48124750

    Peru47534250

    Jamaica59955064

    Iran, Islamic Rep. of50274857

    Tunisia51444755

    Paraguay54664860

    Ecuador49254754

    El Salvador50254855

    Dominican Republic53255158

    Syrian Arab Republic55365261

    Egypt57315458

    Algeria58655263

    Zimbabwe60425662

    Indonesia59825161

    Honduras62216063

    Sri Lanka63915464

    India63915464

    Nicaragua64216265

    Pakistan66106566

    Senegal651035568

    Ghana68206668

    Kenya67116668

    Nepal70116971

    Tanzania, U. Rep. of69016970

    Sudan71017172

    Mozambique72306972

    &R&F\&A

    Uncertainty is estimated considering uncertainty in weights and normalisation method (two triggers), 11256 runs in Sobol sampling

    Countries ranked according to the original TAI

    30

    31

    31

    44

    61

    36

    52

    23

    112

    73

    42

    88

    43

    31

    97

    14

    16

    12

    33

    42

    33

    55

    33

    34

    23

    67

    62

    51

    51

    311

    34

    49

    45

    71

    15

    46

    76

    52

    44

    53

    16

    65

    26

    44

    35

    15

    21

    35

    59

    72

    44

    66

    52

    52

    52

    63

    13

    56

    24

    28

    12

    19

    19

    12

    01

    310

    02

    11

    11

    10

    10

    03

    Composite indicator

    most frequent5th prc.95th prc.Original value

    Finland10314

    United States21315

    Korea, Rep. of21618

    Sweden31326

    Japan54419

    Australia6211417

    Singapore637313

    United Kingdom725512

    Germany924713

    Canada1032712

    Netherlands1163514

    Ireland1234916

    Belgium13131216

    Norway1488622

    Austria16411217

    New Zealand17791026

    France17611118

    Israel18211619

    Spain19331622

    Italy20241824

    Hungary21551626

    Czech Republic22331925

    Slovenia24332127

    Slovakia24322126

    Hong Kong, China (SAR)25432128

    Greece26761932

    Portugal27262533

    Bulgaria27152632

    Poland28152733

    Argentina29172836

    Croatia32432835

    Mexico32942336

    Malaysia351132438

    Romania35513036

    Cyprus36543140

    Costa Rica36643040

    Chile36673043

    Thailand37353442

    Uruguay38253643

    Panama40563546

    Philippines41443745

    South Africa42443846

    China44533947

    Trinidad and Tobago46614047

    Bolivia46514147

    Brazil47624149

    Peru47534250

    Colombia48124750

    Ecuador49254754

    Iran, Islamic Rep. of50274857

    El Salvador50254855

    Tunisia51444755

    Dominican Republic53255158

    Paraguay54664860

    Syrian Arab Republic55365261

    Egypt57315458

    Algeria58655263

    Jamaica59955064

    Indonesia59825161

    Zimbabwe60425662

    Honduras62216063

    Sri Lanka63915464

    India63915464

    Nicaragua64216265

    Senegal651035568

    Pakistan66106566

    Kenya67116668

    Ghana68206668

    Tanzania, U. Rep. of69016970

    Nepal70116971

    Sudan71017172

    Mozambique72306972

    &R&F\&A

    Uncertainty is estimated considering uncertainty in weights and normalisation method (two triggers), 11256 runs in Sobol sampling

    Countries ranked according to the most frequent rank

    30

    31

    61

    31

    44

    112

    73

    52

    42

    23

    36

    43

    31

    88

    14

    97

    16

    12

    33

    42

    55

    33

    33

    23

    34

    67

    62

    51

    51

    71

    34

    49

    311

    15

    45

    46

    76

    53

    52

    65

    44

    44

    35

    16

    15

    26

    35

    21

    52

    72

    52

    44

    52

    66

    63

    13

    56

    59

    28

    24

    12

    19

    19

    12

    310

    01

    11

    02

    10

    11

    10

    03

    Composite indicator

    TAIInnovation-OECDOHSA-WHO

    Significant overlapWorthinessPairsWorthinessPairsWorthinessPairs

    0.10.902580.74840.862485

    0.150.922100.76770.872343

    0.20.931690.78710.882213

    0.250.941450.79690.892054

    0.30.951160.82590.891910

    0.350.96980.83560.901754

    0.40.97820.84530.911604

    0.450.97730.87420.921409

    0.50.98600.87410.931224

    0.550.98440.90310.941015

    0.60.99340.92260.95860

    0.650.99280.93230.96681

    0.70.99230.94180.97532

    0.750.99180.9870.98396

    0.80.99140.9860.99256

    0.851.0071.0010.99140

    0.91.0051.0001.0072

    Technology Achievement Index

    minmax&BAminmax&AHPmeanstd&BAmeanstd&AHPallmethods

    Significant overlapWorthinessPairsWorthinessPairsWorthinessPairsWorthinessPairsWorthinessPairs

    0.10.912260.883030.844100.922100.89276

    0.150.921930.902510.853890.931730.91219

    0.20.941560.921950.863550.941480.93183

    0.250.951260.931680.873300.951240.94153

    0.30.961150.951390.882940.961020.95134

    0.350.96910.951270.902620.97810.95116

    0.40.97730.961110.912340.97650.96103

    0.450.98620.96930.922010.98480.9789

    0.50.98510.97820.941660.99360.9774

    0.550.98400.97730.951390.99300.9859

    0.60.99320.98550.961130.99250.9852

    0.650.99270.98430.96980.99210.9842

    0.70.99200.98400.97720.99160.9932

    0.750.99130.99330.98451.0090.9927

    0.81.00100.99260.99311.0090.9924

    0.851.0050.99210.99181.0040.9918

    0.91.0031.00111.0061.0021.0011

    000

    000

    000

    000

    000

    000

    000

    000

    000

    000

    000

    000

    000

    000

    000

    000

    000

    000

    000

    000

    TAI

    Innovation Index

    Overall Health System Performance Index

    Significant overlap

    Worthiness Index

    00000

    00000

    00000

    00000

    00000

    00000

    00000

    00000

    00000

    00000

    00000

    00000

    00000

    00000

    00000

    00000

    00000

    Meanstd&BA

    Meanstd&AHP

    All methods

    Minmax&BA

    Minmax&AHP

    Significant overlap

    Worthiness Index

    AvRank 50th percentileFirst orderTotal effectAvRank Original valuesFirst orderTotal effectNetherlands-SingaporeFirst orderTotal effect

    Trigger_scaling0.000.070.07Re/St0.000.060.06Re/St0.000.030.03

    Trigger_weighting0.070.520.45BA/AHP0.210.560.36BA/AHP0.140.560.42

    W-patents0.000.010.01W-patents0.000.000.00W-patents0.000.000.00

    W-royalties0.010.030.02W-royalties0.020.020.01W-royalties0.010.040.03

    W-internet0.000.020.02W-internet0.010.030.02W-internet0.020.050.03

    W-exports0.120.550.43W-exports0.140.440.30W-exports0.020.170.15

    W-telephones0.000.020.02W-telephones0.030.050.02W-telephones0.020.070.06

    W-electricity0.010.050.04W-electricity0.040.130.10W-electricity0.170.370.20

    W-schooling0.010.180.17W-schooling0.000.090.09W-schooling0.020.100.08

    W-enrolment0.050.280.23W-enrolment-0.000.140.14W-enrolment0.120.290.16

    sum0.271.731.46sum0.441.531.09sum0.521.691.17

    00

    00

    00

    00

    00

    00

    00

    00

    00

    00

    First order

    Total effect

    Uncertain input factor

    Sobol index

    00

    00

    00

    00

    00

    00

    00

    00

    00

    00

    First order

    Total effect

    Uncertain input factor

    Sobol Index

    00

    00

    00

    00

    00

    00

    00

    00

    00

    00

    First order

    Total effect

    Uncertain input factor

    Sobol' sensitivity measure

    00

    00

    00

    00

    00

    00

    00

    00

    00

    00

    First order

    Total effect

    Uncertain input factor

    Sobol' sensitivity measure

    00

    00

    00

    00

    00

    00

    00

    00

    00

    00

    First order

    Total effect

    Uncertain input factor

    Sobol' sensitivity measure

    Trigger_scalingTrigger_weightingW-patentsW-royaltiesW-internetW-exportsW-telephonesW-electricityW-schoolingW-enrolmentNetherlandsSingaporeNeth-Sing%diffminsum_of_weightsW-electricityW-enrolment

    0.50.50.120.150.120.150.10.120.080.1660.3859.720.660.0110.1200.160

    0.250.750.0760.2880.0920.3020.0540.0280.0550.11557.5654.523.040.061.010.0280.114

    0.750.250.10.050.10.10.150.050.150.161.6557.544.110.070.80.0630.125

    0.1250.6250.2290.1350.0130.070.0410.0190.2520.31749.4652.52-3.06-0.061.0760.0180.295

    0.6250.1250.10.10.10.10.090.090.20.1961.5158.982.520.040.970.0930.196

    0.3750.3750.050.070.090.090.130.050.230.263.9266.40-2.48-0.040.910.0550.220

    0.8750.8750.0450.1530.0650.1710.0130.0120.0940.23463.2566.07-2.82-0.040.7870.0150.297

    0.06250.93750.090.0540.1330.1790.0270.0530.1650.24655.7063.67-7.97-0.140.9470.0560.260

    0.56250.43750.10.130.020.10.150.050.15065.5150.2315.280.300.70.0710.000

    0.31250.18750.050.050.050.140.050.150.20.2562.7672.66-9.90-0.160.940.1600.266

    0.81250.68750.0650.0450.0290.120.0180.0880.0470.22956.5870.00-13.42-0.240.6410.1370.357

    0.18750.31250.20.30.20.30.10.030.20.157.8849.018.870.181.430.0210.070

    0.68750.81250.0780.0410.0290.030.0330.0170.1380.26255.3065.42-10.12-0.180.6280.0270.417

    0.43750.56250.0380.2650.0510.1170.0850.0310.3010.2959.7558.651.100.021.1780.0260.246

    0.93750.06250.050.020.050.30.10.020.050.1259.0570.27-11.22-0.190.710.0280.169

    0.031250.531250.0650.2020.0670.120.030.1160.1760.25958.5760.37-1.80-0.031.0350.1120.250

    0.531250.031250.120.30.20.140.050.10.10.1569.6955.9413.750.251.160.0860.129

    0.281250.281250.070.020.150.10.030.050.150.256.8462.67-5.83-0.100.770.0650.260

    0.781250.781250.1140.0650.0110.1790.0540.0230.3530.2856.6161.07-4.46-0.081.0790.0210.259

    0.156250.156250.050.150.10.150.050.060.150.356.6763.73-7.06-0.121.010.0590.297

    0.656250.656250.0380.0240.0650.1760.0190.1410.2490.29756.7266.86-10.14-0.181.0090.1400.294

    0.406250.906250.1160.040.1030.1540.0660.0390.1270.11558.2260.18-1.95-0.030.760.0510.151

    0.906250.406250.20.050.10.150.10.050.050.1559.3459.77-0.43-0.010.850.0590.176

    0.093750.468750.10.10.050.20.0500.150.1555.8259.76-3.94-0.070.80.0000.188

    0.593750.968750.0940.1350.0260.1820.0230.0190.2450.07263.3454.199.150.170.7960.0240.090

    0.343750.718750.1760.1530.0290.2270.030.0120.2740.24253.6956.44-2.75-0.051.1430.0100.212

    0.843750.218750.20.070.20.20.10.150.250.359.7560.78-1.03-0.021.470.1020.204

    0.218750.843750.0760.0280.0590.1210.0390.4180.2420.15572.4875.73-3.25-0.041.1380.3670.136

    0.718750.343750.10.150.150.150.10.050.20.0566.8553.6813.170.250.950.0530.053

    0.468750.093750.20.20.050.330.200.10.2556.3161.57-5.26-0.091.330.0000.188

    0.968750.593750.1090.2880.1330.2790.0620.0140.2290.38963.7262.980.750.011.5030.0090.259

    0.0156250.7968750.1090.1530.0510.1170.0180.0390.0830.25950.4156.91-6.50-0.130.8290.0470.312

    0.5156250.2968750.20.070.050.150.150.050.20.1558.8956.522.370.041.020.0490.147

    0.2656250.0468750.10.10.20.150.150.060.150.261.1061.46-0.35-0.011.110.0540.180

    0.7656250.5468750.0240.1030.0290.1540.0360.1410.1650.38957.7569.80-12.04-0.211.0410.1350.374

    0.1406250.4218750.20.20.150.140.10.120.20.356.6156.290.320.011.410.0850.213

    0.6406250.9218750.1760.110.0290.120.0410.0150.3010.29757.7158.76-1.05-0.021.0890.0140.273

    0.3906250.6718750.0940.0280.1330.1790.0330.1160.0940.24257.1368.09-10.95-0.190.9190.1260.263

    0.8906250.1718750.10.150.050.100.10.230.1562.7755.577.200.130.880.1140.170

    0.0781250.2343750.10.10.10.250.100.150.1558.6662.10-3.44-0.060.950.0000.158

    0.5781250.7343750.1160.0650.0670.1210.0850.0140.2520.07261.3153.258.050.150.7920.0180.091

    0.3281250.9843750.0250.2020.0110.2790.020.4180.0470.2866.8877.29-10.41-0.161.2820.3260.218

    0.8281250.4843750.120.150.20.20.10.050.20.362.7262.130.590.011.320.0380.227

    0.2031250.6093750.1140.040.0650.1760.0130.0230.1380.15553.5160.56-7.06-0.130.7240.0320.214

    0.7031250.1093750.070.050.10.20.130.150.150.259.9164.32-4.41-0.071.050.1430.190

    0.4531250.3593750.120.150.10.20.0900.080.1655.2356.43-1.21-0.020.90.0000.178

    0.9531250.8593750.0650.0240.1030.2270.0270.0190.0260.21358.6272.35-13.73-0.230.7040.0270.303

    0.0468750.2656250.10.050.020.10.150.050.10.164.3666.98-2.62-0.040.670.0750.149

    0.5468750.7656250.0450.0450.1330.070.030.0310.2740.2659.6162.17-2.56-0.040.8880.0350.293

    0.2968750.5156250.2290.0240.0290.1280.0520.0170.3530.2952.9258.42-5.50-0.101.1220.0150.258

    0.7968750.0156250.10.150.050.090.10.090.150.1263.8655.937.930.140.850.1060.141

    0.1718750.8906250.080.0650.0290.440.0190.0880.2490.31758.2973.84-15.55-0.271.2870.0680.246

    0.6718750.3906250.10.020.10.20.20.150.150.1958.9363.56-4.63-0.081.110.1350.171

    0.4218750.1406250.10.30.150.10.10.050.050.256.3849.806.590.131.050.0480.190

    0.9218750.6406250.090.0410.0510.3020.0390.0130.2420.22957.5365.36-7.83-0.141.0070.0130.227

    0.1093750.7031250.0240.2880.0330.030.0620.0560.2450.24660.1454.705.440.100.9840.0570.250

    0.6093750.2031250.050.050.050.30.030.10.10.258.6869.54-10.86-0.180.880.1140.227

    0.3593750.4531250.10.050.10.30.050.050.080.256.2468.40-12.16-0.220.930.0540.215

    0.8593750.9531250.1090.0280.0660.1170.030.020.1380.36254.8069.04-14.24-0.260.870.0230.416

    0.2343750.0781250.150.070.10.250.10.10.20.358.0566.74-8.69-0.151.270.0790.236

    0.7343750.5781250.0240.1530.0130.1790.0660.0390.1760.26261.2565.54-4.30-0.070.9120.0430.287

    0.4843750.8281250.1230.1350.0380.1710.0230.0530.2290.23455.5859.17-3.60-0.061.0060.0530.233

    0.9843750.3281250.150.10.090.20.0500.10.260.7062.20-1.51-0.020.890.0000.225

    0.00781250.66406250.1230.2650.070.2250.0660.0190.1380.2654.2955.61-1.32-0.021.1660.0160.223

    0.50781250.16406250.050.220.120.330.100.20.167.1859.617.570.131.120.0000.089

    0.25781250.41406250.10.050.10.150.050.150.050.1261.7066.11-4.41-0.070.770.1950.156

    0.75781250.91406250.0240.040.0920.1210.0190.0230.3010.2958.0564.77-6.72-0.120.910.0250.319

    0.13281250.03906250.10.150.150.150.150.050.150.1960.8859.871.010.021.090.0460.174

    0.63281250.53906250.2290.0560.0510.2270.0540.4180.1650.31756.6362.76-6.13-0.111.5170.2760.209

    0.38281250.78906250.0240.2020.3570.1820.1650.0140.0470.22960.1357.912.220.041.220.0110.188

    0.88281250.28906250.050.150.10.10.100.20.264.2059.704.510.080.90.0000.222

    0.07031250.35156250.10.150.110.150.20.10.20.264.5164.200.310.001.210.0830.165

    0.57031250.85156250.090.0490.0660.1540.0620.1160.2520.24657.9962.43-4.45-0.081.0350.1120.238

    0.32031250.60156250.080.1030.0330.1760.030.0150.0940.36249.7067.33-17.63-0.350.8930.0170.405

    0.82031250.10156250.050.050.10.150.10.120.230.260.0062.41-2.41-0.0410.1200.200

    0.19531250.97656250.0780.110.0380.4450.030.1410.0830.26258.9174.45-15.54-0.261.1870.1190.221

    0.69531250.47656250.10.20.090.150.10.060.20.362.6261.401.210.021.20.0500.250

    0.44531250.22656250.10.150.10.20.050.050.150.256.9359.22-2.29-0.0410.0500.200

    0.94531250.72656250.0380.0990.0130.2790.0230.0390.0550.23459.3872.09-12.71-0.210.780.0500.300

    0.03906250.13281250.050.050.120.150.1300.20.0567.0362.434.600.070.750.0000.067

    0.53906250.63281250.0650.110.0560.3960.0130.0530.2420.25959.7766.43-6.66-0.111.1940.0440.217

    0.28906250.88281250.090.0990.0260.030.0270.0390.2290.38951.7562.88-11.13-0.220.9290.0420.419

    0.78906250.38281250.100.20.10.090.10.10.2559.4065.27-5.87-0.100.940.1060.266

    0.16406250.50781250.0380.0490.3570.1710.0540.020.2450.29758.0660.81-2.75-0.051.2310.0160.241

    0.66406250.00781250.050.050.050.20.150.050.150.357.6668.67-11.01-0.1910.0500.300

    0.41406250.25781250.20.150.150.250.10.10.250.1559.7956.802.990.051.350.0740.111

    0.91406250.75781250.0780.0810.0590.070.020.0560.2740.24258.9759.78-0.80-0.010.880.0640.275

    0.10156250.82031250.0940.0450.0110.1280.0850.0130.2490.07263.6361.512.120.030.6970.0190.103

    0.60156250.32031250.10.050.150.090.10.050.050.1562.3961.910.470.010.740.0680.203

    0.35156250.07031250.20.130.20.10.10.150.10.355.9257.75-1.83-0.031.280.1170.234

    0.85156250.57031250.1760.0540.0670.2250.0330.0880.1270.2856.9164.77-7.86-0.141.050.0840.267

    0.22656250.44531250.10.220.10.10.050.030.10.254.1550.893.260.060.90.0330.222

    0.72656250.94531250.0760.2650.1030.3020.0180.0170.1760.15567.1859.687.500.131.1120.0150.139

    0.47656250.69531250.0290.0410.0660.440.0360.0310.3530.21362.4173.30-10.89-0.171.2090.0260.176

    0.97656250.19531250.120.050.110.150.150.020.150.1660.4460.51-0.07-0.000.910.0220.176

    0.02343750.39843750.20.130.150.20.10.020.150.1555.4752.972.500.051.10.0180.136

    0.52343750.89843750.1090.0540.0110.1710.0390.0310.0550.07259.3460.80-1.46-0.020.5420.0570.133

    0.27343750.64843750.0240.0450.0670.070.0620.020.0260.2852.2873.05-20.77-0.400.5940.0340.471

    0.77343750.14843750.10.050.20.250.20.030.080.261.9965.14-3.15-0.051.110.0270.180

    0.14843750.77343750.1760.0410.1030.3960.0520.0880.2520.20458.4766.00-7.54-0.131.3120.0670.155

    0.64843750.27343750.150.30.10.150.10.150.150.1666.7055.4711.230.201.260.1190.127

    0.39843750.02343750.150.020.10.10.150.050.20.162.9560.642.310.040.870.0570.115

    0.89843750.52343750.0240.2650.0650.3020.030.0530.0470.11569.4862.996.490.100.9010.0590.128

    0.08593750.58593750.1160.0990.0560.1820.0230.0120.3010.29754.5561.05-6.50-0.121.0860.0110.273

    0.58593750.08593750.100.120.10.050.050.050.356.0170.29-14.29-0.260.770.0650.390

    0.33593750.33593750.120.050.20.150.10.090.20.1561.7660.521.240.021.060.0850.142

    0.83593750.83593750.0250.1350.0260.440.0660.0170.1380.24260.8870.05-9.16-0.151.0890.0160.222

    0.21093750.21093750.070.150.050.150.050.050.20.0563.5955.298.300.150.770.0650.065

    0.71093750.71093750.1140.0810.1330.1280.030.0280.0830.25960.5364.89-4.37-0.070.8560.0330.303

    0.46093750.96093750.0650.2880.0590.2250.0540.0130.1650.38953.7160.24-6.54-0.121.2580.0100.309

    0.96093750.46093750.120.050.150.330.030.050.150.2559.5566.36-6.81-0.111.130.0440.221

    0.05468750.92968750.0450.1030.0660.2270.0330.0290.2420.2658.0966.54-8.45-0.151.0050.0290.259

    0.55468750.42968750.10.050.110.200.120.20.358.0865.48-7.40-0.131.080.1110.278

    0.30468750.17968750.10.150.10.20.10.050.10.160.7258.422.310.040.90.0560.111

    0.80468750.67968750.0780.0490.0330.1820.0410.4180.2290.2957.1764.39-7.22-0.131.320.3170.220

    0.17968750.30468750.100.20.330.150.150.10.1964.0773.13-9.06-0.141.220.1230.156

    0.67968750.80468750.080.0990.0380.2790.0360.0120.2450.31758.2666.29-8.03-0.141.1060.0110.287

    0.42968750.55468750.090.110.0110.1210.030.0190.1380.22952.6860.89-8.21-0.160.7480.0250.306

    0.92968750.05468750.10.050.150.150.1500.080.261.2364.25-3.02-0.050.880.0000.227

    0.11718750.11718750.050.050.120.150.150.050.15071.6364.686.950.110.720.0690.000

    0.61718750.61718750.0240.0410.070.4450.0270.0140.2490.24658.4669.66-11.20-0.191.1160.0130.220

    0.36718750.86718750.1090.2650.0920.120.0180.1410.1270.36254.6357.95-3.31-0.061.2340.1140.293

    0.86718750.36718750.10.220.10.20.050.060.050.2564.8063.551.250.021.030.0580.243

    0.24218750.74218750.0240.0540.0590.1540.020.0230.2740.26258.5068.38-9.87-0.170.870.0260.301

    0.74218750.24218750.150.150.150.150.10.050.10.362.0362.69-0.66-0.011.150.0430.261

    0.49218750.49218750.150.150.10.150.10.10.150.258.7158.73-0.01-0.001.10.0910.182

    0.99218750.99218750.1230.0560.3570.1760.0850.1160.3530.23462.9458.884.050.071.50.0770.156

    0.003906250.996093750.0240.0990.0330.1710.0130.0310.2740.2857.9967.42-9.43-0.160.9250.0340.303

    0.503906250.496093750.150.150.10.20.050.050.250.261.9958.073.930.071.150.0430.174

    0.253906250.246093750.150.20.110.250.150.090.150.1561.2459.391.850.031.250.0720.120

    0.753906250.746093750.1230.110.0660.070.0270.0170.2450.25959.5958.840.750.010.9170.0190.282

    0.128906250.371093750.10.050.150.30.10.150.10.1563.4169.84-6.43-0.101.10.1360.136

    0.628906250.871093750.0240.1030.0130.3960.0850.0280.2290.11561.3764.60-3.22-0.050.9930.0280.116

    0.378906250.621093750.1090.0490.0380.030.020.0130.2420.20453.5755.95-2.37-0.040.7050.0180.289

    0.878906250.121093750.20.150.20.30.10.050.20.361.7361.660.070.001.50.0330.200

    0.066406250.058593750.10.150.10.10.10.10.150.358.1162.50-4.39-0.081.10.0910.273

    0.566406250.558593750.080.2020.0920.3020.0410.0560.3530.24262.8060.192.600.041.3680.0410.177

    0.316406250.808593750.1160.0560.070.440.0330.020.1760.07259.3766.27-6.90-0.120.9830.0200.073

    0.816406250.308593750.10.150.120.1500.050.10.1565.2059.475.740.100.820.0610.183

    0.191406250.683593750.0450.040.3570.1280.030.0390.1270.38953.6362.13-8.50-0.161.1550.0340.337

    0.691406250.183593750.10.050.10.090.150.10.050.2559.0865.69-6.60-0.110.890.1120.281

    0.441406250.433593750.10.220.150.10.1500.150.0562.4247.7614.650.310.920.0000.054

    0.941406250.933593750.2290.2650.0590.2250.0360.0530.2490.15564.0753.7410.330.191.2710.0420.122

    0.035156250.464843750.070.050.20.150.10.050.20.164.0360.563.470.060.920.0540.109

    0.535156250.964843750.1140.0410.0670.2270.0660.0390.0470.11559.6764.46-4.79-0.080.7160.0540.161

    0.285156250.714843750.0650.2650.0110.1820.0230.1410.1650.2659.0561.45-2.41-0.041.1120.1270.234

    0.785156250.214843750.120.220.150.10.050.060.150.364.3460.363.990.071.150.0520.261

    0.160156250.839843750.1160.0540.0650.2250.0540.0150.3010.23457.5263.89-6.37-0.111.0640.0140.220

    0.660156250.339843750.20.130.10.330.030.120.050.260.9863.11-2.14-0.041.160.1030.172

    0.410156250.089843750.050.050.10.150.050.10.20.1962.5167.99-5.48-0.090.890.1120.213

    0.910156250.589843750.0250.0450.1030.1210.030.0560.1380.31758.2270.18-11.96-0.210.8350.0670.380

    0.097656250.527343750.1760.0810.0260.4450.0190.0140.0550.36248.6769.03-20.36-0.421.1780.0120.307

    0.597656250.027343750.20.150.20.150.200.150.2562.9658.964.010.071.30.0000.192

    0.347656250.277343750.10.050.120.20.10.050.2066.4859.836.650.110.820.0610.000

    0.847656250.777343750.0940.0490.0560.2790.0390.4180.0830.24657.9766.95-8.98-0.151.2640.3310.195

    0.222656250.152343750.200.150.150.150.050.230.1260.6459.291.350.021.050.0480.114

    0.722656250.652343750.1090.0990.0590.1540.030.0230.0940.2958.8066.51-7.71-0.130.8580.0270.338

    0.472656250.902343750.0760.1350.3570.1760.0520.0190.2520.26257.4355.052.380.041.3290.0140.197

    0.972656250.402343750.10.050.050.150.10.10.20.159.8657.931.930.030.850.1180.118

    0.019531250.199218750.10.050.20.330.10.10.080.1262.1767.05-4.87-0.081.080.0930.111

    0.519531250.699218750.0760.2880.0260.2790.0330.1160.1380.2964.0363.550.490.011.2460.0930.233

    0.269531250.949218750.0290.0810.0560.1210.0850.0230.2450.2660.4968.09-7.60-0.130.90.0260.289

    0.769531250.449218750.120.150.120.150.10.050.10.165.6456.938.710.150.890.0560.112

    0.144531250.574218750.0940.1350.0590.2270.0360.4180.2290.22967.7571.85-4.09-0.061.4270.2930.160

    0.644531250.074218750.10.050.150.20.150.050.10.2560.1865.84-5.65-0.091.050.0480.238

    0.394531250.324218750.200.050.20.0500.2056.6952.164.530.090.70.0000.000

    0.894531250.824218750.1760.1530.3570.1820.0180.0140.1760.31764.5060.833.660.061.3930.0100.228

    0.082031250.761718750.0380.2650.0670.1540.0410.0190.3530.36257.4059.05-1.66-0.031.2990.0150.279

    0.582031250.261718750.050.020.20.150.0900.150.261.6964.97-3.27-0.050.860.0000.233

    0.332031250.011718750.20.30.150.150.130.150.10.258.5552.735.820.111.380.1090.145

    0.832031250.511718750.1160.0410.0110.1760.0130.0120.2740.15556.7658.61-1.84-0.030.7980.0150.194

    0.207031250.386718750.120.050.110.150.10.050.050.256.0464.27-8.23-0.150.830.0600.241

    0.707031250.886718750.0650.0450.0660.4450.020.0280.1270.23458.2871.25-12.96-0.221.030.0270.227

    0.457031250.636718750.1140.0540.1030.120.0540.0290.2490.26255.9461.28-5.34-0.100.9850.0290.266

    0.957031250.136718750.070.130.10.20.150.120.150.361.0865.10-4.02-0.071.220.0980.246

    0.050781250.730468750.0780.0560.0920.3960.0390.0130.0470.38951.4274.19-22.77-0.441.110.0120.350

    0.550781250.230468750.20.050.020.150.10.050.20.256.5058.64-2.14-0.040.970.0520.206

    0.300781250.480468750.050.130.050.10.20.150.150.1570.0470.10-0.07-0.000.980.1530.153

    0.800781250.980468750.0380.0540.070.3020.0620.0280.2520.25958.6567.01-8.36-0.141.0650.0260.243

    0.175781250.105468750.10.220.10.20.050.090.080.1558.1856.062.120.040.990.0910.152

    0.675781250.605468750.090.2650.3570.1710.0230.0170.0260.11573.6857.4916.190.281.0640.0160.108

    0.425781250.855468750.080.0410.0510.070.030.0310.0550.29748.3167.07-18.76-0.390.6550.0470.453

    0.925781250.355468750.050.050.150.250.10.050.150.359.7868.51-8.73-0.151.10.0450.273

    0.113281250.292968750.10.050.10.150.0500.150.352.0263.89-11.87-0.230.90.0000.333

    0.613281250.792968750.2290.110.0330.1280.0190.0530.1650.2857.4359.45-2.03-0.041.0170.0520.275

    0.363281250.542968750.0450.0990.0660.2250.0660.0880.0830.07265.0367.56-2.53-0.040.7440.1180.097

    0.863281250.042968750.0500.110.330.10.10.20.1559.1765.89-6.73-0.111.040.0960.144

    0.238281250.917968750.1230.0490.0130.1820.1650.020.0940.21358.5270.34-11.82-0.200.8590.0230.248

    0.738281250.417968750.050.150.10.10.050.030.20.1665.0458.066.980.120.840.0360.190

    0.488281250.167968750.10.050.090.150.030.020.050.250.8061.91-11.11-0.220.690.0290.290

    0.988281250.667968750.0760.1030.0380.440.0540.0560.3010.15560.3863.04-2.66-0.041.2230.0460.127

    0.011718750.332031250.150.020.050.150.050.120.10.2555.3667.88-12.52-0.230.890.1350.281

    0.511718750.832031250.0240.0650.0290.1170.030.0290.1270.38955.7173.50-17.79-0.320.810.0360.480

    0.261718750.582031250.1230.0410.0920.1540.0540.0280.2490.25955.8562.70-6.85-0.1210.0280.259

    0.761718750.082031250.150.30.020.150.030.050.050.0570.5350.9719.560.380.80.0630.063

    0.136718750.957031250.0240.0450.0510.120.0190.0120.3530.24260.0466.38-6.34-0.110.8660.0140.279

    0.636718750.457031250.050.050.150.140.050.150.150.1562.1262.23-0.11-0.000.890.1690.169

    0.386718750.207031250.10.130.10.10.100.10.352.9760.00-7.03-0.130.930.0000.323

    0.886718750.707031250.2290.0540.3570.1790.0660.0190.1760.29761.8560.691.160.021.3770.0140.216

    0.074218750.644531250.080.1530.0660.1210.0520.0140.2290.2855.5559.28-3.73-0.070.9950.0140.281

    0.574218750.144531250.050.070.110.250.0500.10.359.5770.81-11.24-0.190.930.0000.323

    0.324218750.394531250.10.10.10.20.10.050.20.1561.2862.15-0.88-0.0110.0500.150

    0.824218750.894531250.090.1350.0330.2790.030.1410.2420.07262.6057.724.880.081.0220.1380.070

    0.199218750.019531250.10.050.090.20.10.050.250.1662.1065.07-2.97-0.0510.0500.160

    0.699218750.519531250.0450.2880.0380.1760.0390.0230.2740.21366.1857.848.340.141.0960.0210.194

    0.449218750.769531250.0780.0280.0130.2270.0620.1160.2450.15564.6272.51-7.88-0.120.9240.1260.168

    0.949218750.269531250.10.050.10.20.20.10.150.259.8063.28-3.47-0.061.10.0910.182

    0.042968750.863281250.1140.2880.0290.070.020.0560.0940.2950.8750.640.240.000.9610.0580.302

    0.542968750.363281250.070.20.20.10.10.020.230.1268.3954.5713.810.251.040.0190.115

    0.292968750.113281250.120.150.050.090.10.030.20.160.6152.957.660.140.840.0360.119

    0.792968750.613281250.0940.0280.0510.1280.0850.020.2520.2656.4763.01-6.54-0.120.9180.0220.283

    0.167968750.488281250.10.070.020.20.150.10.150.263.3071.67-8.37-0.130.990.1010.202

    0.667968750.988281250.1160.1530.1330.440.0270.0880.0550.22962.5467.04-4.50-0.071.2410.0710.185

    0.417968750.738281250.0250.1350.0290.3020.0180.0530.0830.31755.0172.69-17.68-0.320.9620.0550.330

    0.917968750.238281250.120.10.050.10.0500.20.1960.1458.142.000.030.810.0000.235

    0.105468750.175781250.150.050.20.30.150.050.050.258.0965.28-7.19-0.121.150.0430.174

    0.605468750.675781250.1760.040.0260.030.0360.0120.3010.36253.9360.50-6.57-0.120.9830.0120.368

    0.355468750.925781250.0240.0240.0670.1170.030.0170.1380.24655.4971.43-15.94-0.290.6630.0260.371

    0.855468750.425781250.150.150.10.30.150.090.20.261.8560.731.130.021.340.0670.149

    0.230468750.550781250.1090.0650.0130.1790.0130.0280.0470.23448.5465.54-17.00-0.350.6880.0410.340

    0.730468750.050781250.20.020.10.100.050.20.256.7457.68-0.94-0.020.870.0570.230

    0.480468750.300781250.10.30.10.140.10.050.150.356.3855.560.820.011.240.0400.242

    0.980468750.800781250.0240.2020.0650.1710.0410.0130.1650.26264.3064.090.210.000.9430.0140.278

    0.027343750.597656250.0760.0240.0110.440.030.0530.1270.2956.1578.45-22.29-0.401.0510.0500.276

    0.527343750.097656250.10.150.20.150.100.050.1267.9459.188.760.150.870.0000.138

    0.277343750.347656250.120.050.10.10.050.10.150.160.8858.662.210.040.770.1300.130

    0.777343750.847656250.0290.040.0670.3020.0230.0880.2490.2658.0567.74-9.69-0.171.0580.0830.246

    0.152343750.222656250.10.150.10.10.20.030.150.261.8361.130.710.011.030.0290.194

    0.652343750.722656250.0940.2020.1030.2250.0620.020.3530.22963.3158.235.090.091.2880.0160.178

    0.402343750.972656250.1760.0650.0650.1280.030.0560.1760.31751.0160.33-9.31-0.181.0130.0550.313

    0.902343750.472656250.20.10.10.090.10.020.10.1960.6757.533.150.050.90.0220.211

    0.089843750.410156250.050.150.20.250.150.050.10.2560.7265.44-4.72-0.081.20.0420.208

    0.589843750.910156250.0380.0280.0290.070.0540.0130.2290.36254.4068.84-14.44-0.270.8230.0160.440

    0.339843750.660156250.0780.110.0260.1710.1650.0280.2420.24662.0567.91-5.85-0.091.0660.0260.231

    0.839843750.160156250.20.20.050.20.10.150.2064.9750.5814.400.281.10.1360.000

    0.214843750.785156250.0650.1030.1330.030.0390.0170.1380.23453.7855.28-1.50-0.030.7590.0220.308

    0.714843750.285156250.050.050.050.30.10.090.250.258.8564.86-6.01-0.101.090.0830.183

    0.464843750.035156250.10.150.150.30.050.050.150.355.6863.10-7.42-0.131.250.0400.240

    0.964843750.535156250.1140.1530.0290.1170.0190.0310.2450.26260.3159.500.810.010.970.0320.270

    0.058593750.066406250.10.10.10.140.050.10.20.259.6761.59-1.93-0.030.990.1010.202

    0.558593750.566406250.0780.1350.0920.120.0180.1160.1380.2861.3463.83-2.48-0.040.9770.1190.287

    0.308593750.816406250.0380.1530.0330.1790.0360.0230.3010.07264.4957.826.670.120.8350.0280.086

    0.808593750.316406250.10.070.050.10.150.050.050.1560.4562.16-1.71-0.030.720.0690.208

    0.183593750.691406250.090.0280.0380.1170.0410.4180.1650.11573.2976.58-3.28-0.041.0120.4130.114

    0.683593750.191406250.10.050.090.30.10.050.150.160.8662.40-1.54-0.030.940.0530.106

    0.433593750.441406250.050.050.10.30.100.20.1661.8169.60-7.79-0.130.960.0000.167

    0.933593750.941406250.080.2880.0130.030.0330.0140.0830.20468.6056.2712.330.220.7450.0190.274

    0.121093750.878906250.2290.2020.0290.1760.0540.0560.0260.24248.8252.37-3.55-0.071.0140.0550.239

    0.621093750.378906250.10.30.050.20.150.10.080.1567.3458.339.000.151.130.0880.133

    0.371093750.128906250.050.020.020.20.10.120.150.361.7979.44-17.65-0.290.960.1250.313

    0.871093750.628906250.0450.0650.1330.1540.020.0150.0550.29760.1571.54-11.39-0.190.7840.0190.379

    0.246093750.253906250.050.150.050.250.130.060.20.2562.0568.85-6.80-0.111.140.0530.219

    0.746093750.753906250.1230.0240.0510.1210.0130.1410.0940.38954.0670.02-15.96-0.300.9560.1470.407

    0.496093750.503906250.0240.0450.0290.2790.0270.0390.2520.25959.7274.26-14.54-0.240.9540.0410.271

    0.996093750.003906250.10.050.20.20.090.050.150.0564.6257.826.800.120.890.0560.056

    0.0019531250.5019531250.2290.1350.0650.1540.0410.0170.2490.24251.7252.06-0.34-0.011.1320.0150.214

    0.5019531250.0019531250.10.050.10.150.10.030.150.1560.5260.92-0.40-0.010.830.0360.181

    0.2519531250.2519531250.0500.10.1500.020.050.348.2972.16-23.88-0.490.670.0300.448

    0.7519531250.7519531250.0240.0990.1030.1760.0330.0310.1270.29760.8569.04-8.19-0.130.890.0350.334

    0.1269531250.1269531250.150.050.20.150.150.050.10.258.2860.93-2.66-0.051.050.0480.190

    0.6269531250.6269531250.1230.0490.0670.4450.0180.0530.2740.38956.5468.47-11.92-0.211.4180.0370.274

    0.3769531250.8769531250.0240.0810.0110.2790.0360.0880.3530.25963.0772.58-9.50-0.151.1310.0780.229

    0.8769531250.3769531250.150.150.150.20.150.150.150.1563.5858.465.120.091.250.1200.120

    0.0644531250.4394531250.10.050.150.330.130.090.20.1663.5168.72-5.21-0.081.210.0740.132

    0.5644531250.9394531250.0780.0450.0590.2250.0130.0190.1760.21357.9865.85-7.87-0.140.8280.0230.257

    0.3144531250.6894531250.0450.0540.1330.1280.0270.0120.2290.15559.3660.05-0.69-0.010.7830.0150.198

    0.8144531250.1894531250.10.130.050.150.090.050.10.261.6462.26-0.62-0.010.870.0570.230

    0.1894531250.8144531250.090.2650.0260.2270.0850.0130.2450.2857.2359.08-1.85-0.031.2310.0110.227

    0.6894531250.3144531250.10.220.050.150.10.050.10.362.9363.23-0.30-0.001.070.0470.280

    0.4394531250.0644531250.050.050.20.10.10.050.080.1561.3962.47-1.08-0.020.780.0640.192

    0.9394531250.5644531250.080.0410.0290.1820.020.0280.1380.07259.5360.07-0.54-0.010.590.0470.122

    0.0332031250.0332031250.120.220.10.090.20.050.050.259.7957.352.440.041.030.0490.194

    0.5332031250.5332031250.0250.2650.0130.1280.0190.4180.3010.22963.4858.654.840.081.3980.2990.164

    0.2832031250.7832031250.1160.040.0650.2250.0390.0290.1380.31752.2468.00-15.76-0.300.9690.0300.327

    0.7832031250.2832031250.10.050.090.10.100.20.1959.5060.07-0.56-0.010.830.0000.229

    0.1582031250.6582031250.0940.0560.0330.3020.030.0190.0830.2951.8771.44-19.57-0.380.9070.0210.320

    0.6582031250.1582031250.120.150.050.10.1500.20.1263.5754.439.130.170.890.0000.135

    0.4082031250.4082031250.070.150.10.150.10.150.150.166.4762.563.920.060.970.1550.103

    0.9082031250.9082031250.1140.2020.0920.440.0520.0120.1650.2662.5964.94-2.35-0.041.3370.0090.194

    0.0957031250.9707031250.0240.0490.0290.3960.0660.1410.0550.23462.7682.98-20.22-0.320.9940.1420.235

    0.5957031250.4707031250.10.150.20.30.10.060.150.264.2362.501.730.031.260.0480.159

    0.3457031250.2207031250.20.050.050.30.050.050.080.350.6765.71-15.04-0.301.080.0460.278

    0.8457031250.7207031250.1090.1030.0510.030.0230.0390.0260.26259.2965.26-5.97-0.100.6430.0610.407

    0.2207031250.3457031250.150.050.020.20.030.10.150.256.1265.34-9.22-0.160.90.1110.222

    0.7207031250.8457031250.0240.110.1330.1710.0620.1160.2520.36260.4865.94-5.46-0.091.230.0940.294

    0.4707031250.5957031250.1760.0990.0290.070.030.0230.0470.24644.7952.94-8.15-0.180.720.0320.342

    0.9707031250.0957031250.1500.050.250.050.050.20.255.6763.44-7.77-0.140.950.0530.211

    0.0175781250.2988281250.20.050.10.10.090.150.150.258.3660.62-2.26-0.041.040.1440.192

    0.5175781250.7988281250.1760.1030.0290.1820.0270.0230.2490.22958.3158.87-0.56-0.011.0180.0230.225

    0.2675781250.5488281250.0940.0490.0510.440.0130.0190.1270.31753.1673.19-20.03-0.381.110.0170.286

    0.7675781250.0488281250.10.150.150.150.1300.050.1965.2162.023.180.050.920.0000.207

    0.1425781250.9238281250.0290.0990.1330.1280.020.0140.1760.2954.8362.84-8.01-0.150.8890.0160.326

    0.6425781250.4238281250.1200.020.090.100.10.1254.9960.42-5.43-0.100.550.0000.218

    0.3925781250.1738281250.10.050.050.10.10.050.150.162.4562.320.130.000.70.0710.143

    0.8925781250.6738281250.0760.110.0290.2250.0850.4180.3530.2659.1461.21-2.07-0.041.5560.2690.167

    0.0800781250.7363281250.1140.0410.0130.1710.0330.0150.2420.23454.8964.67-9.79-0.180.8630.0170.271

    0.5800781250.2363281250.070.050.10.200.120.20.259.5263.67-4.14-0.070.940.1280.213

    0.3300781250.4863281250.120.220.090.250.10.10.10.357.2062.66-5.46-0.101.280.0780.234

    0.8300781250.9863281250.0650.2650.0380.070.0410.1160.2290.26264.6658.136.520.111.0860.1070.241

    0.2050781250.1113281250.20.130.050.150.150.050.150.2556.0859.59-3.52-0.061.130.0440.221

    0.7050781250.6113281250.0780.0540.0330.3960.0360.0390.2450.36256.4469.67-13.23-0.231.2430.0310.291

    0.4550781250.8613281250.0380.0560.0660.3020.0180.1410.2740.24662.8573.25-10.40-0.171.1410.1240.216

    0.9550781250.3613281250.050.050.10.10.050.060.25064.5150.9713.540.270.660.0910.000

    0.0488281250.7675781250.080.0540.0290.4450.0190.0390.2520.11561.2270.52-9.30-0.151.0330.0380.111

    0.5488281250.2675781250.050.130.150.150.050.10.20.165.8557.518.340.150.930.1080.108

    0.2988281250.0175781250.10.050.050.20.100.230.1659.9664.86-4.89-0.080.890.0000.180

    0.7988281250.5175781250.090.0450.1330.120.0390.0530.0940.20460.3364.42-4.09-0.070.7780.0680.262

    0.1738281250.3925781250.10.30.050.150.10.10.20.260.7356.034.700.081.20.0830.167

    0.6738281250.8925781250.0380.0410.0510.1540.1650.0560.0830.2857.6870.33-12.65-0.220.8680.0650.323

    0.4238281250.6425781250.0780.2650.0290.1760.0540.020.0550.07257.4348.888.550.170.7490.0270.096

    0.9238281250.1425781250.10.020.20.150.150.050.150.1561.7761.130.640.010.970.0520.155

    0.1113281250.2050781250.100.10.20.050.150.20.2560.8371.42-10.59-0.171.050.1430.238

    0.6113281250.7050781250.0240.1530.0650.2270.030.0880.1380.38960.2669.92-9.66-0.161.1140.0790.349

    0.3613281250.9550781250.1230.1350.1030.1820.0620.0130.3010.25956.7158.36-1.65-0.031.1780.0110.220

    0.8613281250.4550781250.050.050.10.20.030.050.050.0564.2563.420.830.010.580.0860.086

    0.2363281250.5800781250.0450.0810.0670.2250.0230.0310.1650.24256.6068.01-11.41-0.200.8790.0350.275

    0.7363281250.0800781250.050.150.10.330.050.050.150.1564.1463.800.340.011.030.0490.146

    0.4863281250.3300781250.10.050.20.150.10.090.20.359.0365.03-6.00-0.101.190.0760.252

    0.9863281250.8300781250.2290.2880.0260.1210.0660.0170.0470.29763.3558.624.730.081.0910.0160.272

    0.0097656250.1660156250.20.30.050.10.10.060.150.1556.0246.369.650.211.110.0540.135

    0.5097656250.6660156250.1090.2020.3570.1790.0520.1410.2450.11568.3055.6012.700.231.40.1010.082

    0.2597656250.9160156250.0240.0650.0590.120.030.0390.1380.20458.1468.12-9.98-0.170.6790.0570.300

    0.7597656250.4160156250.10.020.150.140.150.050.250.358.4163.55-5.14-0.091.160.0430.259

    0.1347656250.5410156250.1230.0240.0560.030.0390.1160.2420.2857.3864.44-7.06-0.120.910.1270.308

    0.6347656250.0410156250.150.150.120.30.10.10.20.262.1360.681.450.021.320.0760.152

    0.3847656250.2910156250.150.050.20.30.20.050.10.1561.6065.39-3.79-0.061.20.0420.125

    0.8847656250.7910156250.0240.040.0260.1170.0190.0150.2290.25956.3666.74-10.38-0.180.7290.0210.355

    0.0722656250.8535156250.2290.1030.1030.440.030.4180.1760.38960.5470.97-10.43-0.171.8880.2210.206

    0.5722656250.3535156250.10.10.10.20.050.050.10.2560.6465.47-4.83-0.080.950.0530.263

    0.3222656250.1035156250.10.070.110.10.0300.150.0556.9848.018.970.190.610.0000.082

    0.8222656250.6035156250.0450.1530.0660.3020.0620.0140.3530.15562.8859.543.340.061.150.0120.135

    0.1972656250.4785156250.10.150.150.10.0500.150.352.1455.36-3.22-0.0610.0000.300

    0.6972656250.9785156250.1160.0280.0110.070.0230.0190.2490.24254.4460.58-6.14-0.110.7580.0250.319

    0.4472656250.7285156250.080.2880.0670.1280.0660.0230.1270.29754.1754.82-0.64-0.011.0760.0210.276

    0.9472656250.2285156250.10.20.20.090.10.150.050.1567.7958.169.630.171.040.1440.144

    0.0410156250.6347656250.0250.0280.0590.1760.0360.0170.2520.23459.5270.83-11.31-0.190.8270.0210.283

    0.5410156250.1347656250.050.050.150.20.150.030.150.261.4864.99-3.50-0.060.980.0310.204

    0.2910156250.3847656250.20.050.10.20.050.020.20.1953.3356.74-3.41-0.061.010.0200.188

    0.7910156250.8847656250.1160.2880.3570.2270.0180.0310.0940.31768.1761.896.280.101.4480.0210.219

    0.1660156250.0097656250.120.10.120.2500.050.150.156.6556.400.250.000.890.0560.112

    0.6660156250.5097656250.0650.1350.070.1210.0330.0130.0550.11566.1960.385.810.100.6070.0210.189

    0.4160156250.7597656250.1140.1530.0920.2790.0410.0880.0260.2654.3264.13-9.81-0.181.0530.0840.247

    0.9160156250.2597656250.070.070.10.20.10.150.080.359.2268.38-9.17-0.151.070.1400.280

    0.1035156250.3222656250.10.130.090.140.10.050.20.1261.8757.973.910.070.930.0540.129

    0.6035156250.8222656250.0760.0540.0380.120.020.020.0470.2956.0471.86-15.81-0.280.6650.0300.436

    0.3535156250.5722656250.1090.0450.0130.1790.0850.0560.1650.26257.1470.03-12.88-0.230.9140.0610.287

    0.8535156250.0722656250.20.050.10.10.10.10.150.160.0054.465.540.100.90.1110.111

    0.2285156250.9472656250.0940.0410.0660.1170.0270.1410.3010.36258.5368.76-10.22-0.171.1490.1230.315

    0.7285156250.4472656250.10.30.110.150.0900.050.2567.8260.777.050.121.050.0000.238

    0.4785156250.1972656250.20.020.10.150.130.10.2064.7657.307.450.130.90.1110.000

    0.9785156250.6972656250.1760.0650.0330.1540.0130.0390.1380.24656.4562.27-5.81-0.100.8640.0450.285

    0.0253906250.9003906250.1760.0450.0380.1210.0540.0280.2450.22953.9558.93-4.98-0.090.9360.0300.245

    0.5253906250.4003906250.20.050.20.250.150.050.150.2560.0462.25-2.22-0.041.30.0380.192

    0.2753906250.1503906250.10.150.150.20.10.050.25065.6653.7511.910.2210.0500.000

    0.7753906250.6503906250.0940.0240.0110.2790.1650.0130.2740.24656.3464.74-8.40-0.151.1060.0120.222

    0.1503906250.2753906250.120.020.110.20.10.050.20.1261.7064.21-2.51-0.040.920.0540.130

    0.6503906250.7753906250.0290.0650.0660.1760.0390.0120.2420.2958.4666.66-8.19-0.140.9190.0130.316

    0.4003906250.5253906250.0760.2020.0330.1540.0190.0170.2290.2654.9257.28-2.36-0.040.990.0170.263

    0.9003906250.0253906250.10.30.10.20.050.090.10.169.5455.9513.590.241.040.0870.096

    0.0878906250.0878906250.070.050.150.30.10.150.10.263.3172.51-9.21-0.151.120.1340.179

    0.5878906250.5878906250.1140.2880.0590.1170.0660.0880.1760.23465.5757.408.170.141.1420.0770.205

    0.3378906250.8378906250.0650.0280.3570.030.0230.0530.3530.26259.0455.323.720.071.1710.0450.224

    0.8378906250.3378906250.120.050.10.30.050.050.150.357.9267.54-9.61-0.171.120.0450.268

    0.2128906250.7128906250.1160.1530.070.120.0620.0310.2490.36254.2960.27-5.99-0.111.1630.0270.311

    0.7128906250.2128906250.20.070.120.140.20.020.050.260.1260.81-0.69-0.0110.0200.200

    0.4628906250.4628906250.050.10.10.10.10.030.10.259.1363.63-4.50-0.080.780.0380.256

    0.9628906250.9628906250.0380.1350.0920.1790.030.020.1270.15564.8662.682.180.030.7760.0260.200

    0.0566406250.4316406250.050.150.10.30.150.050.150.1564.1567.30-3.16-0.051.10.0450.136

    0.5566406250.9316406250.080.0490.1030.030.0850.0230.1650.11561.9555.916.040.110.650.0350.177

    0.3066406250.6816406250.090.1030.0650.1170.020.1160.0470.29753.4465.87-12.43-0.230.8550.1360.347

    0.8066406250.1816406250.10.050.10.30.10.10.20.358.3066.45-8.16-0.141.250.0800.240

    0.1816406250.8066406250.0380.110.0110.1790.0130.0140.1380.38950.8870.42-19.53-0.380.8920.0160.436

    0.6816406250.3066406250.050.20.150.250.130.050.20.265.3561.373.980.061.230.0410.163

    0.4316406250.0566406250.20.150.20.20.090.060.050.1554.7752.152.630.051.10.0550.136

    0.9316406250.5566406250.0780.0280.0670.1710.0270.1410.3010


Author's personal copy - andrea saltelli · Author's personal copy How to avoid a perfunctory sensitivity analysis Andrea Saltelli*, Paola Annoni Joint Research Center, Institute

Author's personal copy - andrea saltelli · Author's personal copy How to avoid a perfunctory sensitivity analysis Andrea Saltelli*, Paola Annoni Joint Research Center, Institute

Andrea Saltelli · Quantitative Story-Telling (Andrea Saltelli): The objective of the approach is the verification of the narratives used to promote innovation. Narratives can be

Andrea Saltelli · Quantitative Story-Telling (Andrea Saltelli): The objective of the approach is the verification of the narratives used to promote innovation. Narratives can be

Of apples and oranges - Uni Trier · 2013. 8. 18. · Of apples and oranges by Andrea Saltelli andrea.saltelli@jrc.it KEI Workshop on Composite Indicators: Theory, Methodology and

Of apples and oranges - Uni Trier · 2013. 8. 18. · Of apples and oranges by Andrea Saltelli [email protected] KEI Workshop on Composite Indicators: Theory, Methodology and

Revisiting worst-case DEA for composite indicators · assumptions behind the construction of composite indices such as the choice of normalization procedure, aggregation function,

Revisiting worst-case DEA for composite indicators · assumptions behind the construction of composite indices such as the choice of normalization procedure, aggregation function,

SENSITIVITY ANALYSIS IN PRACTICE - andrea saltelli...SENSITIVITY ANALYSIS IN PRACTICE A GUIDE TO ASSESSING SCIENTIFIC MODELS Andrea Saltelli, Stefano Tarantola, Francesca Campolongo

SENSITIVITY ANALYSIS IN PRACTICE - andrea saltelli...SENSITIVITY ANALYSIS IN PRACTICE A GUIDE TO ASSESSING SCIENTIFIC MODELS Andrea Saltelli, Stefano Tarantola, Francesca Campolongo

CONCEPTUAL ISSUES IN CONSTRUCTING COMPOSITE INDICES Nadia Farrugia Department of Economics, University of Malta Paper prepared for the INTERNATIONAL CONFERENCE.

CONCEPTUAL ISSUES IN CONSTRUCTING COMPOSITE INDICES Nadia Farrugia Department of Economics, University of Malta Paper prepared for the INTERNATIONAL CONFERENCE.

DESIGN AND USE OF COMPOSITE INDICES IN ......DESIGN AND USE OF COMPOSITE This paper has benefited from the reviews and comments of Alex de Sherbinin (Senior Research Associate, Center

DESIGN AND USE OF COMPOSITE INDICES IN ......DESIGN AND USE OF COMPOSITE This paper has benefited from the reviews and comments of Alex de Sherbinin (Senior Research Associate, Center

Composite Indicators for Community Vitality Quebec · Composite Indicators of Community Vitality Concepts and Relative Indices Relative Indices The Intergenerational Index (intergen)

Composite Indicators for Community Vitality Quebec · Composite Indicators of Community Vitality Concepts and Relative Indices Relative Indices The Intergenerational Index (intergen)

Andrea Saltelli Centre for the Study of the Sciences and ... · Ethics of quantification Andrea Saltelli Centre for the Study of the Sciences and the Humanities (SVT), University

Andrea Saltelli Centre for the Study of the Sciences and ... · Ethics of quantification Andrea Saltelli Centre for the Study of the Sciences and the Humanities (SVT), University

Andrea Saltelli, European Commission, Joint Research Centre Copenhagen, October 2007

Andrea Saltelli, European Commission, Joint Research Centre Copenhagen, October 2007

1 Constructing Composite Indicators: From Theory to Practice ECFIN, November 11-12, 2010 Andrea Saltelli Ranking and rating: Woodo or Science? Andrea Saltelli,

1 Constructing Composite Indicators: From Theory to Practice ECFIN, November 11-12, 2010 Andrea Saltelli Ranking and rating: Woodo or Science? Andrea Saltelli,

Footprints to nowhere - andrea saltelli · 2014. 10. 24. · 612 M. Giampietro, A. Saltelli / Ecological Indicators 46 (2014) 610–621 and ecological services that the manufactured

Footprints to nowhere - andrea saltelli · 2014. 10. 24. · 612 M. Giampietro, A. Saltelli / Ecological Indicators 46 (2014) 610–621 and ecological services that the manufactured

Matteo Mazziotta and Adriano Pareto Istat - Methodological Office Valentina Talucci University of Rome “La Sapienza” Composite Indices to Measure Poverty.

Matteo Mazziotta and Adriano Pareto Istat - Methodological Office Valentina Talucci University of Rome “La Sapienza” Composite Indices to Measure Poverty.

ROLE OF COMPOSITE INDICES AND SYSTEMIC BIOMARKERS IN COPD

ROLE OF COMPOSITE INDICES AND SYSTEMIC BIOMARKERS IN COPD

Responsible quantification - andrea saltelli...Responsible quantification Andrea Saltelli Centre for the Study of the Sciences and the Humanities (SVT) - University of Bergen (UIB)

Responsible quantification - andrea saltelli...Responsible quantification Andrea Saltelli Centre for the Study of the Sciences and the Humanities (SVT) - University of Bergen (UIB)

Rank Robustness of Composite Indices: Dominance and Ambiguity

Rank Robustness of Composite Indices: Dominance and Ambiguity

An introduction of sensitivity analysis - andrea saltelli · An introduction of sensitivity analysis Andrea Saltelli Centre for the Study of the Sciences and ... Weapons of Math Destruction

An introduction of sensitivity analysis - andrea saltelli · An introduction of sensitivity analysis Andrea Saltelli Centre for the Study of the Sciences and ... Weapons of Math Destruction

Apples and Oranges Composite indicators between analysis and … · 2006. 12. 20. · Apples and Oranges Composite indicators between analysis and advocacy by Andrea Saltelli andrea.saltelli@jrc.it

Apples and Oranges Composite indicators between analysis and … · 2006. 12. 20. · Apples and Oranges Composite indicators between analysis and advocacy by Andrea Saltelli [email protected]

Cointegration and Causality Test of Islamic and Composite Indices in Malaysia

Cointegration and Causality Test of Islamic and Composite Indices in Malaysia

Www.mathsrevision.com Algebraic Operations Adding / Sub Indices Negative Indices Fraction Indices Harder Indices S5 Int2.

Www.mathsrevision.com Algebraic Operations Adding / Sub Indices Negative Indices Fraction Indices Harder Indices S5 Int2.