30 Social indicators: problems of def i ni ti on and of...

No. 30 Social indicators: problems of def i ni ti on

and of selection

cn a 0 C a 0 cn m 0 0 cn a IC

.I

- .I

CI

cn a Q CO Q 'D C CO

L

Unesco

REPORTS AND PAPERS IN THE SOCIAL SCIENCES

The Reports and Papers are intended to present to a restricted public of specialists descriptive or documentary material as and when it becomes available during the execution of Unesco’s programme in the field of the social sciences. They will consist of either reports relating to the Regular Programme of Unesco and its operational programmes of aid to Member States or documentation in the form of bibliographies, repertories and directories.

The authors alone are responsible for the contents of the Reports and Papers and their views should not necessarily be taken to represent those of Unesco.

These documents are published without strict periodicity. Currently available.

SS/CH 11 - SS/CH 15 - SS/CH 17 -

SS/CH 18 - SS/CH 19 - SS/CH 20 - SS/CH 22 - SS/CH 23 - SS/CH 24 - SS/CH 25 -

SS/CH 26 - SS/CH 21

SS/CH 28

SSlCH 29- SS/CH 30

International Repertory of Institutions Conducting Popu- lation Studies (bilingual : English/French), 1959. International Co-operation and Programmes of Economic and Social Development (bilingual : English/French), 1961. International Directory of Sample Survey Centres (outside the United States of America) (bilingual : English/French), 1962. The Social Science Activities of Some Eastern European Academies of Sciences, 1963. Attitude Change : a review and bibliography of selected research, 1964 (out of print in English, available in French). International Repertory of Sociological Research Centres (outside the U.S.A.) (bilingual: English/French), 1964. Institutions Engaged in Economic and Social Planning in Africa (bilingual: English/French), 1966. International Repertory of Institutions Specializing in Research on Peace and Disarmament, 1966. Guide for the Establishment of National Social Science Documentation Centres in Developing Countries, 1969. Ecological data in comparative research : Report on a fiist International Data Confrontation Seminar (bilingual : English/French), 1970. Data archives for the Social Sciences : Purposes, operations and problems, 197 3. DARE - Unesco computerized data retrieval system for documentation in the social and human sciences. International Repertory of Institutions for Peace and Conflict Research. The Unesco Educational Simulation Model (ESM). Social indicators: problems of definition and of selection

Social indicators: problems of definition

and of selection Methods and Analysis Division Department of Social Sciences

Unesco

Published in 1974 by the Unesco Press, 7, place de Fontenoy, 75700 Paris, France

ISBN 92-3-101161-8 French edition ISBN 92-3-201161-1

Printed in the workshops of Unesco Printed in France

0 Unesco 1974 [B]

P ref ace

The present paper needs to be seen within the context of the Unesco project on human resources indicators.

In 1967 a special meeting of experts was held by Unesco in Warsaw with the purpose of discussing alternative approaches to the elaboration of a system of indicators of human resources development, and of giving ad- vice on the choice of methodology. The meeting had discussed papers on the concept and theory of development, on procedures of evaluating high-level manpower data and methods for establishing an order of countries ranked according to their development levels.

the help of an Advisory Panel which met a few times. Working groups discussed the work carried out, which was mainly effected through a number of studies,’ and made an evaluation of the various methods which have been devised or adapted to deal with specific problems, the most important of which has been the identification and selection of indicators. It is this last topic which constitutes the main body of the present paper. The paper also explores the problem of definition, which has been central to the whole project, whether of development on which it hinges, or of the indicators themselves with which it deals. In 1969 when the Advisory Panel met for the first

time, its first task was to review a tentative list of human resources indicators. It was also expected to resolve some controversial points and formulate operational recommendations for future research. The other topics discussed were the concept of human resources, the weighting of component indicators of human resources, and the r61e of human resources in development. At its second meeting, held in 1970, the Panel dis-

cussed papers and studies which were undertaken on the basis of recommendations made at the first meeting. The emphasis shifted however from definitions of concepts, from the descriptive aspects of human resources and the mathematical problems of selecting and weighting indicators, to the interrelationships between human resources and development. Human resources were not understood in the project merely as labour or labour

In the years that followed, the project developed with

potential. The concept used has been broad enough to cover, under the label “human resources indicators”, such indicators as are described in other research programmes as social indicators, socio-economic indicators, indicators of modernization, identified as those indicators which are supposed to affect both labour productivity and the quality of working life.

The ranking, grouping and clustering of countries on the basis of a number of human resources indicators were effected by mathematical methods devised for the purpose.

Studies of gaps and disparities between countries through the use of these various methods have been com- pleted by similar studies within individual countries.

The project on human resources indicators ended in December 1973. The last in a series of meetings to be held under this project took place in December 1973 at the Institute of Development Studies at the University of Sussex, United Kingdom. Two new projects have been launched, the objectives of which have been clearly defined.

The first project relates to the identification of key indicators of social and economic change, for the purpose of appraising socioeconomic progress towards the objectives of the United Nations Second Development Decade.

indicators in development planning, Such indicators will be inore suited to the requirements of planners in an individual country situation, than those which are highly aggregated and tailored for international comparison. The indicators of relevance to this particular project will be drawn from lower levels of aggregation in both the geographical and the social space. They will relate to some of the major sets of determinants of distribution of development and welfare such as: geographical location and ecology, social characteristics, and economic function.

The second project relates to the use of socio-economic

1. See a list of these in Annex on page 27. (These mimeographed papers are avai ‘)le on request).

3

A third project which will be launched in 1975 will methodology of indicator identification and selection, which with concept definition, has occupied a privileged place in the “old” project. The present paper illustrates this effort.

relate to indicators of the quality of life and of the environment. A common concern of these new projects is the

4

Table of contents

Part I . Social Indicators: Problems of Methodology and Selection by Serge Fanchette ................................................. 7

by Zdzislaw Hellwig ................................................. 11

by Branislav Ivanovic ................................................ 21

Part I1 . A Method for the Selection of a “Compact” Set of Variables

Part 111 . A method of Establishing a List of Development Indicators ...................

Annex .................................................................. 27

5

Part I

Social Indicators: Problems of Methodology and Selection

by Serge Fanchette Methods and Analysis Division Department of Social Sciences

U nesco

1. DEFINITION AND PURPOSES OF

The first question which arises is that of definition of a social indicator, followed by a corollary question related to the purposes. Most of the definitions which have been put forward contain reference to the purposes of social indicators thereby making artificial the distinction defini- tionlpurpose. In the rapidly growing literature on social indicators definitions of great diversity are proposed, ranging from the term being used to qualify “good social statistics” obtainable as time-series to series which can fit explicitly into sociological, social policy or social processes models. Within this wide spectrum we find the indicators which under the broad appellation of “social” are described as normative, further distinction being made between input and output variables, i.e. indicators of means as distinguished from indicators of results. There are the ‘‘structural’’ variables characterized by their interrelationships within part or the whole of a social system, as opposed to those which are amenable to some kind of summation into a composite index. And, finally, we find. those indicators which are not viewed within any system and which will rank as performance indicators, social reporting or social intelligence, etc.

tion of social indicators: “Social indicators are needed to find pathways through the maze of society’s interconnections. They delineate social states, define social problems and trace social trends, which by social engineer- ing may hopefully be guided towards social goals formulated by social planning. ”

In the above definition three main elements emerge which define social indicators and their purpose: (i) the descriptive function of the indicators - the

description of social states and of trends in social change;

(ii) their interconnections - which suggests a systems’ approach;

(iii) the analytical tools they can be to the social planner - for the “monitoring” of social change.

SOCIAL INDICATORS

All these aspects are combined in Stuart Rice’s defini-

(i) appears to be the main function served by social indicators in general. (ii) requires the building of a system of indicators or the construction of models which should integrate at least some of the main components of social reality. The conceptual difficulties arising from the many complexities involving among others the interplay of empirical and theoretical factors in this field have limited attempts to build social models. Such attempts have either over-simplified reality in order to capture some relationships that may exist between a very limited number of social variables, or have attempted to introduce into the model so many parameters belonging to a host of social phenomena as to blur out any existing relationships.

By-passing (ii) it has been customary to link up (i) and (ii), going from the descriptive to the prescriptive through an intuitive or a heuristic approach, more often of the former than of the latter.

The definition of Stuart Rice, especially in its prag- matic approach, finds an echo in the following quotation from Sheldon and Moore (eds.), Indicators of Social Change (New York, Russell Sage Foundation, 1968, p.4): “It is especially for those who have undertaken responsi- bility for bringing about publicly approved changes that the notion of “social indicators” is appealing. Such indicators would give a reading both on the current state of some segment of the social universe and on past and future trends, whether progressive or regressive, according to some normative criteria. The notion of social indicators leads directly to the idea of “monitoring” social change. If an indicator can be found that will stand for a set of correlated changes, and if intervention can be introduced (whether on the prime, indicative, variable or on one of its systenlic components), then the programme administra- tor may have been provided a powerful analytical and policy tool.”

The above passage introduces the hypothesis of a set of correlated changes for which an indicator could stand; a similar concept runs through the second and third sec- tions of this paper when it comes to selecting a “reduced”

7

number of indicators from a larger set. In the section by Z. Hellwig a clusterhgof development indicators will be effected by the use of taxonomic graphs on the scale of distances calculated according to correlation coefficients. In the section by B. Ivanovic a method of establishing a list of development indicators is proposed, based on the I-distance.

The definition of Stuart Rice and the quotation from Sheldon and Moore appear to circumscribe directly and precisely the attributes of a socialindicator. However, with the wide interest expressed in social indicators growing into what has been called “ the Social Indicators Movement”, there has been a great laxity in the use of the term, to the extent that most of the social statistics which are usually collected for administrative purposes are being rebaptized “Social Indicators”. There is a good deal of uncertainty as to where to draw the line. At a consulta- tion on social indicators with the secretariats of international organizations, convened in Geneva by the Con- ference of European Statisticians, a body of the United Nations Economic Commission for Europe, an eminent participant with the view of tidying up terminology proposed a classification of statistical series into six types.

GROUP A: (i) Raw statistical series. (ii) “Key series”, e.g. a common sense attempt to pick

out series which one regards as most interesting or important for presentation. The Social Trends, published in the U.K., could be considered as a care- ful, though not rigorous, selection of series.

(iii) The system of social and demographic statistics; which is still basically the raw material but put together in a comprehensive and systematic way.

GROUP B: (iv) Composite indices derived from somehow combin-

ing individual series. This is an application of the classical index number approach. Synthetic “representative” series derived by multi- variate techniques such as discriminant analysis etc. Series which fit explicitly into social models.

(v)

(vi)

The United Nations have been attempting to develop an “Integrated System of Demographic and Social Statis- tics”. In the many documents which have appeared as working papers for meetings held for that project no criteria have been proposed by which “social indicators” and “socio-demographic statistics” are differentiated. Thus the different tables in one of the documents2 show in the first and fifth columns statistics which appear as both “items of data” and “social indicators” without apparent justification. It is true that most of the series listed in these tables could rank within “Social intelligence” or “Social accounting”. Short of a social model into which these “indicators will fit explicitly”, one may still use certain criteria, based on specialist knowledge, to select the social indicators in view of some specific purpose.

A social statistic may rank as a social indicator only in a specific social field for a specific purpose. Instead of drawing up a long comprehensive list of “social indicators” for universal usage it might be appropriate to subject each “candidate” indicator to close scrutiny within the context of specific areas of social concern and through rigorous interdisciplinary treatment. The UN system of social and demographic statistics could be seen as the ordering and classification of the raw material in a comprehensive and systematic way as suggested in Group A (iii) on this page. Some of the data upon which such a system will focus will initiate the construction of certain indicators or may, without further manipulation, serve directly as indicators. The inductive approach underlying much of the current data collection and selection should be superseded by a deductive approach; the identification of demands, of aspirations, of social concerns in general, should influence and consequently precede the collecting and marshalling of relevant data.

The Unesco Social Science work on indicators has until recently focused on human resources indicators. These may not all be termed social indicators and certainly most of the indicators in the system would not if Rice’s definition were accepted in toto. Many of the indicators regarded as human resources indicators in Unesco work would, however, fit the definition of social indicators as implied in some of the tables which appear in the UN document. This definition would be considered by many as being too loose, showing a great laxity in the use of the term. The list of human resources indicators which has been drawn up as the project developed does not refer to the indicators as social indicators. The different fields which have been delineated contain indicators which are described as educational indicators, health indicators, employment indicators and the like. They would be regarded as “definitional” indicators in the first place, i.e. they refer to such phenomena as are wholly or partly identical with the indicatum. Other indicators may be termed “inferential”, i.e. they reflect the directly unobservable or unobserved indicatum indirectly. The educational indicators are termed educational because they relate to education and are therefore definitional. But they would also be inferential indicators of human resources, showing the “flow” aspect of human resources development. As such they would better qualify as indicators, i.e. in the second sense, since many believe

1. The participant later added that “he would not want his proposals to be regarded as too weighty from the point of view of substance” and that there was “quite a lot wrong with this classification”. H e went on to say that “the essence of the social indicator, by definition, must be that it is indicative of something else”, and that this attracted him to confining the term to category (vi). H e thought that in practice that was asking too much, “since people are almost bound to want to use the term for some of the other categories. But what is important is that w e think in terms of a hierarchy of types of series”.

2. An integrated system of demographic and social statistics, etc. (E/CN3/432) - United Nations - Geneva.

8

that the essence of an indicator is that it is indicative of something else (see footnote on page 8 )

The series listed as human resources indicators in the Unesco project would belong to type (ii) in Group A (see page 8).

2. COMPOSITE INDICES

However they may be qualified many series in the Unesco list have been sometimes aggregated into composite indices to provide development measures constructed as better substitutes for GDP, to establish international comparison and to show movement over time at both within- country and international levels. Many sophisticated methods (including Wroclaw Taxonomy, the I-distance method) have been used to derive such indices. In other organizations, a few years back, the eagerness to have a “social” development measure to match the economic development measure, the GNP, led to the construction of various indices. The most interesting have been the different “level of living” indices constructed under the aegis of the United Nations and for which considerable work at both conceptual and methodological levels has been done by the United Nations Research Institute for Social Development, at Geneva. This quest for some sort of “Gross Happiness Product” has been inspired by the conviction that GNP is not everything and does not in itself reflect the welfare of people. This inspiration has unfortunately become an obsession and GNP, despite its many shortcomings, is always present, so much so that the system of social accounts being envisaged tries to be too close an imitation of the economic accounts. Adding up the components of production, with a convenient common measure like their price, is not attended by the many conceptual problems underlying the aggregation of components of welfare, if not components of the ‘good” life. A short article in the 25 December 197 1 issue of the “Economist” entitled “Where the grass is greener”, com- menting on the construction of a “Social thermometer which is the equivalent of the gross national product”, said: “Certainly, social statistics are improving and increasing; statisticians are making ever more ambitious attempts to compare like with unlike and to marshal1 priorities in the social services into neat statistical balances. But the task of selecting and weighting for the calculation of an umbrella index of the quality of life makes the calculation of GNP seem like schoolboy arithmetic...”.

Professor Claus Moser, Director, Central Statistical Office, London, wrote recently: ... “in some writings, it is held that social indicators should necessarily be combinations of series, i.e. index number in the conventional sense, This is an undesirable restriction of the term. Apart from anything else, the technical objections to such indices - most basically the problem of weighting - are substantial. All the inadequacies of concepts, measurement, interpretation are compounded, and in our approach, work on index number combinations is envisaged for later stages...”.

Discussing the different definitions of a social indicator,

Thus it would appear that the construction of the composite indices, however big the difficulties mentioned above and despite the lack of success which has met the many endeavours, both for developing such indices or for finding a use for them, is stiU envisaged by quite a few organizations or institutes in their programme of work on indicators. It is true that on the whole the construction of such indices has lost much attraction, because of the many inadequacies mentioned by Professor Moser. But somehow we know that work for their construction will go on for quite some time. The aggregation that work involves hinges heavily on selection and weighting.

The division into the various fields in the Unesco list of human resources indicators was not only a matter of classification. There was the intention of aggregating the indices for indicators within each field into a composite index for that field. The studies on the problem of weighting had in view, among other aims, the solution to such a problem of aggregation.

The aggregation of several component indicators into one composite index has as many opposers as supporters. The former believe that a composite index tells us much less than the individual indices shown separately and is therefore less useful. For those who believe that development forms a distinctive pattern with progress along its various components behaving rather uniformly and con- tinuously, a profile made up of all the elements separately is more illuminating. It provides a picture of dispersion measures in the human resources components of development, thereby pointing to imbalance and bottlenecks in the system. On the other hand, if the trajectories of development are not parallel and substitutability exists to some extent between indicators of the same field, aggregating several component indicators within that field would provide a unitary measure which could be of great value in international comparison.

3. THE CONSTRUCTION OF SOCIAL MODELS

The hierarchy of series listed on page 8, which proceeds in ascending order of relevance to the social indicator concept, shows the last type of series as comprising those which “fit explicitly into social models”. Accord- ing to a strong body of opinion, these series would better qualify than others for the term social indicator. Some of the protagonists in the “quarrel” over the definition of a social indicator would reserve the term to a series which satisfies expressly the following conditions, i.e. that, to rank as an indicator, it “must have a place within some sociological or social policy model”. Economic indicators fit into economic models, constructed on the basis of economic theories. A social indicator should likewise find an interpretation only within the context of a theory. The analogy in this respect to the economic indicator has to be pursued to a logical conclusion, which appears to many social scientists as the ultimate phase of work in the field of social indicators before they can become useful tools of planning, namely their

9

integration as parameters in social models or models of social change , built upon social theory and policy. But prior to the launching of that phase and implicit in the preliminary work remaining to be done is the analysis by social scientists of all the currently collected components of social accmting, the indicators, so as to bring out their real & cmtextual meaning. There is the constant shift in interest, and some indicators either age quickly or else are discovered irrelevant or useless. Other indicators step to the front of the stage, propelled by concerns of real urgency, or simply by fashion. For over a decade now “quality of life” has. become a major preoccupation without there emerging any indicators which purport to measure it however crudely - indicators which, it is expected, would dethrone GNP or rate of economic growth in the public eye. These economic indicators are still faring well, though ancillary economic indicators like the rate of inflation, loss of pur- chasing power, unemployment or inequality of income distribution, which should be more relevant, seldom enjoy with development economists the status they should have in planning. There has been the failure to develop indicators which might be conceived as describing the “quality of life”. Indicators will eventually be constructed for the description of the environment in its physical, social and cultural aspects and in the interreactions thereof. There is still a long way to go for the construction or identification of the indicators relating to today’s concerns. That work lags far behind the emergence of these concerns which have been asserting themselves for more than two decades. One no longer can invoke fashion to explain the appearance of these “new” preoccupations, or conjecture that with the passage of time some of them will fade away. The environment and the “quality of life”, for which interest is relatively new, will assume more importance within the economy of penury which appears more likely from now on to govern the existence of man on space-ship Earth. For those who are responsible for the badly needed dashboards, the struggle will have to be carried on at two levels: at the higher one, constructing the indicators to serve as dashboard or danger signals relative to these concerns, and at a lower one scrutinize, test, analyse the mass of data which are regularly collected at great cost in a variety of fields, and weed out those for which no utilization has ever been recorded and the collection ever been questioned. An area which has received a great deal of attention

has been the study of interrelationships between socio- economic factors, reflected by many of the favourite indicators, and economic growth. The selection of indicators is relatively much easier to achieve in this case. Many indicators which relate to the “welfare” of a

nation, to the “quality of life” are not always correlated positively with economic growth. As a result, socio- economic models, despite the appellation, will rarely seek to include these, and will most times lean heavily towards the purely economic side. Such models, despite their complexity are still within reach of some of the more enterprising model builders; the relatively easy conceptualization of the ingredient indicators, their ready measurability and availability, coupled with the mammoth capacity of some third generation computers to extricate relationships, have led to a proliferation of models, on which often by hindsight theories are built. One can conjecture that with the “softer”, societal indicators which would go into social models dealing with real social progress, improvement of the “quality of life” and the like, the difficulties facing the social model builder would make the efforts of the econometrician pale into insignificance. All the interacting, conflicting, reinforcing, inhibiting variables changing their directions as time unfolds would all have to be taken in and would indeed make macro-economic models look like schoolboy arithmetic, to use the expression of the article of the Economist. The difficulties are such that the number of those is increasing who consider the difficulties as insuperable and societal models as a wild dream!

tion of such models, if one followed the hierarchy suggested in the listing of types of series on page 8 is the selection of series, within the inductive approach, among those that are currently collected and are sufficiently reliable. Such selections have been based on common sense, intuition or on interdisciplinary treatment and will vary with the specific purpose the series are intended to serve. When it comes to measuring development levels, both at within-country and international levels, there is the necessity of reducing the set of development indicators usually available. Parts I1 and I11 propose methods for the selection of “key” or “core” indicators of socio- economic development. The exercises which have been worked out in the presentation of the methods have more illustrative than substantive value. They had to rely on those series which are usually available for a sufficient number of countries. One should not read too much into the selection of the original series, which was governed more by the availability of data than by the conceptual validity of the series themselves. The methods are evidently applicable to series which may better qualify themselves as social indicators. As such they would prove a useful tool for the study and selection of indicators which as suggested in Sheldon and Moore “stand for a set of correlated changes”.

An important task which remains before the elabora-

10

Part I I

A Method for the Selection of a “Compact“ Set of Variables

by Zdzislaw Hellwig Graduate School of Economics

Wroclaw, Poland

1. EMPIRICAL MODELS OF SOCIO-ECONOMIC DEVELOPMENT

It is a known fact that the problem of economic growth belongs to the most complex, difficult, and thereby controversial part of theoretical and applied economics. This problem has attracted a lot of scientific interest and attention and has been tackled from many view-points by many outstanding economists and mathematicians. The per- spective has ranged from a historical or descriptive purpose at one end of the spectrum to abstract and sophisticated relationships of a causal nature, at the other. An analysis of the practical function of purely descrip-

tive or even more advanced mathematical models usually based upon trend extrapolation will show that while they may be useful for pedagogical reasons they do not help to solve some basic problems of economic development which planners have to face in their every-day work. This has led some investigators to look for alternative approaches to the important problems of economic development and growth. One of these approaches has been presented in the series of studies published as working documents under the Unesco project on human resources indicators.

roughly sketched as follows: Implicit in the approach is a concept which will be

6)

(ii)

(iii)

The object of any scientific study has to be precisely defined. In cases where political, administrative, geographical or social structures are being in- vestigated the most concise and convenient description is by the way of vectors. The object of study, a country for example, or any of its administrative parts, is usually a very complex geo-political and socio-economic object. Precision and concision which are both very important here can find expression in the vector concept. If any socio-economic unit is to be presented and described in vector terms then the most essential part of the exercise will obviously be the appropriate selection of the vector components. One has to distinguish between the optional list of

all variables (the number of which is limited only by data availability) and the many different subsets of variables selected the “best” way so as to throw light on the many facets of the object under study. Having selected optimal sets of variables one can then define countries in vector terms. This allows the division of the whole universe of countries into a number of sub-groups which are more homogeneous than the universe itself. This clustering process is of real importance because it reveals the main differences and proximities between particular countries. Statistical analysis can now allow a deeper investigation of the internal nature of the phenomena concerned (for instance by applying variance analysis and related techniques). When countries have been represented by vectors they can be more easily compared. This may be effected by defining the “distance” of one country from another. This distance can simultaneously play the r61e of a yardstick of socio-economic development (by finding how far a country lags behind the “ideal” country) and may be looked upon as a composite index of the overall achieve- ment of that country in a specific sphere of its activity. Recent studies have shown that the use of certain metrics allows the possibility to go up and construct more general composite indices, solving thereby a very important problem of measure aggregation, or to go down, i.e. split a composite structure into some more simple and elemen- tary parts (see for example the study by S.Fanchette, proposing a method for the aggregation and dis- aggregation of socio-economic measures based on the Euclidean metric).l In this respect, the possibility of both analysis and synthesis has been demonstrated.

(iv)

(v)

1. Study XX: Distance-based analysis, numerical taxonomy and classification of countries according to selected areas of socio- economic development, by Serge Fanchette, Unesco - mimeo. (Ref. SHC/WS/237, May 1972).

11

Nowadays one cannot speak of the development of any country by isolating it from its geographical environment or from the family of countries to which it is bound by geo-political ties. One may go further still and say that its development cannot be studied in ignoring the rest of the world. Since we cannot construct a model which will describe and explain on a world scale all the movements on the map of development we may accept the following fundamental assumption: it can be judged and measured only by making comparisons. There are two main dimensions of comparison: time and space, the latter being understood in a very broad sense. By comparing countries we may discover the pattern of a “well-balanced strategy” of devel3p- ment and also point out this country within a relatively homogeneous group of countries whit ’7

could play for the others the r61e of an empirickLLy verified model of development. One could study with the utmost attention the conscious or uncon- scious development patterns adopted by the leading country. The implication is that instead of trying to construct some more or less primitive (or sophisticated as one may in fact judge) models of development, it would be easier, cheaper, safer and better to have a real, living model. The problem is only how to detect this model in the number of families of homogeneous countries.

Having outlined, and commented upon, this concept, let us consider the methodological aspects and working procedure in building such empirical models. In this procedure the selection of variables will play a very crucial r6le.

In some modern fields of experimentation, e.g. in biology, medicine, agriculture etc., it is beyond our reach to develop a very strong and reliable basis for describing and explaining interrelationships between all the phenomena involved. There are two main reasons for this : (a)

@)

The whole set of phenomena is, at least in theory, unlimited; There is the strong influence of some random factors which distorts a “pure” functional relationship between identifiable and measurable variables which are known to be involved.

To meet this methodological challenge in some applied sciences (sociology and economics offer good illustrations) a great deal of scientific effort has been recently undertaken, aimed at the elaboration of a statistical procedure which leads to the construction of more or less sophisticated models which could simulate with some degree of accuracy the “behavioural pattern” and performance of the object under investigation.

Such a procedure may consist of the following stages : 1. An introductory penetration of the field of interest. 2. Elaboration of definitions of some basic phenomena,

facts, variables and categories.

3.

4. 5. 6. 7. 8. 9.

Development of some intuitive approach toward the problem, formulation of working hypotheses which could provide us with some plausible explanation of the internal structure of the whole system of inter- related phenomena. Designing of statistical experiments. Data collection. Data testing. Introductory statistical data analysis. Setting up correlation tables. Clustering statistical units (which are formally understood as vectors in an N-dimensional mace).

I ,

10. Clustering statistical variables which are also inter- preted as vectors in an N-dimensional space.

11. Selection of the “most representative” variables. 12. Aggregation and construction of some composite

13. Final statistical analysis and building empirical

14. Estimation of parameters. 15. Final tests of coherence between the theoretical

description of the phenomena involved and their true description.

indices.

models.

In the research project on human resources indicators being carried out by the Unesco Department of Social Sciences, steps 1-9 and also 12 have already been dealt with to a greater or lesser extent. In this paper we will concentrate on stages 10 and 1 1.

2. SOME SELECTION CRITERIA

There are many methods available which could be applied for the selection of such a set of variables. One can divide the whole range of these methods into two groups according to the character of the criteria chosen for effecting this selection. If we decide on intuition or on commonly- held views that one of the variables belonging to the complete set of candidate variables should serve as such a criterion then we refer to an “endogenous” criterion. This is the case for instance when we choose per capita GNP as the most valuable indicator of economic performance and use this variable as a yardstick for measuring the relevance of components of a system, of which it is a part, which cuts across economic and social fields, a system of which economic performance measured in the value of goods and services produced is one of the characteristics.

Sometimes having decided on the components of a system reflected by the number of variables chosen, one may turn to an “external” variable to serve as the yardstick to assess the relevance of each of the candidate variables we have defined as likely to be part of the set we should select for describing the system. In this case we speak of an “exogenous criterion. For example a composite index of socioeconomic development identified with the distance from the “ideal country” of Wroclaw Taxonomy 1 may serve as such a criterion.

1. The “ideal country” concept is expounded in Study 111, by Z. Hellwig.

12

This involves a certain amount of circularity, since the taxonomic distance is made up of distances on dimensions represented by all the candidate variables. A better example may be chosen from principal components analysis where the criterion of selecting the most representative variables is the degree of explained total variance.

If we opt for an endogenous criterion we may apply a method developed in some earlier studies carried out under the Uneco project and which relates to the concept of “maximum information capacity” of a predict0r.l Alternatively one may as well ayply a better-known method: that of examining the statistical significance of regression function parameters in a multidimensional case. For the application of this method which falls within a closely-related family of multiple regression analytical methods, the use of the very handy and effective stepwise regression and checking procedure is strongly advocated. In any reasonable well-equipped computational centre there are ready-made programmes available for these operations. It should however be remembered that when choosing this method the checking of some assuptions should be carried out beforehand. These assumptions are implied in the method with re- gard to the multidimensional random variable distribution. This checking is usually a rather dull exercise.

3. CHARACTER (OR TYPE) OF VARIABLES

There are two other points which should be emphasized when analysing the different methods for selecting from the total set of candidate variables the best combination of these. The first point is mainly theoretical and relates to the nature of the variables. All social indicators can be looked upon as continuous variables. This statement has far-reaching methodological consequences. It means among other things that we are justified to apply such basic statistical parameters as the mean value, the standard deviation or the coefficient of correlation. It also means that if we do not involve discrete or categorial variables, then for clustering countries and variables we can use either the Wroclaw taxonomic method or the principal components method to mention but two of them. The use of the majority of socio-economic variables selected for analysing the statistical aspects of socio-economic development rests upon the assumption of their continuous character. However, the range of variables normally applicable to the description of countries may include those we might describe as discrete or categorial. Some of these variables may be, for instance, as following: the number of ethnic groups living in a country with a plural society, or the number of languages or dialects spoken in that country. We have here examples of a discrete variable. The variables may also be categorial ones, e.g. when they can assume the form of a question like “Does the country belong to the UN? ”, which is illustrative of a categorial variable capable of having only the two dichotomized values “yes” or “no”. GNP, life expectancy. birth and death rates which are among the favourite variables

selected for describing socio-economic development are the continuous variables most closely associated with the categorial variable whether the country falls within the denomination L.D.C. or M.D.C. It is very difficult to deal adequately with a mixed bag of continuous, discrete and categorial variables simultaneously. Most times the distinction between those types is clear-cut. Sometimes the line of division is blurred, just as a continuous stream may have successive cascading jumps. Another example is the continuous flow of cars in a city and the interruptions of that flow by traffic lights, converting it from a continuous to a quantum form. Luckily the problem of classification is not intractable enough to suggest the analogy with the contemporaiieous wave theory and corpuscular theory of light which testified differently to its continuous or tQ its discrete nature. Neither will it be necessary to carry the analogy further still with the electron, which partakes of both natures.

to convert by appropriate scaling continuous variables into discrete variables, or vice-versa.

In tilt: first case we would have to stop speaking of the variables and of their numerical values. We would have to translate them into adequate probabilities. It means, for instance, that the expected value E (X) would have to be replaced by np, the standard deviation S (X) by (np (1 -p)) the correlation coefficient r (X,Y) by the coefficient of stochastical dependence :

When our list contains all the types then one has either

1-xyf (Pij-Pi x P.) J d2 =

1 1- . min (s,t)

?1 1’ J (where p-. = probabilities corresponding to (Y, 5); p- p. - marginal probability distributions; s, t - number of rows and columns respectively in the probability contingency table). It means also that instead of applying a conceptu- ally and numerically easy taxonomic method or the less easy but well-known principal components method, we would have, for clustering and selecting the most appropriate variables, to devise some special methods to deal with discrete variables and categories. One of these methods is the “Analyse des correspondances” which can be regarded as an adaptation of the classical “principal components” method to cover a discrete case.

crete ones we operated in the opposite direction, we could expect at least two advantages: (a) Since the number of discrete variables and categories

in the list of socio-economic indicators should be by far smaller than the number of continuous variables, the amount of computational effort to convert discrete and categorial variables into the continuous type

If instead of converting continuous variables into dis-

1. Studies VI and XVI: O n the optimal choice of predictors; approximative methods of selection of an optimal set of predictors, by Z. Hellwig. (Studies and working papers prepared within the Unesco Social Science project on human resources indicators).

13

would be much smaller than if we proceeded the other way round. The continuous case is much simpler from the conceptual, and much better known, from the theoretical, point of view. It means that in this case we are not only better equipped with powerful tools of statistical analysis but we have also many ready-made programmes if we choose to use a computer. There are different methods of converting discrete and

categorial variables into continuous ones. For discrete variables one of the usual techniques is linear interpolation. Recently a new technique has been successfully applied, the so-called linear randomization procedure. To convert categorial variables into continuous ones the best-known and widely recommended method is the so-called “dummy variables” procedure.

4. THE PROBLEM OF WEIGHTS

The modern technique of describing some objects of scientific investigation by means of vectors in an N-dimensional space immediately brings up the question of the relative importance of particular vector components. Experience or intuition, even in the absence of knowledge of a particular field, would suggest that the relative importance of particular variables, whether descriptive or explanatory, cannot be uniform. Data in raw form would at the very start cloud any quick evidence of that relative importance. The common practice before examining by the usual tools of statistical analysis the relative importance of these variables is to standardize the latter. The standardizing procedures are conventional. These do not constitute a difficult task. They however cover a wide range of practices. The common standardizing practice of replacing xi by (xi- %) / s, where 5 is the standard deviation, introduces also the element of weighting. While eliminating the problem of units, it also favours the variable with a smaller standard deviation. This first bias left aside, there comes the real problem of ascribing weights to the variables. This cannot be satisfactorily solved unless there is agreement upon the most appropriate criterion for evaluating the relative importance of particular variables. This brings us to the examination of the different criteria one may consider for differentiating between the variables and for situating them on the various steps of a hierarchical ladder. As however we need the same criteria to select the most meaningful variables and point out the most important among the selected ones, we may conclude that the problem of selecting the variables and that of assessing their relative importance by means of a given system of weights are mutually equivalent and show the two different sides of the same coin i.e. that before one starts speaking about weighting variables one has to determine the most adequate criterion for comparing and selecting these variables.

There is also another important fact about weighting which is worth remembering. Weights are non-negative numbers: pl, p2 ... pn such that >: p. = 1. If _n is a J j = 1

large number it follows that unity is partitioned into many sub-intervals the mean length of which should tend towards zero. Hence if we agree to accept a given number po with 0 <po<l as a threshold value and retain all those variables 5 corresponding to weights p. arranged in a decreasing order which fulfil the condition:

J

E p j 2 1 - p ~ ; no\< n and if we take into account on the one hand that po is an arbitrary value which can vary within a pretty wide interval and on the other hand that we can never measure the relative importance of variables with a high degree of accuracy, then we should adopt a practical rule that all retained variables are of the same high relative importance and should be given the same weights, say p, whereas all rejected variables being of minor importance have their weights, say q , although equal as well but smaller than weights p ascribed to the retained variables (see fig. 1). Hence the problem of selecting the most appropriate variables boils down to the equivalent problem of weighting which in turn consists in finding such a number no = f (p,) that no p) 1 - p, , where of course

no p + (n-no) q = 1

1 1 1 Pj since p = n , pj and q=-

n-no j>no j Sn0 We have attempted to show that the problem of measuring the relative importance of variables (or components of vectors by means of which we define the object of our investigation) is equivalent to the weighting problem and this can always be expressed in terms of reducing the initial tentative list of variables to a much smaller size.

This reduced set of variables should contain only the most important or the most representative ones among the original set. The problem of finding this reduced or “compact” set of variables is at the very heart of cross- sectional analysis and constitutes a primordial element in the whole theory of international comparison in the socio- economic field.

1 2 3...n0 ......... j .......... .. n-1 n

Whether the objective be theory-building around causal hypotheses or, more soberly, the testing of empirical models, the selection of a small number of relevant variables from the forest of those for which data are collected has paramount importance. Both thiorie and empirie are required; the latter is usually within better reach of analysts. The reason is that work usually starts with data that are available. Other variables might be

14

more relevant which are unfortunately not collected. The testing of theoretical or hypothetical constructs would require that data be collected on variables which are absent from international compendia, national reports or administrative files. The problem of selection on the contrary implies a reduction of the number of variables for which data are collected internationally aiid are therefore available. This type of analysis therefore suits empirical investigations better. Put differently empirical studies would come earlier in the day than theory building and testing. The importance of selecting a set of core variables to describe or “predict” some phenomenon, like for example, socio-economic development is for the reasons given above very important. No wonder therefore that so much attention has already been given to the problem and so much effort has gone into the search for satisfactory solutions. Earlier studies carried out within the Unesco project ,’ research studies carried out by various institutes and agencies2have dealt with the methodology of indicator selection and weighting. They have shown most of them varying degrees of sophistication. If+ it is suggested above,selection and weighting are looked at as being in- timately connected, we could propose a rather simplified approach to the whole problem.

This is of course a conclusion the importance of which should not be underestimated. After all the most sophisticated contemporary statistical methods such as Fisher’s test of overall significance of regression parameters, stepwise regression, principal components method, factor analysis, the method of maximum capacity of information, “la mkthode d’analyse des correspondances” are all dealing with the same bunch of questions: how to determine a compact set of variables, how to split the set of variables into some more homogeneous groups and how to find the best representatives for particular groups.

In the next paragraph a very simple and convenient method will be presented which could supply some answers to these questions. The method is graph-based and does not therefore require going through complex computational processes which are unavoidable when utilizing other techniques e.g. the principal components method.

5. VARIABLES CLUSTERING BY MEANS OF THE TAXONOMIC METHOD

To make the method more easily understandable we shall in the course of its presentation illustrate it with numerical examples. (i) The algorithm starts with a correlation matrix

R (n x n) where n stands for the number of variables (see Table 13). In the second step the following transformation is effected d.. = 1- r by means of which the matrix R is mapped into a new matrix D (see Table 2). This matrix is also of (n x n ) type, symetrical and with elements d-

1J

(ii) i~ I ij 1

where i, j = 1,2, ... and d.. > 0) Now the smallest element has to be found in each row of the matrix D. If we denote these elements by di then di = m i n d.. 4 After having found di in the ith row, one finds the number of the column which forms with the given row a cross-cell containing the value di. In this way one gets n pairs of indices:

Let us now define a few terms. The terminology will be very simple. Each of our pairs of indices will be called a link whereas i and ji in (i, ji) are nodes. The first node is the begirinirzg of the link which ends with the second node. The links may therefore be called oriented links. When we draw a graph all nodes will be represented by points and all links by arrows. The number of arrows pointing at a given node will be called the power of that node The node with the highest power is next found. If there are more than one with the highest power the choice will fall on the one to which corresponds the smallest value of - n

d.=l C d.. i n 1J

Now all nodes which are beginnings of links pointing towards the node of the highest power are to be linked with this node. All nodes attached to the node of the highest power can be now looked upon as the ends of some other nodes which are in to be linked with all these ends. By continuing this process of joining together all the nodes related to the node of the highest power, one obtains a certain family of nodes which can be referred to as the “first concentration” of nodes, or as the first cluster of points. Next, one finds all the clusters which correspond to the nodes with powers below the highest one, in a decreasing order of magnitude. The node with the highest power in a cluster is termed the centre of the cluster. If in a cluster nodes of the highest order number more than one, we select the one to which corresponds the highest value of d.. The centre of a cluster can be looked upon as the representative of variables belonging to this cluster. The set of representatives of all clusters will be referred to as a set of core variables. The range of variation of the parameters di should now be divided into three parts:

1J

j

(l,Jl), (2,J2), ...... (njjn)

j - 1

J

1. 2.

3.

5

Studies VI, VII, IX (Unesco mimeographed working documents). A valuable example of selecting key indicators from a wide range of thesc is given in a research report of the United Nations Research Institute for Social Development: Report No. 70.10 Geneva 1970: Contents and measurement of socio-economic development: An empirical enquiry. N.V. Sovani, A.M. Subramanian: A n Index of Socio-Economic Development of Nations, UNRISD, 1969.

6. CONCLUSION

I’ where - d - - 1 ” di ; Sd = [ - 5 (di-d)* n i = l i = 1

All nodes which are beginnings of some links the length d. of which belongs to the first interval can be removed from the graph and the corresponding variables may be called redundant. Nodes which are beginnings of some other links the length di of which falls in the third interval can also be eliminated and the respective variables may be referred to as irrelevant. All remaining variables would form the “compact” set. All the steps which lead to the obtention of particular clusters, the core variables as well as the “compact” set of variables are summarized and illustrated in the diagrams which follow.

The procedure which has been described is simple and liable to easy interpretation. It is equally applicable to variables which are continuous, discrete or categorial. If however one is put off by, or is suspicious of, its great simplicity, there are more sophisticated methods available, like for example, the principal components method, which will provide good solutions to the problem as it has been posed above, but at much greater cost.

3

The selection method which has been proposed, while being a variation on the intercorrelation theme, has the virtue of simplicity and provides a clear picture, through the taxonomic graph, of the relationships and distances between the variables. Furthermore a novel idea has found expression in the elaboration of the method: it refers to the problems of selection and weighting as close correlates requiring a unique solution.

The clustering technique which has been described here has identified clusters of variables and cluster centres. Each centre could be made representative of the whole group. Expanding the number of the more representative variables would mean a movement in concentric circles from the centre towards the periphery so as to include the number of variables which would be retained for any specific purpose. The two exercises which have been carried out have led to taxonomic graphs which are reproduced on the following pages. It may be that in each case the original number of variables has been somewhat too limited to derive many useful conclusions from the clustering. Most of the groupings appear to be consistent with what is already known or suspected of the relative movement of variables as development proceeds.

16

o 5 s 2i w n 0 W

z m N

N

N

3

N

0

N

2 00 3

2 W 3

In 3

b,

m 3

2 3

3

2

m

00

l- Q

m

Tt

m

N

3

m

';? d 9 0

\9 0

Y d T m ? W

b 3

S N T

3

3

17

a

n

c

3

I1 -- .I :3 a

0

II 5- W

43 0

18

w

I \

k

n &I e, +J

s - U

0

(1

20

Part 1 1 1

A Method of Establishing a List of Development I ndicators

by Branislav lvanovic Statistics Division

United Nations Conference on Trade and Development

I. INTRODUCTION

The choice of social and economic development indicators is certainly one of the most important problems in the project for development evaluation and forecasting during the Second Development Decade: the more complete the information about national development levels contained in the indicators selected, the better the results.

There are a certain number of indicators which we always think of as providing the most reliable information about national social and economic development and which are the most frequently used in evaluating a country’s development level, for example, the per capita GDP, the number of illiterates, the percentage of the population not employed in agriculture or the share of the GDP accounted for by industrial products.

However, the lists of development indicators used in various national or international research institutions are not always identical, and there is constant controversy about the value and importance of one or other of these well-established indicators. In actual fact, it may be wondered whether there are any objective reasons why they should all be considered as social and economic development indicators and whether others, so far overlooked or neglected, might not be more suitable than certain of the ones traditionally used. We have thus endeavoured to find new development

indicators, and in so doing we have defined and examined the indicator of “concentration of births in relation to the age of the mother”. I This indicator provides information about the family situation in a given country and is obviously also an important development indicator. First of all, a high degree of concentration of births in relation to the mother’s age in a particular country signifies that the number of children per family is limited, which en- sures a higher standard of living in that country. Next, if the area of concentration is situated between the mother’s ages of twenty and thirty, this means that the parents are of an age which makes it possible for the children to have a good family education, prolonged economic protection, favourable physical development

and a more complete school education. Lastly, where there is a high degree of concentration, one rarely en- counters unfavourable cases such as those of mothers who are too young or too old, children born of a second mar- riage, etc. In short, the more concentrated the births are in relation to the age of the mother, the more ordered is the family life and the more effective the family’s activity from the social and economic points of view.

Quantitative analysis shows that this indicator gives more information about a country’s level of development than certain traditional indicators such as per capita exports of manufactured goods, average expectation of life, number of literates, the contribution of the manufacturing industries to the GDP, the number of school enrolments, etc.

list of development indicators, those which have previously been neglected or overlooked should be added to those appearing in all the lists used hitherto.

This would make a very long list, which we shall call the maximum list of social and economic development indicators. The statistics established for these indicators would give a very complete picture of all aspects of development in the countries concerned.

However, if we examine all the statistical publications of the United Nations Statistical Office, regional economic commissions, and various Specialized Agencies of the United Nations, as well as national publications and other official or semi-official documents, we see that the choice of development indicators is greatly limited by the absence of statistical data about many developing countries.

The conclusion could be that to arrive at an exhaustive

1. In a country j, the measurement of this indicator is given by

where n represents the number of age groups, x.. the number of births in which the mother’s age is in the i-th group, and X. the total number of births in the country j during the year in question.

U

J

21

The absence or poor quality of statistical data hinders the introduction of a large number of development indicators such as those providing information about productivity in industry or agriculture, standard of living, infrastructure, industrial and educational cadres etc.

The introduction of even a single indicator into a country’s statistical system frequently involves setting up a new department with suitably qualified staff and adequate equipment for the compilation, processing and publication of the data. These data should be of a high quality and comparable with data from other countries.

ment indicators consequently requires time, new institutions and great financial efforts. It goes without saying that this problem is even more serious in developing countries.

If we are to keep track of the development of countries during the Second Decade with anything like sufficient accuracy to reach our objectives, some new items ought unquestionably to be included in the list of development indicators. The question is, however, which should be chosen from the many already suggested.

The first problem to be solved in carrying out the project for the evaluation and forecasting of development during the Second Decade will thus be to establish, on a scientific basis, an optimum list of a limited number of development indicators.

The introduction of new items into our list of develop-

2. CRITERION FOR SELECTING SOCIAL AND ECONOMIC DEVELOPMENT INDICATORS

Various statistical indicators may be used to estimate and keep track of the social and economic development of a country. Each of these indicators provides information about one aspect of the level of development, and information about this aspect is not, generally speaking, identical with other information. The indicator “percentage of the population not employed in agriculture”, for example, gives us information about a country’s level of development which is not identical with the information given by the indicator “per capita energy consumption”. A choice must thus be made and a whole set of indicators has to be prepared in order to give a more complete idea of the level of development of the countries concerned.

First conclusion: Increasing the number of indicators also increases the total amount of information about the country’s level of development.

The various development indicators do not contain the same volume of information about the development level of a country, i.e. they are not all of equal importance in relation to development. The “per capita GDP” indicator thus provides more information about the social and economic development level of a country than, for example, the “average life expectancy” indicator. If the quantity of information contained in development indicators is measurable, it should be possible to classify them in order of importance.

Second conclusion: Sets of indicators of the same size (same number of indicators) do not, in general, contain

the same quantity of information about the country’s development level.

Generally speaking, development indicators are not independent of each other, so that information provided by one indicator is partially contained in the information already provided by others. Taking the whole set of indicators selected, therefore, it could be that certain quantities of information are repeated two or more times. It is obvious that these duplicated items would have to be subtracted from the total volume of information so as to leave only the basic information contained in this set of indicators.

(without duplication) given by a set of indicators is generally less than the sum of the quantities of information contained individually in each indicator of that set. Let us take the example of two completely inter-

dependent indicators. If we arrange the countries in question according to the values corresponding to either indicator, we shall obtain two identical lists. This means that both indicators contain the same information about the countries’ development level and that consequently only one of the indicators need be considered, the other being superfluous.

Fourth conclusion: Despite the fact that two given indicators may be very important from the point of view of the information they provide, separately, about a country 3 development level, the contribution of one of them is insignificant if there is a high degree of correlation between the two.

Fifth conclusion : Indicators producing information which is completely contained in the overall information provided by the indicators already taken into account should be discarded.

chosen indicators could contain a greater sum total of information than a large number of badly chosen indicators.

Consequently, to obtain as complete an idea as possible of the development level of the countries under observation, it is not enough to increase the number of indicators. To increase the total amount of information, the importance of each indicator, expressed in the form of a quantity of information, and duplications of information, expressed in the form of interdependence between indicators, must be taken into account.

The optimum selection of a limited number of development indicators should give a maximum total amount of information (without duplication), whilst the sum total of duplications should be reduced to a minimum.

The more information an indicator provides which is not already contained in the sum of the information provided by more important indicators, the greater that indicator’s contribution to the evaluation of the country’s development level will be. Nevertheless, even an indicator which makes quite a considerable contribution will not be very useful if its value is almost constant in all the countries observed. The indicator “percentage of illiterates over ten years of age in the population”, for example, is

Third conclusion: The total amount of information

It follows that a relatively small number of carefully

22

of no significance in the rich countries where it has become stabilized at between l and 3 per cent. On the other hand, in the poor countries this value varies between 5 and 99 per cent, so that this same indicator provides a high degree of discrimination in the evaluation of the development level of developing countries. The opposite would be true if we took “contribution of industrial production to the GDP” as the development indicator.

be discriminatory in all the countries under observation. W e may therefore complete the criterion for selecting a development indicator for a given set of countries as follows:

The greater UFI indicator’s discriminatory capacity in all the countries observed and the greater the quantity of information it provis“7s which is not contained in the sum of the informatioil giwii by more important indicators already taken into condcration, the greater its contribution to an evaluation of the countries ’ development level.

Sixth conclusion: A good development indicator should

3. OBJECTIVE EVALUATION OF THE DEGREE OF IMPORTANCE OF A DEVELOPMENT INDICATOR

The degree of importance or significance of a development indicator may be thought of as the quantity of information about a country’s development level it contains. This quantity of information cannot, of course, be expressed in absolute terms. What we are able to determine is more a relative quantity, dependent on which countries are being observed and which indicators are selected.

If a given indicator is completely dependent on the development level of a country, we shall have the same classification of countries if we list them according to their value in terms of this indicator, or according to their development level. In this case, we may say that all the information about the development level of countries is contained in this single indicator, since one only has to list these countries by their values according to this indicator to obtain at the same time the list by levels of development.

the given indicator is only part of the total information about the development level of the countries.

ance of a development indicator corresponds to the degree of dependence between that indicator and the development level.

If we take the I-distance D1 as the relative measure of a country’s level of development and if we express the degree of dependence between indicator Xi and the development level by the simple correlation coefficient ri, the value ri will express the degree of importance of

If dependence is partial, the information contained in

Consequently, we may say that the degree of import-

Xi. Listing the given indicators according to the order of

magnitude of their coefficients of normal correlation with the I-distance, we obtain the classification of development indicators in order of importance.

Development indicators could thus be classified ob- jectively according to the correlation between the various indicators actually used and the overall index containing the maximum amount of information. We cannot calculate the correlation coefficients directly, however, since before we can do so we need the series of I-distances, to calculate which we need the indicator classification which we are in the process of establishing. As we are thus in a vicious circle, we have to see

whether it is at least possible to resolve the problem of classifying development indicators by an approximate method.

It should be observed, first of all, that there are two possible extreme cases of our problem. In the first, all indicators are independent and the problem of the same quantities of information being duplicated no longer occurs. In the second case, all the indicators selected are completely dependent on one of them, i.e. the quantities of information of all the indicators are contained in the information of the dominant indicator. In both cases, it is possible to establish the series of I-distances without any difficulty and subsequently to order the indicators by their degree of importance.

way between these two extremes. One may say, nevertheless, that one is always either in a position in which one indicator contains most of the sum total of information about the countries’ development level, or in a position in which it is impossible to say that one indicator is obviously dominant in relation to the rest.

The first position is closer to the extreme case of the total dependence of all indicators on one dominant indicator than to the extreme case of the total independence of all indicators. This is why, as a first approximation, we shall take the dominant indicator as containing the sum total of all the information in all the indicators being studied. In other words, we shall assume for the time being that the development level corresponds to the value of the dominant indicator for the country in question.

If we calculate the correlation matrix and list the given indicators in accordance with the order of magnitude of their coefficients of correlation with the dominant indicator, we shall obtain a provisional list of development indicators by degree of importance. This we shall call the first-stage classification.

Using for the time being, the indicators arranged in this order, we are able to calculate the I-distance values for all the countries under consideration. As this I-distance based on all the indicators under consideration contains more information about the development level of the countries than the dominant indicator, we shall, in the second stage, determine the order of the indicators according to the magnitude of their coefficients of correlation with the Idistance of the first stage. This will be the second-stage classification.

In practice, however, one is usually in a position mid-

1.See Research Memorandum No. 41 of 5 November 1970 (UNCTAD).

23

If we calculate the series of I-distances according to the second-stage classification and if we calculate the coefficients of correlation between the indicators under consideration and the I-distance thus obtained, we arrive at the third-stage classification of indicators.

This iterative process is continued until such time as the classification of indicators is identical with that of the previous stage. The classification is then definitive since it remains invariable in all succeeding stages.

In the second case (without a dominant indicator) one is closer to the extreme case of the complete independence of all the indicators than to the extreme case of the complete dependence of all the indicators on the dominant one. We thus assume, as a first approximation, that all the indicators are independent and that there is consequently no duplication of information.

The I-distance is reduced in this case to FrBchet’s distance F, the calculation of which requires no order of indicators since they enter into the F formula on an equal footing.

If we calculate the values of the F distance for all countries under consideration and if we subsequently determine the Coefficients of correlation between the indicators observed and the F distance, this gives us the first-stage classification of development indicators by their degree of importance.

The same iterative method as in the first case is subsequently used to arrive at the definitive list of indicators.

It should be noted that the selection of social and economic development indicators is an instance of the first case, since the indicator “per capita GDP” is quite clearly dominant.

If the development indicators observed {Xl ,X2, ..., are arranged according to the definitive classification and if the I-distance D for each country is calculated according to this classification, the quantitative measurement of the importance of the indicator the formula: Ii = r(%; D).

classification, namely:

will be expressed by

Their order of magnitude corresponds to the definitive

r(X1 ; D) 2 r(X2; D) >, ... >/ 4%; D).

Taking the developing countries and restricting the development indicators to 12, the order of importance of these indicators in 1968 was as follows: Order

1 2 3

4

5

6

Indicator

Per capita GDP Per capita energy consumption GDP of the population employed

Number of doctors per 100,000

Number of newspapers printed per

Percentage of the population not

in agriculture

inhabitants

1,000 inhabitants

employed in agriculture

Degree of importance 0.97 0.95 0.81

0.81

0.80

0.79

7 Concentration of births in relation 0.77

8 Exports of manufactured goods per 0.76

9 Average expectation of life 0.7 1 10 Literacy rate 0.69 11 Share of manufacturing industries 0.66

12 Rate of school enrolments 0.64

to the age of the mother

head of population

in the GDP

4. QUANTITATIVE ASSESSMENT OF THE CONTRIBUTION OF AN INDICATOR TO THE EVALUATION OF A COUNTRY’S DEVELOPMENT LEVEL

4.1 Statistical parameters

Let X = { X1, ..., % } represent the set of n indicators being considered for possible use in evaluating the development level of all the countries under observation, represented by P = { P1, ..., Pm 1 *

Assuming that statistics are available for each indicator of X and for each country of P, if the value of the indicator 3 for country P. is expressed by the formula x.. J 4 ’ these statistics may be presented in the form of the matrix M = [x.. 1, i = 1, ..., n, j = 1, ..., m.

Calculation of the statistical parameters of the indicators X, requires the values x.. to be weighted. Designating

1J the matrix of the weighting coefficients by the formula N = [ n.. 1, the arithmetic mean will be defined by the

4

4 formula: 1 m

C nij j=1 - xi = - 2 nij xij , i = 1 , ...., n

j

and the standard deviations by:

Using the covariance wic of the indicators and Xc, it is possible to calculate normal correlation coefficients:

Wic . ric = -

ai ac i,c=l, ..., n,

partial correlation coefficients: ric - ris rcs

ric- = i- fori<c, i, c=l, ..., nands#i,c, and multiple correlation coefficients:

24

where R represents the determinant of the correlation matrix and RI the minor of R corresponding to the element r 1.

4.2 The discriminatory capacity of an indicator

As a measurement of the discriminatory capacity of the indicator X, in the countries P as a whole, we suggest the coefficient of discrimination which can be expressed algebraically in the following form:

or

It should be noted that this coefficient does not have the same significance as the coefficient of variation which measures the relative mean dispersion of Xi in P.

Thus in our example, which illustrates three distributions in which the discriminatory capacity obviously varies (with respective coefficients of discrimination of 3.22,3.00 and 2.67), the dispersion is practically constant (2.88,2.86 and 2.83).

A . A X1* WWW

.. x2 ~

.e

e. x3 .. I 1 I I I I I I I I I 1 1 1 1 I X * 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

4.3 Total information of a group of development indicators

To determine the total quantity of information contained in the set of k indicators selected XI, x2, ... xk, we must follow the criterion concerning the choice of development indicators formulated in section 2, according to which we must take into account each indicator's degree of importance, the duplications of information contained in the various indicators, and each indicator's discriminatory capacity.

Given that the indicators XI, X2, ... xk are already arranged in order of their importance, the overall contribution of this set of indicators to the evaluation of the development level of the countries under observation is expressed by the formula:

k i- 1 (4.3.1) C (1,2 ,... ,k) = 1 CDi (P) n

i =I j =1 (1-rji.12 ... j-1)

where CDi (P) represents the coefficient of discrimination of indicator Xi in all the countries P and rjie12 coefficient of partial correlation between indicators Xi and X,.

This formula is increasing and homogeneous in relation to the discriminatory capacities of the indicators selected. Each capacity is weighted so that the higher the indicator's rank and the smaller the partial correlations between it and the indicators preceding it, the greater the importance of its contribution in the sum total of information.

If all indicators are mutually independent, the total amount of information becomes the sum of the discriminatory capacities of all the indicators selected, namely:

j-l the ...

k

i=l (4.3.2) C (1,2 ,..., k)= C CDi (P) ,

and if a functional-linear dependence exists between them, the total amount of information is reduced to the discriminatory capacity of the first indicator, namely to:

(4.3.3) C (1J ...,k) = CD1 (P). If the set of indicators selected can be divided into two

mutually independent groups, the total amount of information will be the sum of the total information corresponding to the two groups.

It should be emphasized again that in accordance with the analytical formula for the total amount of information (4.3.1), the discriminatory capacities of all the indicators represent additive quantities and that the quantity of information from indicator Xi already contained in the information from the indicators preceding it in order of importance may be subtracted by using the weighting factor

4.4 Optimum list of a limited number of development indicators

The fundamental question involved in the choice of a list of development indicators is: if we have a maximum list of n indicators, how can it be reduced to a list of k indicators (k < n) such that this set of k indicators shall contain a total quantity of information about the development level of the countries under observation greater than any other set of k indicators selected from the n indicators of the maximum list?

Let us assume that the indicators{ XI, X2, ..., Xn} are already arranged according to the definitive classification

25

and that (4.4.3)

is an ordered sub-set of k indicators in which

The total contribution of this set of indicators to the evaluation of the development level of the countries is, according to formula (4.3.1), as follows:

The optimum list of k indicators will thus consist of

the group 1 x,, xp, ... xk 1 which maximizes formula (4.4.1). In other words, the optimum list will consist of the indicators Xa, Xp , ... xk , if I

It will not be easy to determine this optimum list, even using a computer, if the maximum list contains a large number of indicators.

In practice, however, high-level authorities such as the United Nations General Assembly, UNCTAD's Trade and Development Board, ECOSOC, etc., which decide on and adopt a definitive list of development indicators, could insist on a certain number of indicators being included in advance and leave it to the experts to add others to make up an optimum list. This list would perhaps no longer be the best possible, but from the technical point of view it would be much easier to carry out calculations. Let X1, X2, ..., Xp be the p indicators decided in ad-

vance and let X. , ..., X, be the (k-p) indicators to be chosen from the (n-p) indicators of the maxi- m u m list. We can order the set formed by the union of any combination of (k-p) indicators with the p indicators decided in advance, i.e.

'p+ 175p+2 k

Lastly, if the k indicators are already selected, either by establishing them in advance or by identifying them by the formulae (4.4.2) or (4.4.3), it remains to identify the indicator xk+l which would produce the maximum increment in the total amount of information C (1,2, ..., k).

This will be the indicator Xm if

1 - forik+1 E 1 k+l, k+2, ..., n .

5. CONCLUSION

We have tried in this study to outline an objective procedure for establishing an optimum list of development indicators which would provide statistics of use in evaluating and forecasting development in the Second Develop- ment Decade.

The first step is to establish the maximum list of development indicators by combining the various lists used hitherto with other indicators which have been neglected or overlooked in the past.

The procedure then consists of making a quantitative evaluation of the discriminatory capacity and importance of each indicator in this maximum list.

The total amount of information of a group of indicators is defined as the sum of the coefficients of the discriminatory capacity of all the indicators in this group whose weighting coefficients correspond to the importance of the indicators and do away with duplications of the same quantities of information. The optimum short list of development indicators will then be formed by the indicators which provide the maximum quantity of total information.

Considering the absence of statistics in certain cases or the poor quality of the statistics provided by the large number of indicators in a maximum list, and the expense and time involved in introducing new indicators into the statistical system of developing countries, this procedure makes it possible to draw up a rational plan for determining a limited number of indicators providing the maximum amount of total information.

<] 1, 2, ...)P/ U I ip+l, ip+2, ik 1 > The order of these indices will be established according

to the degree of importance of the corresponding indicators. The optimum list will now be made up of the p indica-

tors decided in advance and the (k-p) indicators

for which 1 x u , x,Uw,..-, x k 1

26

Annex

Progress Report I on the Methodology of Human Re- sources Indicators. (Unesco/COM/(SHC)CS/ 194/5).

Study 11. An Analytical Approach to Educational Attain- ment Indicators Based on the Analysis of School En- rolment by Grade.

Study 111. Procedure of Evaluating High-Level Manpower Data and Typology of Countries by means of the Taxonomic Method, by Zdzislaw Hellwig.

Study IV. On the Definition and Theory of Development with a View to the Application of Rank Order Indi- cators in the Elaboration of a Composite Index of Human Resources (Paper prepared for the Unesco Expert Meeting on the Methodology of Human Re- sources Indicators, held in Warsaw 11-16 December 1967. The paper can also be identified as PRIO- publication No. 6-4 from the International Peace Research Institute, Oslo), by Johan Galtung & Tord Hoivik.

problem areas and difficulties connected with the e!aboration of coherent systems of indices of human resources, by Zygmunt Gostkowski.

Study VI. On the Optimal Choice of Predictors, by Zdzislaw Hellwig.

Study VII. On the Problem of Weighting in Internation- al Comparisons, by Zdzislaw Hellwig.

Study VIII. Indices of Development for Selected Latin American and Middle African Countries. An Experi- mental Exercise using the Hellwig Taxonomic Method, by Frederick H. Harbison, Joan Maruhnic, Jane R. Resnick.

sources Components as Predictors of Economic Growth in Latin America, by The Higher School of Economics, Wroclaw, Poland.

Study X. Concerning Data Availability and Conceptual Validity, Selected Latin American and African Coun- tries, by Frederick H. Harbison, Joan Maruhnic and Jane R. Resnick.

Resources Indicators by means of a Cobb-Douglas

Study V. Discussion of methodological guidelines,

Study IX. Alternative Combinations of Human Re-

Study XI. The Assessment of the Validity of Human

Type Production Function, by H.V. Muhsam, Hebrew University, Jerusalem, Israel.

Setting Based on International Comparisons, by Zygmunt Gostkowski.

The Case of Mexico, by Tord Hoivik, International Peace Research Institute, Oslo.

Development: I. Theory, Methods and Data; 11. The Case of Japan, by Johan Galtung.

Development: The Case of Venezuela, by Kristin Tornes, Internaitonal Peace Research Institute, Oslo.

Study XVI. Approximative Methods of Selection of an Optimal Set of Predictors, by Zdzislaw Hellwig.

Study XVII. Mathematical Methods in the Unesco Project on Human Resources Indicators: (i) An evaluation (ii) Alternative Analytical Methods. by Ludovic Lebart (SHC/WS/219).

Study XVIII. Human Resources and Economic Develop- ment. Some Problems of Measurement by Prof. Wilfred Beckerman (University College, London).

Study XIX. Distance-based analysis, numerical taxonomy and classification of countries according to selected areas of socio-economic development, by Serge Fanchette, Methods and Analysis Division, Unesco.

Study XX. Diachronic Analysis of Relationships between Human Resources Components and the Rate of Economic Growth in Selected Countries (COM/WS/ 13 1) by Johan Galtung.

of Socio-Economic Development, by Zdzislaw Hellwig and Serge Fanchette.

Developing Countries and its Relationship to the Work on Human Resources Indicators, by H.W. Singer (Institute of Development Studies, The University of Sussex, UK).

Study XII. The Use of Taxonomic Measures in Target

Study XIII. Human Resources and Economic Growth:

Study XIV. Human Resources and Socio-Economic

Study XV. Human Resources and Socio-Economic

Study XXI. The Selection of a Set of “Core” Indicators

Study XXII. The Quest for an Employment Strategy in

27

Study XXIII. “Synchronic and Diachronic Approaches in the Unesco Project on Human Resources Indicators. Wroclaw Taxonomy and Bivariate Diachronic Analysis” (SHC/WS/209), by Serge Fanchette - Methods and Analysis Division, Unesco.

Study XXIV. A method of establishing a list of development indicators, by B. Ivanovic (Director of the Office of Statistics of UNCTAD).

Study XXV. Grouping and ranking of 30 countries of Sub-Saharan Africa. Two distance-based methods compared, by B. Ivanovic and Serge Fanchette.

Study XXVI. Aspects of the distribution of income and

wealth in Kenya, by H.W. Singer and Stuart D. Reynolds (The Institute of Development Studies, University of Sussex, UK).

Study XXVII. Distributional patterns of development and welfare: A case study of Zambia, by Charles Elliott (Overseas Development Group, School of Development Studies, University of East Anglia).

Study XXVIII. Typological study using the Wroclaw Taxonomic Method (A study of regional disparities in Venezuela), by JosC F. Silvio-Pomenta (Centro de Estudios del Desarrollo (CENDES), Universidad Central de Venezuela.

28

UNESCO PUBLICATIONS: NATIONAL DISTRIBUTORS

Argentina Australia

Austria Belgium Bolivia Brazil

Bulgaria Burma

Cameroon

Canada

Chile Colombia

Congo Costa Ricn

Cuba

Czechoslovakia

Dahomey Denmark

CYPNS

Egypt

Ethiopia Finland France

French West Indies German Dem. Rep.

Fed. Rep. of Germany

Ghana

Greece Hong Kong

Hungary

Iceland India

Indonesia Iran

Iraq

Ireland Israel

Italy Jamaica

Kenya Khmer Republic

Republic of Korea Kuwait Liberia Libya

Luxembourg Madagascar

Malaysia Malta

Mauritius Mexico Monaco

Netherlands

Netherlands Antilles New Caledonia New Zealand

Japan

Editorial Losada, S.A., Alsina I 131. BUENOS AI=. Publications : Educational Supplies Pty. Ltd., Box 33, Post Office BROOKVALE 2100, N.S.W. Periodicals : Dominie Pty. Ltd., Box 33, Post Office, BROOKVALE 2100, N.S.W. Sub-agent: United Nations Association of Australia (Victorian Division), 5th Floor, 134-136 Flinders St.. MELBOURNE 3000. Verlag Georg Fromme & Co., Arbeitergasse 1-7, 1051 WIEN. Jean D e Lannoy, 112, rue du Trsne, BRUXFLLES 5. CCP 708 23. Libreria Universitaria. Universidad San Francisco Xavier. apartado 212. SUCRE. Fundacgo GetWo Vargas, Serviso de Publicasiies, caka postal ZIIZO Praia de Botafogo 183, RIO DE JANEIRO G.B. Hemus, Kantora Literatura, bd. Rousky 6, SOFIIA. Trade Corporation n.O (9), 550-552 Merchant Street, RANGOON. Le Secrktaire gknkral de la Commission nationale de la Rkpublique fkdkrale du Cameroun pour I’Unesco, B.P. 1061, YAOUND6. Information Canada. OTTAWA (Ont.); Bookshops: 640 Ouest, rue Ste Catherine, MONTR~AL I I I (Que.); 1683 Burrington St. HALIFAX(N.S.); 393 Portage Ave., WINNIPEG (Manitoba), 171 rueslater, OTTAWA (Ont.); 221 Yonye St, TORONTO (Ont.) ; 800 Granville St, VANCOUVER (B.C.). Editorial Universitaria, S.A., casilla IOZZO, SANTIAGO. Libreria Buccholz Galeria, avenida Jimknez de Quesada 8-40, apartado a6reo 49-56, BOGOTA; Distrilibros Ltda., Pi0 Alfonso Garcia, carrera 4’, non 36-119 y 36-125, CARTAGENA; J. Germin Rodriguez N., Calle 17, 6-59, apartado nacional 83, GIRARDOT (Cundinarnarca). Editorial Losada Ltda., Calle 18A. n.O 7-37. apartado akreo 5829, apartado nacional 931. BOGOTA. Sub-depots : Edificio La Ceiba, Oficina 804 MEDELLfN; Calle 37. n.O 14-73 Oficina, 305, BUCARAMANGA; Edificio Zaccour, Oficina 736. CALI. Librairie populaire. B.P. 577, BRAZZAVILLE. Libreria Trejos S.A., apartado 1313. SAN Jose. Telkfonos: 2285 y 3200. Distribuidora Nacional de Publicaciones, Neptuno 674, LA HABANA. ‘MAM’, Archbishop Makarios 3rd Avenue, P.O. Box 1722, NICOSIA. SNTL, Spalena 51. PRAHA I (Permanent display) ; Zahranicni literatura, I I Soukenicka, PRAHA I. For Slovakia only : Alfa Verlag. Publishers. Hurhanovo nam. 6, 893 31 BRATISLAVA. Librairie nationale, B.P. 294, PORTO Novo. Ejnar Munksgaard Ltd.. 6 Nsrregade. 1165 KUBENHAVN K Librairie Kasr El Nil 38, rue Kasr El Nil. LE CAIRE. National Centre for Unesco Publications I Tlaat Harb Street, Tahrir Square. CAIRO. National Commission for Unesco, P.O. Box 2996, ADDIS ABARA. Akateeminen Kirjakauppa, z Keskuskatu, H ~ I N K I . Librairie de I’Unesco, 7 place de Fontenoy, 75700 PARIS. CCP 12598-48. Librairie ‘Au Boul’ Mich‘. L Rue Perrinon and 66 Avenue du Parquet, 972 FORT-DE-FRANCE (Martinique). Deutscher Bucb-Export und -Import G m b H , Leninstrasse 16, 701 LEIPZIG. Verlag Dokumentation, Postfach 148, Jaiserstrasse 13. 8023 ML‘NCHEN-PULLACH. ‘The Courier’ (German edition only): Bahrenfelder Chaussee 160 HAMBURG-BAHRENFELD. CCP 27 66 50. For scientific maps only: GEO C E N T R E , D-7 STUTTGART 80, Postfach 800830. Presbyterian Bookshop Depot Ltd., P.O. Box 195, ACCRA; Ghana Book Suppliers Ltd., P.O. Box 7869, ACCRA; The University Bookshop of Ghana, ACCRA; The University Bookshop of Cape Coast; The Uni- versity Bookshop of Legon, P.O. Box I, LEGON. Anglo-Hellenic Agency, 5 Koumpari Street, ATHINAI 138. Swindon Book Co., 13-15 Lock Road, KOWLOON. Akadhiai Konyvesbolt Vdci U 22, BUDAPE~T V. A.K.V. Konyvtarosok Boltja. NkpkGztPrsasig utja 16, BUDAPFST VI. Snaebjorn Jonsson & Co. H. F.. Hafnarstracti 9. REYKJAVIK. Orient Longman Ltd.; Nicol Road, Ballard Estate, BOMBAY I; 17 Chittaranjan Avenue, CALCUTTA 13; 36A Anna Salai, Mount Road, MADRAS 2; B-3/7 Asaf Ali Road, NEW DELHI I. Sub-depots: Oxford Book & Stationery Co., 17 Park Street, CALCUTTA 16; and Scindia House, NEW DELHI; Publications Section, Ministry of Education and Social Welfare, 72 Theatre Communication Building, Connaught Place, NEW DELHI I. Indira P.T., JI. Dr. Sam Ratulangie 37, JAKARTA. Commission nationale iranienne pour I’Unesco, avenue Iranchahr Cbomali no 300. B.P. 1533. T~H~RAN. Kharazmie Publishing and Distribution Co., 229 Daneshgahe Street, Shah Avenue, P.O. Box 14l1486, TBHBRAN. McKenzie’s Bookshop, AI-Rashid Street, BAGHDAD; University Bookstore, University of Baghdad, P.O. BOX 75. BAGHDAD. The National Press, 2 Wellington Road, Ballsbridge. DUBLIN 4. Emanuel Brown, formerly Blumstein’s Bookstores: 35 Allenby Road and 48 Nachlat Benjamin Street, TEL Avrv; 9 Shlomzion Hamalka Street. JERUSALEM. LICOSA (Libreria Commissionaria Sansoni S.p.A.). via Lamarmora 45. casella postale 552, 50121 FIBENZE. Sangster’s Book Stores Ltd., P.O. Box 366, IOI Water Lane, KINGSTON. Maruzen Co. Ltd.. P.O. Box 5050. Tokyo International, TOKYO. The ESA Ltd.. P.O. BOX 30167, NAIROBI. Librarie Albert Portail, 14. avenue Boulloche. PHNOM-PENH. Korean National Commission for Unesco. P.O. Box Central 64, SEOUL. The Kuwait Bookshop Co. Ltd., P.O. Box 2942, KUWAIT. Cole & Yancy Bookshops Ltd., P.O. Box 286, MONROVIA. Agency for Development of Publication and Distributlon, P.O. Uox 34-35, TRIPOLI. Librairie Paul Bruck, 22 Grande-Rue, LUXEMBOURG. Cornmission Nationale de la RBpublique Malpache, MinistAre de 1’Education Nationale, TANANARIVB. Federal Publications Sdn. Bhd.. Balai Berita, 31 Jalan Riong. KUALA LUMPUFL Sapienza’s Library, 26 Kingsway, VALLETTA. Nalanda Co. Ltd, 30 Bourbon Street, PORT-LOUIS. CILA (Centra Interamericano de Libros Acadhicos), Sullivan 31 bis, MfXlCO 4. D.F. British Library. 30. boulevard des Moulins, MONTE-CARLO. N.V. Martinus Nijhoff, Lange Voorhout 9. ’s-GRAVENHAGE; Systemen Keesing, Ruysdaelstraat 71-75> AMSTERDAM. G. C. T. Van Dorp & Co. (Ned. Ant.) N.V., WILLEMSTAD (Curacao, N.A.). Reprex S.A.R.L., B.P. 1572, NOUM~A. Government Printing Office, Government Bookshops: Rutland Street, P.O. Box 5344, AUCKLAND; 130 Oxford Terrace. P.O. Box 1721, CHRISTCHURCH; Alma Street, P.O. Box 857, HAMILTON; Princes Street, P.O. Box I 104, DUNEDIN; Mulgrave Street, Private Bag, WELLINGTON.

Niger Nigeria

Norway

Pakistan

Peru

Philippines Poland

Portugal Southern Rhodesia

Romania

Senegal

Singapore South Africa

Spain

Sri Lanka Sudan

Sweden

Switzerland Tanzania Thailand

Togo

Turkev

United States Upper Volta

Venezuela

Yugoslavia Zaire

Librairie Manclert, B.P. 868, NIAMEY. The University Bookshop of Ife; The University Bookshop of Ibadan, P.O. Box 286, IBADAN; The University of Nsukka; The University Bookshop of Lagos; The Ahmadu Bello University Bookshop of Zaria. All publications: Johan Grundt Tanum, Karl Johans gate 41!43. OSLO I. ‘The CmUier’ only : A/S Narveqns Litteraturtjeneste. Box 6125, OSLO 6. The West-Pak Publishing Co. Ltd., Unesco Publications House, P.O. Box 374, G.P.O., LAHORE. Showrooms : Urdu Bazaar, LAHORE and 57-58 Murree Highway, G/6-1, ISLAMABAD. Pakistan Publications Bookshop: Sarwar Road, RAWALPINDI; Mina Book Agency, 65 Shahrah Quaid-e-azam, P.O. Box 729 pi”“ 3. The Courier’ only : Editorial Losada Peruana, apartado 472, LIMA. Otk publications: Distribuidora Inca S.A., Emilio Althaus 470. Lince, casilla 3115. LIMA. The M o d e m Book Co., 926 Rizal Avenue, P.O. Box 632. MANILA. Osrodek Rozpowzechniania Wydawnictw Naukowych PAN, Palac Kultury i Nauki, WARSZAWA. Dias & Andrade Ltda., Libraria Portugal, rua o Carmo 70, LISBOA. Textbook Sales PVT) Ltd., 67 Union Avenue, SALISBURY. I.C.E. LIBRI. Calea Victoriei nr. 126, P.O. Box 134-135 BUCURESTI. SubscriptMns to p ~ r i ~ d i ~ ~ l ~ t Rompresfilatelia. Calea Victoriei nr. 29, BUCURESTI. La Maison du Livre, 13, avenue Roume, B.P. 20-60, DAKAR; Librairie Clairafrique, B.P. zoos, DAKAR; Librairie ‘Le Senegal’. B.P. 1594, DAKAR. Federal Publications Sdn Bhd., Times House, River Valley Road, SINGAPORB 9 Van Schaik‘s Bookstore (Pty.) Ltd., Lihri Building, Church Street, P.O. Box 7.24. PRBTORIA. AI1 publications : Editiones Iberoamericanas, S.A., calle de OEate 15, MADRID 20; Distribuci6n de Publi- caciones del Consejo Superior de Investigaciones Cientificas, Vitrubio 16, MADRID 6 ; Libreria del Consejo Superior de Investigaciones Cientificas, Epigciacas 15. BARCELONA. For ‘The Courier’ only : Ediciones Liber, apartado 17. ONDdRROA (Viscaya). Lake House Bookshop, Sir Chittampalam Gardiner Mawata, P.O. Box 244, C O L o m o 2. AI Bashir Bookshop, P.O. Box I I 18, KHARTOUM. All publications: A/B C.E. Fritzes Kungl. Hovbokhandel, Fredsgatan 2, Box 16356,103 27 STOCKHOLM 16. For ‘The Courier’: Svenska FN-Forbundet, SkolgrPnd 2, Box 150 so. s-Io4 65 STOCKHOLM. Europa Verlag. Ramistrasse 5, ZURICH; Librairie Payot, 6, rue Grenus, 1211 GENBVE XI. D a r es Salaam Bookshop, P.O. Box 9030, DAR ES SALAAM. Suksapan Panit, Mansion 9. Rajdamnem Avenue, BANGKOK. Librairie evangPlique, B.P. 378; LoMB; Librairie du Bon Pasteur, B.P. 1164, LoMB; Librairie modeme, B.P.777, LoM~. Librairie Hachette. 469 Istiklal Caddesi, Beyoglu, ISTANBUL. Uganda Bookshop, P.O. Box 145. KAMPALA. Mezhdunarodnaja Kniga, MOSKVA, G-zoo. H.M. Stationery Office, P.O. Box 569, LONDON SEI gNH; Government bookshops: London, Belfast, Bir- mingham, Bristol, Cardiff, Edinburgh, Manchester. Unesco Puhlicstions Center, P.O. Box 433. N e w York, N.Y. 10016. Librairie Attie B.P. 64. Ouagadougou, Librairie catholique ‘Jeunesse d’Afrique’, OUAGADOUGOU. Libreria Historia, Monjas a Padre Sierra, Edificio Oeste 2. n.O 6 (frente a1 Capitolio), apartadode correo8

Jugoslovenska Knjiga. Terazije 27. BEOGW. Drzavna Zalozba Slovenije Mestni Trg. 7.6, LJUBLJANA. La Librairie, Institut national d’etudes politiquen, B.P. 2307, KINSHASA: Commiaaion nationale de la Rdpu- blique du ZaIre Dour I’Unenco, Ministere de I’Mucntion nationale. KINSFIMA.

7320-101, CARACAS.

ISBN 92-3-1 0 1 1 29-4

30 Social indicators: problems of def i ni ti on and of...

Documents

Transcript of 30 Social indicators: problems of def i ni ti on and of...