EUROPEAN COMMISSION EUROSTAT Directorate D: Single Market, Employment and Social statistics Unit D-2: Living conditions and social protection

Doc. D2/HBS/158/2005/EN Doc. EU-SILC 155/05/EN

FIRST MEETING OF THE WORKING GROUP ON

LIVING CONDITIONS

(HBS, EU-SILC AND IPSE)

8-10 June 2005

Eurostat-Luxembourg

HBS and EU-SILC Imputed rent

2

TABLE OF CONTENTS

1. INTRODUCTION................................................................................................... 3

1.1. Rent imputing for dwelling services in the context of the EU-SILC................. 3

1.2. Rent imputing for dwelling services in the context of the HBS........................ 4

2. THE PROBLEM OF RENT IMPUTING FOR DWELLING SERVICES................ 6

2.1. The concept of rent imputing for dwelling services ......................................... 6

2.2. Why imputing a rent to owners-occupiers and free-tenants ............................. 7

2.3. Tenure discount ............................................................................................. 8

3. COMPUTATIONAL METHODS FOR IMPUTING RENT .................................... 9

3.1. Self-Assessment method................................................................................. 9

3.2. User-cost method ......................................................................................... 10

3.3. Stratification method .................................................................................... 10

3.4. Hedonic prices method. Logarithmic regression............................................ 12

3.5. Heckman method ......................................................................................... 14

4. IMPLEMENTATION OF HECKMAN’S METHOD: PRACTICAL APPROACH ......................................................................................................... 15

4.1. Model equations........................................................................................... 15

4.2. Variable selection......................................................................................... 16

4.3. Practical approach: Steps and difficulties ...................................................... 17

4.4. An alternative approach: building quality indexes of the dwelling (QI) .......... 18

4.5. Software implementation.............................................................................. 19

4.5.1. Implementation in STATA.............................................................. 19

4.5.2. Implementation in SAS ................................................................... 19

5. CONCLUSIONS AND RECOMMENDATIONS.................................................. 19

5.1. Conclusions and recommendations for EU-SILC .......................................... 19

5.2. Conclusions and recommendations for HBS ................................................. 20

6. BIBLIOGRAPHY ................................................................................................. 21

3

1. INTRODUCTION

This document has been produced in collaboration between the EU-SILC and HBS teams of Eurostat. The synergy between these two projects is obvious because both deal with the problem of rent imputing for dwelling services.

However, we should highlight that the level of requirements for the solution of this problem are different for both projects. While EU-SILC needs to be able to compare the disposable income of individual households in a reliable way, the main goal of HBS is to measure average household consumption expenditure for relatively large population groups (e.g. households grouped by age of the reference person). Therefore the requirements of accuracy and reliability of the solutions found for individual households are lower for HBS than for EU-SILC. This fact will have some consequences on the choice of methods recommended for each project.

Therefore, although a large part of this document is common for both projects, there are some points which have been written specifically for one of these projects.

Some parts of this document have quite a technical content. In order to avoid giving too many technical details, some bibliographic references, where these details may be found, are mentioned at the end.

1.1. Rent imputing for dwelling services in the context of the EU-SILC

EU-SILC Framework Regulation specifies that the non-monetary income components (with the exception of company car) will be optional from the first year of the survey and compulsory from 2007. Nevertheless, variables required for calculating imputed rent will be collected as from the first year of data collection. It will permit once is decided upon a method to calculate the imputed rent back from the beginning of operation.

During the EU-SILC Task Force meetings held in March and June 2002, the calculation of “imputed rent” was discussed.

Two documents have been produced about this issue: DOC 73/02 “Primary ideas about imputed rent”, where an overview of the different methods to calculate “imputed rent” was presented, and DOC 83/02 “Test results on different methods of imputed rent” where a series of tests were carried out on the basis of ECHP for two types of methods:

- Self-assessment method or subjective method

- Statistical methods (stratification / Heckman econometrics)

No test was made on the basis of the “Estimation based on the selling price of accommodations” because this variable was not available in surveys.

The analysis of the tables presented in document 83/02 showed the extremely high level of the subjective rents and the major consistency for the different methods except for the factor ‘number of years of occupation of the dwelling’, which has not been taken into account. Developments for the Heckman variable, which is often framed by the subjective rent (high levels) and the real rent (Realrent) are also consistent.

The stratification method proved to generally be only coherent for the variables, which were used to define the strata (seniority, number of rooms and dwelling type). However, the

4

number of strata cannot be increased without taking the tenants’ sample size into consideration if a minimum of 30 observations in each stratum is required. This constraint could be overcome by the use of external sources.

In the absence of external sources, the Heckman model is perhaps the best solution to be adopted. An alternative would consist of deducting “all the speculative charges” relating to seniority from the subjective rents but this can lead to highly biased results, as we will show later on in section “3.1 Self-assessment method”.

As a consequence of the mentioned tests, EUROSTAT decided to propose a method for those countries that use the “self-assessment method” for the calculation of imputed rent as well as for those countries that do not have a method for the calculation of this variable.

This document gives an overview of the possible methods that can be used in the calculation of imputed rent, and it proposes the Heckman method for countries not having a national method other than the subjective one.

1.2. Rent imputing for dwelling services in the context of the HBS

The problem of rent imputing for dwelling services has been already discussed in various meetings of the HBS working group and has been analysed in several documents during the recent years. In particular we should mention here the last methodological guides about HBS produced by Eurostat:

– “Household Budget Surveys in the EU. Methodology and recommendations for harmonisation 2003”

– “Household Budget Surveys in the Candidate Countries. Methodological analysis 2003”

The last references were the data collections carried out for the reference year 1999. In this exercise all EU -15 countries imputed a rent for owners-occupiers. Most of the countries carried out the calculation themselves before transmitting their data to Eurostat and only two were assisted by Eurostat. However the methods used by each country were not homogeneous, as shown in the following table 1. The main reasons for such a difference are the different size and characteristics dwelling rental markets in each country. For instance, we may highlight that whereas 55,1% of German households lived in rented dwellings in 1999, only 8,5% of Spanish households did the same. Therefore the development of a reliable an accurate model for imputing rents to owners-occupiers is more difficult in the second case than in the first one.

5

Table 1: Estimation methods of imputed rentals for HBS by country in 1999 1)

Population Method Stratification method

Owner-occupiers Tenants (reduced and free rental)

Self-asse-ssment

Stratifica-tion

Extrapola-tion

Size Loca-tion

Other

BE X - Only free rental X (1) X (1) HBS (1) X (1) X (1)

DK X X (2) Statistics on rents

X X X (3)

DE X X X - X HBS X X X (3)

EL X X Only free rental X (4)

ES X - X X

FR X (5) - X X X X X

IE X - Only free rental X X (6) X (6)

IT X - - X

LU X X X X (7)

NL X - - X (8) Estate offices (each 5 years)

AT X - X Micro-census (March 2000)

X (9) X X

PO X X X X (10)

FI X - X X Statistics on rents

(11)

X X X

SE X - -

X Statistics on rents

(11)

X X X

UK X (12) - - - X (12) X (12) X (12) X (12) X (12)

(1) Belgium: Self-assessment method + stratification method for control and imputation of missing. The main stratification variables are number of bedrooms, type of dwelling, garage, and elevator and location (zones urbane, rural, semi-rural). A supplementary control is operated using the "cadastral" income ("revenu cadastral”).

(2) Denmark: Rent is imputed for free rental. (3) Germany, Denmark: ‘other variable’ includes period of construction and installation.

In Denmark, some recalculations are made to handle the actual expenses on repairs, taxes etc. The strata used are the same as in the Danish National Accounts.

(4) Greece: Self-assessment but the interviewer checks and corrects, where necessary, the answer of the interviewee taking into account the locality, mean of actual rent per locality, number of rooms in the dwelling, area of the dwelling, year of construction and the quality of the building/dwelling.

(5) France: Rents for owners-occupiers are estimated using rents paid by tenants. (6) Ireland : Stratification method using location and number of rooms. Data compared with Labour Force Surveys and

Consumer Price Index Rent Survey. (7) Luxembourg: The households’ declarations are checked a posteriori by making a calculation per stratification in

order to rectify the doubtful statements. (8) Netherlands: national method of evaluation. Regression model based on estate offices (‘0’ and ‘5’ years). (9) Austria: stratification method using the variables ‘regions (Bundesland)’, ‘amenities’, ‘living area in m2’ and ‘year

of construction’. The extrapolation source is the Austrian Microcensus March 2000. The actual rents used include some of the dwelling charges.

6

(10) Portugal: Self-assessment with control tables by region during the interview according to the area, number of rooms, sanitary, water and electricity facilities.

(11) Finland, Sweden: The statistics on rents include some of the dwelling charges. (12) United Kingdom: Imputation for owner-occupiers main dwelling is not an official estimate. It is done for Eurostat

only. Stratification is carried out with a hedonic model estimated by regression on private market rents. Extrapolation is carried out for larger and more expensive properties with small numbers privately rented in the sample. Stratification variables include: size–number of rooms (not floor area), location–region, property tax band, whether furnished (for estimation, set to unfurnished for imputation), whether house or flat and date of interview (for estimation, set to appropriate date for imputation).

Regarding the ten new Member States, Bulgaria, Romania and Turkey, only five countries out of thirteen used some method for imputing rent in 1999. The problem of imputed rent in these countries is generally more difficult than in the EU-15 because many of them have very narrow dwelling rental markets (in some cases less than 3% of households of the country live in rented dwellings). Therefore the approach of calculating imputed rents from the sub sample of households living in rented dwellings might be quite difficult for certain countries.

Considering that imputed rentals make up a substantial part of the budget of the owner-occupier (almost 20 per cent, HBS figure for EU-15 in 1999), harmony of concepts and measurement should be given high priority and requires a systematic and comprehensive analysis. The main purpose of this document is to address this question with a practical approach in order to set a clear and realistic recommendation for the next round of HBS (reference year 2005).

2. THE PROBLEM OF RENT IMPUTING FOR DWELLING SERVICES

2.1. The concept of rent imputing for dwelling services

Conceptual and economical set-out

In ESA 95, which is a basic reference for all the statistics dealing with economical aspects, the purchase of the dwelling as such is regarded primary as capital formation (investment) and not as consumer expenditure. However, the ownership of a dwelling is considered to produce a service – a shelter – which is actually consumed over time by the households. As a consequence, ESA requires the estimation of the price of the shelter, by imputation of a rental, since no monetary transaction is involved. This imputed rental is part of household consumption expenditure because the involved households are actually consuming that dwelling service. On the other hand, since the dwelling owners have allocated those dwellings to be used by their households, we may consider that there they receive a property income equivalent to the imputed expenditure as compensation. Therefore both amounts, imputed income and imputed expenditure, are mutually cancelled and no monetary payment is involved. So, for the HBS and EU-SILC to be consistent with the ESA principles, it is recommended to exclude the acquisition of dwellings, whereas the service of the dwelling should be included, as part of both, the household net income and the household consumption expenditure.

This way of reasoning may be extended too to tenants occupying dwellings for free or paying a reduced rent ( e.g. employment benefits, aid given by social institutions , dwellings let by relatives …)

7

Coverage

Coverage may be defined with regards to several criteria: the use of the dwelling as primary or secondary residence, the inclusion of annexes as part of the dwelling, the tenure status and the actual content of imputed expenditure.

The dwellings that should be included for imputation of rental in ESA, are those used entirely or primarily as residences. Houseboats, barges, mobile homes and caravans used as principal residences of households should also be included, as are historic monuments identified primarily as dwellings. For secondary dwellings (week end and holiday residences) imputation is not required.

Any associated structures of the dwellings, such as garages, gardens and so forth, will be considered as belonging to them, as long as they serve to improve the living conditions of the dwelling occupiers. However, no imputation is to be made for garages used by their private owner only for the purpose of parking near the workplace, which should be classified as a part of transport services.

Regarding tenure status, imputation may be calculated for the following groups of households:

– Owners-occupiers (dwellings owned by their occupiers)

– Free rent (use of dwellings provided for free by employers, relatives, social aid institutions …)

– Reduced rent (dwellings rented at prices lower than market prices by employers, relatives, social aid institutions …)

In case of furnished dwellings, the part of rental paid for this furniture should not be considered for the purpose of calculating imputed rents. Finally, the imputed rent is net of charges for electricity, water, refuse collection etc.

2.2. Why imputing a rent to owners-occupiers and free-tenants

The inclusion of imputed rent in gross and disposable income as well as in final consumption expenditure is a considerable step forward in achieving a more complete measure of income and expenditure. It will be a better basis for comparison of standards of living between households with different housing behaviour patterns between and within ‘Member States’.

Effectively, some people own a house and have no regular expenditure for housing, other than maintenance. Others live in subsidised or rent-free housing and have comparatively small expenditure than tenants. The inclusion of imputed rent will permit to make more accurate welfare comparisons between households that have the same level of monetary income but different housing behaviour patterns.

The imputed rent should be estimated only for those dwellings (and any associated such a garage) used as main residence of household.

8

The imputed rent should be estimated for all households that did not report paying full rent, either because they are owner-occupiers or they live in accommodation rented at a lower price than the market price, or because the accommodation is provided rent free.

The value to impute should be the equivalent market rent that should be paid for a similar dwelling as the occupied, less any rent actually paid (in the case that the accommodation is rented at a lower price than the market price), less any subside received from the government or from a non-profit institution (if the accommodation is free or at lower price than the market price), less any minor repairs or refurbishment expenditures which the owner-occupier households make on the property of the type that would normally be carried out by landlords.

The market rent is the rent due for the right to use an unfurnished dwelling on the private market, excluding charges for heating, water, electricity, etc.

2.3. Tenure discount

Quite often tenants living in the same dwelling for a long time pay less than tenants who have moved recently, even though the characteristics of both dwelling s are identical. There are several economical reasons for this phenomenon:

– Landlords use to trust more in long-term tenants and consider that future payments of these tenants are more reliable. Hence they use to be more willing to accept smaller retributions in the case of old contracts because the economical risk is lower than for new contracts.

– Before offering a dwelling on rent, landlords use to invest some money in fitting it properly in order to suit the expectations of new demand. These investment s are not generally required for on-going contracts. This cost saving allows that smaller rental prices are acceptable for on-going contracts.

– Dwelling utilisation (proportion of time that a dwelling is actually rented divided by the total available time) uses to be higher for long -term contracts than for short -term contracts, because of the time needed to find new tenants and to conclude the contracts. Again, for a given annual turnover expected by a landlord, the monthly rental price may be lower if the utilisation of the dwelling is higher.

– In certain countries there are legal limitations for increasing rental prices.

Anyhow, no matter these reasons, this dependence of rental prices on the duration of the contracts is a matter-of-fact effect which has been confirmed experimentally in some opportunities and has been called by researchers as “tenure discount”. This phenomenon may be particularly important for certain periods of time; e.g. following the introduction of new legal regulations of rental markets or during periods of price instability.

Tenure discount must be taken into account when imputing dwelling rents because it is a specific household circumstance which may modify the price of dwelling services received by households in the same way that other household circumstances may modify the price of other goods or services (e.g. discounts received by number of children in school fees).

9

For this reason, the time for which a household has been living in the same main residence is an essential variable to be considered by the main imputation models. Moreover, this involves that external sources, such as price statistics, which not take into account tenure discount, cannot be directly used for the purpose of calculating imputed rents.

3. COMPUTATIONAL METHODS FOR IMPUTING RENT

There are several approaches that can be followed to calculate the imputed rent. The most important are the following:

• Self-assessment method or subjective method

• User cost method

• Statistical methods

– Stratification

– Econometric methods

– Hedonic prices

– Heckman

3.1. Self-Assessment method

In the self-assessment method information is collected from owner-occupiers (accommodation rent-free or accommodation rented at a lower price than the market price) through a household survey on what they estimate the potential market rent of their dwelling to be.

The main drawback of the self-assessment method is the subjective nature of the estimates. Compared with ‘true’ potential market rents, self-assessed estimates may be subject to large degrees of both over and under estimation, depending on the precise circumstances under which the estimation is made. Such biases are extremely difficult to measure, leaving substantial uncertainty about the quality of the resulting estimates. Furthermore, the error margin over time does not seem likely to remain stable with changing circumstances.

As a proof of the last statement, we can mention some problems shown by a study1 carried out with data from the Spanish HBS for years 1980 and 1990. This work showed an underestimation for the subjective rental of “13%” in year 1980 and an overestimation of “27%” for year 1990, compared to the imputed rental value using Heckman’s method. An explanation suggested for this contradictory behaviour was the different level of awareness about the rental market at both moments by owner-occupiers. Significant legal changes between these two dates made that dwelling owners considered the possibility of renting their

1 Arévalo 2001

10

dwellings differently in both years and, for that reason, owner-occupiers were better informed about the rental market conditions and prices in 1990 than in 1980.

These facts show how particular reasons may produce important biases in the results of subjective assessment.

3.2. User-cost method

The user cost method has been proposed by the working groups on National Accounts and Purchasing Power Parities as an alternative method of estimation of dwelling services whenever the stratification method does not work well. More specifically they propose:

“In the case of privately rented dwellings constituting less than 10% of the total dwelling stock by number and, where there is a large disparity between private and other paid rents (say by a factor of three), as an alternative objective assessment, the user-cost method may be applied.”

In a few words, user cost consists of adding relevant cost items, like intermediate and capital consumption as well as some allowance for the net operating surplus, including the interest on mortgages. A detailed description of this method may be found in the first bibliographic reference.

The user-cost approach avoids the particular problem, inherent in stratification, of imputing owner-occupied rent by comparisons between one type of dwellings and another to determine gross rentals. It does, however, require other types of imputation.

Although this method may solve the problem which arises when the sub-sample of privately rented dwellings is too small, it has some important drawbacks for its implementation in the contexts of EU-SILC and HBS. The most important one is the macro-economic approach of some calculations which prevents from getting accurate imputations at micro-level.

At the moment of writing this document Eurostat is aware of one attempt to adapt this method for producing acceptable results at micro-level. As soon as these results will be analysed, they will be communicated to the EU-SILC and HBS working groups.

3.3. Stratification method

The dwellings are disaggregated in quantity terms into strata (for example, by region, type of dwelling, and type of population) so that the rental value of dwellings within a stratum is as homogenous as possible. Data on actual market rents paid within each stratum are then obtained. The average rent paid in the stratum to which the dwelling belongs is then imputed to the owner-occupier (accommodation rented free, accommodation rented at a lower price than the market price). This is known as the stratification method2.

2 This method is recommended in the Commission Decision 95/309 (July 1995), where is setting out the principles to be adopted to impute rents in National Accounts. The Annex to Commission Decision 95/309 identifies the sets of variables, which may be relevant in the construction of strata, as well as the data sources to be in making estimates of imputed rent. Nevertheless, since the housing market will differ from one Member State to another, the choice of stratification factors will differ too and it is up to each to determine which are the most significant factors in their particular circumstances.

11

With some variants, this method was the most frequently used for the HBS round of 1999 by the EU-15 countries (see table 1). The results of the stratification method may be satisfactory for the HBS purposes in many countries and has been recommended as an acceptable method for the collection round of the reference year 2005. Compared with Heckman, it has the advantage of being computationally more simple and requiring less information. However, it has some drawbacks that may do its practical application quite difficult (particular in a context of requirements of high quality imputations at the level of individual households, such as EU-SILC does):

– For surveys with small sample sizes, it could be difficult to define strata where the rental value of dwellings within each stratum is homogenous enough.

– The same problem may be found for countries with large surveys but small rental markets. Please note that there are some countries where the proportion of households living in rented dwellings and paying full rent is less than 10%.

Analysis of the stratification method in the context of EU-SILC

The adoption of the stratification method means that no data have to be collected in EU-SILC to provide a direct measure of imputed rent. Instead, data will be required in order to place each sampled dwelling candidate to be imputed into its correct stratum cell, and on the average actual rent to be applied to that stratum cell. This means that at the micro level, each owner-occupied dwelling (rent-free or accommodation rented at a lower price than the market price) within a particular stratum cell will be allocated the same value for imputed rent.

It will therefore be important first to ensure that within the housing section of EU-SILC, questions are asked on each of the variables used to stratify dwellings for the purposes of imputed rent.

The second requirement for estimation of imputed rent is data on actual market rents in order to calculate the average imputed rent to be applied to the owner-occupied dwellings (rent free or accommodation rented at a lower price than the market price) sampled in EU-SILC in each stratum cell. Although actual rents will be collected in EU-SILC, for reasons of sample size these are unlikely to provide a sound basis for the estimation of imputed rent within each stratum cell. It will probably be desirable instead to use other sources (housing census, housing survey or household budget survey, preferably the same source(s) used in the national accounts).

Analysis of the stratification method in the context of HBS

The Commission Decision of 18 July 1995 (95/309/EC, EURATOM) specifies the principles for estimating dwelling services in the ESA. HBS is not mentioned in this Decision. Nevertheless, it has already been agreed by the Member States that HBS should estimate imputed rentals for owner-occupiers, using the stratification method whenever possible. The use of such a method should insure the coherence between National Accounts and HBS results, and would objectively get more precise results than the self-assessment method.

The methods used to determine which variables have a significant impact vary in sophistication between a straightforward tabulation of the statistical information available to advanced statistical techniques such as multiple regression analysis. These are also described in the Annex to the Commission Decision (section 1.2.2).

12

Although this Decision states a clear set of basic principles to be applied, enough room for interpretation is left in order to fit the various circumstances of housing stock and housing markets of each Member State. Therefore, some important details, such as the actual dwelling features used for stratification purposes, the stratification criteria or the number of strata, should be determined at country level.

Possible features of dwelling to be used for stratification purposes

– seniority of dwelling occupation (time living in the same dwelling);

– size of a dwelling, i.e. area and number of rooms;

– amenities, i.e. bathroom, balcony/terrace, central heating, air-conditioning;

– period of construction or last major renovation;

– availability of garage, lift, swimming-pool, garden;

– dwelling-type (detached, semi-detached house, flat, vintage);

– population density of the area where the dwelling is located;

– location of the dwelling (distance from centre, situated in flat or mountainous land, transport facilities and nearby or far away shops and schools).

3.4. Hedonic prices method. Logarithmic regression

Rosen (1974) established the micro-economic bases of the Hedonic Regression. It provides a model of determination of the price of a differentiated and indivisible good under conditions of “perfect competition”, where the price is the result of the intersection of offer and demand of all characteristic of the good.

Although the offer/demand of characteristics cannot be observed, the partial derived of the equation can be interpreted as the implicit marginal price of the prevailing characteristics in a model of balance.

The hedonic regression can be used to explain the ‘rent’ for the tenants as a function of independent variables (such as, location variables (region, degree of urbanization), characteristics of the dwelling, and seniority) which determine the rent.

The model can be enunciated as:

Y = X 㬠 + 㭐

Where:

Y is the logarithm of variable “current rent related to occupied dwelling” (HH060 in EU-SILC and HE04.1.1.1 in HBS)

X are independent variables which determine the rental. Typically the most explicative variables are among the following:

13

• LOCATION:

region, climatic zone, degree of urbanization, living conditions of the area (noise, pollution, security, proximity to public services or communication infrastructures …), …

• CHARACTERISTICS OF THE DWELLING:

type of dwelling (flat, semidetached house, detached house, farm house, number of stories, …), size (number of rooms or number of bedrooms, useable surface or average room size …), availability of lift and annexes (balconies, terraces, garage, lumber room …), sanitary comfort (bathroom, shower, indoor flushing ...), thermal comfort (isolation, heating, air conditioning ...), security (alarm, safety door, video surveillance …) luxury elements (fireplace, verandas, external decoration such as columns or statues, facade lighting, garden, sauna, swimming pool …), preservation status (humidity, cleanliness, façade aspect …) …

• SENIORITY: year of contract

Sometimes testing with the squares of some of the variables might be useful too.

㭐 is an independent error term following a N (0,㰰) distribution

㬠 is the parameter that measures the effect of the characteristics X on the rent (implicit price of each X)

The estimation of imputed rent will be3:

â = E (a/x)= E (exp(x βⱠ )* exp(ε )) = exp (x βⱠ )* exp (㰰2/2)

Under normal circumstances, i.e. with an estimate of a rent variable, which is specified for tenants only, not for the whole population, and without knowing pertinent factors which are not observable, the best specification of a logit model would be unable of giving satisfactory results. It would be unable to eliminate the selection bias resulting from a lack of knowledge of factors explaining tenant or owner status or the transition from one to the other.

3 As Y= ln a = ln “current rent” = X 㬠 + 㭐, then: a = exp(y)= exp(x 㬠) * exp (㭐).

If 㭐 ~ N (0, 㰰), exp(㭐) ~ log N (exp(㰰2/2), exp 㰰2 (exp 㰰2 -1))

Then: â = E (a/x)= E (exp(x βⱠ )* exp(㭐)) = exp (x βⱠ )* exp (㰰2/2)

14

3.5. Heckman method

Effectively, the sample selection problem occurs when the observed sample is not a random sample but systematically chosen from the population.

The classical example “income is only observed for employed persons but not for the ones that decide to stay at home”, can be applied to the rent. Dwelling rental is only observed for tenants paying full rental and not for other categories of the tenure status.

The Heckman method resolves the problem of selection bias. This involves the resolution of a probit model with tenant or non-tenant status as the dependent variable and conventional explanatory variables. The coefficients found for the inverse of Mills ratio are re-injected into a regression model. If the specification is "correct", the method guarantees that there will be no selection bias.

Consider a model with two latent variables yi* and di

* which linearly depend on observable

independent variables xi and zi, respectively

di* = zi

’㬰+µi

yi* = xi

’ 㬠+ 㭐i

The error terms µi and 㭐i are independently (across observations) and jointly normaly distributed with covariance 㰐㰰㭐, ( 㰐= corr (µ,㭐)) .

The two latent variables cannot be observed by the researcher. She/he only observes and indicator di when the latent variable di

* is positive. The value of the variable yi = yi

* is only observed if the indicator is 1.

di = 1 if di* >0, 0 otherwise

yi = yi* if di =1, n.a. otherwise

In other words, the first equation (the decision equation di*) explains whether an observation

is in the sample or not. The second equation (the regression equation y i*) determines the

value of yi.

The expected value of the variable yi is the conditional expectation of yi* conditioned on it

being observed ( di = 1)

E (yi /xi , zi ) = E (yi* / di

* = 1, xi , zi) = xi’’㬠+ 㰐㰰㭐 (㱠(zi

’㬰)/㩠(zi’㬰)) = xi

’ 㬠 + 㰐㰰㭐㮰(zi’㬰)

where 㮰(㬐) = 㱠(㬐)/㩠(㬐) is called the inverse of Mills ratio.

To resolve the estimation, Heckman proposed a two-step procedure which only involves the estimation of a standard “probit” and a “linear regression model”. The two step procedure draws on the conditional mean

E (yi /xi , zi ) = xi’’㬠+ 㰐㰰㭐 (㱠(zi

’㬰)/㩠(zi’㬰)) = xi

’ 㬠 + 㰐㰰㭐㮰(zi’㬰)

of the fully observed yi’

15

Step 1 is the consistent estimation of 㬰 by ML (Maximum Likelihood) using the full set of observations in the standard probit model

di* = zi

’㬰+µi

di = 1 if di* >0, 0 otherwise

We can use this to consistently estimate the inverse Mills ratio for all observations.

Step 2 is the estimation of the regression equation with the inverse Mills ratio as an additional variable

yi = xi’ 㬠+ 㬠㮰 λⱠ + ui

4. IMPLEMENTATION OF HECKMAN’S METHOD: PRACTICAL APPROACH

4.1. Model equations

Let,

yi* = xi

’ 㬠+ 㭐i the regression model,

where xi are the independent variables which determine the rental

yi is the ln ai(ln of rent)

㭐i ~ N (0, 㰰)

yi = yi* is only observed if the tenure status is ‘tenant’

Let,

di* = zi

’㬰+µi

where i ∈ “tenant” ⇔ di =0 zi’㬰+µi >0

i ∉“tenant” ⇔ di =1 zi’㬰+µi ≤ 0

under the hypothesis that µi ~ N (0, 㰰), di* = zi

’㬰+µi correspond to a probit model, so:

Pr (i ∈ “tenant”) = Pr (di =1) =㩠(zi’㬰), and

Pr (i ∉“tenant”) = Pr (di = 0) =1 -㩠(zi’㬰)

where, 㩠 is the distribution function of a normal standard.

16

4.2. Variable selection

Choosing the right variables is probably the main difficulty of this method and it is strongly dependent on national circumstances.

Here will only give some general indications instead of describing a detailed procedure. The actual work of variable choosing and interval definition for each variable depends on a detailed descriptive analysis that can only be carried out properly by people knowing well the national circumstances of their rental markets (including the legal ones) and the structure of their survey. The experience of Eurostat shows that this task may be quite time consuming.

We have already explained what the variables y and d are:

yi are the logarithms of the actual rentals paid for their main residence by all households whose tenure status is ‘tenant paying full rent’ as required by a logit model

di are binary variables giving the likehood of a dwelling being rented as required by a probit model

The explicative variables x of the logit model may be the same as proposed for the hedonic methods (please refer to section 3.4). As we have already explained, we need three types of variables: linked to dwelling location, dwelling characteristics and seniority of the dwelling occupation by the same household.

The explicative variables of the probit model z may include all variables x but they cannot be exactly the same. Usually, the best practice consists in adding some variables connected with the household features which may help to explain their decision about living in a rented dwelling or not. These variables might be one or several of the following:

– condition of recent emigrant (e.g. when household moved to that population in period of time shorter than three years),

– occupation of the reference person,

– age of the reference person,

– level of education of the reference person,

– household size,

– …

Variables x and z must be discrete. This means that continuous variables , such as those measured in time units, monetary values or surface, must be introduced in this model as range variables. This involves splitting the full range of each continuous variable in a subset of mutually excluding sub-ranges or intervals. Then we replace each original continuous variable by a set of new variables giving information about whether a value lies into each of the adjoining intervals. Since the membership to one interval depends on the membership to the other intervals of the same original variable, in practice, we must remove one membership variable per continuous variable in order to avoid autocorrelation problems in the model.

17

4.3. Practical approach: Steps and difficulties

In order to simplify the implementation of the Heckman model we are going to enumerate the main steps that must be carried out. Please note that sampling weights must be taken into account in all steps.

(a) Definition of the independent variables xi’ and zi

’

Variables xi’

Three types o f variables are proposed (for more detailed information about variables x, please refer again to section 3.4):

• LOCATION: region, climatic zone, degree of urbanization …

• CHARACTERISTICS OF THE DWELLING: type of dwelling, size, sanitary comfort, thermal comfort, security, preservation status …

• SENIORITY: year of contract4

A descriptive analysis on the basis of average rent should be done before choosing the variables and their breakdowns. This analysis might be difficult and quite time consuming.

The actual variables to be included in this model depend on the country.

Variables zi’

Before choosing the variables a descriptive analysis on the basis of % of tenants, % of non-tenants should be done.

At least a different variable than the variables chosen for xi’ is necessary in

order to get an “identifiable model”.

(b) Treatment of “missing” values of “outliers” in the tenants sub-sample

Whenever a missing value for a variable is found, it must be treated properly: either by imputing an estimate or by removing this household from the sample.

Outlier values should be detected and eliminated as well.

(c) Verification of the hypothesis of the model, i.e.

- ln a ~ Normal distribution

- no heterocedasticity

- 㭐 is symmetric

4 “Tenure discount” : Miron (1990), Borsch-Supan (1994), Hubert(1995), Arévalo and Ruiz-Castillo (2004)

18

- µi is symmetric

(d) Calculation of the probit model

(e) Calculation the inverse Mills ratio and study its significance

Please note a non-significant Mills ratio means that there is not selection bias, and therefore the Heckman’s method is not actually required. In this case Heckman’s method becomes the hedonic prices method. Eurostat is aware of at least one country were this circumstance is given.

(f) Estimation the regression model with the inverse Mills ratio as an additional variable (compensation of selection bias)

(g) Treatment of “missing” values in the “owners sub-sample”

Whenever a missing value for a variable is found, it must be treated properly before imputing of a rental.

(h) Imputation of a rental on the basis of the coefficients of the model for all households which not pay a full rental.

4.4. An alternative approach: building quality indexes of the dwelling (QI)

In some cases, when too many x variables are found, there is an alternative way of proceeding that might be more advantageous. This approach would consist in replacing all the variables xi linked to the characteristics of the dwelling by a few quality indicators.

The method5 used to synthesize the set of characteristics in a few QI is the multiple correspondences analysis.

The advantages of the QI are that it:

• avoids the problem of multi-co linearity (e.g. surface and number of rooms),

• allows to include the effect of low frequency variables,

• makes the estimation of crossed effects between variables easier,

• makes the selection of relevant information to explain the rent easier,

• reduces the volume of information to manage in relation to the level of quality of the dwelling (the number of variables is dramatically reduced).

On the other hand, the practical application of multiple correspondence analyses for deriving these QI is rather complex and time-consuming.

5 An application of this method can be seen in “ Arevalo and Ruiz-Castillo (2004)”.

19

4.5. Software implementation

4.5.1. Implementation in STATA

Stata has the Heckman method as a built in function. Therefore the development of a program for implementing this method with Stata is trivial. The following program template covers the steps c), d), e), f) and h) of the section 4.3:

heckman depvar [varlist], select ( depvar_s = [varlist_s])

where depvar = y, varlist = x, depvar_s = d and varlist_s = z.

Stata allows calculating the estimates in two-step by adding the option twostep. In that case, the intermediate results of Heckman’s model are output too.

4.5.2. Implementation in SAS

SAS does not include Heckman’s method as a built in function. However, it includes powerful tools for developing it easily. In particular, SAS provides procedures to solve probit and logit models. At the moment of writing this document, Eurostat is developing some test programs in SAS that could serve as examples for the Heckman implementation. These programs will be sent to all members of the HBS and the EU-SILC working groups by the end of 2005.

5. CONCLUSIONS AND RECOMMENDATIONS

5.1. Conclusions and recommendations for EU-SILC

This document has presented and discussed some methods for the calculation of imputed rentals for dwelling services of owners-occupiers (rent-free or accommodation rented at lower price than the market price). The selection and the application of these methods will be a national responsibility, because the variable selection and definition strongly depend on national circumstances. Nevertheless, for EU-SILC purposes Eurostat will develop SAS programmes that will allow the calculation of “imputed rent” on the basis of Heckman’s method. The Heckman’s method must be applied for those countries using the “self-assessment method” or the “user-cost method” in the calculation of imputed rent.

The SAS programs will be sent to all members of the EU-SILC working groups by the end of 2005.

20

5.2. Conclusions and recommendations for HBS

This document has presented and discussed some methods for the calculation of imputed rentals for dwelling services of owners-occupiers and tenants not paying full rent. The selection and the application of these methods will be a national responsibility, because the variable selection and definition strongly depend on national circumstances. For HBS purposes Eurostat recommends to consider these methods in the following order of priority:

(1) Stratification method as proposed by the Commiss ion Decision 95/309 (July 1995). If any country preferred to test an econometric method first, Eurostat would not be opposed to this idea.

(2) Where stratification cannot yield acceptable results, the methods of Heckman’s and hedonic prices (when Mills ratio is not significant) are recommended.

(3) Only in exceptional cases, after all the other methods have failed, Eurostat is open to discuss with the concerned country the use of self-assessment or a method derived from user-cost.

If required, Eurostat will assist a countr y in the tasks of choosing and implement ing of a suitable method for rent imputing, but in any case Eurostat considers the possibility of doing no-imputations.

21

6. BIBLIOGRAPHY

- Arévalo, R. (2001). El Mercado de la Vivienda en España, unpublished Ph.D. dissertation, Universidad Complutense de Madrid

. - Arévalo, R. and Ruiz-Castillo J. (2004). “The Rental Equivalence Approach to Non- rental Housing in the Consumer Price Index. Evidence from Spain.” Working Paper 04-17, Economic Series 04, Universidad Carlos III de Madrid, available in http://docubib.uc3m.es/WORKINGPAPERS/WE/we041704.pdf

- Börsch-Supan, A. (1986). “On West German tenants protection legislation”, Journal of Institutional and Theoretical Economies, 142 (2), pp. 380-404

- Börsch-Supan, A. (1994): Aging in Germany and the United status:Internacional Comparisons in D.A Wise (ed.), Studies in the Econimics of Aging, University of Chicago Press: Chicago.

- Eurostat; “Task Force Report on Alternative Estimation Methods for Dwelling Services in the Candidate Countries”; Meeting of the Working Parties on National Accounts and Purchasing Power Parities held in Luxembourg on the 19 November 2002; Doc NA-PPP 02/6

- Heckman J. (1979), “Sample selection Bias as a specification error”; Econometrica, Vol. 47, No. 1, January 1979

- Hubert, F. (1995): “Contracting with costly tenents”, Regional Science and Urban Economics 25, 631-654.

- Miron, J.R. (1990): “Segurity of tenure, costly tenants and rent regulation”, Urban Stadies, 27, 2, 167-184.

- Marchand O. and Skhiri E. (1995), “Prix hédoniques et estimation d’un modèle structurel d’offre et de demande de caracteristiques”; Économie et Prévision, nº 121, 1995-5

- Markier R. (2003), “Imputation de loyers fictifs aux propiétaires occupants. Quel impact sur les contoursde la population pauvre ?”; INSEE, Série des Documents de Travail de la Direction des statistiques démographiques et sociales, NºF0309, December

- Peña D. and Yohai V. (1995), “Detection of Influential Subsets in Linear Regression by using an Influence Matrix”; J.R. Statistical Society B, 57, No 1

- Rosen S. (1974), “Hedonic Prices and Implicit Markets: Product Differentiation in Pure Competition”; J. Pol.Econ., January/February

- Schmidheiny K. (2004), “Limited Dependent Variable Models”; Université de Lausanne, HEC, Applied Econometrics II