Document of The World Bank

Report No: 27428-RO

IMPLEMENTATION COMPLETION REPORT(SCL-43190)

ON A

LOAN

IN THE AMOUNT OF US$ 30 MILLION

TO THE

GENERAL INSPECTORATE OF COMMUNICATIONS AND INFORMATION TECHNOLOGY, ROMANIA

FOR A

TELECOMMUNICATIONS REFORM AND PRIVATIZATION SUPPORT PROJECT

March 9, 2004

Infrastructure and Energy Sector UnitEurope and Central Asia Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective December 5, 2003)

Currency Unit = Lei 1000 Lei = US$ 0.03

US$ 1 = 33,213 Lei

FISCAL YEARJanuary 1 to December 31

ABBREVIATIONS AND ACRONYMS

EBRD - European Bank for Reconstruction and DevelopmentEIB - European Investment BankEU - European UnionGICIT - General Inspectorate of Communications and Information TechnologyGOR - Government of RomaniaICB - International Competitive BiddingITU - International Telecommunications UnionMOCIT- Ministry of Communications and Information TechnologyNRAC - National Regulatory Agency for CommunicationsNSMS - National Spectrum Managament SystemSEMOS - Spectrum Engineering and Management Operation SupportUSAID - United States Agency for International DevelopmentWTO - World Trade Organization

Vice President: Shigeo KatsuCountry Director Anand K. SethSector Manager Henk Busz

Task Team Leader/Task Manager: Varadarajan Atur

ROMANIATELECOMMUNICATION REFORM AND PRIVATIZATION SUPPORT PROJECT

CONTENTS

Page No.1. Project Data 12. Principal Performance Ratings 13. Assessment of Development Objective and Design, and of Quality at Entry 24. Achievement of Objective and Outputs 45. Major Factors Affecting Implementation and Outcome 146. Sustainability 157. Bank and Borrower Performance 158. Lessons Learned 169. Partner Comments 1710. Additional Information 19Annex 1. Key Performance Indicators/Log Frame Matrix 20Annex 2. Project Costs and Financing 21Annex 3. Economic Costs and Benefits 23Annex 4. Bank Inputs 24Annex 5. Ratings for Achievement of Objectives/Outputs of Components 26Annex 6. Ratings of Bank and Borrower Performance 27Annex 7. List of Supporting Documents 28

Project ID: P008788 Project Name: TELECOMMUNICATIONTeam Leader: Varadarajan Atur TL Unit: ECSIEICR Type: Core ICR Report Date: March 9, 2004

1. Project DataName: TELECOMMUNICATION L/C/TF Number: SCL-43190

Country/Department: ROMANIA Region: Europe and Central Asia Region

Sector/subsector: Central government administration (100%)Theme: Infrastructure services for private sector development (P); Regulation

and competition policy (P)

KEY DATES Original Revised/ActualPCD: 09/15/1992 Effective: 09/29/1998 09/21/1998

Appraisal: 01/21/1997 MTR: 09/30/2000 07/27/2001Approval: 04/28/1998 Closing: 12/31/2002 06/30/2003

Borrower/Implementing Agency: GENERAL INSPECTORAT OF COMMUNICATIONS/GENERAL INSPECTORAT OF COMMUNICATIONS

Other Partners:

STAFF Current At AppraisalVice President: Shigeo Katsu Johannes LinnCountry Director: Anand K. Seth Kenneth LaySector Manager: Henk Busz Franco BatzellaTeam Leader at ICR: Varadarajan Atur Varadarajan AturICR Primary Author: Elliott Hurwitz; David Satola;

Varadarajan Atur

2. Principal Performance Ratings

(HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HL=Highly Likely, L=Likely, UN=Unlikely, HUN=Highly Unlikely, HU=Highly Unsatisfactory, H=High, SU=Substantial, M=Modest, N=Negligible)

Outcome: S

Sustainability: L

Institutional Development Impact: SU

Bank Performance: S

Borrower Performance: S

QAG (if available) ICRQuality at Entry: S

Project at Risk at Any Time: No

Introduction

Starting in 1996, the Government of Romania (GOR) began to take major steps to liberalize and reform its telecom sector, licensing a number of private firms to provide mobile telephone service. It broadened these actions by its commitments to the World Trade Organization (WTO) and the European Union (EU), the latter in the context of preparing for EU accession in 2007, that would further reform the sector. The Telecommunications Reform and Privatization Support Project (TRPSP) was designed to support and facilitate the Government’s program of telecom reform.

3. Assessment of Development Objective and Design, and of Quality at Entry

3.1 Original Objective:The objectives of the TRPSP were: (a) to assist Romania in reforming the telecom sector, including privatization of existing public sector telecom facilities; (b) to provide the necessary infrastructure to enable GICIT to maximize the productive use of the radio frequency spectrum and to improve frequency assignment criteria for further expansion of wireless communications services provision by the private sector; and (c) to support GICIT in performing regulatory functions over all networks and services in the telecom sector.

3.2 Revised Objective:The objectives were not revised.

3.3 Original Components:The project consisted of two components:

1. Sector Reform (SAR cost US$1.8 million):a) Development of policy, legal and regulatory frameworks:

Policy Framework: Further develop and support the basic policy framework, building on Romania's agreement with the WTO and to ensure consistency with the EU acquis communitaire. The effort was intended to focus on such key areas as industry structure, competition policy, ownership policy, pricing policy, regulatory policy, and the extent to which existing entities could participate in various service areas.Legal Framework: Provide legal underpinning for transparent regulation and the chosen policy alternatives. Regulatory Framework: Strengthen technical regulation (quality of service, typeapproval of equipment, frequency spectrum management) and economic regulation (prices, license fees).

b) Privatization of Rom Telecom

2. Regulatory Agency Development (SAR cost US$34.2 million)This component focused on the initial concept of strengthening the General Inspectorate of Communications (GIC) (later organized into the General Directorate of Communications and Information Technology, GICIT) as the single regulatory body for the telecom sector, including modernization of its infrastructure for radio spectrum management and monitoring. A program was designed and implemented under the TRPSP, consistent with the overall sector policy

- 2 -

framework, with the following governing principles:

a) Independence of GICIT from both entities which would be regulated and policy makers;

b) Structure to define the legal, organizational and administrative arrangements under which GICIT function as an independent body;

c) Autonomy, consisting of both operational autonomy (e.g. appointment of regulators on a transparent basis and meeting minimum qualifications) and financial autonomy (e.g. independence from the national budget and financing of GICIT from regulatory services);

d) Area of Responsibility defining GICIT's purview of regulation as well as the necessary instruments (e.g. economic regulation, technical regulation, interconnection regulation, service quality regulation, monopoly operator regulation, promotion of competition, spectrum assignment and licensing, etc.)

It was also envisioned that the TRPSP would develop a National Spectrum Management System (NSMS)—a complete spectrum management system—with three main subcomponents:

a) Installation of hardware and operating system software. This was to be achieved by GICIT, and operated on twin, redundant servers.

b) Installation of application software for the Radio Frequency Spectrum Management System. This would enable the authorities to assure that allocation of frequencies would be done in accordance with International Telecommunications Union (ITU) guidelines, and would also be used to evaluate new applications for potential interference with existing services, and compliance with national regulations and cross-border frequency coordination. It would further have functionality for administrative and financial uses.

c) Frequency monitoring system. To ensure efficiency and functionality, a system needs to be periodically checked to assure that transmissions are in conformance with their allocations and consistent with the provisions of their license and to assess the actual occupancy of the radio frequency spectrum. For this purpose, the program provided for the purchase of 22 fixed monitoring stations, 10 mobile stations, 15 transportable stations, 1 high-frequency station, and enabling software.

3.4 Revised Components:The components were not revised.

3.5 Quality at Entry:Quality at entry was satisfactory. The TRPSP was specifically cited in the 1997 CAS, where it directly addressed the CAS objective of “providing needed infrastructure and increase private sector role in the telecom sector.” The project emerged from a close collaboration between the Bank and the government, which was keen to privatize Rom Telecom and thereby gain privatization revenues and at the same time mobilize private capital in modernizing the sector. The National Spectrum Management System (NSMS) was essential to upgrade the country’s rudimentary spectrum management and monitoring capabilities at a time when expanded use of the spectrum was envisioned. The original project concept in 1992 involved investment financing

- 3 -

support to expand Rom Telecom's fixed telephone network, but since such financing became available from EIB and EBRD, the Bank's project preparation activities were suspended for some time, and eventually evolved into a technical assistance project. The TRPSP thus was comprised of an appropriate mix of policy and investment components sufficient to facilitate the fundamental transformation and modernization of the sector. A strength of the project design was the provision for an international consultant to provide full-time on-site technical management for the NSMS. A 1999 QAG Rapid Supervision Report stated that the project design was sound, the project was fully ready for implementation at approval, and that Borrower commitment was strong.

4. Achievement of Objective and Outputs

4.1 Outcome/achievement of objective:Introduction and Overview

Recognizing that most major relevant objectives were achieved, albeit with some shortcomings & delays, the TRPSP outcome is rated as satisfactory on balance. Significant positive outcomes include:

• Rom Telecom was privatized, with 35% of its shares sold in 1998, and an additional 19% in 2002• The number of wireless subscribers rose from 640,000 in 1998 to around 6.4 million in 2003, a ten-fold increase over 5 years (see figure 3). This provided effective competition for fixed wire telephony, improved service for millions of users, and introduced new services• The market for voice telephony was liberalized on schedule on January 1, 2003 in accordance with both EU and WTO commitments, which permitted access to numerous new entrants• Comprehensive EU-compliant telecom legislation was enacted which provided the basis for provisionally closing Chapter 19 of the acquis communitaire, in November, 2002. Romania thereby became the first country (including existing EU members) to meet the new EU requirements.• A competent and independent regulatory agency, the National Regulatory Agency for Communications (NRAC) in advance of liberalization. The agency is also financially independent, funded by a tax on 0.2% on the revenues of operators.

Not all of these outcomes are directly attributable to the TRPSP. However, this ICR argues that the TRPSP either achieved, or was a significant facilitating factor, in these highly positive outcomes, and that consequently the benefits achieved by the TRPSP considerably outweigh its shortcomings (see discussion of attribution at the end of section 4.1).

Regulatory Structure: It is important to note that while the Bank recommended that the GoR establish a single regulator, the GoR eventually established a framework involving two regulatory agencies, GICIT and NRAC. The original concept is depicted in Figure 1, below, while that which was eventually implemented is shown in Figure 2. This arrangement is discussed in more detail in the next paragraph.

- 4 -

GORPrime Ministry

Cabinet

Ministry of Communications

GIC•Economic/ competition regulation•Licensing•Technical regulation (frequency monitoring, etc)

Operating companies

Figure 1:Regulatory Structure as

Conceived in SAR

Ministry of Communications and Information Technology•Spectrum and numbering policy•Licensing of spectrum usage

GICIT•Technical Regulation•Monitor spectrum•Assign frequencies for non-governmental use

Figure 2:Regulatory Architecture

as Implemented

GORPrime Ministry

Cabinet

NRAC•Economic/competition regulation•Manage authorization process•Additional licensing functions

Operating Companies

The TRPSP originally envisioned that the single regulator would be the General Inspectorate of Communications (GIC), the predecessor of GICIT, an arrangement agreed between the GoR and the Bank. However, the subsequent government was less committed to this arrangement than its predecessor, and after an intermediary stage, it decided on the following allocation of responsibilities:

- 5 -

• MoCIT: responsible for policy development and implementation (with the exception of some residual regulatory functions noted below)

• NRAC: responsible for administering the regulatory regime, as well as economic and competition aspects of regulation

• GICIT: responsible for spectrum management, including assignment of frequencies to nongovernmental spectrum users, and spectrum monitoring

After careful consideration, the Bank decided that this was an acceptable arrangement. In assessing the current structure it should be recognized that what is critical to the regulatory function is that the regulator can perform its functions independently from the Ministry and from the system operators, and has financial autonomy. Romania has largely achieved this status, and has received international recognition of this achievement. In November, 2002, in provisionally closing Chapter 19 of the acquis communitaire, the EU signaled that it did not have concerns over Romania’s regulatory scheme, provided that the residual issue of ownership and regulation within the same agency is dealt with prior to accession (see next paragraph). The June, 2003, IBM report states (p. 103): “NRAC is an independent entity with regard to structure, operations, and financing.” (see section 10 for full report citation)

However, there is a residual issue. While NRAC presently handles the majority of regulatory functions, MoCIT still has responsibility for frequency allocation. At the same time, MoCIT still owns 46% of Rom Telecom and other telecom infrastructure assets including National Radiocommunications Company (Rom Radiocom). The GoR has agreed to review its regulatory structure prior to its prospective entry into the EU in 2007, and resolve this remaining issue. When this is completed, Romania’s regulatory framework will be completely consistent with EU requirements and international good practice.1

In utilizing two regulatory agencies, the GoR was following the practice extant at that time of such countries as the United Kingdom, which also had two agencies performing these functions. However, the UK is a mature market economy, with established institutions for dispute resolution, including independent judiciary, and recent legislation in the UK (adopted subsequent to the adoption of the legal framework in Romania) has now largely abandoned this bifurcated institutional structure.

Achievement of Specific Objectives

Objective (a): Assist Romania in reforming the telecom sector, including privatization of existing public sector telecom facilities.

The Transformation of Romania’s Telecom Sector

During the period of the TRPSP, the Romanian telecom sector was not so much reformed as transformed, with explosive growth in mobile services, complete decontrol of the sector, and improvement in nearly all indicators of the breadth, quality, and efficiency of service. Achievement of objective (a) was highly satisfactory.

As noted earlier, Rom Telecom is now private; OTE of Greece purchased 35% of its shares in

- 6 -

1998, and a further 19% in 2002. The remaining 46% is owned by MoCIT. The company has made progress in service quality, productivity, and penetration (described below).

In accordance with its WTO obligations, on January 1, 2003, Romania completely liberalized the telecom sector, removing Rom Telecom’s remaining exclusive rights to offer voice telephony and leased wire services, and implemented a simplified procedure to permit new operators to enter the market. To start a telecom business, a firm need only notify the regulator, NRAC, which then acknowledges that notification has been received, and the required data provided. In the case of wireless, the new entrant may then apply to GICIT to be assigned part of the spectrum.

Fixed Line Service: Rom Telecom is still the predominant provider of fixed line service, with 97% of this service. However, 2 new fixed line operators are in an advanced stage of preparation and are expected to begin offering service in December, 2003. In addition, the electric transmission utility and the railway—both state-owned—have formed subsidiaries to offer telecom services and both have recently installed advanced fiber optic telecom networks along their rights of way. The electric transmission company upgraded its 8,700 km telecom network (by comparison, Rom Telecom has an fiber optic network of 11,000 km) with the assistance of a loan from the EIB and EBRD (the total loan for upgrading all infrastructure was US$175 million); the firm has now received a grant of US$186,000 from USTDA to delineate the characteristics of the telecom services it can best offer using its network. The railway has interconnected its 3,600 km telecom network with Hungary, and will be interconnected with Bulgaria and Turkey starting in January, 2004. The railway is currently selling telecom services to a large number of business customers—mostly suppliers, affiliates, and other transportation companies—and is also selling video conferencing services to two affiliates. The railway telecom network was recently upgraded with the support of a US$30.5 million loan from the World Bank (the Bank’s overall Railway Rehabilitation Project was US$195 million, with EBRD and PHARE contributing additional funding).

Mobile Service: The number of mobile telephones increased approximately ten-fold from 1998 to 2003, as shown in Figure 3, and mobile service became a serious competitor for fixed line carriers. At the time of the ICR mission the mobile subsector consisted of two GSM carriers that held around 96% of the market, one additional GSM carrier in bankruptcy (a Rom Telecom subsidiary), and a CDMA carrier that began operations in 2001 and is growing quickly. The mobile sector is very dynamic; one of the two large carriers reported that it was gaining more than 100,000 new customers per month.

The commercial boom of mobile systems in Romania went far beyond the most optimistic predictions. For example, the SAR (1998) predicted that there would be 1.2 million mobile phones by 2008. This dramatic expansion in the use of the spectrum made upgrading the NSMS much more critical, and a number of participants in the market observed that without the enhancement in frequency monitoring made possible by implementation of the NSMS—even though the capacity developed was less than envisioned—wireless expansion could not have proceeded as rapidly as it did.

The large wireless firms also offer a package of services to medium and large businesses that

- 7 -

includes digital transmission of all of their traffic—voice, fax, and data—both within and outside the country. Customers for such a package would have a microwave antenna installed on a company building, and then route all of their transmissions through the wireless firm’s network.

Figure 3: Number of Mobile Telephones, 1998-2003

Number of Mobile Telephones, 1998-2003, in 000s

0

1000

2000

3000

4000

5000

6000

7000

1998 1999 2000 2001 2002 2003 (est,11/03)

Source: GICIT (1998-2002), project staff estimate (11/03)

While there were approximately 6,400,000 mobile phones in use in November, 2003, because some individuals have more than one mobile phone, the number of mobile phone users was lower. In discussions with management of the large mobile carriers, regulators, and government officials, a consensus estimate of around 5,600,000 mobile users emerged. Thus the mobile penetration rate was around 25%, dramatically higher than the past few years, but still the lowest of 13 EU candidate countries (IBM report, p. 45). From 16% in 1998, the penetration rate of fixed line telephones rose modestly to 19.4% in 2002, also the lowest of the EU candidates. Market participants expect mobile penetration to continue to increase—although the rate of increase will slow—thus narrowing the gap between Romania and other EU candidates.

Additional Indicators of Telecom Sector Progress: Additional data reflecting change in the telecom sector include:

• The average call completion rate for fixed line service rose from 50% in 1998 to 90% in 2003• Rom Telecom reduced the number of its employees from 50,000 in 1998 to 23,500 in 2003• The waiting list for fixed line installation decreased from 0.97 million customers in 1998 to 0.54 million in 2002 • The number of households receiving cable TV service rose from 2.4 million in 1998 to 3.3

- 8 -

million in 2002, while the proportion of “households passed” reached 4.1 million, or nearly 80% of all households• The number of cable channels offered in 2003 was: urban areas, 35 (standard service) and 50 (premium service); rural areas, 16• There are now more than 200 internet service providers. The proportion of the population using the internet increased from a very low figure in 1998 to 14% in 2003• With the deregulation of voice telephony in January, 2003, the price of international calls began to fall substantially; by November, 2003, it was estimated that the average decrease was around 35 to 40%, compared with the same period last year, although some new operators offered prices as much as 80% lower• In addition, starting in 2003, Voice over Internet Protocol (VoIP) service began to be offered at a large number of establishments, with charges for international calls reported to be 5 US cents or less per minute• Mobile service coverage includes areas where approximately 90% of the population lives (but 70% of the land area of the country)• The number of pay telephones in the country grew from 42,000 at the end of 2001 to over 51,000 at the end of 2002• An updated National Frequency Register was issued in 1999, 2001, and again in 2003 (extended to 275 GHz), to make it more consistent with the European Common Allocation Table.• Although data were not available, interviews with operators and regulators indicated that interference was not a significant problem, and did not impede their operations

It should be noted that Rom Radiocom remained state-owned despite extensive efforts to divest it. This firm owns substantial microwave network infrastructure, used mainly for the radio and television broadcast subsector. While the privatization of the firm was not an explicit objective of TRPSP, a privatized National Radiocommunications Company might be in a position to offer significant competition to Rom Telecom in the fixed wire area.

Objective (b): (NSMS) Provide the necessary infrastructure to enable GICIT to maximize the productive use of the radio frequency spectrum and to improve frequency assignment criteria for further expansion of wireless communications services provision by the private sector

Most of the main components of the NSMS have been installed and are operational, and as described below, are producing important benefits. However, deficiencies in system performance—still being addressed by the contractor at the time of the ICR mission—result in a rating of objective (b) (the NSMS) as unsatisfactory.

The GICIT computer system was delivered and began operation in 1999 with 2 servers and 4 regional offices linked to headquarters via leased lines. The initial functionality was in office automation, which improved agency operational efficiency and customer service. Subsequently the NSMS software was installed and in 2001 the system began use as intended, for compliance monitoring, location of illegal radio activity, and spectrum occupancy measurement (though with lower performance than anticipated). To date, the NSMS has detected many cases of illegal frequency use. Deployment of the system is summarized in Table 1, below.

- 9 -

However, system operation is not as efficient or effective as intended. While the receivers are functioning acceptably in most respects, testing by GICIT determined that they were producing an unacceptable rate of false positive signals, i.e., indicating that a problem is present when it is not. This necessitates more work by the system operator, and the contractor has agreed to replace the existing receivers with upgraded ones in the spring of 2004. Also, the financial management and administrative modules of the system are not yet in use. Finally, while the individual parts of the system are working as described above, the system as a whole has not been operated or tested. The effectiveness of the parts of the NSMS that are currently in use is estimated to be around 30% of the efficiency and effectiveness that can be achieved with the fully functional NSMS. While these shortcomings are likely to be remedied within the next 6-9 months, until this has been accomplished, NSMS cannot be regarded as satisfactory.

Table 1: Main Components and Status of Radio Monitoring SystemItem Contract

QuantityDelivered Installed and

operational*Fixed stations 22 22 22Mobile stations 10 10 10

Transportable stations 15 15 15HF station** 1 1 0

NSMS application software

1 1 1

*As explained in the text, individual monitoring stations are operational, but functioning at a lower level of efficiency than required in the contract. Also, the NSMS is not yet operating as an integrated system.**The government has encountered zoning issues on the land intended for this station.

Difficulties and delays were encountered throughout the design and implementation of NSMS. First, during the post-effectiveness design review, GICIT negotiated more than 30 design changes with the contractor. The Bank reviewed these, and found that in many cases they were unnecessary and overly complex, and raised technical risk. The Bank stated that it would not finance them, however, GICIT decided to go ahead and finance them with their own funds. The process of introducing design changes, and the attendant Bank review, caused a delay of around 7 months. Second, after some components had been delivered and others were being manufactured, the contractor had a fire (March, 2000), which damaged components in the plant and caused a second major delay to manufacture and recondition monitoring system components. Testing following delivery established that both components that had been delivered before the fire and those that had been repaired were substandard and unreliable. Consequently, many components were completely replaced with new models, which introduced more delay and some software incompatibilities.

The delays in bringing NSMS to a fully satisfactory status were serious, and delayed and diminished the flow of benefits the system was intended to provide. Part of the responsibility for the delay lay with GICIT, which did not take full advantage of the on-site presence of a technical consultant funded by the TRPSP to assist GICIT in implementing NSMS. Reorganization of GICIT and personnel turbulence within the agency, as well as inconsistent support from the

- 10 -

government, were also important factors in GICIT's performance. Additional responsibility lies with the system contractor which--beyond the force majeure circumstances of the fire in their facility--continued to provide faulty components. Efforts by GICIT to work with the contractor to obtain satisfactory components distracted the agency from other important responsibilities.

In large measure as a result of delays in bringing NSMS on line, several private firms utilizing the spectrum developed, at their own expense, some functionalities that NSMS was not yet providing, i.e., ensuring that there was little or no interference on the frequencies on which they operated. While it was unfortunate that this effort was necessary, these actions demonstrated the importance of avoiding interference to these operators and confirmed the value of the NSMS component of the project.

Even though the performance of the NSMS is rated unsatisfactory, the system is producing substantial benefits (Table 2 below). As noted, the NSMS has detected a significant number of cases of illegal spectrum use, and appropriate actions were initiated. Even at its current level of functioning, the NSMS can monitor the spectrum and provide other services much more efficiently and effectively than the manual systems it replaced; spectrum monitoring prior to initiation of NSMS was rudimentary. GICIT has been validating the existing license database against actual use, identified unoccupied frequencies, and thereby enable services growth in crowded frequency bands. GICIT has also verified that proposals for usage conform to protocols for cross-border frequency coordination. Considering the explosive growth in the use of the spectrum from 1998 to the present, a number of service operators and regulators interviewed stated that they did not believe that an acceptable level of quality could have been maintained using the antiquated pre-NSMS systems and practices.

Table 2: Utilization of NSMS (2002)1. Number of times NSMS utilized in license authorization 5,9362. Number of times NSMS utilized in assignment of frequency band: of which: Number of times NSMS utilized in detecting frequency interference

1,240

178

3. Number of times NSMS utilized in application processing 6,2064. Number of times NSMS utilized in identifying sources of interference

131

5. Number of times NSMS utilized in monitoring compliance with license authorization

3,850

Source: GICIT

Objective (c) Support GICIT in performing regulatory functions over all networks and services in the telecom sector.

TRPSP assistance to the three agencies involved with sector regulation was significant, although not as extensive as envisioned because of changes in the regulatory structure and the availability of grant-funded support from other donors. Achievement of objective (c) is assessed as satisfactory.

- 11 -

As noted earlier, after a delay, the government adopted a regulatory structure in which NRAC holds principal responsibility for economic regulation, competition concerns, and administration of the regulatory regime, MoCIT is responsible for policy development and implementation, and certain residual regulatory functions, while GICIT is principally charged with technical responsibilities. Consequently, Bank assistance related to regulatory matters was redirected from GICIT to NRAC and MoCIT.

TRPSP provided critical support to MoCIT and NRAC for development of an EU-compliant legal framework—the Framework Law and others—later modified and approved by the Parliament, which established (1) the legal framework for the sector; and (2) conditions of access to public communications networks and interconnection between these networks. This provided the basis for provisionally closing Chapter 19 of the acquis communitaire, which Romania achieved in November, 2002, thereby becoming the first country (including existing EU members) to meet the new EU requirements. It should be emphasized that once the issue of the sector’s regulatory structure was clarified, the Bank moved quickly to provide support in drafting and refining the law that permitted NRAC to be created and start operations in September, 2002, only 3 months before deregulation.

After NRAC began operations, the GoR preferred to use grant funding from USAID (and later the EU), rather than TRPSP funds, to strengthen agency capacity. USAID provided considerable training to NRAC (and to a lesser extent, to MoCIT) on the structure of regulatory agencies, their role in the marketplace, and required staffing. A study tour to pertinent US agencies was organized for 6 top regulatory officials, and USAID continued to provide specialized advice on an informal basis through the first half of 2003. USAID grant assistance on regulatory matters totaled US$0.5 million through June, 2003. Starting in late 2002, the EU PHARE program provided advice and assistance on overall agency operations, economic concentration and competition, access to networks and interconnection, and facilitating access to facilities used by multiple operators (e.g., transmission towers, rights of way).

Due to the change in the regulatory structure and the GoR preference to use grant assistance, several TRPSP technical assistance subcomponents to GICIT achieved less impact than envisioned, or were dropped. The Spectrum Engineering and Management Operation Support (SEMOS) contract advised GICIT on monitoring and assignment of the spectrum, fees for spectrum use, spectrum management policy, and GICIT organization. While the contract strengthened the agency, SEMOS’ impact was less than expected because of reorganization of the agency and personnel turbulence within GICIT in the context of changing GoR views of the regulatory structure. And in light of the grant assistance provided by USAID and the EU, TRPSP plans to provide technical assistance in the regulatory and licensing areas were not undertaken.

Attribution

The TRPSP was not the principal driving force behind the transformation of the telecom sector. Rather, it was a timely facilitating instrument—an enabler—that permitted the sector’s transformation to occur more rapidly and more smoothly than would otherwise have been possible, especially during the interim period before the current government came into power.

- 12 -

There were four principal drivers behind the sector’s transformation:

1. The privatization of Rom Telecom2. Key government policy decisions in 1996 and 1997 to open the mobile market to new private entrants3. The country’s obligation to WTO to liberalize the telecom market in January, 20034. Romania’s candidacy to join the EU, which led the country to adopt EU-compliant legislation and regulatory framework.

The privatization of Rom Telecom was an explicit TRPSP objective. Privatization advisors were appointed within the framework of the project, and terms for the sale to OTE were concluded within 3 months of the signing of the loan.

The TRPSP project documents describe the government’s ambitious reform strategy for the sector. They also note the country’s agreement with WTO and its prospective EU candidacy, and TRPSP explicitly set out a program of complementary actions designed to facilitate progress toward fulfilling the country’s obligations in these areas.

TRPSP provided support for an EU-compliant legal framework—the Framework Law—which provided the legal foundation for the transparent regulation and implementation of the chosen policy alternatives by the principal regulatory agency, NRAC. As noted, once the regulatory structure was clarified, timely Bank legal assistance was critical in enabling NRAC to start operations. It is not evident that other resources were available that could have achieved this goal during this critical time.

Finally, the TRPSP also provided hardware, software, and technical advice to assist GICIT in development of the NSMS. Considering the explosive growth in the use of the spectrum from 1998 to the present, service operators and regulators interviewed during the ICR mission stated that they did not believe that an acceptable level of quality could have been maintained using the antiquated pre-NSMS systems and practices.

In summary, while the TRPSP was not the main driving force behind the transformation of the sector, it was a significant facilitating mechanism in the rapid transformation of the sector.

4.2 Outputs by components:Output by components has been described in section 4.1.

4.3 Net Present Value/Economic rate of return:N/A

4.4 Financial rate of return:N/A

4.5 Institutional development impact:Institutional development impact was substantial:

- 13 -

• Rom Telecom was privatized• The telecom sector was completely liberalized, with simplified rules for entry of new

firms• The NSMS, although not yet operating at the level envisioned, substantially upgraded

the country’s capacity in spectrum management and monitoring• The Framework Law and related instruments were enacted, a signal achievement

which enabled the country to provisionally close Chapter 19 of the acquis communitaire achieving another EU accession milestone

• The key regulatory agency, NRAC, was established, funded, and began operation prior to liberalization. The agency met the SAR-specified criteria of independence, autonomy, structure, and clear area of responsibility

• Basic regulations were issued by NRAC• GICIT capacity was strengthened

1 See, for example, “Telecommunications Regulation Handbook,” World Bank infoDev program, p. 1-6.

5. Major Factors Affecting Implementation and Outcome

5.1 Factors outside the control of government or implementing agency:The availability of grant funding from USAID and the EU reduced the utilization of TRPSP technical assistance. Project progress was also hindered by the poor performance of the NSMS contractor. Beyond the force majeure circumstances of the fire in their facility, the firm supplied substandard components.

5.2 Factors generally subject to government control:The change in government's commitment to a single regulatory agency and subsequent shifts and uncertainty regarding the regulatory structure, delayed progress in establishment of a regulatory agency. Also, inconsistent government support for GICIT hampered its progress.

5.3 Factors generally subject to implementing agency control:Major delays in bringing NSMS to a fully satisfactory status diminished the benefits from the system. Part of the responsibility for the delays lay with GICIT, which--hindered by several reorganizations and personnel turbulence--did not make satisfactory progress despite the on-site presence of a technical consultant funded by TRPSP.

5.4 Costs and financing:Procurement of software and equipment for the NSMS was less costly than originally estimated, and consequently, US$7 million of the original loan amount of US$30 million was canceled on March 1, 2000. Of the remaining US$23 million, US$13.55 million was disbursed, leaving around US$9.45 million undisbursed at project close. The funds not utilized represent mainly support envisioned for regional laboratories and workshops, and for capacity building in the regulatory area, as discussed earlier. After the regulatory structure was clarified and the new role of GICIT defined, the Bank and the GoR could not reach agreement on the scope of these facilities. The Bank proactively identified and discussed with MoCIT certain additional areas for utilizing the loan savings; however, MoCIT did not act on this proposal, but instead developed a different set of proposals only six months before loan closing. The Bank's assessment was that these proposals did not reflect a consistent MoCIT strategy, and lacked the strong rationale for

- 14 -

Bank financing of the original proposal.

6. Sustainability

6.1 Rationale for sustainability rating:Sustainability of the benefits achieved is likely. NRAC and GICIT are functioning effectively, and are funded independently, utilizing fees levied on operators. Support for reform from MoCIT continues to be strong. As Romania moves toward EU accession in 2007, it is very unlikely to backtrack on the reforms it has realized. Liberalization of the sector has brought in powerful private firms which have invested large sums, and which are moving rapidly to expand their customer base and introduce new services. These firms, and other private actors, are a powerful voice for continued reform. And the transformation of the sector has brought improved service to millions of customers, who are unlikely to tolerate a deterioration of service.

6.2 Transition arrangement to regular operations:Transition to regular operations is essentially complete, with agencies operating on a routine basis. The NSMS is likely to achieve its intended level of performance within 6-9 months.

7. Bank and Borrower Performance

Bank7.1 Lending:As described earlier, quality at entry was satisfactory; the TRPSP was well designed and well-timed to support and facilitate the transformation of the sector. Government support for sector reform, evident during preparation and identification, remained strong throughout the project. The vehicle of an investment loan that also included some policy components worked well.

7.2 Supervision:Supervision dealt effectively with the change of government in 2000, and the change in the regulatory structure preferred by the new government. While the Bank favored a single agency as both technical and economic regulator—and advocated that position—when the government determined its new structure, the Bank worked with the government, stipulated criteria for an acceptable regulatory regime, and eventually determined that the GoR arrangement was satisfactory. The Bank team's skills were well-suited to the project, and a 1999 QAG Quality of Supervision Assessment rated supervision as satisfactory, with the Task Team's focus on development impact found to be highly satisfactory.

As noted above, the project closed with US$9.3 million undisbursed. There was a substantial dialogue between the Bank and the MoCIT on potential uses for these funds, but the Bank elected not to fund the proposals put forth by the government, judging that they could be implemented either using funds from other projects or from the GoR.

Romania’s impetus to reform its telecom sector was heavily influenced by its status as a candidate for EU accession. The Bank formed a close working relationship with the EU, and the comprehensive legal framework developed for the sector also satisfied the requirements of chapter 19 of the acquis communitaire. The Bank also coordinated closely with USAID in the provision of technical assistance in the regulatory area.

- 15 -

7.3 Overall Bank performance:Overall Bank performance was satisfactory.

Borrower7.4 Preparation:The Borrower was very active and helpful in preparation. Prior to project initiation, policy decisions to permit the entry of new mobile operators set the stage for later progress. When the TRPSP was prepared, the then Minister of Communications, under whose guidance the first 35% of Rom Telecom was privatized in 1998, played an active role in the project design and envisioned that GICIT would be the sole regulatory agency for all telecommunications services in Romania.

7.5 Government implementation performance:The new government preferred a different regulatory structure, and after a transitional period, developed the system now in place. As noted earlier, these shifts and uncertainties over the regulatory structure delayed progress in the establishment and build-up of capacity of a regulatory agency. For around 2 years during the TRPSP, relations between GICIT and MoCIT were strained, and project progress slowed. However, the government followed a consistent line of strong support for reform, established a regulatory structure that was internationally recognized as satisfactory, and generally took actions that permitted the transformation of the telecom sector as described earlier. On balance, government implementation performance is assessed as satisfactory.

7.6 Implementing Agency:As noted earlier, during the course of the project GICIT’s mandate changed several times, it was reorganized, and it experienced considerable personnel turbulence. As noted earlier, this adversely affected the impact achieved by the project's SEMOS contract. The closing date of the project was extended once for up to six months. Better performance by GICIT could have reduced the delays experienced by the project, and hence increased the benefits achieved.

7.7 Overall Borrower performance:Taking the above factors into account, overall Borrower performance was satisfactory.

8. Lessons Learned

Stability of policy and agency structure contributes to project achievement: At lappraisal, the country's policy development structure seemed clear. However, the subsequent government preferred a different arrangement, and considerable time passed before the current government and the new structure was clarified, and considerable TRPSP momentum was lost as a result.

Whether new agencies are more suitable for regulatory functions: TRPSP envisioned lthat GICIT, heretofore a technical agency, was also capable of being transformed and performing the administrative, legal, economic, and competition aspects of regulation. In retrospect, the project should have given equal consideration to the establishment of a new agency—which would work in close collaboration with the technical agency—to perform those functions.

- 16 -

A competent spectrum management and monitoring system is necessary for an lexpanding, modern telecom sector: Spectrum management and monitoring prior to TRPSP was rudimentary, and would not have been sufficient to support the expansion of wireless services that occurred. The increased capabilities accorded by the NSMS—although not yet as robust as envisioned—were essential during a period of rapid growth in the use of the spectrum.

Close donor coordination is key to reform progress: The Bank maintained very close lrelations with the EU and USAID, both of which were active in providing assistance to the sector. This ensured that no duplication of effort occurred, and facilitated provision of timely advice. On the other hand, the availability of grant assistance from the other agencies reduced the utilization of TRPSP technical assistance.

9. Partner Comments

(a) Borrower/implementing agency:GICIT Assessments of outcomes

Objective (a) assist Romania in reforming the telecommunications sector and privatizing existing public sector telecommunications facilities

During the period of the TRPSP the Romanian telecom sector experienced a period of high development of radiocommunication network and services in Romania. Taking into account the full liberalization of the Romanian telecommunication market, effective from 1st of January 2003, the trend for near future seems also to be optimistic, at least for some of the mobile and fixed networks and services. During 2002 Rom Telecom was privatized and new technologies were implemented, in order to provide the local loop by wireless means, mainly in areas with low penetration of the wireline network, i.e. in rural or sparsely populated areas, but also in some urban areas as well.

The analogue nationwide radio-link network, maintained and operated by the Rom Radiocom, has been gradually turning to digital, at least on its main routes, connecting the major telecommunication and radiocommunication distant nodes of the country. This network is of primary importance and is used mainly for leasing capacities, between any two nodes, to interested parties (other telecom operators or private companies) and as carrier for public radio and television programs, from the central and regional studios of the public radio and television companies to the national and regional networks of radio and TV transmitters.

Objective (b) Provide the necessary infrastructure to enable GICIT to maximize the productive use of the radio frequency spectrum and to improve frequency assignment criteria for further expansion of wireless communications services provision by private sector.

All the main components of the NSMS were installed and are operational, however deficiencies in the system performances, and the changes in the regulatory framework, makes this component of the project not fully operational.

- 17 -

The delays in NSMS implementation, beyond the design changes, and the force majeure event are due to faulty components, therefore poor system performance. Nowadays the deficiencies are still being addressed by the contractor.

Objective (c) Support GICIT in performing regulatory functions over all networks and services in the telecom sector.

The year 2002 was marked by important milestones on the way of harmonizing Romanian telecommunication sector with the EU regulatory regimes and practices, as the Romanian Government adopted two important emergency ordinances, later modified and approved by laws of the parliament, which are laying down: the general regulatory framework of the Romanian telecommunications sector, thus implementing the framework directive and the authorization directive of the European Union, and the conditions of access to the electronic communications public networks and interconnection between these networks, thus implementing the access directive of the EU.

The Romanian Regulatory Authority for Communications was founded in summer 2002 under the authority of the Prime Minister. Also in mid 2002, the Romanian Parliament adopted the new Audiovisual Act, replacing the previous law dating from 1992.

Early 2003, the Romanian Government adopted a decision concerning the radio equipment and the telecommunications terminal equipment, with the aim of implementing the R&TTE directive of the EU. Currently, the secondary regulatory framework is undergoing a reviewing process with the aim of revising and updating it, bearing in mind the provisions of the new general regulatory framework in the telecommunication sector adopted in 2002.

A revised and updated list of spectrum usage fees was elaborated by MoCIT and adopted in September 2003 by Ministerial Ordinance. The national table of frequency allocations was also reviewed in order to bring it closer to the European Common Allocations table. The last version of the national table of frequency allocations was issued in November 2003 extended to 275 GHz.

Due to the change in the regulatory architecture, the 2 assistance subcomponents (RDP, SEMOS) achieved less impact than expected. The SEMOS reports concerning the spectrum management policies are under review and confirmation process. Policies, practices and procedures are in the phase of correlation with the new regulations in force. Some of the SEMOS reports directly linked with the current responsibilities of GICIT were reviewed and based on those recommendations operations manuals were prepared, spectrum monitoring procedures were established, etc.

TRPSP supported Romania to adopt an EU - compliant legal framework and also provided Infrastructure to enable GICIT to keep up with the high development in the telecommunications sector.

Taking into consideration the above [GICIT] are satisfied with the ICR.

- 18 -

(b) Cofinanciers:There were no cofinanciers.

(c) Other partners (NGOs/private sector):There were no other partners.

10. Additional Information

Sources:

IBM report, 6/03: “Third Report on Monitoring of EU Candidate Countries (Telecommunications Services Sector),” prepared for the European Commission by IBM Business Consulting

- 19 -

Annex 1. Key Performance Indicators/Log Frame Matrix

Outcome / Impact Indicators:

Indicator/Matrix

Projected in last PSR1

Actual/Latest Estimate

1. Privatization of Romtelecom June 1998 December 19982. Telecom Benchmarks per WTO approved by Government

January 1998

3. Expiry of Rom Telecom exclusivity January 1, 2003 January 1, 20034. Clarify GIC status January 2000 July 20025. Establish Regulatory Agency June 2000 October 2002

Output Indicators:

Indicator/Matrix

Projected in last PSR1

Actual/Latest Estimate

GICIT computer system operational February 1999 April 2000

Final SMS Operation April 2000 November 2002

Integrated NSMS in Operation July 2001 March 2004 *

1 End of project* Only parts of NSMS are in operation due to faulty supplies by contractor; Full integration projected based on GIC estimate.

- 20 -

Annex 2. Project Costs and Financing

Project Cost by Component (in US$ million equivalent)AppraisalEstimate

Actual/Latest Estimate

Percentage of Appraisal

Component US$ million US$ millionSector Reform 1.80 1.80 100Regulatory Agency Development 37.70 27.94 74

Total Baseline Cost 39.50 29.74 Physical Contingencies 2.90 Price Contingencies 1.60

Total Project Costs 44.00 29.74Total Financing Required 44.00 29.74

Project Costs by Procurement Arrangements (Appraisal Estimate) (US$ million equivalent)

Expenditure Category ICBProcurement

NCB Method

1

Other2 N.B.F. Total Cost

1. Works 0.00 0.00 0.00 0.00 0.00(0.00) (0.00) (0.00) (0.00) (0.00)

2. Goods 34.10 0.00 0.00 4.90 39.00(29.00) (0.00) (0.00) (0.00) (29.00)

3. Services 0.00 0.00 3.20 1.80 5.00(0.00) (0.00) (1.00) (0.00) (1.00)

Total 34.10 0.00 3.20 6.70 44.00(29.00) (0.00) (1.00) (0.00) (30.00)

Project Costs by Procurement Arrangements (Actual/Latest Estimate) (US$ million equivalent)

Expenditure Category ICBProcurement

NCB Method

1

Other2 N.B.F. Total Cost

1. Works 0.00 0.00 0.00 1.60 1.60(0.00) (0.00) (0.00) (0.00) (0.00)

2. Goods 14.41 0.00 0.00 7.73 22.14(10.82) (0.00) (0.00) (0.00) (10.82)

3. Services 0.00 0.00 3.09 2.91 6.00(0.00) (0.00) (2.73) (0.00) (2.73)

Total 14.41 0.00 3.09 12.24 29.74(10.82) (0.00) (2.73) (0.00) (13.55)

1/ Figures in parenthesis are the amounts to be financed by the Bank Loan. All costs include contingencies.2/ Includes civil works and goods to be procured through national shopping, consulting services, services of contracted staff

of the project management office, training, technical assistance services, and incremental operating costs related to (i) managing the project, and (ii) re-lending project funds to local government units.

- 21 -

Project Financing by Component (in US$ million equivalent)

Component Appraisal Estimate Actual/Latest EstimatePercentage of Appraisal

Bank Govt. CoF. Bank Govt. CoF. Bank Govt. CoF.Sector Reform 1.80 1.80 100.0Regulatory Agency Dev. 30.00 12.20 13.55 14.39 45.2 118.0Total 30.00 14.00 13.55 16.19 45.2 115.6

Note: Amount shown under Govt. includes amounts financed by GICIT

- 22 -

Annex 3. Economic Costs and Benefits

NA

- 23 -

Annex 4. Bank Inputs

(a) Missions:Stage of Project Cycle Performance Rating No. of Persons and Specialty

(e.g. 2 Economists, 1 FMS, etc.)Month/Year Count Specialty

ImplementationProgress

DevelopmentObjective

Identification/Preparation12/1996 3 Team Leader (1); Telecom

Specialist (1); Spectrum Consultant (1)

2/1997 4 Team Leader (1); Telecom Sector Specialist (1); Regulatory Specialist (1); Privatization Specialist (1);

Appraisal/Negotiation4/1997 5 Team Leader (1);

Privatization Specialist (1); Telecom Specialist (1); Legal Counsel (1); Operations Officer (1)

Supervision11/1998 2 Project Team Leader (1);

Legal Counsel (1)S S

05/04/1999 3 Project Team Leader (1); Regulatory Consultant (1); Privatization Expert (1)

S S

10/30/1999 3 Project Team Leader (1); Legal Counsel (1); Telecom Engineer (1)

S S

06/15/2000 2 Project Team Leader (1); Operations Officer (1)

S S

7/25/2000 4 Project Team Leader (1); Legal/Regulatory Specialist (1); Project Officer (1); Spectrum Consultant (1)

S S

11/03/2000 5 Project Team Leader (1); Legal/Regulatory Specialist (1); Telecom Specialist (1); Project Officer (1); Spectrum Consultant (1);

S S

6/28/2001 4 Project Team Leader (1); Spectrum Consultant (1); Project Officers (2)

S S

07/16/2001 2 Project Team Leader (1) Legal Counsel (1)

S S

09/21/2001 4 Project Team Leader (1); Legal Counsel (1); Spectrum Consultant (1); Project Officer (2)

S S

5/16/2002 1 Frequency Spectrum Specialist (1)

S S

- 24 -

6/24/2002 4 Project Team Leader (1); Legal/Regulatory Specialist (1); Project Officer (1); Spectrum Consultant (1)

S S

09/24/2002 5 Project Team Leader (1); Legal/Regulatory Specialist (1); Project Officer (1); Spectrum Consultant (1); Legal Counsel (1)

S S

4/11/2003 2 Legal Counsel (1); Radio Frequency Specialist (1)

S S

ICR11/14/2003 1 Consultant (1) S S

(b) Staff:

Stage of Project Cycle Actual/Latest EstimateNo. Staff weeks US$ ('000)

Identification/Preparation 100.0Appraisal/Negotiation 90.0Supervision 409.2ICR 20.6Total 619.8

Note: The original FY03 project concept as an investment project was dropped in FY06 and a revised concept as a TA loan was approved in FY07. Hence, costs during FY03-06 have been excluded and adjusted for lending (identification through negotiation) during FY07-08 for TRPSP.

- 25 -

Annex 5. Ratings for Achievement of Objectives/Outputs of Components(H=High, SU=Substantial, M=Modest, N=Negligible, NA=Not Applicable)

RatingMacro policies H SU M N NASector Policies H SU M N NAPhysical H SU M N NAFinancial H SU M N NAInstitutional Development H SU M N NAEnvironmental H SU M N NA

SocialPoverty Reduction H SU M N NAGender H SU M N NAOther (Please specify) H SU M N NA

Private sector development H SU M N NAPublic sector management H SU M N NAOther (Please specify) H SU M N NA

- 26 -

Annex 6. Ratings of Bank and Borrower Performance

(HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HU=Highly Unsatisfactory)

6.1 Bank performance Rating

Lending HS S U HUSupervision HS S U HUOverall HS S U HU

6.2 Borrower performance Rating

Preparation HS S U HUGovernment implementation performance HS S U HUImplementation agency performance HS S U HUOverall HS S U HU

- 27 -

Annex 7. List of Supporting Documents

IBM, "3rd Report on Monitoring of EU Candidate Countries (Telecommunication Services Sector), June 2003

- 28 -

- 29 -