An assessment of the renewable energy potential using a clustering based data mining method. Case...

14
An assessment of the renewable energy potential using a clustering based data mining method. Case study in Romania Gheorghe Grigoras * , Florina Scarlatache Gheorghe AsachiTechnical University of Iasi, Electrical Engineering Faculty, Power System Department, Bd. Dimitrie Mangeron, No. 21-23, 700050 Iasi, Romania article info Article history: Received 18 April 2014 Received in revised form 17 December 2014 Accepted 20 December 2014 Available online 14 January 2015 Keywords: Data mining Clustering Renewable energy potential Maps Romania abstract This paper presents an assessment of the renewable energy potential in Romania using a clustering based data mining method. The available data on installed capacity, level voltage, type of renewable technology and geographical location the renewable energy potential for electricity generation was mapped into representative zones using K-Means clustering algorithm. For each zone, the potential was assessed on voltage level and renewable energy generation technologies (wind, solar, hydro, biogas, biomass, and cogeneration). The zones obtained can be a useful working tool for retrotting substations, upgrading of transmission and distribution lines and also for redesigning them at different parameters with respect to the overload. This information may enable the creation of specic programs to improve planning and development of the electric networks in Romania. © 2014 Elsevier Ltd. All rights reserved. 1. Introduction Currently, 75% of total global energy demand is supplied by the burning of fossil fuels. But with increasing air pollution, global warming concerns, diminishing fossil fuels and their increasing cost have made it necessary to look towards renewable sources as a future energy solution. The decentralized electricity generation is intended to provide small-scale power close to users, using a broad range of renewable technologies. In the past decade, there has been an enormous interest in many countries on renewable energy sources (RES) for power generation. These renewable energy sources are considered as some of the most promising options to provide a more secure, clean and more efcient energy supply [1]. The RES potential responds to global sustainability and envi- ronmental, safety, social, and economic goals. Interconnecting to the electrical network of RES could have signicant effects on the system in terms of power ows, voltage prole and reliability. The benets brought by these sources are numerous: energy efciency, rational use of energy, competition policy, diversication of energy sources, availability of modular generating plants, ease of nding locations for smaller generators, shorter construction times, and lower capital costs [1e3]. The availability of renewable energy sources can be analysed in function by the type of potential considered. In the literature, various types of potentials are dened: theoretical, technical, eco- nomic and market [4e7]. Theoretical potential represents the highest level of potential that takes into account restrictions with respect to natural and climatic parameters. Technical potential represents the theoretical potential reduced due to technical limi- tations, land-use constraints and topographic constraints. The economic potential is the technical potential at cost levels consid- ered competitive. The market potential is the total amount of renewable energy that can be implementation in the market taking into account the demand for energy, the competing technologies, the costs and subsidies of renewable energy sources. Between the entire theoretical potential and the technical, economic and market feasible one, there is nonetheless a signicant gap, creating great discrepancies between the theoretical hierarchy of the different renewable sources potential and the practical orientation of pro- jects [7]. In the literature, there are many studies that present different approaches in the assessment of renewable resource potential. Comparing their results is very difcult because they use different assumptions, methodologies, reporting units, and analysis time periods. In these studies the assessment of renewable energy * Corresponding author. Tel.: þ40 232 278683; fax: þ40 232 237627. E-mail address: [email protected] (G. Grigoras). Contents lists available at ScienceDirect Energy journal homepage: www.elsevier.com/locate/energy http://dx.doi.org/10.1016/j.energy.2014.12.054 0360-5442/© 2014 Elsevier Ltd. All rights reserved. Energy 81 (2015) 416e429

description

computanional math

Transcript of An assessment of the renewable energy potential using a clustering based data mining method. Case...

  • pin

    tem

    Available online 14 January 2015

    ssmvailtheK-

    voltage level and renewable energy generation technologies (wind, solar, hydro, biogas, biomass, and

    the overload. This information may enable the creation of specic programs to improve planning anddevelopment of the electric networks in Romania.

    rgy demcreasinsil fueowards

    intended to provide small-scale power close to users, using a broad

    system in terms of power ows, voltage prole and reliability. Thebenets brought by these sources are numerous: energy efciency,rational use of energy, competition policy, diversication of energysources, availability of modular generating plants, ease of nding

    respect to natural and climatic parameters. Technical potentialto technical limi-constraints. The

    cost levels consid-total amount ofthe market takingting technologies,rces. Between thenomic andmarket

    feasible one, there is nonetheless a signicant gap, creating greatdiscrepancies between the theoretical hierarchy of the differentrenewable sources potential and the practical orientation of pro-jects [7].

    In the literature, there are many studies that present differentapproaches in the assessment of renewable resource potential.Comparing their results is very difcult because they use differentassumptions, methodologies, reporting units, and analysis timeperiods. In these studies the assessment of renewable energy

    * Corresponding author. Tel.: 40 232 278683; fax: 40 232 237627.

    Contents lists availab

    Ener

    els

    Energy 81 (2015) 416e429E-mail address: [email protected] (G. Grigoras).range of renewable technologies. In the past decade, there has beenan enormous interest in many countries on renewable energysources (RES) for power generation. These renewable energysources are considered as some of the most promising options toprovide a more secure, clean and more efcient energy supply [1].

    The RES potential responds to global sustainability and envi-ronmental, safety, social, and economic goals. Interconnecting tothe electrical network of RES could have signicant effects on the

    represents the theoretical potential reduced duetations, land-use constraints and topographiceconomic potential is the technical potential atered competitive. The market potential is therenewable energy that can be implementation ininto account the demand for energy, the compethe costs and subsidies of renewable energy souentire theoretical potential and the technical, ecofuture energy solution. The decentralized electricity generation is highest level of potential that takes into account restrictions withClusteringRenewable energy potentialMapsRomania

    1. Introduction

    Currently, 75% of total global eneburning of fossil fuels. But with inwarming concerns, diminishing foscost have made it necessary to look thttp://dx.doi.org/10.1016/j.energy.2014.12.0540360-5442/ 2014 Elsevier Ltd. All rights reserved. 2014 Elsevier Ltd. All rights reserved.

    and is supplied by theg air pollution, globalls and their increasingrenewable sources as a

    locations for smaller generators, shorter construction times, andlower capital costs [1e3].

    The availability of renewable energy sources can be analysed infunction by the type of potential considered. In the literature,various types of potentials are dened: theoretical, technical, eco-nomic and market [4e7]. Theoretical potential represents theKeywords:Data miningcogeneration). The zones obtained can be a useful working tool for retrotting substations, upgrading oftransmission and distribution lines and also for redesigning them at different parameters with respect toAn assessment of the renewable energybased data mining method. Case study

    Gheorghe Grigoras*, Florina ScarlatacheGheorghe Asachi Technical University of Iasi, Electrical Engineering Faculty, Power SysRomania

    a r t i c l e i n f o

    Article history:Received 18 April 2014Received in revised form17 December 2014Accepted 20 December 2014

    a b s t r a c t

    This paper presents an assedata mining method. The aand geographical locationrepresentative zones using

    journal homepage: www.otential using a clusteringRomania

    Department, Bd. Dimitrie Mangeron, No. 21-23, 700050 Iasi,

    ent of the renewable energy potential in Romania using a clustering basedable data on installed capacity, level voltage, type of renewable technologyrenewable energy potential for electricity generation was mapped into

    Means clustering algorithm. For each zone, the potential was assessed on

    le at ScienceDirect

    gy

    evier .com/locate/energy

  • potential is based either on the complex knowledge of thegeographic regions, both in terms of components and the charac-teristics of the natural environment (topography, climate, hydrog-raphy, soils, and vegetation) either on the interpretation of complexspatial modelling maps e GIS (Geographic Information System)documents.

    In Ref. [5], the results of a spatial analysis calculating renewableenergy technical potential, reporting available land area (squarekilometres), installed capacity, and electricity generation fordifferent renewable technologies (solar, wind, biomass, andgeothermal) are presented. Each technology's system, installedcapacity, and land-use constraints were identied using the pub-lished researches, subject matter experts, and the analysis of aresearch laboratory. The assessment relies heavily on a geographicinformation system (GIS) and linear programming model. A mapwith geographical spread of renewable energy resources that aretheoretically available on the entire Romanian territory is pre-sented in Ref. [7]. This map is only based on the natural and climaticfactors. A review of how geographic information and remotesensing techniques are applied to reduce uncertainties surroundingrenewable energy development, with emphasis on policy andplanning needs is presented in Ref. [8]. In Ref. [9] the authorsdeveloped a method for assessment of renewable energy potentialusing GIS information and publicly available digital spatial data.This method is based on the simulation of the meteorological pa-

    expected energy production from the renewable energy sources topower stations. The inputs of the proposed method are availableinformation from the projects in Technical Connection Approval(CTAs) phasewithin the validity period (i.e., with some potential forachievement) given between the years 2012e2014 in Romania bydistribution and transmission companies: installed capacity,voltage level (low e 0.4 kV, medium e 20 kV, high e 110 kV andextra high e 220, 400 kV), renewable energy generation technol-ogies (hydro, biomass, biogas, cogeneration, photovoltaic, andwind), and geographical locations. Using these data, proposedmethod transforms them into an easy understandable structurecharacterized by zones with different technical exploitable poten-tial of renewable sources. The division in zones based on K-Meansclustering algorithm led to an identication of technical exploitablepotential more accurate than in other analyses, given in Refs. [6,7]and [11], where the assessment is made only for total technicalpotential of Romania, without a detailed assessment per zones,level voltages and type of renewable energy generation sources.The algorithm is able to learn and to establish a very good depen-dence between considered variables for each CTAs.

    The remainder of this paper is organized as follows. In Section 2,the current renewable energy status in EU countries is presented.Section 3 presents an overview of the renewable energy potentialin Romania. Section 4 presents the problem of Mining Data in

    renewable energy sources in primary energy consumption to 20%

    G. Grigoras, F. Scarlatache / Energy 81 (2015) 416e429 417rameters and extraction of theoretical potential areas with differentrestrictions (meteorological conditions, geographical features andsocial environment). An information system for the assessment ofthe technical and economical exploitable RES (Renewable EnergySources) potential is presented in Ref. [10]. This tool is based on aGeographical Information System (GIS) giving the ability to analysethe prefeasibility of RES investments in wide geographical areas.

    Thus, there is a clear need for information management pro-grams that can assess the renewable energy potential fromdifferent geographic areas. In this paper, a clustering based datamining method for to assess the technical exploitable potential ofrenewable energy sources for electricity generation is proposed.The technical exploitable potential is dened as the capacity andFig. 1. Wind power installed in Eurand a mandatory target of a 20% share of renewable energy sourcesin the gross nal energy consumption by 2020 [2,3,12,13,19e22]. AEuropean Environment Agency analysis of the 20% target showedthat electricity will cover approximately 45% of the EU's renewableknowledge discovery from large database and how it can be used inassess of renewable energy potential. Section 5 shows the results oftesting the proposedmethod in Romania. Finally, Section 6 containsthe concluding remarks.

    2. Renewable energy status in EU

    To avoid or at least to reduce climate change, the EuropeanUnion has proposed an ambitious plan to increase the share ofope by end of 2013 [MW] [24].

  • -con

    G. Grigoras, F. Scarlatache / Energy 81 (2015) 416e429418energy production, heating and cooling 43%, and transport 12%[20].

    The electricity target of many EU countries is based on the windand photovoltaic energy considered to have the largest potential inelectricity production from renewable energy sources. Energy As-sociation (EWEA) estimates that, in 2020, the wind power capacityin the EU will be about 230 GW. By 2020 the electricity productionfrom wind energy will be equivalent to the total electricity con-sumption of all households in France, Germany, Poland, Spain andthe United Kingdom together [23]. In 2013, the total wind powerinstalled capacity was by 117,289 MW, of which 12,030 MW wasinstalled across Europe (11,159 MW was in the European Union e9592 MW was onshore and 1567 MW offshore). In Fig. 1, the windpower installed in EU countries by end of 2013 is presented [24].

    Regarding photovoltaic (PV) sources, the energy policies and

    Fig. 2. Cumulative installed gridpublic support programs in Europe differ substantially fromcountry to country. Amongst the European countries, Germany hasbeen and continues to be the European PV market leader with themost installations (35,886 GW at the end of January 2014). Ingeneral the Mediterranean region is an area of strategic importancefor PV markets and can potentially play an important part in theintegration and uptake of this technology especially due to the

    Fig. 3. New hydro potential in the three main European regions [TWh/year] [19].favourable conditions for PV in the region [18]. Since 2000, solar PVelectricity generation capacity has increased 373 times from185 MW in 2000 to 69 GW at the end of 2012, as shown in Fig. 2[25]. The SET for 2020 study carried out by European Photovol-taic Industry Association, outlines that, provided some boundaryconditions are met, PV could supply up to 12% of the electricitydemand in Europe by 2020, thus representing 390 GW of installedcapacity and 460 TWh of electricity generation [26]. the hydro-power development in Europe has attained a relatively maturestage, a signicant potential for hydropower remained still today[27]. The gross theoretical capability, presented in Fig. 3, representsa calculation based on the topography and precipitation in thecountries. This is the amount power that could be extracted if allrun-off was turbined down to the lowest level of the speciedcountry (sea-level).

    nected PV capacity in EU [25].The potential for new hydroelectric capacity in Europe is oftensaid to be limited. However, some estimates put the untappedpotential of economically feasible yet-to-develop capacities at165 TWh/year in Western Europe. The residual potential in SouthEastern Europe is estimated at 145 TWh/year and in Central EasternEurope is 6 TWh/year [19].

    Biomass represents another renewable source that has thecapability to contribute strongly to meeting the European Union's

    Fig. 4. Biomass electricity production capacity in 2005, 2010 and 2020 [29].

  • renewable targets for both heat and electricity in 2020. A signicantmajority of the biomass required can be produced within the EU. In2005, the total capacity of biomass power generation was 15.7 GW[28]. A comparison between status of the EU member in 2005,interim 2010 and estimation for 2020 is presented in Fig. 4 [29].

    In the scenarios published by the European Commission for howEurope could meet its targets, annual biomass heat and electricity

    consumption is predicted to grow by a full 850 TWh by 2020 to atotal of 1650 TWh [29].

    3. The renewable energy potential in Romania

    The integration of renewable sources into a power system is thebiggest challenge for specialists from all countries. Romania is acountry located between Central Europe and South Eastern Europe,bordering the Black Sea, Fig. 5. Geostrategic role of Romania inEurope and the important energy routes from southeast of thecountry are issues more and more discussed in the last ve years.Romania is ranked on 10 in the top of most attractive countries inthe world on investment in wind power and on 13th in totalrenewable energy sources, into a classication compiled by Ernst& Young, which comprises 40 countries, including United States,Germany, China and United Kingdom [31]. The development ofrenewable energy sector in Romania is based on the existence of animportant and diversied theoretical potential regarding therenewable energy sources.

    Currently, wind energy has the largest interest because Romaniapresents the highest potential from the Southeast of Europe.Further, southeast of Romania ranks on the second place across theentire continent. In the past 10 years, thewind potential of Romaniawas reassessed. Wind distribution map of Romania, presented inFig. 6, includes average annual speeds calculated from the height of50 m above the ground. The areas with a solid wind potential are:the Southeast (wind speed is around 8e9 m/s), and the Eastern

    Fig. 5. Location of Romania e dark green (grey e on the European continent, green ein the European Union) [30]. (For interpretation of the references to colour in thisgure legend, the reader is referred to the web version of this article.)

    G. Grigoras, F. Scarlatache / Energy 81 (2015) 416e429 419Fig. 6. Wind energy potential in Romania [32].

  • oten

    / EnFig. 7. Solar energy p

    G. Grigoras, F. Scarlatache420(wind speed is about 6e8 m/s). Considerable wind speeds are re-ported also in other smaller areas in the west of the country. In thestrategy of exploiting renewable energy resources the wind po-tential is of 14,000 MW (installed capacity), that can provide aquantity of energy of about 23,000 GWh/year. These values repre-sent an estimation of the theoretical potential and should benuanced according to the technical and economic possibilities ofexploitation.

    After the great development of the wind sector, Romania hasstarted to attract investors in the solar energy production domain.Fig. 7 shows the map distribution of the average annual ow of thesolar energy incident on horizontal surface in Romania. Five areasdifferentiated by average annual ow values of solar power can beobserved. It nds that more than half of the country benets of anaverage annual ow by of 1275 kWh/m2. According to the NationalEnergy Strategy, the Romanian solar potential can generate1.2 TWh of electricity per year, which means 2.5% of the currentconsumption [32]. The South and Southeast regions of Romania arethe most suitable areas for such investment.

    Hydro energy already represents a substantial source of power,28% from energy production in Romania. Geographically, the hy-droelectric reserves of Romania are concentrated along the Danubeand in the valleys of rivers emerging from the mountain core of thecountry. Other hydro resources include more than 2500 lakes,ranging from the glacial lakes of the mountains to those of theplains. Hydro energy potential, currently at 5900 MW for domesticrivers, could reach 8000 MW with additional investments on theDanube, according to experts' estimations. In Fig. 8, with orangecolour it can be seen the estimated potential of small hydropowerplants in execution. At the end of 2011 the total installed capacity inhydropower plans was approximately 6528 MW [33].

    The potential of developing biomass projects in Romania is quitelarge; thanks to the vast density of forestry that covers the countrytial in Romania [32].

    ergy 81 (2015) 416e429thus, highlighting the prospect of developing a successful biomassindustry. In Fig. 9 are presented the geographical distribution of theforest and agricultural biomass resources potential for each countyfrom Romania. The richest areas in forest biomass resources be-longs to Oriental Carpathians, with green colour, and the richestzones in agricultural biomass resources belongs to Eastern andSouthern of Romania, with yellow colour. Romania is estimated tohave a biomass energy potential corresponding to some 19% of thetotal average primary consumption. Biomass is used in Romaniamostly for thermal energy generation (more than 50% from thegenerated heat had as source forest biomass burning) [35]. But,forest biomass is also used in cogeneration plants for electricitygeneration. According to [37], in 2013 a number of 13 plants havecontracts for connecting to the national network (total installedcapacity by 46 MW) and electricity production was by 0.37% oftotal.

    Based on the estimated potentials from renewable sources, theRomanian Government has committed to the European Union thatthe share of electricity produced from renewable resources in thetotal electricity consumption to be 24% in 2020. The estimation oftotal contribution expected of each renewable technology (wind,hydro, photovoltaic and biomass) to achieve mandatory targets for2020 and the indicative interim trajectory for electricity weightfrom renewable energy sources is presented in Table 1 [32].

    This estimation was made at the beginning of 2005. It can beseen an exponential growth until 2015e2016, after which growthwill be weighted. Most projects require a connection to high andextra high voltage networks of 110 kV, 220 kV, and 400 kV [37]. But,the situation is different in real case. The installed capacity inrenewable sources was smaller than the estimated value. In 2010,65 producers of renewable sources were registered: 32 of hydro-power, 28 of wind energy, 3 in exploitation of biomass and one ofsolar energy. The total installed capacity was 561.2 MW [39]. At the

  • / EnG. Grigoras, F. Scarlatacheend of 2012 the installed capacity from renewable sources amounts2301 MW compared to 1225.6 MW in 2011, of which: 1794 MW inwind power plants, 427 MW in small hydropower plants with aninstalled power not more than 10 MW, 26.5 MW in biomass,2.4 MW in power plants that use waste fermentation gas, and51 MW in photovoltaic power plants. If these data are comparedwith the values from Table 1, it can be said that the targets wereachieved for wind, hydro and photovoltaic power plants, but thereis a signicant difference in the case of biomass (26.5 MWcompared to 165 MW).

    4. Mining Data in knowledge discovery from large database

    Nowadays, the need to extend the capabilities of human analysisfor handling the large number of data that we may collect hasbecome increasingly necessary. The ability of the human operatorsto analyse the large databases manually is far exceeded. Sincecomputers have allowed storing more data, it is only natural toresort to computational techniques to help us to discover mean-ingful patterns and massive structures volume data. In these con-ditions, it is necessary to automate the information (knowledge)discovery to assist the human operator [40,41].

    Several denitions are currently being used for both datamining(DM) and knowledge discovery in databases (KDD). While in somesituations they are used as equivalent terms, data mining is one ofthe steps in the knowledge discovery process [42].

    Knowledge discovery, generally, is the process of nontrivialextraction of information from database, information that is

    Fig. 8. Potential of small hydropoergy 81 (2015) 416e429 421implicitly present in that data, previously unknown and potentiallyuseful for the user [43]. Knowledge is a pattern that is sufcientlyinteresting to user and sufciently certain. As regards KDD, thisrefers to the overall process of discovering useful knowledge fromdata. The KDD process is interactive and iterative, involvingnumerous steps withmany decisionsmade by the user, as shown inFig. 10 [43,44].

    More specically, with reference to the ow-chart of Fig. 10:Data Selection. In this step, the goal and the tools of the data

    mining process are chosen, identifying the data to be mined, thenchoosing appropriate input attributes and output information torepresent the task.

    Preprocessing. The target in this step is to eliminate the noises ormissing values from the data set. The special techniques are usedfor the treatment of missing data.

    Data transformation. The kinds of transformations of the pre-processed data depend on the task of the human operator: orga-nizing data in desired ways, converting one type of data to another,dening new attributes, reducing the dimensionality of the data,removing outliers and normalizing.

    Data mining. The transformed data is subsequently mined, usingone or more techniques to extract patterns of interest. The humanoperator can signicantly aid the data mining method by correctlyperforming the preceding steps.

    Result interpretation/validation. For understanding the meaningof the synthesized knowledge and its range of validity, the datamining application tests its robustness, using established estima-tion methods and unseen data from the database.

    wer plants in Romania [34].

  • G. Grigoras, F. Scarlatache / Energy 81 (2015) 416e429422Incorporation of the Discovered Knowledge. In this last step, theresults are presented to the human operator who may check/resolve potential conicts with previously extracted knowledgeand apply to the new discovered patterns.

    As it can be seen, an important step in the KDD process is rep-resented by Data mining (DM). DM involves tting models ordetermining patterns from observed data [41].

    In the literature, there are available many tools for data mining,but the most important are [40e45]: classication methods, clusteranalysis, search for association rules, aggregation and approximationmethods, time series analysis, dependency analysis, and predictionanalysis. Another classication, based on the machine learning eld,is given by division of the data mining methods into two groups:supervised and unsupervised learning methods. In supervisedlearning methods, a functional model is built for one of the variables(modelled variable). The model will relate the modelled variablewith the rest of variables. Inside this category, the classication andregression methods are usually considered. In unsupervisedlearning, some knowledge about the variables is extracted from thedatabase. This knowledge is represented by relationships betweenvariables. The advantage of the unsupervised learning methods is

    Fig. 9. Biomass potential in Romania per g

    Table 1Estimation of the installed capacity in renewable energy sources [32].

    Renewable technologies Installed capacity [MW]

    2005 2010 2011 2012 2013

    Wind 1.32 560 1250 1850 2480Hydro 6289 6413 6537 6687 6857Photovoltaic 0 0 8 43 78Biomass 0 14 90 165 250that interesting structures can be found directly from the datawithout any background knowledge. From this category, the mostfamous methods are clustering techniques [45e47].

    The clustering methods have as input a set of elements. Anelement (entity, observation, record, element, datum, or featurevector) is a single data item used by the clustering algorithm. Thesemethods build a tree which approximates the similarities betweenelements, where each leaf is an element and intermediate nodesrepresent groups of elements. There are more clustering methodswhich directly build a partition of elements. From these, K-meansclustering algorithm is one of the most used unsupervised learningalgorithms especially for databases.

    The K-means algorithm classies the elements xi, i 1, , M,from whole set S {xi, i 1,, M} based on attributes/features ofthem into K clusters (K is positive integer number). A rstassumption is made for the cluster centroids ck, k 1,, K (usuallychosen at random among the elements of data set). The k centroidsclassify the elements, in the sense that the element xi belongs to thecluster Ck if the distance d(xi, ck) is the minimum of all the K dis-tances between xi and the cluster centroids. The function is asquared error function.

    eographical regions and counties [36].

    2014 2015 2016 2017 2018 2019 2020

    2880 3200 3400 3600 3750 3900 40007087 7287 7387 7452 7513 7621 7729113 148 183 200 220 240 260340 425 510 540 565 590 600

  • max bj ; aj

    G. Grigoras, F. Scarlatache / EnminE min0@X

    K

    k1

    Xx2Ck

    dxi; ck

    1A (1)

    where:

    cke the centre of cluster Ck (c {c1,, ck}, C {C1,, CK}, Ck3 S,k {1, , K});d(xi, ck) is the Euclidean distance between a point xi2S, i 1,,M, and ck.

    Thus, the criterion function E attempts to minimize the distance

    Fig. 10. Flow-chart of the KDD process.of each point from the centre of the cluster to which the pointbelongs. More specically, the algorithm begins by initializing a setof K cluster centres. Then, it assigns each object of the data set to thecluster whose centre is the nearest, and recomputed the centres.The process continues until the centres of the clusters stopchanging. In the following, a validation of results must be made.

    There are many approaches to cluster validation, but internalcluster validation tests are more popular in practice of cluster

    Fig. 11. The silhouette plot for Koptimal 6.the validity period (i.e., with some potential for achievement) givenin the years 2012e2014 [37]. The database had initially a totalnumber of approximately 2000 CTAs, but after data pre-processingstep, only 960 CTAs remained. The reduction of database was madedue to following reasons: some connection technical approvalswere cancelled in this time period or overlapped inside of analyseperiod due to extension of these approvals (until 2015e2016).

    In our study, K-Means clustering algorithmwas used to identifythe zones with renewable energy generation potential in functionby the information included in the database. The proposed algo-rithm is able toworkwith large databases regardless by the amountof data, new information that might to be inserted, or current datathat might to be updated and/or deleted.

    The information used is referred to: installed capacity, voltagelevel (low e 0.4 kV, medium e 20 kV, high e 110 kV, and very highe 220, 400 kV), type (hydro, biomass, biogas, cogeneration,photovoltaic, and wind), and the geographical positions.

    From the clustering process, 6 clusters were obtained. Thevalidation of the results was made using the internal cluster vali-dation test based on the calculation of Silhouette Global Coefcient(SGC). The value of this coefcient was SGC 0.71, representing theoptimal solution of clustering process. The silhouette plot is pre-5. Study case

    The assessment of technical exploitable potential of RES forelectricity generation was made using a database containing theprojects in Technical Connection Approvals (CTAs) phase withinaj e mean distance between element xih, i 1, , M, and ele-ments of the same cluster k {1,, K};bh e minimum mean distance between element xij, i 1, , Mand elements in cluster closest to cluster k {1,, K};

    In our study, the clustering techniques are used to identify thezones with highest renewable energy potential in Romania.analysis. From these, the test based on the Silhouette Global Indexcalculation is one of the most used [48e50]. This calculates thesilhouette width for each sample, average silhouette width for eachcluster and overall average silhouette width for a total data set.Using this approach each cluster could be represented by so-calledsilhouette, which is based on the comparison of its tightness andseparation. The average silhouette width will be applied for eval-uation of clustering validity and also will be used to decide deter-mination of optimal number of clusters. The silhouette globalcoefcient is calculated with relation [48]:

    SGC 1K

    XK

    k1Sk (2)

    where:

    Sk e silhouette local coefcient is dened as:

    Sk 1nk

    Xnkh1

    sh (3)

    sj e the silhouette width index for element j is:

    sj bj aj (4)

    ergy 81 (2015) 416e429 423sented in Fig. 11.

  • Table 2The potential installed capacity of renewable power plants per cluster and zones.

    Cluster Zone Voltage[kV]

    Hydro[MW]

    Biomass[MW]

    Biogas[MW]

    Cogeneration[MW]

    Photovoltaic[MW]

    Wind[MW]

    Total[MW]

    C4 ZC41 0.4 0.00 0.00 0.00 0.00 0.05 0.00 0.05 9.9ZC42 0.00 0.00 0.00 0.06 0.36 0.00 0.42ZC43 0.00 0.00 0.00 0.00 0.04 0.07 0.11ZC44 0.00 0.00 0.00 5.00 1.14 0.00 6.14ZC45 0.00 0.00 0.00 0.00 0.25 0.00 0.25ZC46 0.00 0.00 0.00 0.00 0.26 0.00 0.26ZC47 0.07 1.52 0.00 0.25 0.81 0.00 2.65

    C1 ZC11 20 4.79 7.13 3.50 0.00 30.22 28.90 75.54 696.3ZC12 16.43 9.10 3.74 7.36 305.95 52.57 395.16ZC13 4.00 6.50 0.56 0.60 166.56 48.40 226.62

    C3 ZC31 25.80 19.97 6.08 0.00 308.73 31.22 391.79 714.5ZC32 23.90 13.80 4.79 1.31 278.90 0.00 322.70

    C6 ZC61 0.00 1.35 3.41 14.80 77.75 0.00 97.31 831.1ZC62 12.72 0.00 0.00 2.29 524.62 0.00 539.63ZC63 9.53 7.50 0.23 0.00 177.41 0.00 194.67

    C2 ZC21 110e220 0.00 0.00 0.00 0.00 0.00 1112.10 1112.10 5459.9ZC22 39.30 13.10 0.00 0.00 92.00 309.35 453.75ZC23 0.00 20.00 0.00 0.00 276.35 1005.10 1301.45ZC24 0.00 0.00 14.00 17.60 404.36 30.00 465.96ZC25 0.00 0.00 0.00 0.00 214.19 87.00 301.19ZC26 21.85 0.00 0.00 0.00 344.36 662.50 1028.71ZC27 7.70 0.00 0.00 0.00 178.97 0.00 186.67ZC28 36.72 0.00 0.00 0.00 486.40 87.00 610.12

    C5 ZC51 220e400 300.00 0.00 0.00 0.00 0.00 1095.00 1395.00 6222.7ZC52 0.00 0.00 0.00 0.00 0.00 1409.40 1409.40ZC53 0.00 0.00 0.00 295.00 0.00 2577.30 2872.30ZC54 0.00 0.00 0.00 0.00 0.00 96.00 96.00ZC55 0.00 0.00 0.00 450.00 0.00 0.00 450.00

    G. Grigoras, F. Scarlatache / Energy 81 (2015) 416e429424Each obtained cluster from clustering process contains anumber of zones, denoted by Z. Each zone name contains twoparts that describe each zone in detail. The rst part of the zonename is the cluster (denoted by Ck) and the second part of thename describes the zone in the cluster (denoted by i). In our

    case, i 1, , number of zones from cluster Ck, where k 1, ,

    Table 3The number of renewable power plants per cluster and zones.

    Cluster Zone Voltage[kV]

    Hydro[no.]

    Biomass[no.]

    Biogas[no.]

    C4 ZC41 0.4 0 0 0ZC42 0 0 0ZC43 0 0 0ZC44 0 0 0ZC45 0 0 0ZC46 0 0 0ZC47 1 1 0

    C1 ZC11 20 5 2 2ZC12 3 3 4ZC13 1 1 3

    C3 ZC31 13 5 4ZC32 16 2 3

    C6 ZC61 0 1 1ZC62 11 0 0ZC63 8 1 1

    C2 ZC21 110e220 0 0 0ZC22 1 1 0ZC23 0 1 0ZC24 0 0 1ZC25 0 0 0ZC26 1 0 0ZC27 1 0 0ZC28 1 0 0

    C5 ZC51 220e400 1 0 0ZC52 0 0 0ZC53 0 0 0ZC54 0 0 0ZC55 0 0 06. The characteristics (installed capacity and number of powerplants per each type of renewable technology and voltage level)for each region and zone are indicated in Tables 2 and 3 andFigs. 12e23. The clusters have been arranged in tables accordingto the voltage level, from the low (0.4 kV) to the extra high

    (400 kV).

    Cogeneration[no.]

    Photovoltaic[no.]

    Wind[no.]

    Total[no.]

    0 7 0 7 501 4 0 50 2 2 42 5 1 80 7 0 70 5 0 52 10 0 140 17 6 32 2263 93 11 1171 53 18 770 97 3 122 2331 89 0 1112 29 0 33 2633 139 0 1530 67 0 770 0 20 20 1410 2 9 130 19 22 421 9 1 120 10 2 120 11 9 210 5 0 60 13 1 150 0 10 11 470 0 8 81 0 25 260 0 1 11 0 0 1

  • Fig. 15. Number of renewable power plants from zones ZC11 e ZC13 (Cluster C1).

    G. Grigoras, F. Scarlatache / Energy 81 (2015) 416e429 425Fig. 12. Potential installed capacity [MW] of renewable power plants from zones ZC41 eZC47 (Cluster C4).From Tables 2 and 3, it can observe that the clusters are char-acterized by the voltage level: C4 e low voltage (0.4 kV); C1, C3 andC6 emedium voltage (20 kV); C2 e high voltage (110e220 kV) andC5 e extra high voltage (above 220 kV).

    The analysis by clusters and zones highlights the followingaspects:

    - cluster C4 (low voltage level): The potential installed capacity inall zones is small (below 1.5 MW), excepting zone ZC44 , with5 MW in cogeneration power plants.

    - clusters C1, C3, and C6 (medium voltage level): The values arehigh for potential installed capacity in the photovoltaic power

    Fig. 13. Number of renewable power plants from zones ZC41 e ZC47 (Cluster C4).

    Fig. 14. Potential installed capacity [MW] of renewable power plants from zones ZC11 eZC13 (Cluster C1).plants (over 300MW) from zones ZC12 , ZC34 , and Z

    C67 . For the other

    types of renewable technologies, the values are less than50 MW. The highest installed capacities are in zones ZC62(539.6MW), ZC12 (395.1MW) and Z

    C31 (391.8MW). In these zones

    is also the largest number of renewable power plants: ZC62 (153power plants), ZC12 (117 power plants) and Z

    C31 (111 power

    plants).- cluster C2 (high voltage level): The installed capacities are highfor photovoltaic and wind power plants. Thus, in the zones ZC21and ZC23 , the installed capacity in wind power plants is over1000 MW, and in zone ZC26 is around 660 MW. The installedcapacity of photovoltaic power plants is over 400 MW in zonesZC24 and Z

    C28 , and zone Z

    C26 has a potential by approximately

    Fig. 16. Potential installed capacity [MW] of renewable power plants from zones ZC31 eZC32 (Cluster C3).

    Fig. 17. Number of renewable power plants from zones ZC31 e ZC32 (Cluster C3).

  • Fig. 18. Potential installed capacity [MW] of renewable power plants from zones ZC61 eZC63 (Cluster C6). Fig. 21. Number of renewable power plants from zones ZC21 e Z

    C28 (Cluster C2).

    G. Grigoras, F. Scarlatache / Energy 81 (2015) 416e429426350 MW. For other types of renewable technologies, values areless than 40 MW. The highest total installed capacity is in zonesZC23 , Z

    C21 and Z

    C26 (over 1000 MW). In zone Z

    C23 is the largest

    number of renewable power plants (42 power plants).- cluster C5 (extra high voltage level): At this voltage level, thelargest part of potential installed capacity is in wind powerplants in all zones. The highest installed capacity is in zones ZC53(2872.3 MW), ZC51 and Z

    C52 (with around 1400 MW). In zone Z

    C53

    is the largest number of power plants (26).

    The obtained zones from clustering process can be viewed asmaps. The representation of the zones is given in Figs. 24e27.

    Fig. 19. Number of renewable power plants from zones ZC61 e ZC63 (Cluster C6).

    Fig. 20. Potential installed capacity [MW] of renewable power plants from zones ZC21 eZC28 (Cluster C2).Fig. 22. Potential installed capacity [MW] of renewable power plants from zones ZC51 eZC55 (Cluster C5).As can be seen from Figs. 24e27, the obtained zones are spreadacross the entire surface of Romania. Thus, the clustering baseddivision led to an identication of technical exploitable potentialmore accurate, having a detailed assessment by zones, level volt-ages and type of renewable energy generation sources.

    Further, the analysis made in function by geographical locationhighlighted that the highest values of renewable energy potentialare in eastern part (zones ZC11 , Z

    C12 ,emediumvoltage level; Z

    C41 , Z

    C42

    e low voltage level; ZC21 , ZC22 e high voltage level; Z

    C51 , Z

    C52 e extra

    high voltage) with a total renewable energy potential by4841.42 MW (34.74% from total renewable energy potential), fol-lowed by Southeaster region (zones ZC13 e medium voltage level;ZC43 e low voltage level; Z

    C23 e high voltage level; Z

    C53 e extra high

    voltage) with 4400.48 MW (31.58% from total renewable energypotential), and Southern part (zones ZC62 , Z

    C63 , e medium voltage

    level; zones ZC44 , ZC45 e low voltage level; Z

    C24 , Z

    C25 e high voltage

    Fig. 23. Number of renewable power plants from zones ZC51 e ZC55 (Cluster C5).

  • lust

    G. Grigoras, F. Scarlatache / Energy 81 (2015) 416e429 427level) with 1507.84 MW (10.82% from total renewable energypotential).

    6. Concluding remarks

    Romania is a country with a great potential of renewable energysources as resulted from the analysis performed. Considering alarge database that contains the status of CTAs between years

    Fig. 24. Map of zones from c2012e2014 from Romania, the technical exploitable potential forelectricity generation from renewable energy sources (hydro,

    Fig. 25. Map of zones from cluster C1,biomass, biogas, cogeneration, photovoltaic, and wind) wasassessed using a clustering based Data Mining method.

    The proposed method extracts information from an enormousdatabase (containing data from all distribution and transmissioncompanies relating to status of CTAs) and transforms it into an easyunderstandable structure. The structure refers to zones for whichthe technical exploitable potential is identied more accurate thanin other approaches [6,7,11], where the assessment is made only for

    er C4 (Voltage level: 20 kV).total technical potential of Romania, without a detailed assessmentper zones, level voltages and type of renewable energy generation

    C3 and C6 (Voltage level: 0.4 kV).

  • ter C

    G. Grigoras, F. Scarlatache / Energy 81 (2015) 416e429428sources. As a consequence, the method is comfortable for CTAscrossing administrative boundaries of the counties/countries.

    On the other hand, the method is able to learn and to establish avery good dependence between considered variables for each CTAs:type of renewable source, installed capacity, voltage level, andgeographical position. The learning process isn't sensitive atdimension of database, new information might be inserted orcurrent data might be updated and/or deleted.

    Depending on the information included in database, the future

    Fig. 26. Map of zones from clustechnical exploitable potential was divided into 6 representativeclusters using proposed method. The most renewable energy po-tential is at the extra high voltage level (cluster C5 e 6222.7 MW,44.66%), followed by the high voltage level (cluster C2 e 5450 MW,

    Fig. 27. Map of zones from cluster C39.18%), medium voltage level (clusters C1, C3 and C6 e 2241.9 MW,16.09%), and low voltage (cluster C4 e 9.9 MW, 0.07%). Among alltypes of renewable sources that were analysed, the highest po-tential is given by wind energy (8631.9 MW, representing 61.95%from total renewable energy potential). On the second position, it isphotovoltaic energy (3869.2 MW, representing 27.77% from total).The others positions are occupied by: cogeneration (789.3 MW,5.66%), hydro (503.8, 3.61%), biomass (100 MW, 0.72%), and biogas(36 MW, 0.26%). From viewpoint of geographical location, the

    2 (Voltage level: 110e220 kV).highest values of renewable energy potential are in eastern partwith a total renewable energy potential by 4841.42 MW, followedby southeastern region with 4400.48 MW, and southern part with1507.84 MW.

    5 (Voltage level: 220e400 kV).

  • [18] Makrides G, Zinsser B, Georghiou GE, Schubert M, Werner JH. Potential ofPhotovoltaic Systems in Countries with High Solar Irradiation. http://www.researchgate.net/publication/222661460_Potential_of_ photovoltaic_

    G. Grigoras, F. Scarlatache / Energy 81 (2015) 416e429 429From analysis, a technical exploitable potential by 16.9 TWh/year was identied representing 52% of Romania's target. But, thispotential should lead to the main investments in electrical distri-bution and transmission networks from Romania in perspective2014e2020. In this regard, the maps obtained can be a usefulworking tool for retrotting substations, upgrading of transmissionand distribution lines and also for redesigning them at differentparameters with respect to the overload. The upgrading processwill be done, taking into consideration the technical infrastructurethat is divided into two: the distribution system and the trans-mission system. The renewable sources with small installed ca-pacities are connected to the low or medium voltage distributionsystem, where traditionally only consumers have been connected.The introduction of large amounts of themwill require investmentsnot only at the voltage level where the units are connected but alsoat higher voltage levels (110, 220 or 400 kV). The variation in pro-duction from renewable sources will conduct to new power qualityphenomena, typically at lower voltage levels (0.4 kV and 20 kV).Moreover, this variation as well as the difculty in predictingrenewable sources production will have an impact in the designand operation of electrical distribution networks. By upgrading ofelectrical transmission networks and building of the new lines, itwill ensure the evacuation of power from southeastern, eastern andsouthern parts and increasing the interconnection capacity withneighbour countries (Bulgaria, Republic of Moldova and Serbia).These investments will ensure the evacuation of power from southeastern part and increasing the interconnection capacity. The re-covery of all investments can be made through transmission tariffestablished by the competent authority on the basis of cost justiedin terms of a reasonable prot.

    References

    [1] Chicco G, Mancarella P. Distruibuted multi-generation: a comprehensive view.Renewable and Sustainable. Enery Rev 2009;13:535e51.

    [2] Zervos A, Lins C, Muth J. A 100% renewable energy vision for the EuropeanUnioneRe-thinking 2050. http://www.rethinking2050.eu/leadmin/documents/ReThinking2050_full_version_nal.pdf, [accessed March 2014].

    [3] Renewable Energy Progress Report Report from the Commission to the Eu-ropean Parliament, the Council, the European Economic and Social Committeeand the Committee of the Regions. 2013 [Brussels].

    [4] Hoogwijk M., Graus W., Global Potential of Renewable Energy Sources: ALiterature Assessment, http://www.ecofys.com/les/les/report_global_potential_of_renewable_energy_sources_a_literature_assessment.pdf,[accessed August 2014].

    [5] Lopez A, Roberts B, Heimiller D, Blair N, Porro G. U.S. renewable energytechnical potentials: a GIS-based analysis. Technical Report NREL/TP-6A20e51946. 2012.

    [6] Assessment of the Technical & Economical Potential of Wind Energy along theBlack Sea Coast of Bulgaria & Romania, Project NNE5/2001/733-SI2.318354,http://www.cres.gr/kape/pdf/download/Black_Sea.pdf.

    [7] Berger R., Green Energy In Romania http://rbd.doingbusiness.ro/en/5/latest-articles/1/373/green-energy-in romania, Roland Berger Strategy ConsultantsS.R.L, [accessed August 2014].

    [8] Calvert K, Pearce JM, Mabee WW. Toward renewable energy geo-informationinfrastructures: applications of GIS science and remote sensing that buildinstitutional capacity. Renew Sustain Energy Rev February 2013;18:416e29.

    [9] Wakeyama T, Ehara S. Estimation of renewable energy potential and use e acase study of Hokkaido, Northern-Tohoku area and Tokyo Metropolitan, Japan.In: World Renewable Energy Congress 2011, 8e13 May, Linkoping, Sweden;2011.

    [10] Kilias V, Nikolopoulos P, Vlachos J, Messaritakis S, Kabouris J, Tigas K.,Development of a Geographical Information System to Assess the Technicallyand Economically Exploitable RES Potential in Greece, http://www.gipsynoise.gr/HellasGI/2oSynedrio/Papers/Kilias_etal.pdf, [accessed August 2014].

    [11] Bujac F, Blarke MB. Evaluating the potential of renewable energy sources inRomania, program of sustainable energy planning and management. AalborgUniversity; 2011.

    [12] Lund H. Renewable energy strategies for sustainable development. Energy2007;32:912e9.

    [13] Sri M, Huld TA, Dunlop ED, Ossenbrink HA. Potential of solar electricitygeneration in the European Union member states and candidate countries. SolEnergy 2007;81:1295e305.systems_in_countries_with_high_solar_irradiation/le/e0b49517e0bec4e248.pdf, [accessed January 2014].

    [19] Brizard N. Hydro and wind: friends or foes?. http://www.leonardo-energy.org/hydro-and-wind-friends-or-foes, [accessed February 2013].

    [20] ENDS Europe, Renewable Energy Europe. A special report on the NationalRenewable Energy Action Plans outlining goals and measures to boostrenewable energy use, http://www.endseurope.com/docs/100930a.pdf,[accessed September 2010].

    [21] RES e Renewable Energy sources in Italy. INVITALIA- Government Agency forInward Investment PromotionandEnterpriseDevelopment.www.invitalia.it/site/eng/home/media.../resources/documento1395.html, [accessed February 2014].

    [22] Spain's National Renewable Energy Action Plan 2011e2020. http://pvtrin.eu/assets/media/PDF/EU_ POLICIES/National%20Renewable%20Energy%20Action%20Plan/202.pdf, [accessed January 2014].

    [23] Levitan D. Renewable Europe: Wind Power in EU to Triple by 2020., http://spectrum.ieee.org/energywise/green-tech/wind/renewable-europe-wind-power-in-eu-to-triple-by-2020, [accessed March 2014].

    [24] Wind in power 2013 European statistics. The European Wind Energy Asso-ciation; 2014.

    [25] Waldau AJ. PV Status Report 2013, European Commission, DG Joint ResearchCentre. http://iet.jrc.ec.europa.eu/remea/sites/remea/les/pv_status_report_2013.pdf, [accessed December 2013].

    [26] Solar Photovoltaic Electricity. http://www.erec.org/renewable-energy/photovoltaics.html, [accessed December 2013].

    [27] Hydro in Europe: Powering Renewables. Synopsis Report 2011; http://www.eurelectric.org/media/26440/hydro_report_nal_110926_01-2011-160-0005-01-e.pdf, [accessed January 2014].

    [28] Biomass for heat and power, Opportunity and economics. http://www.europeanclimate.org/documents/Biomass_report_-_Final.pdf, [accessedDecember 2013].

    [29] Biomass 2020: Opportunities, Challenges and Solutions. http://www.eurelectric.org/media/26720/resap_biomass_2020_8-11-11_prenal-2011-113-0004-01-e.pdf, [accessed January 2014].

    [30] http://en.wikipedia.org/wiki/Romania, [accessed December 2013].[31] Renewable energy country attractiveness indices. Ernst & Young; 2012.[32] National Action Plan for Energy from Renewable Sources. http://www.iee-

    interpares.net/wp-content/uploads/PNAER_29-iunie_2010_nal_Alx.pdf,[accessed December 2013].

    [33] Contents annual report. 2011. Hidroelectrica S.A. http://www.hidroelectrica.ro/showData.aspx? typedownload&docRAPORT_2013.pdf [accessedFebruary 2014]

    [34] Romanian Society Small Hydropower, http://www.asociatiamhc.ro/mhc-in-romania?langro [accessed September 2014].

    [35] Borz S., Derczeni R., Popa B., Nita, M. D. Regional Prole of the Biomass Sectorin Romania, http://www.foropa.eu/les/country_reports/country%20report%20romania.pdf, [accessed August 2014].

    [36] Maps and databases on the biomass potential. http://www.fao.org/uploads/media/biomass_maps_databases.pdf, [accessed November 2013].

    [37] Transelectrica S.A., www.traselectrica.ro, [accessed September 2014].[39] Annual report of National Authority for energy regulator. 2013. www.anre.ro

    [accessed January 2014].[40] Anders KH. Data Minning for Automated GIS data Collection. http://www.ifp.

    unistuttgart.de/publications/phowo01/Anders.pdf, [accessed November2013].

    [41] Usama F, Piatetsky-Shapiro G, Padhraic S. From data mining to knowledgediscovery in databases. AI Mag 1996;17:37e54.

    [42] Torra V, Domingo-Ferrer J, Torres A. Data mining methods for Linking DataComing from Several Sources. http://www.iiia.csic.es/~vtorra/publications/unrestricted/confUNECE.2003.143.150.pdf, [accessed March 2014].

    [43] Devedzic V. Knowledge Discovery and Data Mining in Databases. Handbookon Software Engineering & Knowledge Engineering, http://repository.binus.ac.id/2009-2/content/M0724/M072442942.pdf, [accessed March 2014].

    [44] Olaru C, Geurts P, Wehenkel L. Data mining tools and applications in powersystem engineering. In: 13th power systems computation conference, vol. 1;1999. p. 324e30.

    [45] Mirkin B. Clustering for Data mining. A data recovery approach. New York,USA: Chapman&Hall/CRC, Taylor and Francis Group; 2005.

    [46] Ca^rtina G, Grigoras G, Bobric EC. Clustering techniques in Fuzzy modeling.Power systems applications (in Romanian). Iasi: VENUS Publishing House;2005.

    [47] Jain K, Murty MN, Flynn PJ. Data clustering: a review. http://cermics.enpc.fr/~keriven/vision/articles, [accessed December 2013].

    [48] Rousseeuw PJ. Silhouettes: a graphical aid to the interpretation and validationcluster analysis. J Comput Appl Math 1987;20:53e65.

    [49] Yatskiv I, Gusarova L. The methods of cluster analysis results validation.Transp Telecommun 2005;6:75e80.

    [50] Gordon AD. Classication. 2nd ed. NewYork, USA: Chapman & Hall; 1999.

    An assessment of the renewable energy potential using a clustering based data mining method. Case study in Romania1. Introduction2. Renewable energy status in EU3. The renewable energy potential in Romania4. Mining Data in knowledge discovery from large database5. Study case6. Concluding remarksReferences