The Next Generation of Electric Power Unit Commitment Models

THE NEXT GENERATION OF

ELECTRIC POWER

UNIT COMMITMENT MODELS

INTERNATIONAL SERIES INOPERATIONS RESEARCH & MANAGEMENT SCIENCEFrederick S. Hillier, Series EditorStanford University

Saigal, R. / LINEAR PROGRAMMING: A Modern Integrated Analysis

Nagurney, A. & Zhang, D. / PROJECTED DYNAMICAL SYSTEMS ANDVARIATIONAL INEQUALITIES WITH APPLICATIONS

Padberg, M. & Rijal, M. / LOCATION, SCHEDULING, DESIGN ANDINTEGER PROGRAMMING

Vanderbei, R. / LINEAR PROGRAMMING: Foundations and ExtensionsJaiswal, N.K. / MILITARY OPERATIONS RESEARCH: Quantitative Decision MakingGal, T. & Greenberg, H. / ADVANCES IN SENSITIVITY ANALYSIS AND

PARAMETRIC PROGRAMMINGPrabhu, N.U. / FOUNDATIONS OF QUEUEING THEORY

Fang, S.-C., Rajasekera, J.R. & Tsao, H.-S.J. / ENTROPY OPTIMIZATIONAND MATHEMATICAL PROGRAMMING

Yu, G. / OPERATIONS RESEARCH IN THE AIRLINE INDUSTRYHo, T.-H. & Tang, C. S. / PRODUCT VARIETY MANAGEMENTEl-Taha, M. & Stidham , S. / SAMPLE-PATH ANALYSIS OF QUEUEING SYSTEMSMiettinen, K. M. / NONLINEAR MULTIOBJECTIVE OPTIMIZATIONChao, H. & Huntington, H. G. / DESIGNING COMPETITIVE ELECTRICITY MARKETSWeglarz, J. / PROJECT SCHEDULING: Recent Models, Algorithms & ApplicationsSahin, I. & Polatoglu, H. / QUALITY, WARRANTY AND PREVENTIVE MAINTENANCETavares, L. V. / ADVANCED MODELS FOR PROJECT MANAGEMENTTayur, S., Ganeshan, R. & Magazine, M. / QUANTITATIVE MODELING FOR SUPPLY

CHAIN MANAGEMENTWeyant, J./ ENERGY AND ENVIRONMENTAL POLICY MODELINGShanthikumar, J.G. & Sumita, U./APPLIED PROBABILITY AND STOCHASTIC PROCESSESLiu, B. & Esogbue, A.O. / DECISION CRITERIA AND OPTIMAL INVENTORY PROCESSES

Gal, T., Stewart, T.J., Hanne, T./ MULTICRITERIA DECISION MAKING: Advances in MCDMModels, Algorithms, Theory, and Applications

Fox, B. L./ STRATEGIES FOR QUASI-MONTE CARLOHall, R.W. / HANDBOOK OF TRANSPORTATION SCIENCEGrassman, W.K./ COMPUTATIONAL PROBABILITYPomerol, J-C. & Barba-Romero, S. / MULT1CRITERION DECISION IN MANAGEMENTAxsäter, S. / INVENTORY CONTROLWolkowicz, H., Saigal, R., Vandenberghe, L./ HANDBOOK OF SEMI-DEFINITE

PROGRAMMING: Theory, Algorithms, and ApplicationsHobbs, B. F. & Meier, P. / ENERGY DECISIONS AND THE ENVIRONMENT: A Guide

to the Use of Multicriteria MethodsDar-El, E./ HUMAN LEARNING: From Learning Curves to Learning OrganizationsArmstrong, J. S./ PRINCIPLES OF FORECASTING: A Handbook for Researchers and

PractitionersBalsamo, S., Personé, V., Onvural, R./ ANALYSIS OF QUEUEING NETWORKS WITH BLOCKINGBouyssou, D. et al/ EVALUATION AND DECISION MODELS: A Critical PerspectiveHanne, T./ INTELLIGENT STRATEGIES FOR META MULTIPLE CRITERIA DECISION MAKINGSaaty, T. & Vargas, L./ MODELS, METHODS, CONCEPTS & APPLICATIONS OF THE ANALYTIC

HIERARCHY PROCESSChatterjee, K. & Samuelson, W./ GAME THEORY AND BUSINESS APPLICATIONS

THE NEXT GENERATION OF

ELECTRIC POWERUNIT COMMITMENT MODELS

Editors

Benjamin F. HobbsThe Johns Hopkins University

Michael H. RothkopfRutgers University

Richard P. O’NeillFederal Energy Regulatory Commission

Hung-po ChaoElectric Power Research Institute

KLUWER ACADEMIC PUBLISHERSNEW YORK, BOSTON, DORDRECHT, LONDON, MOSCOW

eBook ISBN: 0-306-47663-0Print ISBN: 0-7923-7334-0

©2002 Kluwer Academic PublishersNew York, Boston, Dordrecht, London, Moscow

Print ©2001 Kluwer Academic Publishers

All rights reserved

No part of this eBook may be reproduced or transmitted in any form or by any means, electronic,mechanical, recording, or otherwise, without written consent from the Publisher

Created in the United States of America

Visit Kluwer Online at: http://kluweronline.comand Kluwer's eBookstore at: http://ebooks.kluweronline.com

Dordrecht

CONTENTS

Acknowledgments

I. The Evolving Context for Unit Commitment Decisions

vii

1.

2.

3.

Why This Book?: New Capabilities and New Needs for UnitCommitment ModelingB. F. Hobbs, W. R. Stewart Jr., R. E. Bixby, M. H. Rothkopf, R. P.O'Neill, H.-p. Chao

Regulatory Evolution, Market Design and Unit CommitmentR. P. O'Neill, U. Helman, P. M. Sotkiewicz, M. H. Rothkopf, W. R.Stewart Jr.

Development of an Electric Energy Market SimulatorA. Debs, C. Hansen, Y.-C. Wu

II. New Features in Unit Commitment Models

1

15

39

4.

5.

6.

7.

8.

Auctions with Explicit Demand-Side Bidding in CompetitiveElectricity MarketsA. Borghetti, G. Gross, C. A. Nucci

Thermal Unit Commitment with a Nonlinear AC Power FlowNetwork ModelC. E. Murillo-Sánchez, R. J. Thomas

Optimal Self-Commitment under Uncertain Energy and ReservePricesR. Rajaraman, L. Kirsch, F. L. Alvarado, C. Clark

A Stochastic Model for a Price-Based Unit Commitment Problemand Its Application to Short-Term Generation Asset ValuationC.-L. Tseng

Probabilistic Unit Commitment under a Deregulated MarketJ. Valenzuela, M. Mazumdar

53

75

93

117

139

III. Algorithmic Advances

9. Solving Hard Mixed-Integer Programs for Electricity GenerationS. Ceria 153

vi The Next Generation of Unit Commitment Models

10.

11.

12.

An Interior-Point/Cutting-Plane Algorithm to Solve the Dual UnitCommitment Problem -- On Dual Variables, Duality Gap, andCost RecoveryM. Madrigal, V. H. Quintana

Building and Evaluating Genco Bidding Strategies and UnitCommitment Schedules with Genetic AlgorithmsC. W. Richter, Jr., G. B. Sheblé

An Equivalencing Technique for Solving the Large-Scale Ther-mal Unit Commitment ProblemS. Sen, D.P. Kothari

167

185

211

IV. Decentralized Decision Making

13.

14.

15.

16.

Strategic Unit Commitment for Generation in Deregulated Elec-tricity MarketsA. Baíllo, M. Ventosa, A. Ramos, M. Rivier, A. Canseco

Optimization-Based Bidding Strategies for Deregulated ElectricPower MarketsX. Guan, E. Ni, P. B. Luh, Y.-C. Ho

Decentralized Nodal-Price Self-Dispatch and Unit CommitmentF. D. Galiana. A. L. Motto, A. J. Conejo, M. Huneault

Decentralized Unit Commitment in Competitive Energy MarketsJ. Xu, R. D. Christie

Index

227

249

271

293

317

ACKNOWLEDGMENTS

This volume contains papers that were presented at a workshop entitled"The Next Generation of Unit Commitment Models", held September 27-28, 1999 at the Center for Discrete Mathematics and Theoretical ComputerScience (DIMACS), Rutgers University, Piscataway, NJ. The editors grate-fully acknowledge the financial support of the co-sponsors of the workshop:DIMACS (funded by the National Science Foundation under grant NSFSTC 91-19999); and the Electric Power Research Institute (EPRI), whichsupported the publication and distribution of this book. The editors alsothank Sarah Donnelly of DIMACS for her organizational support of theworkshop and the subsequent book.

The editors would also like the many speakers and other participants ofthe workshop for their ideas and hard work. All the papers were subjectedto anonymous peer review by at least two referees and two editors. Thereferees included authors of other papers in this volume, along with PaulSotkiewicz and Judith Cardell of the Office of Economic Policy of the Fed-eral Energy Regulatory Commission.

Technical editing for the volume was ably provided by Debi Rager ofThe White Cottage Company. Liz Austin of Johns Hopkins Unviersitycompiled the index. Funding for B. Hobbs' involvement in the workshopand book came from NSF Grants ECS 96-96014 and 00-80577. Partialsupport for M. Rothkopf was provided by NSF grant SBR 97-09861.

This page intentionally left blank

Chapter 1

WHY THIS BOOK?NEW CAPABILITIES AND NEW NEEDS FORUNIT COMMITMENT MODELING

Benjamin F. HobbsThe Johns Hopkins University

William R. Stewart Jr.The College of William & Mary

Robert E. BixbyRice University and ILOG


Richard P. O'NeillFederal Energy Regulatory Commission

Hung-po ChaoElectric Power Research Institute

Abstract: This book presents recent developments in the functionality of generation unitcommitment (UC) models and algorithms for solving those models. These de-velopments, the subject of a September 1999 workshop, are driven by institu-tional changes that increase the importance of efficient and market responsiveoperation. We illustrate these developments by demonstrating the use ofmixed integer programming (MIP) to solve a UC problem. The dramaticallylower solution times of modem MIP software indicates that it is now a practi-cal algorithm for UC. Participants in the workshop also prioritized the featuresthat need to be considered by UC models, along with topics for research anddevelopment. Among the highest research priorities are: market simulation;bid selection; reliability and reserve constraints; and fair processes for choos-ing from alternative near-optimal solutions. The chapter closes with an over-view of the contributions of the other chapters.

2 The Next Generation of Unit Commitment Models

1. PURPOSE OF THE BOOK

The unit commitment problem can be defined as the scheduling of pro-duction of electric power generating units over a daily to weekly time hori-zon in order to accomplish some objective. The problem solution must re-spect both generator constraints (such as ramp rate limits and minimum up ordown times) and system constraints (reserve and energy requirements and,potentially, transmission constraints). The objective function should accountfor costs associated with energy production, ramping, start-ups and shut-downs decisions, along with possible effects upon revenues or customercosts of those decisions. The resulting problem is a large scale nonlinearmixed integer program.

For many years, the electric power industry has been using optimizationmethods to help them solve the unit commitment problem. The result hasbeen savings of tens and perhaps hundreds of millions of dollars in fuelcosts. Things are changing, however. Optimization technology is improv-ing, and the industry is undergoing radical restructuring. Consequently, therole of commitment models is changing, and the value of the improved solu-tions that better algorithms might yield is increasing. The purpose of thisbook is to explore the technology and needs of the next generation of com-puter models for aiding unit commitment decisions.

Because of the unit commitment problem's size and complexity and be-cause of the large economic benefits that could result from its improvedsolution, considerable attention has been devoted to algorithm development[1,2]. Heuristics such as priority lists have long been used by industry; but inthe last three decades, more systematic procedures based on a variety of al-gorithms have been proposed and tested. These techniques have include dy-namic programming, branch-and-bound mixed integer programming (MIP),linear and network programming approaches, and Benders decompositionmethods, among others. Recently, metaheuristic methods have been tested,such as genetic programming and simulated annealing, along with expertsystems and neural networks.

The solution approach that has been most successful, and which is mostwidely used at present, is Lagrangian relaxation. This procedure decomposesthe problem by multiplying constraints that couple different generators (suchas energy demand and reserve constraints) by Lagrange multipliers and plac-ing them in the objective function. Given a set of multiplier values, the prob-lem is then separable in the generating units, and a dynamic program of lowdimension can be used to obtain a trial schedule for each unit. A process ofmultiplier adjustment is used to search for feasible near-optimal solutions.

Lagrangian relaxation has proven useful for quick development of good,if not optimal, generator schedules. Recent improvements in integer pro-

Introduction 3

gramming codes and other algorithms suggest, however, that it may be pos-sible to find better solutions more rapidly. Further, such codes can morereadily incorporate additional coupling constraints, such as transmission lim-its and emissions caps. Meanwhile, restructuring has sharpened the appetiteof generation owners for more efficient operation. In the past, utilities soldpower on a regulated cost-plus basis and so may not have put as much prior-ity on squeezing out the last few percent improvements in the objective func-tion. Furthermore, system operators realized that cost functions were ap-proximate, so the operators were perhaps more likely to be satisfied withgood solutions or marginal improvements that were technically suboptimal.Now, with restructuring, we have schedule coordinators making commitmentdecisions in a market environment, and independent system operators (ISOs)dealing with bids. Bids are precise, and small improvements in solutions canresult in significant changes in payments to bidders. Further, the fact thatoptimization models are, in some cases, being used to determine which gen-erators will be operated and thus paid implies that there is a greater incentiveto get exact answers to make the bidding process fair and legitimate (and todisempower the bid-taker).

In other words, electric markets are changing rapidly, as is the role ofunit commitment models. How UC models are solved and what purposesthey serve deserve reconsideration. The goal of the workshop that led to thisbook was to bring together people who understand the problem and peoplewho know what improvements in algorithms are really possible. The papersin this book summarize the participants' assessments of industry needs to-gether with new formulations and computational approaches that promise tomake unit commitment models more responsive to those needs.

In Section 2 of this chapter, we present an example to show how the ca-pabilities of commercially available integer programming software to solvelarge unit commitment problems to optimality have dramatically improvedin recent years. This example illustrates how improvements in software maymake it possible to solve bigger problems in less time, while simultaneouslyincluding more of the complications that users want to represent. Section 3then summarizes the results of a survey of workshop participants in whichthey were asked to identify what issues concerning unit commitment model-ing are most in need of further research and development. Finally, in Sec-tion 4 we give an overview of the other chapters in this book. There, we de-scribe how the papers contribute to our two goals of articulating the emerg-ing needs of the restructured power industry and describing model develop-ments that can make unit commitment models more responsive to thoseneeds.


2. EXAMPLE OF NEW CAPABILITIES: SOLVINGUNIT COMMITMENT PROBLEMS USING MIP

When formulated as a mixed integer program (MIP), the unit commit-ment (UC) problem is a large and complex mathematical programming prob-lem. As a result, optimal solutions have been hard to obtain for practicallysized problems due to the exponential behavior of solution times as problemsgrow larger. As a result, exact solutions to large UC problems have beenunavailable, and approximate techniques are employed to produce solutionsthat are near optimal (within 0.5%-2%). In practice, Lagrangian relaxationmethods have performed well, but the non-convexity and overall size ofmost practical problems have prevented solving such problems to optimalityor even providing a bound on the optimal solution.

Approximate solutions have two problems. First, since an approximatesolution will be dispatching units that are slightly more costly than the onesthat would have been dispatched had an optimal solution been available, theapproximate solution almost certainly will be inefficient economically and adeserving facility may have been passed over by the approximate solution.The second, related problem consists of the political implications of passingover cost-effective units in a world in which generating units are owned bydifferent competing entities, as opposed to the historical situation where allgeneration was centrally owned by a single, large regulated utility.

The purpose of this section is to report on how the technology for solv-ing MIPs to optimality (branch-and-bound, cutting planes, etc.) has im-proved computational times in the past several years, and the implications ofthose improvements for solving UC problems. Historically, MIPs have beennotoriously difficult to solve due to the presence of multiple near-optimalsolutions such as would occur when a system contained several generatorswith similar operating characteristics and costs. Traditional branch-and-bound techniques would have to find all of these optimal solutions explicitlybefore an optimal solution could be verified. For this reason and because ofthe many near-optimal solutions in a typical large MIP, solution times growvery quickly with the size of the problem because of the large number ofnodes in the branch-and-bound tree that must be fathomed. However, anumber of theoretical improvements have been incorporated into commercialMIP codes, and further improvements are anticipated. Consequently, thesecodes can now solve to optimality problems thought impossible just a fewyears ago.

As a concrete example of these improvements, we summarize the resultsof applying several recent generations of a widely used MIP code, CPLEX®,to a test problem. The problem comes from a paper by Johnson et al. [3]. Itrequires the scheduling of 17 generators to meet electricity demand over a

Introduction 5

seven-day (168-hour) period. The generators have different operating char-acteristics and the resulting MIP contains 25,755 variables (including 2,856integer variables) and 48,939 constraints. While this may be a small prob-lem by unit commitment standards, until recently it would have been consid-ered intractable for direct solution by commercial MIP solvers using a linearprogramming/branch-and-bound/cutting plane technology. Recent advanceshave substantially accelerated these approaches, however; as a result, solu-tion times for problems such as the one below are approaching the rangewhere the MIP approach can be considered viable.

Test UC Mixed Integer Linear Program:

Minimize

subject to:

where:is the MW of energy produced by generator i in period t,is a binary variable that is 1 if generator i is dispatched during t,is 1 if generator i is started at the beginning of period t,is 1 if generator i is shut down at the beginning of period t,

is the MW of spinning reserves available from generator i in t,and are the fixed cost of operating ($/period), the cost of

generation ($/MW/period), and the cost of start-ups ($), respec-tively, for generator i during t,

and are the minimum and maximum MW ca-pacities of plant i, and its maximum reserve contribution,respectively, and


and are the maximum ramp rates (in MW/period) forincreasing and decreasing output, respectively, from unit i.

The seven-day MIP problem is non-convex, with one integer variablefor each generator for each hour of the week. The rest of the variables

are continuous. In addition to the formulation as shown in (1)-(10), the ob-jective function coefficients for some of the generators grow with out-put level. These are approximated by piecewise linear cost functions. Thereis also a set of constraints not shown in (1)-(10) that controls for the mini-mum number of periods a generator can be up and running and the minimumnumber of periods that a generator must be shut down.

The objective of UC is to choose a set of generators and operating levelsso as to minimize the total cost of the dispatch (1) subject to meeting hourlydemand (2), having sufficient spinning reserves (3) (arbitrarily set to 3% ofdemand in this problem), generation at each operating unit at or above itsminimum run level and below its maximum output level (4)-(5), and spin-ning reserves below the maximum level for each generator (6). Constraints(7)-(8) force hour-to-hour changes in generation to respect ramp rate limits.Constraint (9) requires that a generator be started if it was not dispatched theprevious period and will be this period, while (10) mandates that a unit beshut down this period if it is was dispatched the previous period and will notbe used this period.

The results for solving two related test problems are reported in Tables 1and 2. The first problem is a one-day (24 time periods) problem and theother is the full seven-day problem. The one-day problem has one-sevenththe number of continuous variables, discrete variables, and constraints thatthe week-long problem has. The main computational comparison is how thesolution times for these problems improve as they are solved by later ver-sions of a specific solver (CPLEX). In addition, we show the number oftimes a linear program is solved in the branch-and-bound portion of the algo-rithm (number of nodes).

Introduction 7

The marked improvement in times seen in Version 6.5 (Table 1) can beattributed to several factors: the inclusion of cutting planes eliminates a lot ofextraneous fractional solutions that were previously explicitly considered;later versions of the dual simplex algorithm in CPLEX just solve the linearprogram faster (a couple of orders of magnitude faster on the seven dayproblem, see below); and the inclusion of pre-solve reductions substantiallytightens the problem before branch-and-cut starts. The improvement oftimes from CPLEX 6.0 to CPLEX 6.5 of about an order of magnitude givespromise that this approach can be useful for solving UCPs in the future.

The seven-day (168 time periods) problem was unsolvable in a reason-able amount of time in CPLEX 4.0; indeed, it took an hour just to get a solu-tion to the linear programming relaxation. Yet, as Table 2 shows, that prob-lem was solved in less than two hours using CPLEX 6.5 and under 30 min-utes using a developmental version of CPLEX on a 500 MHz DEC Alpha.These results show that innovations in algorithms and their implementationswill continue to have a strong impact on solution times for MIPs and thatrealistic unit commitment problems can be solved to optimality by off-the-shelf software.

3. NEW NEEDS: RESEARCH PRIORITIES

The potential of improved algorithms was one important focus of theworkshop. The other was the functionality of unit commitment models. Theworkshop attendees participated in an afternoon meeting to evaluate thefunctions that unit commitment models should perform in restructured markets.The meeting was structured as a nominal group [4] to maximize the efficiencyof information exchange and to ensure that all forty or so participants had achance to voice their ideas. The steps of a nominal group exercise are asfollows: participants silently write down ideas in response to one or morequestions; the ideas are posted without attribution; the ideas are discussed inround robin fashion; and participants numerically rate the ideas either at themeeting or in a questionnaire after the meeting.

The workshop participants performed the following tasks:


Identify features, characteristics, or requirements of system operation andbidding that are desirable to include in unit commitment models, and issuesinvolved in representing these issues in those models.Describe the extent to which additional research is needed to incorporatethose features in unit commitment models. This was done by grouping thefeatures into three categories:

I

II

III

This feature/characteristic/requirement is modeled by existingmodels and software;Models have been proposed for capturing this feature/characteristic/requirement, but require development andimplementation; andThere remain fundamental disputes or uncertainties as to howthis feature/characteristic/requirement should be modeled. Inthis case, fundamental research is needed.

Rate the importance of including each feature/characteristic/requirement onthe following Likert scale:

1 Minor importance3 Somewhat important5 Very important7 Crucial

Approximately 40 of the workshop attendees addressed the first two tasksduring the afternoon session. Eighteen participants returned the follow-upquestionnaire on importance rating. The latter group included nine industryrepresentatives (primarily consultants, but also including generating companyand EPRI representatives); eight university researchers; and a regulator. Fromthe results of these tasks, a set of research priorities can be distilled.

The following two tables summarize the results. Table 3 ranks the featuresby the mean importance rating given by the group, while Table 4 categorizesthem in terms of whether further research is needed. Where participantsdisagreed concerning the category, a range is shown.

Table 3 indicates near unanimity concerning the crucial importance ofincluding the commodities of energy and ancillary services along with dynamicconstraints and decision sequencing. The next most important issue wasmarket prediction, interpreted as the projection of prices, and how the marketmight respond to alternative bidding strategies.

Some importance ratings were mildly surprising. In particular, transmissionissues were rated relatively low. It would seem that research on transmission is

Introduction 9

Introduction 11

not a high priority with this group, as inclusion of AC representations fell incategory II and received a moderate importance rating, while DCapproximations were in I and received one of the lowest average importanceratings. Other features that received low ratings included distributed


generation; emissions modeling; and particular stochastic methods formodeling reliability, uncertainty, and variability.

Table 4 reveals that the group unanimously identified 11 features as re-quiring fundamental research (category III), of which nine received averageimportance ratings of 4.0 or more. These features are extremely diverse,ranging from risk and reliability modeling to strategic bidding, new productvaluation, and dealing with the fairness and institutional implications of mul-tiple near-optimal solutions.

Another six features were thought by some but not all group members tobe in category III and so are identified as being in category “II-III.” Severalfacets of market prediction were highly rated in that category. These facetsinclude price prediction, bidding strategy assessment, and inclusion of price-responsive demand. Meanwhile, in the “requires development” category (II),there are two very important topics: inclusion of ancillary services in unitcommitment models and explicit accounting for the sequential nature of de-cisions (such as multi-settlement systems). Research of a more applied na-ture would seem to be justified in these cases.

4. BOOK OVERVIEW

The chapters of this book are grouped into four sections. The first sec-tion includes this introduction and two other chapters that summarize theevolving institutional context that has motivated the functional and algo-rithmic developments described in the rest of the book. O'Neill, Helman,Sotkiewicz, Rothkopf, and Stewart (Chapter 2) review the recent history ofshort-term electricity markets in the U.S., focussing on alternative marketdesigns and the implications for unit commitment modeling. They also sug-gest some principles for designing the next generation of UC market models.An alternative approach to presenting the evolving context of unit commit-ment is presented in Chapter 3. There, Debs, Hansen, and Wu present a gen-eral modeling framework that encompasses all the functions of short-termenergy markets, including commitment, with a focus on whether the marketparticipant is an ISO/RTO, generating company, market administrator, loadserving entity, or even an energy service company.

The other three sections of the book describe novel applications and fea-tures in UC models, new algorithms for solving those models, and modelingapproaches that represent decentralized commitment by independent generat-ing firms. Chapter 4 is the first chapter in the second section; there, Bor-ghetti, Gross, and Nucci show how demand-side bidding can be included inLagrangian-relaxation-based unit commitment models, and how such bid-ding can dampen price volatility and mitigate market power. Their formula-

Introduction 13

tion represents load recovery, hour-by-hour bids for demand reduction, andmultiple bidders. Chapter 5 by Murillo-Sanchez and Thomas is the secondchapter on new model features. They describe how a nonlinear AC powerflow representation can be incorporated, with both active and reactive powersources. They also discuss a parallel processing implementation.

The last three chapters of the second section concern the inclusion ofprice uncertainty in UC models. All use variations of a stochastic dynamicprogramming-based commitment model; when generators cannot individu-ally influence price, the models neatly decompose into a single optimizationmodel for each generator representing their self-commitment problem. InChapter 6, Rajaraman, Kirsch, Alvarado, and Clark describe how uncertain-ties in both reserve and energy prices can be considered in such models. InChapter 7, Tseng shows how such models can be used to quantify rigorouslythe worth of operating flexibility (“option value”) for a single generatingasset. Finally, in Chapter 8, Valenzuela and Mazumdar present a model forsingle generator optimization that uses probability distributions of marketprices directly derived from assumptions concerning demand variability andgenerator availability in the whole market.

In the third section of the book, we turn our attention to improved algo-rithms for solving the UC problem. Four distinct approaches are representedin this section: mixed integer programming, Lagrangian relaxation, geneticalgorithms, and aggregation approaches.

Chapter 9 by Ceria, like Section 3 of this introduction, addresses the use-fulness of MIP for UC, along with recent developments in MIP technologythat have drastically improved solution times. He also briefly reviews twoactual applications by utilities in Europe. Chapter 10, authored by Madrigaland Quintana, proposes an interior-point/cutting-plane algorithm to solve theLagrangian relaxation problem and demonstrates its computational advan-tages over subgradient and other methods traditionally used to update La-grange multipliers. They also offer some observations on several issues in-volved in using UC models to clear power markets, including duality gaps,cost recovery, and the existence of multiple solutions. Chapter 11, contrib-uted by Richter and Sheble, reviews a range of considerations involved increating bidding and commitment strategies. They then propose genetic al-gorithms and finite state automata-based simulations for strategy develop-ment and testing. Chapter 12 by Sen and Kothari shows how aggregation ofgenerating units into a few sets of similar units can be exploited to improvesolution times. The algorithm involves three basic steps: aggregation; solu-tion of the simplified UC problem using dynamic programming or anotheroptimization method; and disaggregation to create schedules for individualunits.


The fourth and final section of the book contains four chapters that ad-dress modeling and algorithmic issues associated with decentralized com-mitment decision processes. Important questions include opportunities forstrategic manipulation of prices by market participants, coordination algo-rithms, and the ability of decentralized processes to approach optimality.

The first two chapters of the fourth section focus on decision-making byindividual firms. Baillo, Ventosa, Ramos, Rivier, and Canseco devote Chap-ter 13 to a model for committing a firm’s units while recognizing how com-mitment and dispatch decisions may affect market prices. Rival firms areassumed to behave according to price-elastic supply functions, which allowsfor derivation of total firm revenue as a function of its output. They useMIP to solve their model. In Chapter 14, Guan, Ni, Luh, and Ho describetwo general approaches to bid development in decentralized markets. One isbased on “ordinal optimization” for obtaining satisficing bidding strategies,and a second uses stochastic optimization to self-schedule and manage riskswhile considering interactions among different markets.

The last two chapters of the fourth section turn to the issue of coordina-tion of decentralized decisions. Galiana, Motto, Conejo, and Huneault(Chapter 15) propose a coordination process in which locational prices areannounced, generating firms self-dispatch to maximize their individual prof-its, and prices are adjusted to ensure that demands and network constraintsare satisfied. The authors use a Newton algorithm to update prices, and im-pose a “convexifying rule” to facilitate convergence. Case studies illustratethe process. In Chapter 16, Xu and Christie consider the interactions of stra-tegic behavior by individual generating firms with a price-based coordinat-ing mechanism. Firms optimize bidding strategies with the help of a simpleprice prediction model. The combined effects of multiple firms using thatapproach is explored with a market simulator, which reveals that conver-gence, feasibility, and price stability can be difficult to achieve, but that pricecycling can be dampened with alternative price prediction models.

REFERENCES

1.

2.

3.

4.

G. Sheblé and G. Fahd. Unit commitment literature synopsis. IEEE Trans. Power Syst.,9(1): 128-135, 1994.S. Sen and D.P. Kothari. Optimal thermal generating unit commitment: a review. Elec.Power Energy Syst., 20(7): 443-451, 1998.R.B. Johnson, S.S. Oren, and A.J. Svoboda. Equity and efficiency of unit commitmentin competitive electricity markets. Utilities Policy, 6(1): 9-20, 1997.A. Delbecq, A. Van de Ven, and D. Gustafson. Group Techniques for Program Plan-ning — A Guide to Nominal Group and Delphi Processes. Glenview, IL: Scott Fores-man and Co., 1975.

Chapter 2

REGULATORY EVOLUTION, MARKET DESIGNAND UNIT COMMITMENT

Richard P. O’Neill, Udi Helman, and Paul M. SotkiewiczFederal Energy Regulatory Commission


William R. Stewart Jr.The College of William and Mary

Abstract: In the context of competitive wholesale electricity markets, the unit commit-ment problem has shifted from a firm level optimization problem to a marketlevel problem. Some centralized market designs use it to ensure reliability anddetermine day-ahead market prices. This chapter reviews the recent history ofshort-term electricity markets in the United States to evaluate the experiencewith alternative market designs and the implications for unit commitment mod-eling. It presents principles for the design of the next generation of unit com-mitment-based markets.

1. INTRODUCTION

Competitive wholesale electricity markets now operate in several majorU.S. markets, confirming the analysis and recommendations of prescienteconomists, electrical engineers, and others over the past two decades.1 Sincegeneration comprises approximately 75 percent of all electricity costs, com-

1 Seminal contributions on competitive wholesale electricity markets include [1,2]. As ofJanuary 1, 2000, regional markets with centralized wholesale electricity exchanges are opera-tional in California, the Pennsylvania-New Jersey-Maryland (PJM) interconnection, NewEngland, and New York.


petition in generation promises large efficiency gains and cost savings to con-sumers. The unit commitment problem, the traditional method by which regu-lated utilities and power pools conducted internal scheduling of generation tomeet demand at least cost over a multi-hour to multi-day time frame, is nowembedded, in various ways, in competitive markets. Potentially, unit com-mitment models will be used by different market participants and institutions:individual firms, centralized auctioneers, decentralized aggregators of genera-tion schedules, and transmission system operators. This environment presentsa new set of modeling requirements and market design challenges.2 The mar-ket level unit commitment problem is typically much larger in scale than thefirm level problem. Speed and accuracy are important if an auctioneer usesthe solution by an auctioneer to determine market prices. Evaluating the char-acteristics of the solution, such as the presence of duality gaps (implying alack of market clearing prices) and alternative optima, becomes of direct fi-nancial interest to market participants.

The new electricity markets, and hence new applications of the unit com-mitment problem, are being developed within an evolving regulatory context.Indeed, an important driver of market designs is the guidance given by theregulator. The Federal Energy Regulatory Commission (henceforth “theCommission”) initiated regulatory reform of transmission in 1996, with theobjective of encouraging competitive regional electricity markets that pro-mote economic efficiency without compromising system reliability. The regu-latory approach, embodied in a series of orders described below, has been toprovide an open market architecture where alternative market designs are im-plemented, evaluated, and changed when necessary. Research into the unitcommitment problem has largely been reactive to the new regulatory envi-ronment and the emerging issues in market design. A more proactive ap-proach is needed. Among the issues that need consideration and research arethe choices between simultaneous and sequential optimization of several en-ergy and ancillary service products, alternative bidding rules for differentproducts, different mechanisms for congestion pricing, and inter-regional co-ordination. In addition, unit commitment modeling now has to confront theissue of economic incentives in various market settings, which requires amore extensive familiarity with economics and game theory.

In response to the regulatory evolution it has set in motion, the regulatoralso needs to adapt institutionally and develop its technical capabilities. Thisis imperative because the Commission is taking an oversight role in marketdesign decisions across the United States. Several wholesale markets operatecentralized unit commitment auction markets (e.g., PJM, New England, and

2 The literature on market design in electricity markets is extensive; for a survey, see the arti-cles in [3].

Regulatory Evolution, Market Design, and Unit Commitment 17

New York), the market design of which is the focus of this chapter. Thesemarkets also allow bilateral trading and types of self-scheduling. Followingapproval of the basic design, the Commission provides oversight for a floodof subsequent adjustments and refinements in the search for well functioningmarkets. The underlying unit commitment model is often either an implicit orexplicit matter in these market rule decisions.

The objective of this chapter is to describe regulatory evolution and themarket design challenges for unit commitment modeling. The chapter focuseson day-ahead markets, but much of the discussion is also applicable to real-time markets. Section 2 of the chapter describes the key regulatory develop-ments and the design and recent experience of the major regional wholesaleelectricity markets. Section 3 focuses on principles that should guide the de-sign of day-ahead energy and ancillary service markets. Finally, Section 4offers conclusions.

2. REGULATORY EVOLUTION AND THEORGANIZATION OF ELECTRICITY MARKETS

The recent history of electricity regulatory reform in the United States be-gan when the Commission issued Orders 888 and 889 in 1996 [4,5]. Theseorders required an open access transmission regime, based on non-discriminatory transmission rates and transparent posting of available trans-mission capacity (ATC). Order 888 also included fairly broad organizationalprinciples for an independent system operator (ISO), an institution whichseparates ownership from control of the grid and can perform market func-tions. Between 1997 and 2000, ISOs and power exchanges (PXs) were formedin California and in the three tight power pools of the eastern United States.These ISOs established day-ahead and real-time markets for energy, ancillaryservices, and transmission (in California, the day-ahead energy market is con-ducted by several separate scheduling coordinators, including the CaliforniaPower Exchange). An ISO has also been established in the Midwest, but is notyet operational. Other regions of the country have been less successful or un-willing to centralize grid operations, and electricity trading remains bilateral,with a vertically integrated utility performing the balancing and reliabilityfunctions.

By early 1999, a certain amount of inertia was evident in the developmentof wholesale markets. Electricity traders expressed dissatisfaction with thetraditional methods of transmission grid management still employed in largeparts of the United States. Specifically, there was substantial concern about


frequent curtailments of transactions, justified on the basis of reliability butoften questioned by parties to the transactions.3

On December 15, 1999, the Commission took a step toward clarifying theappropriate transmission access and market institutions with Order 2000,which requires the formation of regional transmission organizations (RTOs)[6]. The order establishes in the RTO many of the features that had emergedin the ISO markets as well as additional characteristics and functions that ad-dress unresolved issues in both ISO and non-ISO electricity markets. TheRTO is required to serve a region of sufficient scope and configuration toprovide for a reliable, efficient electricity market. With respect to the unitcommitment problem, some of the important features of the RTO are that itmust have exclusive authority for maintaining short term reliability, act asprovider of last resort of ancillary services, address parallel path flows, pro-vide real time energy balancing, and ensure development of market mecha-nisms for congestion management.

In many ways, the functions assigned to the RTO are based on the princi-ples of market design embodied in the better functioning ISOs that haveemerged (in fact, RTOs will subsume existing ISOs). Section 2.1 surveyssome of these ISO market design lessons; Section 2.2 discusses further theobjectives of Order 2000 and some market design issues.

Before considering the details of open access and market design, an im-portant question is: Why should the regulator remain involved in the designand oversight of the emerging competitive markets? Recent experience hasmade clear that, in the near term, the Commission has a continuing role for atleast three reasons:

1.

2.

3.

There is the ongoing development of open access itself, including theprovision of short-term reliability services linked to transmission andfuture expansion of the grid. There are public good aspects to reliabil-ity and transmission expansion.4

Where not currently available, there must be an efficient pricingmechanism for transmission congestion (i.e., pricing of the external-ities created by parallel path flows).There must be mitigation of market power in markets for electricityand ancillary services and in provision of transmission capacity. Mar-ket power is the ability of firms to raise prices above competitive lev-

3 Such curtailments are supposed to follow the North American Electricity Reliability Coun-cil’s (NERC) Transmission Loading Relief (TLR) procedures, which provide criteria for themanagement of congested transmission facilities.

4 A public good is a good that is non-rivalrous and non-excludable. In the case of reliability, aload’s or generator’s consumption of reliability in no way prevents others from enjoying thesame reliability, and if all of the loads and generators are interconnected on the same system,they cannot be prevented from enjoying the benefits of reliability.


els. Firms can exercise market power in electricity markets because ofboth structural factors (e.g., firm concentration or transmission con-straints) and opportunities offered by the market design (see [7,8] fora survey).

The Commission’s policy heretofore has been to approve implementationof the markets (through more liberal standards for granting market-basedrates) while at the same time evaluating the markets’ operational experienceand providing guidance on market designs as a means to promote efficiencyand competition. In addition, most of the responsibility for day-to-day moni-toring and mitigation of market power has shifted to the ISOs and the futureRTOs. The regulatory approach, then, seeks to balance the current reality -some firms can exercise a degree of market power generally or under certainsystem conditions – with the expectation that entry of new firms and moreefficient market designs will substantially mitigate future market power.5

2.1 Experience with ISO and Bilateral Markets

Order 888 outlined principles for, but did not require, a particular struc-ture for competitive wholesale energy markets. Two broad types of marketstructures have developed. The first type consists of markets with ISOs, whichmay or may not include one or more scheduling coordinators or PXs.6 TheISO markets with PXs take the form of either a centralized ISO/PX or a de-centralized ISO and PX(s). The centralized ISO/PX markets use unit com-mitment models for creating the day-ahead schedule (which incorporates bi-lateral transactions and self-schedules) while the decentralized ISO marketsrequire self-commitment by the PXs. The second type of market has no ISO;rather, the transmission system and much of the generation continues to beoperated by vertically integrated utilities. Power is traded bilaterally.

This section focuses on the various market designs and performance ofthe existing ISO markets, as well as offer some conclusions and recommenda-tions on what designs will work best. It also includes a brief review of the per-formance of the bilateral, non-ISO markets.

Market Functions and Design of ISO Markets. Competitive wholesale

5 There is a large body of literature on market power due to both structural and market designcharacteristics of electricity markets. For analysis of regional energy and ancillary servicemarkets in the United States, see [8-12].

6 Scheduling coordinators or power exchanges (PXs) are power trading operations functionallyseparate from the ISO. These terms will be used interchangeably.


electricity markets can be complex, with multiple interdependent productssold on different time frames and differentially priced at different locations.Existing ISO markets can be characterized by the (1) number and types ofdifferent products (energy, ancillary services, capacity), (2) bidding andscheduling process, (3) relationship of temporal (forward and real-time) mar-kets, (4) market clearing and settlement rules, and (5) type of congestionmanagement and transmission rights. Each of the existing ISOs has also es-tablished market power monitoring and mitigation, but this function will notbe examined here.

The ISO carries out the basic function of assessing the feasibility of pro-posed generation schedules. The ISO also serves as the buyer, through con-tracted rates and bid-based auctions, of reliability services, including short-term ancillary services (voltage support, operating reserves, and automaticgeneration control) and possibly longer-term capacity products. The PX facili-tates and conducts a forward auction market for electric energy. PX functionscan either be carried out by the ISO itself, as in New York, New England, andPJM, or by one or more separate, unaffiliated PXs, as in California.

With the exception of the California ISO, the ISOs run a unit commitmentmodel to determine which units will be scheduled to provide energy and an-cillary services during the following day.

In California, the ISO and PX (which is one of several scheduling coordi-nators) are separate, a decision intended to keep the transmission system op-erators (who may have been affiliated with an incumbent utility) functionallydistinct from the market. In the California PX market, generation owners self-commit their units through scheduling coordinators. This market structure hasexperienced certain disadvantages. One problem is that the scheduling coor-dinators’ submissions can be physically infeasible. The ISO must then engagein a time consuming iterative process with the scheduling coordinators to re-solve the infeasibilities. The PX auction algorithm is such, however, that self-committed units are often asked to start-up and stop, disregarding minimumrun and down times, with potentially adverse results.7 Another problem is thatthe separation of the ISO and PX raises the transaction costs for market par-ticipants. In addition, because the ISO’s decisions about congestion and pur-chases of ancillary services cannot be remedied by the PX, market participantsmay bid strategically into the ISO’s congestion market to ensure that profit-able transactions are not curtailed.

Bidding and Scheduling. The ISO markets began, largely, with onlysupply-side bidding for energy and certain ancillary services. While demand-side bidding for energy is allowed in some markets (currently the CaliforniaISO and PX and New York, but planned for the other ISOs), the energy de-

7Conversations with California ISO staff confirm this problem.


mand function is largely inelastic – that is, not price responsive – due to bothtechnical limitations and historic rate designs. As more price responsiveness isintroduced into demand bid functions (through installation of meteringequipment and technological advances in distributed generation and informa-tion technology), there should be a reduction in both price volatility and thepotential for exercise of market power, particularly during peak hours.

The structure of bids is another market design issue that has attracted at-tention. Bids in current ISO energy markets vary in the number of cost com-ponents and required technical parameters, such as ramp rates, high and lowoperating limits, and so on. In the so-called “one-part” incremental energybid, the bidder must factor its start-up, no load, and other costs into its day-ahead energy bid for each megawatt-hour (MWh) offered.8 Even so, genera-tors face the risk that they may not cover all of their costs in the auction. One-part bids require generation owners to internalize this risk in some fashion,which in turn increases their costs (use of the real-time market to make ad-justments can eliminate part of this risk).9 One-part bids also result in ineffi-ciency if they are the only costs the dispatcher considers in commitment.

In a three-part bid, the start-up and no-load costs can be separated out,allowing generators to bid actual operating costs more precisely and allowingfor a more efficient unit commitment. In PJM and New York, generators areguaranteed to at least recover all of their bid costs if they are committed torun.10 This mechanism eliminates the uncertainty of whether a generator willbe committed and dispatched only to lose money, and it allows for a moreefficient dispatch.

Market-Clearing and Settlement Systems. Market-clearing rules andsettlement systems are the procedures that determine quantities produced andconsumed, who pays, and who gets paid. As discussed above, ISOs typicallyoperate multiple markets, including energy, several types of ancillary ser-vices, and transmission products. There are two basic ways to clear these dif-ferent markets, sequentially or simultaneously, with variations on eachmethod. In general, sequential auction markets clear each product separately

8 Currently, the California PX requires one-part energy bids without technical parameters, suchas minimum run times, ramp rates, and so on. This could be called a pure one-part bid. NewEngland currently requires one-part bids with technical parameters.

9 At its worst, the need to internalize risk in the one-part bidding system could lead to a greaterincentive to internalize via bilateral contract or merger to avoid higher transactions costs.High market concentrations lead to market power concerns. Further, not being allowed to bidmarginal cost is an easy defense to an inquiry on market power abuse.

10 Generators will receive an “uplift” payment to recover their costs only if the revenues theyreceive from the energy and ancillary services markets are less than their total bid costs.


in a sequence, even though several of the products may be alternative (substi-tute) uses of the same generator. Variations of this approach were adoptedinitially in California and in the interim New England market. In contrast, asimultaneous auction, adopted in PJM and New York, clears the relevantmarkets at the same time, minimizing the joint bid cost of providing energyand ancillary services. This method explicitly takes into account that someproducts are substitutes.

One of the clearest market design lessons from ISOs is that sequentialmarket clearing without product and/or quantity substitution is economicallyinefficient and offers opportunities for strategic behavior. In California, en-ergy, regulation, 10-minute spinning reserves, 30-minute non-spinning re-serves, and replacement reserves are cleared in the order given.11 It has some-times been the case that the price for ancillary services with lower productioncosts exceeds the price of ancillary services (and energy) with higher produc-tion costs (for example, providing spinning reserve requires using some fuel,whereas providing non-spinning reserve requires simply being on stand-by).The reason for some of these “price inversions” is that generators can strategi-cally bid high prices in the last markets knowing that there will not be muchcapacity remaining after the other markets clear. Hence, the ISO must takethese high price bids. To combat this design flaw, the California ISO has insti-tuted a pre-processing algorithm, called the Rational Buyer Protocol, whichwill allow it to substitute higher quality services for lower quality services ifand only if it reduces its ancillary service procurement cost. New Englandalso initially used sequential market clearing and experienced problems simi-lar to California; temporary solutions have included rolling over of bids be-tween substitute services, as in California, as well as price caps.

Experience with simultaneous market clearing is limited. The simultane-ous market clearing method will largely, but not entirely, avoid the price in-versions seen in the California ISO and New England markets because thesoftware used to clear the markets automatically clears all remaining arbitrageopportunities. In addition, generators which bid strategically, as in California,would be far less likely to be selected to provide ancillary services at thehigher price due to the greater substitution possibilities in the simultaneousmarket clearing. While more complex computationally, simultaneous market-clearing appears to be emerging as the better system from an efficiency stand-point.

11 In California, regulation refers to automatic generation control, but is defined in terms ofwhether generation output is increased (regulation up) or decreased (regulation down). Spin-ning reserves is reserve capacity available in a specified time period from a generator syn-chronized with the grid. Non-spinning reserves are reserve capacity available in a specifiedtime period from a generator not synchronized with the grid. Replacement reserves are re-serves that can be available within 60 minutes.


The settlement systems can be characterized as either single settlement ormulti-settlement based on the number of temporal markets the ISO runs. InCalifornia, there are three temporal markets: day-ahead, hour-ahead, and real-time with a financial settlement for each. New York and PJM run two tempo-ral markets: day-ahead and real-time. New England currently has only real-time markets (bids are due day-ahead but financial settlement only takes placeat real-time prices), but is scheduled to implement day-ahead markets withfinancial settlement. A clear market design lesson is that single-settlementsystems, which require generators to submit bids and stand-by day-aheadwhile awaiting financial settlement at real-time prices, create problems inscheduling and often require additional rules to constrain generator incentivesto change their bids. The multi-settlement system has been adopted by all theISOs in recognition of the value of the forward market as a financial hedge forreal-time conditions. Also, the forward market should facilitate demand-sideresponses by giving demand that has bid to reduce load more time to react toprice signals.

Congestion Management and Pricing. In the ISO context, there are twogeneral ways to manage congestion: locational pricing and non-locationalpricing. Locational pricing can be sub-divided into approaches defined by thelevel of aggregation used to calculate the price. In the typical “nodal pricing”method, an energy price is calculated at each generation and load bus (node).The transmission congestion price between any two busses is the difference inenergy prices at the busses.

At higher levels of aggregation, the busses in the system operated by anISO can be gathered into one or more congestion zones. Zones are intended tobe indicative of the historical pattern of congestion in the system on the pre-sumption that congestion will take place between the zones with little or nocongestion within a zone. The price of congestion between zones is the differ-ence in energy prices between the zones. If congestion occurs within a zone,the costs of managing it (typically through generator re-dispatch) are sharedby all market participants inside the zone using a system of subsidies.12 Thisintrazonal congestion management method could be considered a type of non-locational pricing.

12 Several ISOs have attempted to operate as single zones (PJM, New England), but have sub-sequently made the transition to locational pricing. If the single zone system has consistentcongestion between sub-regions (that is, should be at least two zones), this can create oppor-tunities for generators to leave the spot market and use bilateral contracts to take advantageof the system price. This was the experience in PJM before it implemented a nodal system;the ISO was required to adopt administrative measures to curtail the bilateral transactions.


The zonal approach has been adopted in California, which currently hastwo (soon to be three) congestion zones but is experiencing congestion thatshould trigger new zones (also, each import point effectively creates a newzone). The California ISO manages inter-zonal congestion through adjustmentbids submitted by generation and load. These bids indicate the price at whichthe market participant is willing to be ramped up or down in order to alleviatecongested lines. If this fails to relieve the congested lines, then the ISO mustcall on generators with cost-based contracts to relieve congestion.13

California manages intra-zonal congestion by re-dispatch (which incorpo-rates the transmission constraints into the original, transmission unconstraineddispatch) with the resulting costs averaged over load in the zone. Persistentintrazonal congestion indicates that the zones are not properly defined; in ad-dition, the averaging of congestion costs within the zone is inefficient, sincethe congestion costs are also paid by participants not causing the congestion.14

In the long run, zonal pricing as practiced in California can lead to pricesignals that distort decisions on siting new generation and transmission as-sets.15 Neither maintaining fixed zones in the face of intrazonal congestionnor continuous re-zoning are efficient methods of congestion management.

Zonal market design in California has been instituted in part under therationale that it lowers market power. Both in theory and practice this assump-tions has been proved wrong. Market power cannot be reduced by the declara-tions of large zones. If this were so, there would be no market power problem.Transmission constraints and generation costs determine the size of the mar-ket, not the declaration of zones. The California ISO rules recognize this byproviding for dispatch orders and out-of-market payments to generators in thesame zone separated by constraints.

In contrast, despite opposition from some generators and marketers, nodalpricing has been adopted in New York and PJM and approved for New Eng-land (these systems actually use nodal prices for generators and zonal aver-ages for loads). Nodal pricing eliminates the problem of properly defining

13 These are called Reliability-Must-Run (RMR) contracts. RMR contracts are intended toensure that the ISO has sufficient generation capacity to meet various system contingencies,such as congestion relief and voltage support.

14 The California ISO can create a new congestion management zone if the cost to alleviatecongestion over the previous 12 months exceeds 5 percent of the approximate annual reve-nue requirement of the transmission operators. In order to be considered an active congestionzone, the markets on either side of the congested interface must be “workably competitive”for significant portions of the year.

15 For example, the Commission has rejected a California ISO proposal (Tariff AmendmentNo. 19, filed June 23, 1999) that new generators upgrade transmission capacity to alleviateintrazonal congestion which might arise from their entry on the grounds that it could createfurther barriers to entry and market distortions. A similar New England proposal was alsorejected.


zones and the need to average the costs of any intrazonal congestion. In theshort run, load receives the proper price signals about how much to consume,and the long-run decisions can be made much more easily. Even though loadspay a zonal price, the nodal price information remains available for decisionmaking. For financial markets, nodal prices for a region can be aggregatedinto fewer “hub” prices, which are weighted averages of the underlying nodalprices. For example, PJM has two hub prices.

Transmission Rights. Transmission rights have traditionally been used toreserve access to the transmission system and to ensure that energy transac-tions would be curtailed only in extreme circumstances. These rights werephysical rights – the right to transmit physically a specific amount of powerover the system for the access charge paid. With the advent of congestionpricing (whether zonal or nodal), most ISOs have provided both physicalrights and financial rights that can be used as a hedge against congestion costs(the stochastic nature and potentially high cost of congestion makes financialhedging necessary).16 In all the markets with locational congestion pricing,payment of congestion prices is essentially a physical right to transmit be-tween nodes or zones (although not a right that is bought in advance). On theother hand, financial rights are typically purely financial mechanisms thatprovide revenues but confer no physical priority. They can be traded on a sec-ondary market.

For example, in New York and PJM, financial transmission rights givethe holder the right to collect congestion rents between a designated point ofinjection and point of withdrawal, so that if a transaction incurs congestioncosts, those costs would be offset by the revenues from the financial right.Auctions for these rights are typically held regularly. The California ISO hasimplemented a similar type of zone-to-zone right, but which also conferssome physical priority.17

Performance of ISO Markets. As discussed above, none of the ISOmarkets has reached a stable point in terms of market design; some are under-taking major market re-designs while others are in the process of implement-ing major components of their market design. There is a convergence in mar-ket design in many areas: all the ISOs have implemented either sequentialauctions with substitutions or simultaneous auctions for energy and ancillaryservices; most ISOs have established multi-settlement systems or will shortly.Most ISOs offer some form of financial transmission right; in the East coast

16 Transmission rights can take the form of either options or obligations.17 If the California energy markets fail to clear, the holder of a transmission right usually gets a

better position in the curtailment queue than a generator not holding a right.


ISO markets, nodal pricing is used for generation or is planned for future im-plementation.

Given these ongoing changes, the preliminary performance of the marketsvaries by product and time period. In transmission, the ISOs have recordedfew curtailments. There has been some concern, however, that the number ofbilateral transactions has decreased in nodal congestion management systems(because point-to-point congestion may be difficult to hedge with the avail-able transmission rights). The energy markets seem to be functioning fairlywell, although prices under certain system conditions reflect varying levels ofmarket power [8-13]. Entry of generation, transmission capacity expansion,and demand-side bidding should lower prices and lessen volatility.

The ancillary service markets have been more problematic. Reserve mar-kets in particular have experienced price spikes and price inversions, reflect-ing the greater vulnerability of these markets to market power and to marketdesign flaws that exacerbate strategic behavior [9,12]. Temporary price andbid caps and more permanent market re-designs should help solve some ofthese problems. Other general market problems include limitations in soft-ware implementation and technical capabilities (such as using telephonerather than electronic communications for dispatch), and conflicts that emergewhen system operators depend on rules of thumb to dispatch the system ratherthan the outcomes of the auction. In general, however, many market design orimplementation problems are amenable to satisfactory resolution, somethrough admittedly short-term “band-aid” solutions, but most with a longerterm fix available. Business confidence is not equally robust in each ISO mar-ket (PJM appears to be the market with the fewest problems to date), butshould increase as the markets mature.

Performance of Bilateral (Non-ISO) Markets. The largely bilateralmarkets, especially those in the Midwest, have experienced many potentialreliability problems as evidenced by the frequency of curtailments underTransmission Loading Relief (TLR) procedures. These may also be attribut-able to the lack of independence of the system operator and market partici-pants.18 Market participants have complained that they could not get access tothe transmission system even when capacity appeared to have been available.As described below, Order 2000 requires the implementation of more efficientcongestion management practices.

18 The curtailment of transactions in the presence of prices 10 to 100 times the annual average,due to TLRs and voltage reductions concurrent with power outages, indicate markets are notworking in harmony with reliability constraints. For example, in the summer of 1999 theECAR region with bilateral trading called 87 TLRs and the adjacent PJM ISO called three.For a general review of these complaints, see [6].


2.2 Order 2000 and RTOs

A Regional Transmission Organization (RTO) is a transmission systemoperator that is independent of market participants, controls transmission fa-cilities within a region of appropriate scope and configuration, and is respon-sible for operating those facilities to provide reliable, efficient, and non-discriminatory service. All transmission owners must file a proposal to par-ticipate in a RTO or provide reasons for delaying or avoiding participation.Order 2000 explicitly notes that the designs for bid-based markets in the fourISOs operational before the year 2000 should form a basis for the design ofRTO markets. Yet the open architecture adopted in Order 2000 does not pro-pose a single market model and offers sufficient leeway for further experi-mentation within the RTO design principles.

With respect to the unit commitment problem, the RTO has certain rele-vant functions. The RTO must have exclusive authority for maintaining short-term reliability. To fulfill this function, Order 2000 makes clear that the RTOrequires knowledge of the operational status of generators and load.19 Thisincludes control over interchange schedules, the authority to require re-dispatch of generation connected to the grid, and approval over scheduledoutages.

The RTO will determine the required amount of each ancillary serviceand the location where the service is to be provided. It will also act as pro-vider of last resort of ancillary services. That is, market participants can self-supply or purchase ancillary services from third parties, but the RTO musthave the capability to provide any residual. The RTO or a third party unaffili-ated with market participants must provide real time energy balancing.

With regard to transmission, the RTO must ensure development of marketmechanisms for congestion management and must develop procedures to takeinto account parallel path flows. The RTO will sell physically feasible, short-and long-term, tradable transmission rights. The RTO may choose to expandthe transmission system and/or invest in advanced technology to increasecapacity.20

19 Such knowledge includes technical information supplied by generators such as ramp rates,upper and lower operating limits, whether the unit is running or not, start-up times and timebetween start-ups. In real-time and for day-ahead planning, the RTO must have informationon generator injections and load withdrawals of energy in order to balance the system.

20 In the comments on Order 2000 [6], various policy suggestions were made regarding increas-ing transmission capacity, including overbuilding the transmission system (see Joskow com-ments) and/or investing in the high tech Flexible AC Transmission System (FACTS) andWide-Area Measurement System (WAMS) to allow more robust competition to develop.


Finally, the RTO is required to monitor for market power abuses andmarket design flaws. It should also evaluate and implement potential effi-ciency improvements in the markets it operates.

Beyond these requirements and guidelines, specific market designs areleft to the RTO market developers (subject to the proviso that they not limitthe RTO’s ability to improve efficiency further). The remainder of this sectiondiscusses some issues about the conceptualization of the role of the RTO withrespect to financial and physical transactions as well as the relationship ofRTO-operated auction markets in relation to other energy markets. In addi-tion, some pressing market design issues are reviewed, including pricing ofreserves and inter-regional coordination. Section 3 then draws on the ISO ex-perience and other sources to outline some principles for the design of day-ahead RTO markets.

Relationship Between Physical and Financial Transactions. An issuethat has remained contentious in the preliminary design and operation of ISOmarkets is the relationship between physical and financial markets –specifically, the concern that the centralized ISO markets and nodal conges-tion pricing would inhibit development of the decentralized financial mar-kets.21 An important principle underlying the future RTO markets is that well-functioning physical markets promote robust financial markets. For our pur-poses, physical trades are trades that the RTO has registered as feasible, con-sidering all other physical trades and required ancillary services. This includesbids into the ISO markets, bilateral transactions and self-schedules that havebeen cleared in the ISO day-ahead schedule (even though these day-aheadtransactions are actually financial contracts until physical delivery). Financialtrades are trades that are not physical trades, but take the form of forward con-tracts, futures contracts, or options contracts. They are not considered physicaltrades until they are confirmed as physically feasible by the RTO. Indeed, theRTO should be concerned primarily with physical market transactions; itwould not operate purely financial markets and need not be involved in anyfinancial markets unless the transaction goes to delivery.

Financial markets can and must exist in harmony and equilibrium withphysical reliability markets. If not, the financial markets’ ability to reduce riskis diminished. Multiple PXs and bilateral trading can fit easily into this market

velop. The latter appears more promising because it promises not only more capacity andless greenfield construction, but also better system control (see EPRI and EEI comments).

21 One argument has been that uncertainty over nodal congestion prices, calculated hourly inreal-time, increases the risk of bilateral deals concluded prior to the hour. Another is thatsome rules regarding three-part bids, in which the start-up payments made by the ISO areaveraged over all electricity load in the system (e.g., in New York), effectively amounts to asubsidy to generators in the ISO auction market.


framework. Each PX would act as a single scheduler, submitting schedulesand technical information for generation and load to the RTO.

The market design of the physical market should allow full, but optionalinteraction with financial markets. To become physical transactions, a marketparticipant need only self-schedule, that is, submit quantities to the day-aheadmarket. If the transaction includes the necessary transmission rights and ancil-lary services, no charges will be assessed. This allows for fully hedged finan-cial transactions. In addition, payment would be received for any additionalservice provided. Otherwise, the market participant will be billed for conges-tion, losses, and ancillary services caused by the bilateral transaction.

If not self-supplied, a bilateral trader can place price limits on what it iswilling to pay for transmission and ancillary services. If the price limits arenot met, the transaction will not be scheduled. If all voluntary adjustments aretried and reliability constraints are still not met, the transaction will be can-celed in the day-ahead market. This gives ample time for parties to make ad-justments. This cancellation avoids a potential TLR and the resulting scheduleis very likely to be physically feasible.

Physical markets provide real-time price signals and additional liquidity.Without good price signals from the physical markets, the financial marketscan become unstable and encourage more speculation and less hedging.

Even though bilateral trading may be highly discriminatory (that is, sell-ers may charge different prices for the same delivered product to differentbuyers), the opportunity for buyers to participate in an efficient,nondiscriminatory RTO auction market will tend to discipline the bilateralmarket. RTO auction markets create options for all buyers and sellers andthereby allow for a light-handed approach to the regulation of thesetransactions. In sum, the benefits of well-designed RTO markets includelighter-handed regulation of financial markets, more liquidity, less gamingand risk, more visible prices, lower transactions costs, fewer curtailments, andcompatibility with financial markets.

Reserves Markets. The establishment of efficient ancillary service mar-kets is an ongoing market design challenge. As the cost of reliability increasesand in the absence of a way to represent willingness to pay for ancillary ser-vices, the RTO system operator can relax reserve margins and transmissionconstraints. This is, in fact, written into the market rules in some ISOs; it re-mains a contentious issue, largely because it has involved system operatordiscretion that results in changes in market prices. The relaxation of theseconstraints increases the probability of a system failure; as such, it should bepart of the operational parameters of the auction decided in advance of theday-ahead auction so that actions of the system operator are not seen as arbi-


trary.To be effective, reserves must be able to respond to loads that need them.

If transmission is congested between generation and load, reserves on thegeneration side are of little help to the load side. Both transmission and gen-eration can be used to meet reserve requirements. They have substitutecharacteristics (strengthening transmission connections within a region canlower the total generation reserves needed in a region) and complementarycharacteristics (if the reserves for a load are not located at the same node,transmission capacity between the reserves and load will also need to be setaside). This set-aside is similar to the capacity benefit margin (CBM) concept.The auction algorithm would set aside transmission capacity to allow reservesto respond. The modeling of this process is very similar to the modeling of theenergy market itself. Prices for reserves would have a locational componentand the transmission price would reflect the set aside transmission capacity.

Using this locational method of allocating reserves, it could appear thattransmission capacity is being withheld. One method to deal with this is tohave a “use or lose” requirement of transmission rights. The set-aside “use” oftransmission for reserves markets would be considered a “use,” not withhold-ing, and is under the control of the auction process and the operator. Dealingwith the combination of congestion constraints and soft reserve constraintssimultaneously requires operator independence and transparency in the mar-ket environment, as a means to promote trust in the market.

Over time, as demand becomes more responsive to price, generation re-serve margins can decline as offers to reduce demand can substitute for re-serves. Currently the costs of reserves are averaged among all end users.Some of these costs can be more directly assigned to specific generating unitsand individual customers. For example, a unit with a good reliability recordshould be responsible for a lower reserve margin or be charged less for re-serves based on size and the historic probability of unit failure. Payments ordiscounts that differentiate more reliable from less reliable generators shouldbe handled in the pre-day-ahead markets.

RTO and Expanded Inter-regional Coordination. Spatial boundaryconditions – also called the “seam” or interface problem – are becoming animportant design issue as trading between ISOs increases. ISO coordinationefforts on this matter are in the nascent stages. In terms of system representa-tion, some ISOs have included a reasonably detailed representation of the in-terconnection with control areas outside the ISO as part of the boundary con-ditions. Approaches to the seam problem have included proposed interfaceauctions for inter-control area exchanges [16,17]. In day-ahead markets, thereis some time to coordinate these interfaces offline via iterative trading rules.Research into this problem generally is in its infancy. PJM, New York, NewEngland, and Ontario are examining a broad set of inter-regional market de-


sign and procedural issues, and these markets have signed a memorandum ofunderstanding on inter-regional coordination (on the need for such coordina-tion generally, see [6]). Order 2000 anticipates that RTOs will assist in creat-ing larger regional markets in which seams issues are resolved.

3. DESIGN PRINCIPLES FOR DAY-AHEAD RTOAUCTIONS

In the early phases of electricity market design and implementation, thevarious disciplines – notably economics and electrical engineering – have notundertaken adequate inter-disciplinary research or sufficient professional ex-changes. For example, ISOs have several times misunderstood the incentiveissues in electricity and ancillary service auction designs, as evidenced in theremedial actions and market re-designs described in Section 2, above. At thesame time, market design has sometimes proceeded with the economics basi-cally correct but without an adequate consideration of reliability and technicalconstraints. As a result, there is sometimes little understanding of what basicprinciples ought to underlie these complex markets. This section attempts toelaborate such principles for day-ahead markets. These principles can be com-pared with the market design principles in [14].

This section addresses the prospective RTO day-ahead market, which isdefined as the market in which the initial bidding to provide energy and ancil-lary services for reliability, congestion management, and energy balancingtakes place. As is done currently in ISO markets, this market would be con-ducted on the day prior to the dispatch day. The dual objectives of the day-ahead market are to achieve economic efficiency and ensure system reliabil-ity. The day-ahead market is a physical market where all expected balanc-ing/ancillary services are scheduled. The design for the day-ahead market isdiscussed below and it assumes that there is a real-time market, in which ad-justments are made to energy and ancillary services reflecting the differencesbetween day-ahead expectations and real-time conditions.22

22 The real-time market will not be examined in depth here. Efficient market design requires,however, that most principles be adhered to with respect to the relationship of the real-timeand day-ahead markets. Bids should be submitted separately into the real-time market, andmarket prices based just on those bids. Deviations from the day-ahead market should pay thereal-time price unless there is a reliability problem. If a bidder does not deviate from theday-ahead schedule, there are no additional costs to pay based on the real-time market. Fi-nally, the market operator needs to keep the system in balance at the nodal level using bidsto the extent possible.


The following is a list of recommended design principles derived from theforegoing analysis and experience to date, with short explanations and clarifi-cations.

Principle 1: Maximize economic efficiency. The RTO auction objective isto maximize economic efficiency through voluntary market bids, bilateraltransactions, and self-scheduling, given the physical and reliability con-straints.

Not all current ISOs adopt this principle. For example, the California ISOcannot adjust power schedules submitted to it by the California PX to improveeconomic efficiency. This principle requires that if market participants use theRTO markets, the resulting prices are efficient. The market-clearing proce-dure will balance the system and the purchase of ancillary services using thebids it has received. If the financial markets are efficient, there may be fewadditional trade gains in this market and this market becomes a reliabilitycheck. Efficiency requires that prices be consistent (e.g., no price inversionsdue to market design flaws) and that arbitrage opportunities reflected in thebids be exhausted.

Principle 2: Ensure physical feasibility of market transactions and systemreliability.

Without physical feasibility, reliability problems cannot be fully ad-dressed. For ancillary services, the market design should require that genera-tors committed to provide these services are located so that their capacity isavailable when and where they are needed.

Principle 3: Remove disincentives to market participation. Participation inan RTO market should involve low transaction costs and create minimaladditional risks.

Minimal participation in the market is the submission of generation andconsumption quantities (that is, bids which are taken at any price, or “at mar-ket”). Any unit dispatched should be guaranteed bid-cost recovery.

Principle 4: Bidding protocols should promote flexibility of participation.All market participants should be allowed, but not required, to submitmulti-part bids that reflect short-term marginal costs. Market partici-pants should be allowed to self-schedule, that is, allowed to submit quan-tity-only bids.

This principle requires that all resources have the option to bid areasonable approximation of their short-term marginal cost function, includ-ing start-up, no load, and energy costs (in addition to technical parameters,such as minimum and maximum load limits, ramp rates, and minimum shut-down time). Although a bid function will seldom serve as a perfect match for


actual marginal (incremental or going forward) costs, a good approximationshould be available.

This principle will require changes in some current unit commitmentauctions, which allow only one-part energy bids (of course, in a multi-partbidding rule, generators can still submit one-part bids, by bidding zero forstart up and no load). As discussed above, there are technical, financial, andeconomic reasons for adopting multi-part bidding.

While the multi-part bid allows for more accurate representation of mar-ginal costs and thus, in the absence of market power, should result in a moreefficient solution, it also results in a non-convex supply function. In turn, thismakes the market equilibrium and prices harder to derive. This complicationcan be addressed and made manageable with some simplifying assumptionsabout which generators are allowed to bid non-convex costs, whether someparameters should be fixed for a specified period (such as ramp rates, maxi-mum and minimum output), and what should be fixed in the bid function.

Demand bid functions are essentially the mirror images of generator bidfunctions but will not be discussed in detail here. Consumers need more ex-plicitly defined contracts to participate properly in the market and should beallowed bid functions similar to the generators.

Principle 5: Make clear the distinction between financial and physicalcommitments. If accepted, bids are financially binding. If needed for reli-ability, bids are physically binding.

The RTO schedules are physical and financial commitments subject toliquidated damages. That is, if market participants deviate from their com-mitments, they are liable for making the affected market participants whole.Under emergency conditions, the RTO may exact other penalties for non-performance and/or issue perform-to-contract orders.

Principle 6: Minimize opportunities for arbitrage between different productmarkets.

This principle addresses generally the problems that arise when genera-tors can bid into different product markets (energy and ancillary services) thatare sequentially cleared. As discussed above, simultaneous markets eliminateopportunities for arbitrage.

If the market design allows opportunities for lower quality products to bepriced higher than high quality products, then there may be cases where gen-erating units are paid more for not generating than for generating. Preferably,this should not be the case. The New England ISO attempted to establish thisprinciple administratively by proposing that the energy price always act as acap for operating reserves prices. This proposal was rejected by the Commis-


sion; administrative measures should be at best transitions to market designswhich can efficiently reach the same outcome. As discussed above, the simul-taneous auction design will automatically allocate energy and reserves effi-ciently, but cannot guarantee the elimination of price inversions in the pres-ence of market power.

Principle 7. Prices should be unbundled where possible to minimize averag-ing (i.e., socializing) of costs.

Averaged prices (whether for congestion, reserves, or other costs) do notsend the correct price signals for the entry of new generation. Uplift charges,the mechanism used for passing through costs of energy and ancillary servicesnot covered by market prices, should be coupled with incentives for the RTOto minimize the use of such charges.

Principle 8. Market-clearing information should be made available as soonas possible. Fuller information on bids should follow with a suitable de-lay.23

Market-clearing prices and quantities are the basic results of the bid ac-ceptance process. They enable market participants and potential future marketparticipants to assess the market and plan their businesses efficiently. Theyalso allow market participants to spot and correct obviously erroneous bidacceptance and rejection decisions.

Disclosure of individual bids should be made eventually, but not immedi-ately. Such disclosure will allow detection of subtle bid acceptance errors andit will also allow study of the market by independent analysts and market par-ticipants. It may lead to the exposure of the exercise of market power. Imme-diate disclosure of individual bids is undesirable because it might facilitatecollusion by the market participants. Immediate disclosure might reveal in-formation about market participants who wish to keep their costs confidential.After 6 months or a year, the information on individual bids has essentially novalue for collusion and discloses little new about any bidder’s current costs,but the information would have high value for auditing and independentanalysis.

The auction software should be available to the market participant or pub-lic at a reasonable cost. Improvements to the software are desirable, and thebest way to accomplish this is by making the software available with a set oftest problems.

23 The California ISO, PJM, and New England appear to set the benchmark with updated pricesevery five minutes. At a minimum, prices and aggregate quantities should be available be-fore the next round of bids with enough lead time to allow a reasonable response to the newinformation.


Principle 9: Minimize the incentives for market participants to engage instrategic behavior. The design should not favor market participants withmarket power.

Market designs can only imperfectly address structural sources of marketpower. An auction does not eliminate the ability of a large firm to withholdcapacity profitably. Auctions have been devised to “pay-off” market power,but these require significant computation and may not be revenue sufficient.24

Hence, absent a market design solution, the basic problem of structural marketpower has to be addressed using both structural remedies (vertical and hori-zontal dis-integration, encouragement of entry) and regulatory remedies, suchas market power monitoring and mitigation. Of course, structural remediesmay be wrapped up in market design issues. For example, an ISO may seek topromote rules on transmission interconnection for new generation which ap-pear to favor incumbent generators.

While market design cannot necessarily mitigate structural market power,it can certainly exacerbate it; market design can also create opportunities forstrategic behavior by generators other than the obvious large players. An ex-ample, discussed above, is the sequential clearing of energy and ancillary ser-vice markets without substitution in both the initial California and New Eng-land markets. Even a small generator can try to take advantage of shortagesof certain types of reserves in this type of market to raise prices.

4. CONCLUSIONS

The considerable developments in the design of electricity markets overthe past few years have provided the groundwork for the next generation ofshort-term markets. This chapter has emphasized that unit commitment mod-els are now embedded in a variety of market contexts governed by an evolv-ing regulatory framework that presents new requirements for modeling, in-cluding incorporation and understanding of market design issues. The openarchitecture promoted by the Commission allows for continued experimenta-tion with RTO market design, but within parameters reflecting lessons learnedheretofore from the ISO markets as well as the non-ISO bilateral markets. Themarket design principles presented in the paper are intended to reflect thoselessons.

24 The Vickrey-Clark-Groves (VCG) mechanism, which has been applied to electricity auctionsby [15], is a method to elicit truthful bids from players with market power.


The other primary argument in the chapter is that a well-designed RTOday-ahead auction market with unit commitment complements “decentral-ized” financial markets. The computer does the work of resolving reliabilityconstraints and will ensure that no offered trade gains are missed. Financialmarkets are free to create whatever innovative deals they can. The only re-striction is that if they go to delivery they must satisfy any reliability con-straints. Also, the opportunity for buyers to participate in an RTO auctionmarket will tend to discipline the financial market. This will allow for lighter-handed regulation of financial transactions. The paper identified several otherbenefits of well-designed RTO markets: fewer curtailments, more visibleprices, lower transactions costs, and less gaming and risk.

The open architecture also allows for continued progress in efficient pric-ing, so that causality has financial consequences – prime candidates are trans-mission rights and reliability. Over time, as price signals are sent and acted onin real-time, accurate pricing can allow the public good aspects of these mar-kets to shrink in importance and the private good aspects to grow.

ACKNOWLEDGEMENTS

This paper reflects ongoing discussions among the authors and CarolynBerry, Judith Cardell, Benjamin Hobbs, Thanh Luong, David Mead, WilliamMeroney, and Roland Wentworth.

REFERENCES

1.2.

3.

4.

5.

6.

7.

8.

P. Joskow and R.M. Schmalansee. Markets for Power. Boston: MIT Press, 1983.F.C. Schweppe, M.C. Caramanis, R.E. Tabors, and R.E. Bonn. Spot Pricing of Electricity.Norwell, MA: Kluwer Academic Press, 1988.H. Chao and H.G. Huntington, eds. Designing Competitive Electricity Markets. Boston:Kluwer Academic Press, 1998.Federal Energy Regulatory Commission. “Promoting Wholesale Competition ThroughOpen Access Non-discriminatory Transmission Services by Public Utilities and Recoveryof Stranded Costs by Public Utilities and Transmitting Utilities.” Order No. 888, 61 FR21, 540, May 10, 1996.Federal Energy Regulatory Commission. “Open Access Same-Time Information System(formerly Real-Time Information Networks) and Standards of Conduct.” Order No. 889,61 FR 21,737, May 10, 1996.Federal Energy Regulatory Commission. Regional Transmission Organizations. Order No.2000, 89 FERC 61,285, December 20, 1999.C.A. Berry, B.F. Hobbs, W.A. Meroney, R.P. O’Neill, and W.R. Stewart. Analyzing Stra-tegic Bidding Behavior in Transmission Networks. Utilities Policy, 8(3): 139-158, 1999.S. Borenstein, J. Bushnell, and C.R. Knittel. Market Power in Electricity Markets: Be-yond Concentration Measures. Energy J., 20(4): 65-88, 1999.


9.

10 .

11.

12.

13.

14.15.

16 .

17.

California ISO. “Annual Report on Market Issues and Performance.” Prepared by theMarket Surveillance Unit, California Independent System Operator, June 6, 1999.R.E. Bohn, A.K. Klevorick, and C.G. Stalon. “Second Report on Market Issues in theCalifornia Power Exchange Energy Markets.” Prepared for the Federal Energy RegulatoryCommission by The Market Monitoring Committee of the California Power Exchange,March 9, 1999.S. Borenstein, J. Bushnell, and F. Wolak. “Diagnosing Market Power in California’s De-regulated Wholesale Electricity Market.” PWP-064, Univ. of Cal. Energy Institute, Berke-ley, CA, Revised, March 2000.P. Cramton and J. Lien. “Eliminating the Flaws in New England’s Reserve Markets.”Working Paper, Department of Economics, University of Maryland, College Park, MD,March 2, 2000.J.E. Bowring, W.T. Flynn, R.E. Gramlich, M.P. Mclaughlin, D.M. Picarelli, and S. Stoft.“Monitoring the PJM Energy Market: Summer 1999.” PJM Market Monitoring Unit, un-dated draft.R.D. Wilson. “Design Principles.” In [3].B.F. Hobbs, M. Rothkopf, L. Hyde, and R.P. O’Neill. Evaluation of a truthful revelationauction in the context of energy markets with non-concave benefits. J. Regulatory Econ.,18(1): 5-32, 2000.M.D. Cadwalader, S.M. Harvey, W.W. Hogan, S.L. Pope. “Coordinating Congestion Re-lief Across Multiple Regions.” PHB Hagler Bailly Inc., Navigant Consulting Inc., andJ.F.K. School of Government, Harvard University, Cambridge, MA, October 7, 1999.B.H. Kim and R. Baldick. Coarse-grained distributed optimal power flow. IEEE Trans.Power Syst., 12(2): 932-939, 1997.

Chapter 3

DEVELOPMENT OF AN ELECTRICENERGY MARKET SIMULATOR

Atif Debs and Charles HansenDecision Systems International

Yu-Chi WuNational Lien-Ho Institute of Technology and Commerce

Abstract: The paper outlines the development of an electricity market operationsimulator (EMOS) that can be used by competing market participants aswell as independent system operators and independent market operatorssuch as power exchanges. The uses include training personnel to makecontrol and operating decisions and to assess bidding strategies by energytrading entities for energy and ancillary service products. We base thesimulator is based on the power system model used in the operator trainingsimulator. In the EMOS, market participants use unit commitment to as-sess bidding strategies and/or to schedule generation in a pool-type elec-tricity grid. The EMOS provides a framework for users to implement theirown applications with minimal effort and to model their competitors in ageneric manner.

1. INTRODUCTION

This chapter reports on efforts to develop a realistic electricity market op-eration simulator (EMOS) for the emerging electric energy market. Severalkey features of the market are:

(a)

(b)

An independent system operator (ISO) coordinates supplies and de-mands with a focus on system security and reliability;A market clearing entity determines which bids by market partici-pants are accepted. In some cases this entity is an independent powerexchange (PX), such as the California Power Exchange. In other


(c)

cases, the ISO and the market clearing entity are integrated into oneorganization (e.g., the PJM Interconnection); andMarket participants (MPs) consist of a mix of unregulated and aver-age-cost regulated entities. The unregulated entities compete againsteach other and submit daily, hourly, and even real-time bids to themarket clearing entities and/or the ISO. They are generally generationcompanies (GENCOs), energy service companies (ESCOs), or sched-uling coordinators (SCs). The regulated entities are mainly distribu-tion companies whose aim is to purchase the energy through the mar-ket at competitive prices.

In order to compete, each MP has to follow the rules of the market areainvolved and make appropriate “bids” to meets its goals. The bidding is nor-mally based on the MP’s estimates of market prices and other factors associ-ated with various risks involved.

The purpose of the EMOS is to simulate the overall market composed ofthe entities just mentioned for very short-term operations (one day ahead, upto real-time). The working tool is the power system model (PSM) of theEPRI-operator training simulator (OTS) [1-7]. Each MP may have its ownmodel of the system, but with outputs consisting of its bids into the market.The market clearing entity generates market clearing prices (MCP) and alsomost of the information needed to schedule system operation. The ISO doesthe final balancing of the market and the provision of needed “ancillary ser-vices” for system balancing, security, reliability, and real-time control [8-10].

Through the use of many existing software packages, standard applicationprogram interfaces, and communication protocols, the EMOS can be realizedin a variety of configurations. It is, therefore, usable by any of the entitiesmentioned above.

How do unit commitment (UC) and other forms of system optimization fitinto the overall scheme of things? The approach given here uses UC at boththe MP level and the market clearing entity level, whenever applicable. TheEMOS also recognizes the fact that network optimization through the optimalpower flow (OPF) is essential for such market considerations as: congestionmanagement, reactive power/voltage scheduling, maximization of transfers,and others.

Section 2 of this chapter provides a background summary of the emergingelectricity market with a focus on short-term scheduling. Section 3 provides aconceptual framework for the development of MP models. Section 4 describesthe overall market model, and Section 5 contains concluding remarks.

An Electric Energy Market Simulator 41

2. ELEMENTS OF EMERGING ELECTRICITYMARKETS

2.1 Generation Scheduling in the Competitive ElectricityMarket

The traditional approaches of UC scheduling, hydro scheduling (HS), or acombination thereof (so-called hydro-thermal coordination (HTC)) [11] arebeing replaced rapidly by various forms of short-term competitive biddingschemes [8]. The majority of such schemes are based on a 24-hour, day-aheadscheduling process – a Forward Day-Ahead Market in the new jargon.

In all the new schemes of electric utility restructuring, the generationcomponent, as compared with the transmission and distribution components,is strictly market-driven and competitive. As a result, the central dispatcherplays the role of implementing the rules of generation scheduling as obtainedthrough a market auction (bidding) mechanism. The main differences betweenthe traditional and the “market-driven” approaches for generation schedulingconsist of the following [11-13]:

(a) Market prices are the result of the market design and associated rulesand regulations under consideration. These rules may vary from onesystem to another. What complicates the picture is that the “products”being traded are many – starting with MWH energy and continuingthrough a host of ancillary services (A/S) and interconnected opera-tion services (IOS). Some bidding schemes require full “multi-part”bids. Others require single-part bids for energy, and either simultane-ous or sequential bids for the ancillary and IOS services.Individual for-profit MPs aim to improve their short- and long-termmarket positions through various strategies for increasing their prof-its, controlling their market share, and/or other objectives. In the areaof short-term (e.g., day-ahead and same-day) scheduling their mainfocus is on profit maximization. For the long-term strategies, theytend to use different mixes of tools including financial ones (futures,options, and swaps), or strategic ones (mergers and acquisitions, in-vestments in other markets, etc.).The new markets can become more or less effective through the exis-tence of catalysts such as a power exchange, scheduling coordinators,energy brokers, service providers, e-commerce, and independent sys-tem operators.

(c)

(b)


2.2 Network and Other “System” Issues in the New Envi-ronment

Principles that have been followed in restructuring transmission include:1. In order to allow generating companies – as well as customers and

distribution companies – to compete effectively, an open-access re-quirement has been suggested and implemented. The functions ofgeneration scheduling have been “separated” from those of networkaccess and control.Certain markets (e.g., the PJM Interconnection, New Zealand,NYISO) use locational marginal prices (LMP). These depend on a so-lution of both a resource scheduling program (unit commitment, hy-dro-scheduling, or hydro-thermal coordination) and a security-constrained optimal power flow (SCOPF). The resulting locationalprices reflect a combination of “charges,” such as transmission losses,congestion, reactive support, reserves and, others [14,15]. (See Figure1.)

2.

Even in markets that do not use LMPs (or equivalent), network informa-tion and the ancillary services protocols can be of great value for all marketplayers, e.g., the California ISO [8].


In short, different market players will have to adapt to the operating rulesfor the systems served by different types of ISOs. In many cases, knowledgeof real-time and forecasted network conditions can be critical in assessingvarious risk exposures and in evaluating hedging instruments.

2.3 Market Prediction is a “Learning Process”

Competitive markets do not behave like centrally dispatched systems.Price discovery is attained through the auction mechanism, which has its owndynamics [12]. Prices tend to be uncertain with a random component and asystematic component. The random component stems from external uncer-tainties (unpredictable weather, forced outage of a unit, system disturbance,fluctuating fuel prices and interest rates, etc.). The systematic component isrelated to physical constraints, market rules, auction clearing mechanisms, etc.It is also related to “gaming” and market strategies by various players (e.g., tomanipulate prices and increase prices).

Within this systematic component of market price prediction, a “learningprocess” becomes necessary to predict the behavior of the competition. Sinceenergy markets clear prices very frequently, the learning process can be for-mally analyzed [13]. In most cases, however, it becomes the purview of “ex-perts,” like skilled traders who earn their income from being very good learn-ers.

2.4 Rationale of a Simulation-Based Approach

The foregoing discussion points to the fact that various analytical modelsmay not provide the full answer to all the questions. Given the complexity ofthe power system and hence the electric energy market, there is a need in ouropinion for a simulation tool to help in the learning process. Such a tool mustby necessity represent both the systematic and random parts of the process.From a modeling viewpoint, we have considered the following to be critical[1-8,13]:

The power system under consideration should be modeled realisti-cally.Control and decision mechanisms by the main dispatcher (ISO, TSO,etc.) should be customized in accordance with the rules of the systemunder consideration.The market bidding and price clearing mechanisms should also bemodeled for each respective market.


The transitions from one market regime to the next (i.e., day ahead tohour ahead to real-time) should be clearly modeled and accommo-dated,The various tools used by individual market participants to play themarket should be modeled. The simulation should have a growing li-brary of such tools as one learns about the bidding strategies of vari-ous players.The simulation should explicitly model random and systematic riskexposures.

3. MARKET-PARTICIPANT MODELS

The simulation model recognizes that the power system is run as a resultof decisions made by groups of various market participants (MPs). At aminimum, these MPs would represent the following entities:

Generating companies (GENCOs) who are competing against eachother,Demand-side entities,Independent System Operators (ISOs)Energy Service Providers (ESCOs)

Figure 1 gives the functional structure of MP models. This structure is inturn specialized for each of the categories of market participants as given inFigure 2. Figure 3 in Section 3.6 displays the ESCO functional structure.

3.1 Generating Companies (GENCOs)

GENCOs comprise a variety of entities: (a) the generation component ofan integrated utility that is required to have functional separation of genera-tion trading from other services, (b) a pure generating company with re-sources scattered over several geographic areas, or (c) an independent powerproducer (IPP). All these entities share similar needs and requirements. As aresult, we designed the GENCO model to consist of three basic modules: (1)input module, (2) bid optimization module and (3) post-mortem analysismodule. These are now presented in some detail as they form the core of theoverall model.

3.1.1

The GENCO will not be able to compete successfully unless it is able toforecast energy and ancillary service prices. Other inputs are related to spe-

Input Module


cific market rules for the clearing of bids. We discuss these in some detailbelow.

The GENCO should be able to forecast market-clearing prices based onhistorical information and its relative market position [12]. The approach usedin our modeling is a regression model, which relates market-clearing pricesto: (a) overall system demand and (b) available generation by the GENCO.Given the system demand forecast, the GENCO can then predict the corre-sponding price forecast. Several price-forecast scenarios are provided basedon uncertainties in the demand forecast. At a minimum, these are expected,pessimistic, and optimistic scenarios with associated probabilities of occur-rence.1 Price forecasts are done for both energy (MWh) prices as well as an-cillary service prices.

A given GENCO sits somewhere between the two extremes of being (a) apure price leader or (b) a pure price follower [12]. Furthermore, the GENCOmay be constrained by socio/economic/regulatory factors, which would limitits ability to manipulate price, even if it has market dominance.2 The output

1 An alternative to this is fuzzy modeling of the uncertainty.2 For example, the GENCO may be concerned with anti-monopoly laws or excess profit regula-

tion, which may arise out of aggressive practices.


of this sub-module consists of: (1) limits on market share and (2) sensitivityof market prices and energy supply to GENCOs bid prices. Both of these areto be used in the bid optimization module.

3.1.2 Bid Optimization Module (BOM)

The BOM consists of two components, described below.

Component 1: Deterministic bid optimization based on specific input sce-narios

The optimization is performed for two specific markets: (a) day ahead and(b) hour ahead markets.3 The market rules vary from market to market. Ourapproach models initially two types of market rules – (1) those which arebased on LMP considerations, e.g., the PJM Interconnection, and (2) thosebased on bidding by scheduling coordinators4 using a PX, e.g., the CaliforniaISO. Thus, there are four basic sub-modules to consider.

Day Ahead/LMP-Based. The bidding system for LMP-based day-ahead mar-kets consists of a multi-part bidding process whereby the generation (andload) bidders are required to submit typically the following:

Energy multi-block price curves which are piecewise constantNo-load price bidsOperating range (min and max MW)Minimum up-timeMinimum down-time

The ISO control center as a result performs a combination of unit com-mitment and contingency-constrained optimal power flow computations toyield an optimal solution. The performance criterion is the minimization ofoverall price to meet customer demand. When combined with elastic demandbids, it is possible to use the same software to maximize social welfare. Ide-ally, if every generation bidder submits his marginal cost data (plus a speci-fied mark-up for profit), the result would be almost the same as that of a fullyintegrated utility system. The constrained OPF would minimize system losses.In case the ISO would pay for reactive power production, then the OPF wouldattempt to maximize profit from the use of reactive power production.

From the perspective of the GENCO, the objective is to submit a bid thatmaximizes profit – at least at this initial deterministic level. Thus for each

3 It is possible to extend the functionality of this module to cover both real-time bids andlonger-term forward market bids.

4 Scheduling coordinators (SCs) are allowed to submit strictly balanced generation/load bidsfor the energy part of the market.


forecasted scenario, the control variables consist of the above bid parameters,as long as they do not violate technical limitations (such as bidding minimumup-time which is always greater than or equal to the technical minimum up-time). One of the key issues here is to ensure that the GENCO Market-Participant is able to model its own cost information, including their non-convex price incremental cost curves.

Hour-Ahead/LMP-Based. Under this regime, the basic bidding process con-sists of incremental/decremental bids to allow for adjustments of schedulesbased on deviations from the day-ahead schedules. The main tool is strictlythe OPF with an objective function to maximize market surplus. From thebidder’s perspective, the goal is to maximize profit.

Day-Ahead/SC/PX5-Based. In the SC/PX-based market, the typical biddingprocess is decomposed into hourly energy price bid curves and other bids forcongestion management and ancillary services. However, there is a distinctionbetween a scheduling coordinator (SC) and the power exchange (PX).

In California, the SCs are exposed to network congestion and ancillaryservice risks (and opportunities). Their main bid parameters to improve prof-itability consist of incremental/decremental (INC/DEC) bids for congestionmanagement and ancillary service bids. In the simulation model the focus isstrictly on congestion management bids.

For GENCOs bidding through the PX, the main objective is to optimizehourly energy price bid curves. Furthermore, there is a question here whetherportfolio bidding should be allowed. If portfolio bidding is permitted, thenthe GENCO has to perform optimization at two levels: one that uses a singlecurve for the entire company and a second that provides generation schedulesat the cleared market prices. Again, the GENCO has the opportunity to bidinto the congestion management and ancillary services market.

Hour-Ahead/SC/PX-Based. This is similar to the day-ahead bidding systemwith the exception that portfolio bidding may not be allowed.

Component 2: Bid optimization through risk managementFor each of the four bidding regimes, the GENCO optimization is per-

formed for a set of scenarios. In the input module, there is a probability asso-ciated with each scenario. The key output here is a profit probability profile.The analysis would select a scenario that yields bid prices for the bestprofit/risk trade-off.

5 PX = Power exchange, whether independent or part of the central market clearing organiza-tion.


3.2 Demand-Side Entities

For the purposes of the model under consideration, we distinguish the fol-lowing demand-side entities:

(a) Major Load Serving Entities (LSE) (e.g., distribution company, orcommercial or industrial loads): An LSE may be able to bid a declin-ing multi-block purchase price curves based on the composition of itsload and the demand-side management systems in place (typically in-terruptible loads). The objective here for the LSE is to reduce the costof purchases based on its price forecasts (and contracts already inplace).Pumped Storage Power Plants: These plants try to combine their gen-eration bids with their demand bids (for pumping). The result is acomposite optimization.

(b)

3.3 Independent System Operators (ISOs)

In our model, the ISO represents the component that implements the re-sults of the bidding process to create a full schedule of system operation thatmeets regulatory reliability and security requirements. Normally, the ISO con-trols only its own area of responsibility (control area). Again, we distinguishbetween two types of ISOs: LMP-based and SC/PX-based.

3.3.1 LMP-Based ISO

The model of the LMP-based ISO permits the following functions to beperformed:

Issuance of ISO-demand forecast for the day-ahead and hour-aheadmarkets: The demand forecast is a net forecast for all demands to bemet by participants in the day-ahead and hour-ahead markets. In es-sence, the forecast is adjusted by subtracting any bilateral contractsand export/import contracts, which are also bilateral in nature.Hourly optimization of the generating system and the networkthrough a combination of unit commitment and contingency-constrained OPF, as described above, based on the totality of the bidsby the market participants. The optimization would yield LMPs,which are published and released to the market.Computation of congestion charges and firm transmission rights(FTR) payments.


Similar functions for (1,2, and 3) the hour-ahead market (no unitcommitment is involved, however).Real-time system control through automatic generation control(AGC). The AGC utilizes an economic dispatch component which isbased on the hour-ahead bid prices, or alternative bid prices for sys-tem regulation as an ancillary service

3.3.2 SC/PX-Based ISO

In this case, the following functions are supported:Demand forecast similar to the one in the previous section for theLMP-based ISOCongestion management for interzonal congestion as well as intra-zonal congestion for both the day-ahead and hour-ahead marketsSimultaneous and sequential ancillary services bid clearingAutomatic generation control based on ancillary service bids for sys-tem regulation and load following

3.4 Power Exchange (PX) Model

There is no need for a separate PX model for the LMP-based system,simply because the ISO in this case clears the market using the various multi-part bids. The PX model for the SC/PX-based market is quite simple as it justmatches supply and demand curves. The PX, however, may engage in ancil-lary service markets and options/futures markets and these have to be takeninto account.

3.5 Energy Service Provider (ESCO)

For the purposes of the market simulation model, the ESCO is a combina-tion of a GENCOs and LSEs. A large ESCO may be bidding in multiple mar-kets and these have to be modeled separately. The trading performed by theESCO may combine a portfolio of contracts/bids to manage risk.

3.6 Overall Market Model

Figure 3 shows the resulting EMOS overall market model. We have de-signed the model for the following uses:


(a)

(b)

(c)

Studies and training by each market participant entity described aboveto improve its performance and meet its own needs.Studies at regional or national levels to evaluate different market de-signs.On-line activities by MPs, including energy bidding strategies, ancil-lary services bidding, market surveillance, etc., whenever the systemis linked to real-time data sources. Again, a given market participantshould be able to model its own activities using its own proprietarysystems.

3.7 Note on Implementation

The EMOS is implemented using standard software applications devel-oped primarily by EPRI and other developers. These include:


EPRI-OTS by EPRI for the primary PSM modelingEPRI-DYNAMICS unit commitment program for various market par-ticipant modulesEPRI ANN-STLF for short-term load forecastingDSI-OPF for the multi-area optimal power flow interfaced withEPRI-DYNAMICS

Other components of the EMOS are still under development. The overallsystem uses an effective database system which is compliant with EPRI’scommon information model [1].

4. CONCLUDING REMARKS

The development of an overall market model for the electric energy in-dustry under restructuring is a major challenge mainly because the electricphysical system is highly complex, changes constantly, and creates a signifi-cant amount of physical risk to market participants (MPs). The overall marketmodel consists of an interacting population of MPs. Market rules and designsare aimed at coordinating all efforts through ISO’s. The model implementa-tion is based on such developments to allow for effective interoperability andplug-compatibility among various applications.

We expect that the developments reported here can be used in a variety ofways by each MP type: GENCOs, LSEs, ISOs, PXs and ESCOs. The mainuses consist of:

Just-In-Time training of personnel – traders, dispatchers, operationsengineers and managersEnhanced mechanisms for improving competitive positions by tradersAbility of ISOs and market designers to analyze the behavior of mar-ket participants and test improvements in market rules and overallmarket design

Unit commitment as such is used as a tool whenever applicable. The mainusers of UC are the GENCOs, ESCOs, and some ISOs with multi-part biddingmarket rules.

ACKNOWLEDGEMENTS

The developments reported in this chapter are under the sponsorship ofthe National Science Foundation (NSF) under SBIR Phase II Grant, DMI-98010161. The EPRI-OTS developments and related enabling technologies havebeen partially sponsored by EPRI, Palo Alto, California.


REFERENCES

A. Debs, Y.-C. Wu, and C. Hansen. “Enhancement of the EPRI-OTS for the RestructuredElectric Utility,” Project Reports, Decision Systems International, Atlanta, Georgia, underSmall Business Innovations Program of the National Science Foundation.A. Debs and C. Hansen. “The Total Power System Simulator: A Comprehensive Tool forOperation, Control and Planning.” In Proc. Arab Electricity ’97 Conference & Exhibition,PennWell Europe and DSI, Bahrain, 1997.A. Debs and C. Hansen. “The EPRI-OTS as the Standard for Training and Studies in theNew Era: Strategy for Global Application.” Presentation at the First Asia Pacific Confer-ence on Operation and Planning Issues in the Emerging Electric Utility Environment,sponsored by EPRI, Kuala Lumpur, Malaysia, 1997.M.K. Enns, et al. Considerations in designing and using power system operator trainingsimulators. EPRI EL-3192: 1984.V. Calovic and A. Debs. “The Fully Integrated DTS for ESB, Dublin. ” In Proc. EPRIFirst European Workshop on Power System Operation and Planning Tools, Amsterdam,The Netherlands, 1995.C. Hansen and M. Foley. “Power System Model Enhancement at ESB, Dublin.” In Proc.EPRI First European Workshop on Power System Operation and Planning Tools, Am-sterdam, The Netherlands, 1995.S. Lutterodt and A. Debs. “Issues in Use of Operator Training Simulators.” Paper pre-sented at the CEPSI Conference, Kuala Lumpur, Malaysia, 1996.A. Debs and F. Rahimi. “Modern Power Systems Control and Operation in the Restruc-tured Environment.” Class notes for intensive short course by Decision Systems Interna-tional, San Francisco, CA, 1999.A. Debs, P. Gupta, C. Hansen, A. Papalexopoulos, and F. Rahimi. “System Planning inthe Context of Competition and Restructuring.” Class notes for intensive short course byDecision Systems International, San Francisco, CA, 1996.M. Ilic, F. Graves, L, Fink, and A. DiCaprio. Operating in the open access environment.Elec. J., 9(3): 61-69,1996.A. Debs. Modern Power Systems Control and Operation. Norwell, MA: Kluwer Aca-demic Publishers, 1988.B. Sheblé. Computational Auction Mechanisms for Restructured Power Industry Opera-tions. Boston, MA: Kluwer Academic Publishers, 1999.A. Papalexopoulos. “Design of the Wholesale Market in the USA.” In Proc. EPRI Sec-ond European Conference – Enabling Technologies and Systems for the Business-DrivenElectric Utility Industry, Vienna, Austria, 1999.Y.C. Wu, A.S. Debs, and R.E. Marsten. A direct nonlinear predictor-corrector primal-dualinterior point algorithm for optimal power flows. IEEE Trans. Power Syst., 9(2): 776-883,1994.H. Kim and R. Baldick. Coarse-grained distributed optimal power flows. IEEE Trans.Power Syst., 12(2): 932-939,1997.A. Debs. “The OPF in the Deregulated Environment.” Lecture notes for intensive shortcourse by Decision Systems International: Modern Power Systems Control and Operation,San Francisco, CA, 1996.A. Papalexopoulos, et al. Cost/benefits analysis of an optimal power flow: The PG&Eexperience. IEEE Trans. Power Syst. 9(2): 796-804, 1994.Y.C. Li and A.K. David. Optimal multi-area wheeling. IEEE Trans. Power Syst., 9(1):288-294, 1994.

1.

2.

3.

4.

5.

6.

7.

8.

9.

10.

11.

12.

13.

14.

15.

16.

17.

18.

Chapter 4

AUCTIONS WITH EXPLICIT DEMAND-SIDEBIDDING IN COMPETITIVE ELECTRICITYMARKETS

A. BorghettiUniversity of Bologna, Italy

G. GrossUniversity of Illinois at Urbana-Champaign

C. A. NucciUniversity of Bologna, Italy

Abstract: This paper focuses on the development of a model for and the simulation ofelectricity auctions with demand-side bidding (DSB) explicitly considered. Wegeneralize the competitive power pool (CPP) framework developed in [1] toinclude DSB. In order to allow customers to play a proactive role in the pricedetermination process, the DSB provides the opportunity for them to submitbids for load reductions in specific periods. We study the behavior of DSB in-clusion in electricity auctions simulations obtained with a specially developedLagrangian relaxation scheme that effectively takes advantage of the structureof the problem. We present some numerical results for a 24-hour simulation ona small system. This case study is effective in illustrating the various economicimpacts of DSB including system efficiency effects, changes in the systemmarginal price and the load recovery effects.

1. INTRODUCTION

This paper deals with the development of a model for the simulation ofelectricity auctions with demand-side bidding (DSB) explicitly considered.The model is a generalization of the competitive power pool (CPP) frame-work developed in [1] to include DSB. Such a mechanism has been used attimes in the England & Wales Power Pool by large industrial consumerswho can offer their ability to reduce load directly to the Pool and receive apayment for actually making such a reduction possible [2,3].


In the CPP with only supply-side offers, the point of intersection of theaggregate supply curve and the fixed forecasted load determines the market-clearing price. To improve competition, the consumers may be allowed toparticipate in the market price definition process by providing them the op-portunity to submit bids. Without DSB, the system marginal price (SMP) isdetermined by constructing the supply curve from the offers submitted bythe supply-side bidders and determining the uniform price paid to all bidderswho are selected to serve the fixed forecasted load demand. The basic idea inthe application of DSB is to use the demand profile to lower SMP by cuttingload during peak periods. DSB may take advantage of large industrial cus-tomers who can cut load or shift load to benefit from lower electricity prices.Since industrial customers are not expected to reduce significantly their dailyenergy consumption, the overall effectiveness of DSB depends on the how,the how much, and the when of the energy recovery. Clearly, the more flexi-bility such a customer has, the more possible it becomes for him to partici-pate in DSB.

The demand recovery is typically a function of the weather, of the eco-nomic conditions as well as of the number of reduction periods, and theamount of load cut in each period. In [4] an analysis has been presented ofdata relevant to a four-year period for a regional electricity company (REC)in the United Kingdom that quantifies the extent of intertemporal substitu-tion in electricity consumption across pricing periods within the day for fivegroups of different industries. Every industry shows electricity substitutionpossibilities with its own firm-specific characteristics. We have constructedan econometric model from the results of this analysis. This model is for theuse of the REC to formulate its demand-side bids and as such takes into ac-count the specific pricing scheme and rules adopted in the England & WalesPower Pool. There are additional data available on the characteristics of theload recovery from experiences gained with the application of commer-cial/industrial and residential load control programs [5-7]. In this case, thereis electricity substitution only across adjacent load periods, i.e., the energy isrecovered in the periods that follow right after the reduction periods.

In this paper, we simulate and study the inclusion of DSB in electricityauctions to examine various economic and policy aspects in DSB. The simu-lation tool implements a Lagrangian relaxation-based scheme, which takesadvantage of the problem structure with the inclusion of DSB. The suffi-ciently detailed representation of the supply-side bidders allows the model-ing of the unique physical characteristics of the power generation system. Inparticular, it takes into account the main operating considerations includingthe operational limits and up- and down-time constraints. The supply-sidebidders are also allowed to include in their bids a separate price for start-upin addition to the price for the MWh commodity. In the simulation tool, theimpact of DSB is taken into account by incorporating specific energy con-

55Auctions with Explicit Demand-Side Bidding

straints and load recovery into the Lagrangian relaxation algorithm. Re-searchers have applied Lagrangian relaxation methods widely in recent yearsfor the solution of unit commitment problems for large-scale systems due totheir ability to include a more detailed system representation than would bepossible with other techniques [8].

Some aspects of DSB have been analyzed in [9,10]. The studies indicatethat the specific characteristics of the demand-side bidders have a substantialimpact on the system economies. However, additional work is required tobetter quantify such impacts. This chapter aims to bring additional insight tothe study of DSB. The principal combinations are the development of a gen-eralized framework for the analysis of various aspects of DSB and the illus-tration of the salient characteristics through the simulations obtained with thetool developed for the implementation of this framework.

The chapter is organized as follows. In section 2, we present the gener-alization of the CPP structure with the inclusion of DSB into the CPP frame-work and a description of the Lagrangian relaxation-based algorithm for itssolution. In section 3 we present some numerical results obtained for a 24-hour simulation on a small system. We compare costs and price signals ob-tained from DSB in addition to supply-side bidding (SSB) to costs and pricesignals obtained from the use of only supply-side bidders. The paperprovides an analysis of the numerical results. Section 4 offers final thoughtson this research, and the Appendix presents details concerning the imple-mented code and the power system data.

2. GENERALIZATION OF THE COMPETITIVEPOWER POOL FRAMEWORK FOR INCLUSIONOF DEMAND SIDE BIDDING

We start with the extension of the CPP framework developed in [1] toinclude DSB in addition to SSB. With the addition of DSB, the CPP dis-patcher problem is to select the winning bids and offers from the set of bothsupply-side offers and demand-side bids. We use henceforth the term bid torefer to both the supply-side offer and the demand-side bid and we refer toeach player as a bidder. Each bid has three components:

the bid variable price which describes the per hour price as a

function of MW provided/reduced. The function is here assumed tobe a quadratic or piece-wise linear function;the bid start-up price which is charged whenever the bidder is

committed; andthe bid offered capacity which is a vector whose

component is the maximum capacity offered by the bidder for usein period t, expressed as a fraction of the maximum output.

The supply- and demand-side bids have different characteristics.In particular, for the case of SSB, we have made the assumption that the

operational data1, including the minimum and maximum output and

and minimum up- and down-times and need to be provided byeach bidding plant i.

Moreover, the bid start-up price which is charged whenever unit iis started in period t, is assumed to be a function of the down time of unit iat period t-1

where is the thermal time constant of unit i, I is the number of supply sidebids, is a constant related to the cold start charge, and is a constantrelated to the fixed operating and maintenance charges [11].

The characteristics of the demand-side schemes require the specificationof the following information for each demand-side bid j,j=1, ..., J:

the bid start-up price that is considered a constant value

the subset of the operator-designated load-reduction periods

that is the subset of the periods in which bidder j may un-dertake load reduction

the minimum and maximum demand and that can be re-

duced2 by bid jthe subset of the operator-designated load-recovery periods that

is the subset of the periods in which bidder j may under-take a load recoveryload recovery at period h, that is related to all the load reductions

in each reduction period of by the following expression

1 We do not take ramp and network constraints into account in the model discussed in thischapter.

2 Note that a requirement is introduced in the problem formulation to ensure that the sum ofthe maximum load reductions of all the demand-side bids in a period t cannot exceed theload forecasted for that period.


where represents the load recovery ratio in period h of the load

reduction in period t.

The above representation of load recovery is similar to that used in [9]. Itallows the representation in a flexible and simple way of the electricity sub-stitution characteristics of consumers who may participate in load modifica-tion. It allows, moreover, load recovery to occur both in pre- and post-loadreduction periods. In [9] it is pointed out that the beneficial effects of DSBare greatly attenuated if some bidders recover their load in the same periodsin which others are reducing it. Therefore, we shall assume that there are nooverlapping periods in which the load reductions and the load recoveriesmay cancel each other out.

By taking into account both load reductions and load recoveries, the ac-tual load in reduction period t is given by

where and are the forecasted loads in period t and h, respectively.

The additions of the relations for the DSB participation do not changethe basic structure of the framework in [1]. Consequently, we can extend theLagrangian relaxation framework developed in [1] to the more generalmodel described here. For this purpose, we use the following notation:

and the actual load in recovery period h is given by

Auctions with Explicit Demand-Side Bidding 57

the number of supply-side biddersthe number of demand-side biddersthe number of time periodsthe T-dimensional vector of the load demands in each period t in thescheduling horizonthe T-dimensional vector of the zero-one variables indicatingwhether supply-side bidder i is committed in period t or notthe T-dimensional vector of the zero-one variables indicatingwhether demand-side bidder j is committed in period t or notthe T-dimensional vector of the amount of power that the supply-side bidder i is producing in period tthe T-dimensional vector of the amount of power that the de-mand-side bidder j is reducing in period t.

I :J:T:D:


The CPP dispatcher problem determines the most economic commitmentthat satisfies the forecasted demands D without violating physical and oper-ating constraints of the generation equipment and demand specifications.

We refer to the optimization problem in equations (4)-(7) as P.While the demand-side bids are defined similarly to supply-side offers,

problem P differs from problem with only supply-side bidding. In fact, theincorporation of DSB adds the two decision variables w and y that arecharacterized by specific constraints specified in (7).

The solution approaches with and without DSB are similar. Also withthe introduction of new DSB decision variables w and y, the Lagrangian re-laxation approach allows the decomposition of P into I+J sub-problems as-sociated with each supply-side and demand-side bid. With the inclusion ofDSB, we use the following Lagrangian dual function

or, more explicitly, taking into account (4),

subject to initial conditions and, by taking into consideration equa-tions (2) and (3), subject to


subject to the specified initial conditions and to constraints (6) and (7).t= l,...,T, are non-negative Lagrange multipliers. The dual function is rear-ranged as

and, for demand-side bid j, by explicitly taking into consideration equations(2) and (3),

subject to the specified initial conditions and to the constraints (6) and (7).Problems (10) and (11) are the I + J sub-problems, one for each bidder,

that need to be solved. In order to find the optimal values of the multipliersto use in equations (10) and (11) the following dual problem is solved

where, for supply-side bid i

The Next Generation of Unit Commitment Models

In fact, L * provides a lower bound of the final solution of the primal prob-lem P [12].

As a by-product of the process of maximizing L we obtain the optimalLagrange multipliers and a system schedule {u, w, p, y} resulting from thesolution of the Lagrangian relaxation for In certain cases, this systemschedule does not satisfy the conditions imposed by the demand constraintand, therefore, a practical approach for computing a near-optimal schedulehas to be implemented [13].

It may be useful to note that the optimal schedule, obtained with the so-lution of the proposed framework, is the one that minimizes the total “pro-duction” costs, i.e., the costs paid to satisfy the forecasted demand, on thebasis of the bids of the supply-side and demand-side bidders. The solutioncan differ, in general, from the schedule that minimizes the consumer pay-ments, for the various reasons cited in [14].

We implemented the solution approach based on the extended frame-work for the inclusion of DSB as a software package. Some of the salientaspects of this package are summarized in Appendix A. This package hasbeen used to perform many numerical studies to assess the characteristicsand the impacts of DSB in a competitive power pool.

To illustrate the application of the generalized framework, we presentsome typical results of our simulation studies. We use the 10-unit system in[13] as the supply side of the resources. The operational data and the bidfunctions used for the supply-side bidders are given in Appendix B. The be-havior of the power pool is simulated for the 24-hour period load profile ofTable B.1 in Appendix B. Figure 1 plots the hourly loads. The load reduc-tions are dispatched on the basis of the demand-side bids that represent theprices at which the bidders wish to reduce their consumption by the specifiedamounts. The bidder that is scheduled to reduce its consumption in a certainperiod is also allowed to recover its load partially or totally in other periods.The model specifies the periods in which load reduction and load recoveryare allowed. These periods have been defined in the previous section as the

operator-designated load-reduction periods and load-recovery periods

To show the influence of such a constraint, the results of a sensitivity

study are presented. The study was carried out by varying only the followingtwo triggering levels:

the load-reduction triggering level as a fraction of the peak load,specifies the load level above which the load reductions are allowed;and

60

3. NUMERICAL RESULTS


the load-recovery triggering level as a fraction of the differencebetween the peak and the base load, specifies the load level belowwhich the load recoveries are allowed.

For each given and values, the corresponding permitted reduction andrecovery periods, and respectively, are then established. The load

recovery is considered uniform throughout the recovery period.We discuss the results of several simulations on this system with a single

demand-side bidder in addition to the ten supply-side bidders. The demand-side bidder limits are and The demand-side bidis 1 $/MWh with 1 $/h being the no-load bid. These values were selectedspecifically to provide a lower cost per unit of energy to the CPP dispatcheron the demand side than the supply-side. We start out with a case in which,the DSB is allowed whenever the load exceeds For this case isset at 0.5. The periods when load reductions are allowed, are

The corresponding recovery periods are

and the recovery is implemented to spread the re-

covery of load in equal amounts in each of the periods in


To assess the impact of DSB, we use the no-DSB case as the referencebasis. We show in Figure 2 the modified loads as a result of DSB under dif-ferent levels of load recovery. These levels of load recovery are indicated bythe recovery fraction value, that is the total amount of load recovery over thetotal amount of reduced load. In Figure 3, we depict the corresponding im-pact on the system marginal prices (SMP). For the supply-side bidding strat-egy used, the system marginal prices track closely the modified systemdemands Furthermore, DSB, in effect, acts to “smooth” out the pricevariations across periods.

We now evaluate the impacts of varying some of the parameters in DSBimplementation. One parameter that has a major impact on deployment ofDSB is the load reduction recovery factor. This parameter is the ratio of therecovery energy to the cut energy. As this factor decreases from 1 to 0, thesavings increase with respect to the costs for the case without DSB (what weshall call reference costs). Figure 4 shows results for the test system.

The assessment of DSB on the variability of the SMP is important. DSBtends to impact prices in different periods lowering the prices in periods withhigher loads and increasing them in periods with lower loads. We evaluatethe hourly average SMP, as well as the volatility impacts in terms of thestandard deviation as a function of recovery fraction. Figure 5 gives theseresults with the no-DSB case as reference. While there is a reduction in thevolatility, a reduction in the average value of the SMP is not obtained for all


recovery fractions. The reason is that the schedule is obtained without aminimization of consumer payments. In the case of DSB without load recov-ery, the reduction of the volatility is only due to the reduction of the SMP inthe peak periods. Therefore, the reduction in the volatility is lower in thiscase than for the case in which there are both the reduction in the price in thepeak periods and the increase of the price in load recovery periods. We notethat at the higher fraction of load recovery, DSB is committed in only a lim-ited number of periods, and as such both the savings and the reduction inprice volatility are limited.

An important aspect to examine is the impact of different triggering lev-els for the definition of the operator-designated load-reduction and recoveryperiods. For the test system we evaluate the savings with respect to the refer-ence DSB case for sixteen different triggering levels under the fixed loadrecovery fraction of 0.8. Table 1 gives the data, and Figure 6 presents theplots as a function of the reduction triggering levels for different valuesof Due to the fewer number of periods in which the demand-side biddermay be committed, higher reduction triggering levels result in lower savings.On the other hand, the effect of the recovery triggering level is less uniform:as the recovery triggering level increases, the number of recovery periodsincreases and the recovery of the load decreases in each recovery period. Inthe case of the higher recovery triggering levels, the recovery periods are inthe shoulder load periods resulting in a decrease in the savings compared tothe lower values of


We examine the impact on loads and SMPs for as a functionof Figure 7 shows the load modifications, and Figure 8 depicts the im-pact on the SMPs. These plots make more concrete the observations in theprevious paragraph on the impacts of different

We next discuss the impacts of a single-supply unit’s bidding strategy onthe market. We focus on the unit 9, and we obtain the results in Figures 2-8with the bid of unit 9 declaring unavailability in periods 10 to 21. A supply-side bidder for various reasons or market objectives may select such “strate-gic” behavior. When unit 9 changes its bid and declares availability for theperiods 10-21, the total costs without DSB change $537,271. This change

represents a saving of $3,186 with respect to the reference case of no-DSBwithout the participation of unit 9 for the periods 10-21. From Figure 4, wecan see that such a saving is lower than the saving obtained with DSB recov-ering the entire energy. We may, therefore, interpret the impacts of DSB toinclude its ability to mitigate the market power on the supply-side.

Thus far, we have concentrated on displaying the salient characteristicsof DSB through a single demand-side bidder. The rationale for this was toprovide a good insight into the interaction of the DSB with the electricitymarket. We next study the case with multiple DSB bidders. To illustrate thebehavior of DSB with multiple demand-side bidders, we consider two cases:(i) five DSB bidders and (ii) nine DSB bidders. We compare the behavior inthese two cases to that in the case of the single bidder discussed above. Forthese cases, the demand-side bidders have and the loadrecovery fraction is 0.9. Recall that for the single DSB bidder, the bid is1 $/MWh with 1 $/h being the no-load amount and In case(i) the bids are 1, 2, 3, 4, and 5 $/MWh, respectively, with 1 $/h the no-loadamount for each bidder. In case (ii) the bids are 1, 1.25, 1.5, 1.75, 2., 2.25,2.5, 2.75, and 3 $/MWh with 1 $/h the no-load amount for each bidder. Inthe two cases is 200 MW for each DSB bidder.

Figures 9 and 10 provide plots of the results obtained with the five andnine demand-side bidders together with the results obtained with the singleDSB bidder. Figure 9 gives the impacts on the load modification, and Figure10 displays the impacts on the hourly SMPs. The number of demand-sidebidders has been chosen to exceed the number that is committed in the mul-tiple-bidder cases: in (i) only two are committed, and in (ii) only five arecommitted with the fifth being committed only for a single period in thehour 10.


4. CONCLUSIONS

This chapter focused on the integration of DSB into electricity markets.We discussed the development of a general framework for the inclusion ofDSB in addition to supply-side bidders. We used the Lagrangian relaxationbased solution approach to examine the impacts of DSB. The tool we devel-oped for simulation is very useful for investigating a wide spectrum of pol-icy issues. In our case, we have used this tool to examine the ability of DSBto mitigate the potential for exercise of market power by the supply-sidebidders. In addition, the impacts of DSB on smoothing system marginalprices and on mitigating price volatility are important attributes as indicatedby the numerical results.

The numerical studies of the paper provide good insights into variousaspects of DSB. In particular, it is important that load reductions and loadrecoveries do not cancel each other out. Therefore, the specification of per-mitted load reduction periods and permitted load recovery periods has beenadded to the model. The results of a sensitivity analysis show that the speci-fication of appropriate triggering levels, that define the permitted load reduc-tion periods and permitted load recovery periods, is particularly critical tomake effective use of DSB.

In the chapter, we provide a simple example to study multiple DSBplayers. The impacts of competition and possible collusion among DSBplayers need to be investigated in more depth. Also, there are many addi-tional policy issues that remain to be examined. One key issue for furtherstudy is that of DSB remuneration, including the needs for such incentivesand the levels at which to set them. The changes in supply-side bidder be-havior brought about by the inclusion of DSB are another key issue that re-quires investigation.

We are grateful for the financial support for the research reported herefrom the Italian National Research Council. George Gross received supportfrom the Power System Engineering Research Center administered by Cor-nell University.

G. Gross and D.J. Finlay. “An Optimization Framework for Competitive ElectricityPower Pools.” In Proc. PSCC, 815-823, 1996.F.A. Wolak and R.H. Patrick. The impact of market rules and market structure on theprice determination process in the England and Wales electricity market. Working PaperPWP-047, University of California Energy Institute, 1997.D.W. Bunn. Demand-side participation in the electricity pool of England and Wales.Decision Technology Centre report, London Business School, 1997.R.H. Patrick and F.A. Wolak. Estimating the customer level demand for electricity underreal time pricing. Working Paper, Department of Economics, Stanford University, 1997.D.V. Stocker. Load management study of simulated control of residential central airconditioner on the Detroit Edison Company system. IEEE Trans. Power Apparatus Syst.,4: 1616-1624, 1980.A.I. Cohen, D.H. Oglevee, and L.H. Ayers. An integrated system for residential loadcontrol. IEEE Trans. Power Syst., 3: 645-651, 1987.C.N. Kurucz, D. Brandt, and S. Sim. A linear programming model for reducing systempeak through customer load control programs. IEEE Trans. Power Syst., (11)4: 1817-1824, 1996.


ACKNOWLEDGMENTS

1.

2.

3.

4.

5.

6.

7.

REFERENCES

The Next Generation of Unit Commitment Models

A.J. Svoboda and S.S. Oren. Integrating price-based resources in short-term schedulingof electric power systems. IEEE Trans. Energy Conversion, (9)4: 760-769, 1994.G. Strbac, E.D. Farmer, and B.J. Cory. Framework for the incorporation of demand-sidein a competitive electricity market. In IEE Proc. – Generation, Transmission andDistribution, (143)3: 232-237, 1996.G. Strbac and D. Kirschen. Assessing the competitiveness of demand side bidding. IEEETrans. Power Syst., (14)1: 120-125, 1999.J. Gruhl, F. Schweppe and M. Ruane. “Unit Commitment Scheduling of Electric PowerSystems.” In System Engineering for Power: Status and Prospects, LH Fink and K.Carlsen eds., Henniker, NH, 1975.A. Geoffrion. Lagrangian relaxation for integer programming. Math. Prog. Study, (2):82-114, 1974.J.F. Bard. Short-term scheduling of thermal-electric generators using Lagrangianrelaxation. Oper. Res., (36)5: 756-766, 1988.S. Hao, G.A. Angelidis, H. Singh, and A.D. Papalexopoulos. Consumer paymentminimization in power pool auctions. IEEE Trans. Power Syst., (13)3: 986-991, 1999.R. Fourer, D.M. Gay and B.W. Kernighan. A modeling language for mathematicalprogramming. Manage. Sci., (36): 519-554, 1990B.A. Murtagh and M.A. Saunders. “MINOS 5.1 User’s Guide.” Technical Report SOL83-20R, Systems Optimization Laboratory, Department of Operations Research,Stanford University, 1987.

We have developed a computer program incorporating both supply-sideand demand-side bidding. The program allows multiple demand-side biddersand supply-side bidders to interact for meeting a load profile over a specifiedperiod. The generality of the program allows us, in particular, to study thecharacteristics of the demand-side bidders and the influence of their parame-ters on the results.

For the Lagrangian relaxation implementation we have followed the pro-cedure proposed in [13]. To solve subproblems (10) and (11), the values ofare, at first, considered assigned as well as the values of the zero-one deci-sion variables and It follows that these subproblems, for supply-sidebid i and demand-side bid j, respectively, become

70

8.

9.

10.

11.

12.

13.

14.

15.

16.

A.

APPENDICES

Development of a Computer Program for Validationof the Proposed Approach


subject to upper/lower limits specified in equation (6) (without the minimumup/down constraints) and in equation (7).

These problems are piece-wise or quadratic problems (it depends on how

the bid variable price function is defined), and can be solved with a

linear or quadratic programming technique. These optimizations give thevalues of power productions and load reductions To find the optimalvalues for the decision variables and we solve subproblems (10) and(11) using the values of and in a forward dynamic programming, takinginto account the start-up prices and the minimum up-/down-time constraints.

In the case of demand-side bidder problem (11) the dynamic program-ming approach differs from that of supply-side bidders. In particular, to takeinto account the load recoveries, we add the following term to the valuescorresponding to the states in which demand-side bid j is committed at pe-riod t

where, taking into consideration equations (2) and (3),

This term represents the contribution to the function of all the load re-coveries due to the scheduled load reduction at period t that has to be

minimized in equation (11). Because of the presence of this additional term,the solution of the problem (A.2) and the dynamic programming procedureare included in a loop that ends when two consecutive solutions result in thesame commitment, i.e., in the same values of the zero-one decision variables

To find the maximum value L * of equation (12), we use a standard sub-gradient technique [12]. With this technique, the solution process is iterated,starting from a set of tentative values of multipliers and at each iterationk the multipliers are updated using


and denote the current solution (iteration k) of the dual problem andthe lowest available solution of the primal one, which represent the lowerand upper bound of the final solution at iteration k, respectively. is a sca-lar parameter that is progressively reduced whenever the solution of the dualproblem is not improved.

To implement such a technique, we need to find a feasible solution forthe primal problem P at each iteration. This is accomplished by adopting thepriority-list heuristic procedure proposed in [13], based on the calculation ofthe marginal costs associated with each supply-side bidder and each period.The heuristic procedure is implemented in such a way that it does not changew, i.e,. the commitment of the demand-side bidders. The procedure here de-scribed is stopped when the relative duality gap (defined as

is lower than a specified value.We implement the program in the algebraic modeling language for

mathematical programming called AMPL [15]. This language is particularlyuseful for the piecewise linear representation of the cost functions. A majoradvantage of the modular structure resulting with the AMPL implementationis the ability to use a library of solvers. In this paper the MINOS [16] solverhas been used to solve the problems (A.1) and (A.2).

Ten generators, considered as supply-side bidders, compose the supplysystem. Tables B.2, B.3, and B.4 show the supply-side bidder data and havebeen adapted from those used in [13]. Table B.1 shows the values of theminimum and maximum power outputs the minimum up- anddown-times and the initial status of the units i.e., the number of

B. Case Study System Data

Table B.1 gives the load profile.

Auctions with Explicit Demand-Side Bidding

In Table B.3 the quadratic expression of the cost data of [13] is converted

into piecewise linear form so that the bid variable price is specified by

6 parameters: the no-load price three incremental prices andtwo elbow points respectively.

Table B.4 displays the values of the components of the bid start-up price

defined in equation (1).

73

periods the unit had been up or down

The Next Generation of Unit Commitment Models74

Chapter 5

THERMAL UNIT COMMITMENT WITHA NONLINEAR AC POWER FLOWNETWORK MODEL

Carlos E. Murillo-Sánchez and Robert J. ThomasCornell University

Abstract: This chapter presents a formulation of the thermal unit commitmentproblem that includes nonlinear power flow constraints, thus allowing amore accurate representation of the network than is possible with DCflow models. This also permits potential VAr production to be used as acriterion for commitment of otherwise expensive generators in strategiclocations. We use a Lagrangian relaxation framework with duplicatedvariables for each active and reactive source, permitting the exploitationof the separable structure of the dual cost. Results for medium-sizedsystems in a parallel processing environment are available.

INTRODUCTIONThe central theme surrounding the use of Lagrangian relaxation for the unit

commitment problem is that of separation. Ever since the early papers [1, 2]this separability was the objective and for a good reason: the unit commitmentproblem, as a mixed-integer mathematical program, suffers from combinatoriccomplexity as the number of generators increases. This is what dooms otheralgorithms intended for solving it, such as dynamic programming: the combinedstate space of several generators in a dynamic program is too large to be ableto attack many realistic problems, even with limited memory schemes.

Classical Lagrangian relaxation permits the decomposition of the probleminto several one-machine problems at each iteration; the coupling to other con-straints involving more machines is achieved by sharing price information cor-responding to the relaxed constraints, which is updated from one iteration toanother. The complexity of a given iteration becomes linear in the number ofdiscrete variables. The price to pay is that of switching to an iterative method

1.


rather than a direct method, as well as giving up certainty of global optimalityif there is a duality gap. This last drawback is not as bad as it seems, however;very small duality gaps are routinely obtained using Lagrangian relaxation.

The unit commitment problem can be formulated generally as:

where

The production cost function F is assumed to be convex (in fact, quadratic)and separable over each generator and time period:

For our purposes, the constraints in the problem have been classified into threekinds. Category groups constraints that pertain to a single generator, butmay span several time periods. These include minimum up or down timesand ramping constraints. Category groups constraints that span the com-plete system but involve only one time period, such as load/demand matching,voltage limits, reserve constraints, and generation upper/lower limits. Finally,category groups constraints that involve more than one generator and morethan one time period. A typical example is the infeasibility of turning on morethan one unit at a time in a given location because of crew constraints.

To show the specific type of separation achieved by means of Lagrangianrelaxation, we take a look at the work of Muckstadt and Koenig [2]. Theyconsidered a lumped one-node network with losses modeled as fixed penaltyfactors. They also considered reserve constraints. We write an example formu-lation including demand and reserve constraints, the relaxation of which yieldsthe Lagrangian

Length of the planning horizonNumber of generators to scheduleReal power output for generator at timeReactive power output for generator at timeOn/off status (one or zero) for generator at time

The total production costThe sum of any startup costsA set of dynamic generator-wise constraintsA set of static instantaneous system-wide constraintsA set of nonseparable constraints.

Thermal Unit Commitment with Nonlinear AC Power Flow 77

where is the real power demand in period is the desired minimumtotal committed capacity for the same period, and is the upper operatinglimit for the ith generator. Consider the dual objective

and the corresponding dual problem

which can be written explicitly in the following form after collecting terms ona per generator basis:

Thus, for fixed and evaluation of amounts to solving separate,single-generator dynamic programs of the form

The dynamic programs can readily accommodate constraints suchas minimal up or down times. Transition costs can easily include cold start orwarm start costs, derived from either banked or total shut-downs. Ramp-rateconstraints can also be introduced by discretizing the generation domain forthe unit, though the dynamic program grows considerably. For a detailed de-scription of a dynamic programming graph including most of these constraints,see reference [3].

After confirming that the evaluation of the dual functional is simple, a dualmaximization algorithm can be readily applied to since the sub-gradientis easy to compute from Equation (3). A method as simple as a Poljak’s sub-gradient ascent or as sophisticated as a bundle method [4] can be utilized.

A proper algebraic constraint structure is the key to the separability ofthe dual functional. Note that the step from Equation (3) to Equation (6)was possible due to the fact that the constraints are affine in the optimizationvariables. We want to separate the variables on a per-generator basis, so, failingaffinity, we need at least separability into sets of variables related to a singlegenerator each. The unit commitment problem has such separability as long asthe system-wide constraints are separable by generator. Thus, generalnonlinear constraints cannot be readily included in the model, at least notdirectly. We will show a way around this later.


THE CASE FOR AN AC POWER FLOW MODELIn the past 20 years, many advances have been made, enhancing the num-

ber and type of constraints that can be treated, addressing convergence issueswhen the production costs are not strongly convex, and so on. In particu-lar, transmission line constraints have been included by using the so-called DCpower flow network models. These constraints, being linear, can be relaxedwith appropriate multipliers and the dual functional will still be separable bygenerator. There are, however, genuine engineering considerations for includ-ing a nonlinear AC flow model. These considerations stem from the fact thatsome very important system constraints can only be modeled accurately witha full-blown AC power flow model. For example, in an actual transmission linethe limit is best expressed as a limit in the current, whereas a transformer’scapacity is best described by its MVA rating. Both actual MVA loading andcurrent depend on its orthogonal active and reactive components. The DC flowmodel can only predict (and even then, only for relatively small angle deviationsand under a nominal voltage assumption) the active component, and thereforeit is easy to find situations in which it does a poor job of modeling importantconstraints. Active branch limits inextricably tie together active and reactivedispatch restrictions; if the reactive component is large, the dispatch obtainedby the DC flow-based unit commitment algorithm may have to be altered tocomply with actual line limits, or conservative limits may have to be used.

Other limits that cannot be modeled accurately via linear approximationsare voltage limits. Appropriate voltage levels are crucial to the operation ofmost electrical apparatus, and it is not uncommon to find so-called “mustrun” generators that are needed not so much because of their actual powergeneration capacity, but because of their reactive power capacity. In order toraise the voltage to levels that are adequate for consumers, sometimes largeamounts of reactive power are needed at specific locations in the network, evenif the only sources are generators that are costly to operate.

Granted, common practice to this day is to use linear network models forconstrained economic dispatch, and in many cases accuracy is good enough.The reason why they work, however is that usually the only branches that aremodeled are tielines, which tend to have controllable reactive compensationin order to keep power factors near unity. Linear congestion models workbest if unity power factor is maintained. They become inaccurate if used toapproximate thermal limits over broad ranges of voltage and reactive and activeinjections, especially if the branch being represented does not have dispatchablereactive compensation.

In summary, current Lagrangian relaxation algorithms using DC flow-basednetwork constraints or other linear approximations may do poorly enforcingvoltage limits and true current or MVA limits in grids requiring large reactiveinjections. As a result of the limited accuracy of this network representation,it is quite possible that a solution obtained by such an algorithm is actuallyinfeasible, requiring additional generators to be committed in order to meetvoltage or congestion constraints.

2.


We find some of the works that have followed the DC flow formulation in theirincorporation of line limits to the dual maximization in [5, 6, 7, 8, 9, 10, 11].In [12], the authors use linear approximations to include the voltage constraintsin the formulation, but it is not clear that a linear approximation will workunder large excursions in the reactive dispatch. Baldick [8] uses a general for-mulation that could in principle be used to address AC flow constraints, and heelaborates a little more in the speculative paper [13], although the authors donot seem to have actually implemented their scheme. The formulation proposedin this work was first reported in [14]. It uses fewer multipliers and, by explic-itly employing an augmented Lagrangian, should have improved convergenceproperties.

3. FORMULATION AND ALGORITHMOur approach has its origins in the variable duplication technique credited to

Guy Cohen in [6] by Batut and Renaud. Baldick [8] later used this technique inhis more general formulation of the unit commitment problem. The principalachievement of this work is the inclusion of reactive power output variables tothe formulation, so that better loss management may be performed and gener-ators that are necessary because of their VAr output but not their real powerare actually committed. This is the logical next step in the development of La-grangian relaxation techniques. At this point, when typical algorithms reducethe duality gap to figures close to 1% [15, 10], it is important to recognize thata better handling of the reactive power considerations at the unit commitmentstage may have a payoff that is higher than those last few percentage points inthe duality gap.

The variable duplication technique exploits the fact that the problem

is equivalent to

Notice that the last constraint is linear and therefore amenable to relaxation.We start by defining two sets of variables. The dynamic constraints will

be posed in terms of the dynamic variables, whereas the static or system-wideconstraints will be posed in terms of the static ones:

Dynamic variables:

Commitment status {0,1} for generator at time

Real power output for generator at time

VAr output for generator at time


Static variables:

Real power output for generator at time

VAr output for generator at time

We define the optimization problem as:

subject to:(1.) constraints

(2.) constraints

(3.) and the following additional constraints


where is the minimum combined capacity that is acceptable for thezone in the period and is the set of indices of generators in the zone.

A word about the representation of the minimum up-time or down-timeconstraints is in order. While it is perfectly possible to state these restrictionsin terms of additional state variables obeying state transition rules that reflectthese constraints (see [10]), we have chosen not to include any more variablesbecause of the already overburdened notation. These constraints can readilybe incorporated in the structure of the dynamic programming graph used tosolve the resulting subproblems, as in [3]. Thus, it will be assumed that we canenforce the constraints (9–11) on the D variables and the constraints (12–14) on the S variables, so that we only relax the three last constraints (15–17),which leads to the following Lagrangian:

where are multipliers on the relaxed equalities of the two kinds ofvariables, is the multiplier associated to the zone’s reserve requirementat the period, and returns the index of the zone to which generatorbelongs.


The separation structure of the Lagrangian is obvious upon looking at equa-tions (19) and (20). It makes it possible to write the dual objective as

By looking again at (19) and (21), we see that the first term can be computed bysolving dynamic programs again; the second term separates into optimalpower flow (OPF) problems with all generators committed but with special costcurves for generator at time Notice that also has a price.We assume that the solutions of the dynamic programs meet the constraintsand that the solutions of the optimal power flows meet the constraints.

It would be tempting to apply a dual maximization procedure to the dualobjective as stated, but there are some issues that prevent us from doing thatwithout some modification of the Lagrangian. The first issue is that the costof reflected in the dynamic programs, being linear, is not strongly convex;this can cause unwanted oscillations in the prescribed by the dynamic pro-gram [6]. Therefore, we set out to fix this before addressing any other problemsby augmenting the Lagrangian with quadratic functions of the equality con-straints. This will introduce nonseparable terms. At this stage, we will invokethe Auxiliary Problem Principle described by Cohen in [16] and [17]. Thisprinciple is a rather general formalism characterizing optimality in an implicitmanner for convex programming, and an algorithm of the fixed point type canbe inferred from it. Convergence for the convex case with affine constraints hasbeen proved by Cohen. We proceed to write the new augmented Lagrangian as


Within the context of the auxiliary problem principle, it is possible to substitutethe augmentation terms by the following at iteration (see [17] for the generaltechnique and [6, 7] for the first documented application to unit commitment):

where and are the values obtained at the itera-tion. Since (22) is separable, we can collect terms of the augmented Lagrangianon a per-generator basis, so that at the iteration we are faced with


Notice that (24) has the same separation structure of (19).Now that the separability issue has been resolved, we propose the following

Algorithm: AC Augmented Lagrangian relaxation

Step 0:

Step 1:

Step 2a:

Step 2b:

Step 3:

Step 4:

Step 5:

Step 6:

Initialize to the values of the multipliers on the power flowequality constraints at generator buses when running an OPF with allunits committed. Initialize to zeros. Initialize a databaseof tested commitments to be empty; it will later be filled with thecommitment schedules obtained in the course of the algorithm andtheir corresponding feasibility status.

Compute

by solving one-generator dynamic programs.

Compute

by solving OPFs in which all generators are committed, their gen-eration range has been expanded to include and the specialcost is used. Note: all tasks in steps 2a and 2b canbe solved in parallel.

If the commitment schedule Û is not yet in the database of testedcommitments, perform a cheap primal feasibility test. If the resultsare not encouraging (i.e., not enough committed capacity), store theschedule in the database and label it as infeasible, then go to Step 6.

Perform a more serious primal feasibility test by actually attemptingto run OPFs with the original constraints. If all OPF’s aresuccessful in finding a feasible dispatch, store the commitment in thedatabase, together with the primal cost including startup costs. Elselabel the commitment as infeasible, store it in the database, and go toStep 6.

If the mismatch between the two sets of variables is small enough, andthere is already a set of feasible commitment schedules, stop.

Update all multipliers using sub-gradient techniques, and


Step 7: Go to Step 2.

The proposed algorithm is very OPF-intensive: the major computationalcost is that of computing OPFs for every iteration in order to solve the staticsubproblems, plus extra OPFs in selected iterations when a given commitmentis promising in terms of primal feasibility. Thus, every effort possible must bemade to try to alleviate the burden of OPF computation. The first thing thatcan be done is to use as a starting point for the OPF the result of the previousiteration for the same time period. Most of the times, the only difference inthe data for the OPF would be a small change in the costs (reflected by thechange in from one iteration to another). This, in theory, should result infewer iterations needed for the OPF.

Another drawback of the algorithm is that a different set of OPF compu-tations must be performed to compute the value of the dual objective and tocompute the value of the primal. Thus, before even trying to compute the valueof the primal objective, one should make sure that such a costly computationis worth doing. Some of the cheap tests include verifying that the reserve con-straint is met and that the mismatch between the S and the D variables issmall, since feasibility is a given if the mismatch is zero. With respect to thelatter, we have found that if a smaller mismatch should be specifiedas requisite to feasibility than if More costly feasibility tests wouldinvolve power flow problems starting from appropriate initial values. Currentlya constrained power flow is being performed at this stage.

4. COMPUTATIONAL RESULTSAn implementation of the algorithm has been written in the MATLAB™

environment. The dynamic subproblems can accommodate minimal up or downtimes, warm start and cold start-up costs and are solved using forward dynamicprogramming. We solve the static subproblems using a version of MINOS [18]that has been incorporated in the MATPOWER package [19] by the first au-thor. It incorporates box constraints on the generator’s active and reactiveoutput, piecewise-linear or polynomial cost functions for both P and voltageconstraints, line MVA limits, and, of course, the power flow equations. Addi-tionally, any linear constraint on the optimization variables can be imposed. Apreconditioner for MINOS that performs a constrained power flow is used ifnecessary. It implements a Levenberg-Marquardt-like minimization of the sumof squares of the power flow constraints, with penalty functions on some otherbox contraints and constrained variables in the case of voltage limits. Thus,each iteration involves the solution of a problem rather than a Newton step.We solve the problem using a MEX-file version of BPMPD [20], an interiormethod solver.

The specific sub-gradient technique being used at this point is a simple Poljacstep size schedule, with an iteration-dependent weight that is inversely propor-tional to iteration number. We chose simplicity initially because convergenceconditions for Poljac’s method are “mild” and well documented: it is advanta-


geous to be able to blame any convergence problems on the formulation itselfand not on the particular kind of sub-gradient update. The obvious drawbackto this decision is that the convergence of the dual iteration process may notbe very fast. More sophisticated gradient updates will be investigated once thebasic features of the algorithm are well understood.

The program was originally tested on a modified IEEE 30-bus system [21]with six generators and a planning horizon of length six. We modified the testcase so that generator #4, located at bus #27, is needed for voltage support formany load levels even though it is most uneconomical to operate. For compar-ison purposes, we also wrote a version of the Lagrangian relaxation algorithmwith DC flow-based relaxed line limits. The AC-based algorithm correctly iden-tified this unit as a must-run for those time periods, even providing some priceinformation on the MVArs that this unit produced by means of the correspond-ing The number of iterations required was usually in the vicinity of onehundred. In contrast, the DC flow-based algorithm failed to commit unit #4for any period, producing a commitment schedule that was infeasible in lightof the AC power flow constraints.

The importance of proper selection of the parameters was ap-parent from the beginning. After several trial runs, we obtained good resultswith and Values very different fromthese, however, tended to produce somewhat smooth, damped oscillations inthe values of some of the

To highlight one of the new features found in the algorithm, we show theevolution of versus iteration number for a typical run in Figure 1.The multipliers with the higher values are all P-type multipliers. Those withthe smaller values correspond to the Most of them settle to zero, indicatingthat is essentially free almost always. Yet, a few of them actually have highprices: these belong to generators and time periods where the OPF tries to usetheir MVArs in order to force feasibility or guided by economic considerations,but the generators are not actually committed. In the course of the algorithm,these may grow so large that they trigger the respective unit on. Once thishappens, such multipliers tend to approach zero again, since is now plentiful.In Figure 1 there are two clear examples of this behavior, corresponding tounit 4 being committed for certain time periods. As the multiplier approacheszero, the static copy will approach the dynamic

A slightly more ambitious test has been performed using the IEEE 118-bussystem, with 54 generators and a time horizon of 24 periods, correspondingto two “weekdays” and one day of the weekend, each with 8 three–hour pe-riods. The total variation of the load relative to the base case is –50% and+40%. Figure 2 shows the behavior of the norm of the active and reactivemismatches between the two sets of variables. Figure 3 depicts the evolutionof the multipliers in this case.


5. PARALLEL IMPLEMENTATIONOne of the purported advantages of separation by Lagrangian relaxation

is that it allows for simultaneous solution of all of the static and dynamicsubproblems in a given dual iteration, making the computation amenable toparallelization. Profiling of the algorithm has indicated that more than 95%of the computation time in this particular implementation is allocated to thesolution of the optimal power flows. In order to test larger systems, a parallelimplementation was deemed not just convenient, but in fact necessary sincerunning times were already several hours long. Fortunately, MultiMATLAB, aparallel processing toolbox for MATLAB is being developed at Cornell Univer-sity by John Zollweg (see [22] and the original work of Trefethen et. al. [23]);this allowed us to take advantage of much of the existing code.

Although the dynamic programs are also parallelizable, their time-granularityis much smaller than that of the OPFs, so communications overhead is likelyto reduce the efficiency of their parallelization. We have opted to parallelizeonly the OPF computation at this time.

MultiMATLAB works by having several copies of MATLAB, each runningin a different node, communicate by means of a subset of the Message PassingInterface (MPI) library [24]. The calls to the MPI functions are implementedby means of MEX (MATLAB-Executable) files. A master node performs themain algorithm and the dynamic programming subproblems and when a setof OPFs need to be carried out it sends the OPF data to the other nodes,


working in a master/slaves configuration. The workers’ only task is to receiveOPF input data, run the OPF solver and report the results back to the master.

Two parallel scheduling strategies have been programmed so far. The firstone is a straightforward round-robin scheme in which the master cycles throughthe workers, receiving the results of any previously assigned OPF by means ofa blocking receive (MPI’s Recv function), which ties the master until data doesarrive. Then the master sends the worker data for the next OPF and turns itsattention to the next worker. Clearly, having the master wait for the worker isnot optimal, but it has the advantage of employing only MPI’s Send and Recvcalls, whose implementation is stable in MultiMATLAB. A second, more so-phisticated parallel scheduling strategy makes use of MPI’s Irecv non-blockingreceive function. Immediately after sending data to a worker, the master nodeposts a non-blocking receive, setting aside an incoming message reception area.The master can then turn its attention to other workers and assign further work.Every now and then the master calls MPI’s Testany function, which informsthe master if any pending Irecv’s have been completed. If so, the master readsthe data from the corresponding buffer and assigns more work if needed. Thisstrategy promises the most efficient use of the workers. In limited experimentswith up to seven workers, the master is actually free most of the time, testingfor incoming data. This means that it is efficiently keeping the workers busy.The efficacy of this scheduling strategy can be further improved by performingthe OPFs in order of decreasing expected execution time. Unfortunately, as ofnow the Testany implementation is not sufficiently stable to allow running acomplete problem.

The round-robin strategy has been tested using the 118-bus system witha much longer time horizon (168 periods) and longer start-up and shut-downtimes on the generators. Figure 4 shows a plot of the evolution of the mis-matches.

6. FUTURE WORKHistorically, the best justification for using Lagrangian relaxation has been

the regular achievement of small relative duality gaps in larger problems. Sofar, our algorithm seems to behave correctly with proper tuning of parameters,but there is a need to test much larger (i.e., more than 300 generators) systemsto verify whether small duality gaps are also routine. At that point, comparisonto the results obtained using a DC-flow or linear network model formulationshould also be performed, with special attention to the cost of any commitmentcorrections needed to make the schedule generated by the linear network modelalgorithm to be AC-feasible. It would be worth it to incorporate other kinds ofconstraints and continue improving the robustness of the optimal power flowsubsystem, which has been the weakest link so far.

Three major efforts are being undertaken. First, John Zollweg is implement-ing several changes that should make the non-blocking receive implementation


in MultiMATLAB more stable, allowing the use of improved parallel schedulingstrategies. Secondly, work is being done to improve the robustness of the OPFsolver. While MINOS has been found to do a good job of finding optima givena good starting point, the early stages of the Lagrangian relaxation algorithm,when prices are being adjusted with larger steps, result in OPF problems wherethe solution lies far away from the starting point. We have found MINOS notto behave as well in these cases, especially for larger (i.e., 3000 buses) systems,even with the constrained power flow preconditioner. So the preconditioner isbeing turned into a first-stage optimizer intended to locate binding constraintsand provide a better, feasible starting point. Finally, work is being done onhow to include ramping constraints into the formulation. There are two basicvariations in the literature: the first one involves discretizing the generationrange for the dynamic copy of the active sources and disallowing transitionsthat violate ramping constraints in the dynamic program. While relativelystraightforward, this approach would result in much larger dynamic programs.The second method involves relaxing the linear ramping constraints and addingthem to the Lagrangian. There is, however, some concern about the speed ofconvergence of the corresponding multipliers, and special updating strategiesmay be part of the solution.

Finally, there remains the question about whether such an algorithm is usefulanymore. Lagrangian relaxation methods have not been used to clear energymarkets that also produce commitment schedules due to fairness problems:since the cheapest commitment schedule obtained may depend on parametertuning, a Lagrangian relaxation solution may lack the clarity and uncontesta-bility required in a method used to clear a market. In addition, the issue ofcomputation time also arises, since the algorithm requires the solution of many


complex AC OPFs; a method used to clear a market should be expeditious dueto practical considerations. This second consideration is actually an issue thatwill become moot in time, given the exponential growth of typical computingcapacity and the algorithm’s amenability to parallel computation. The firstissue is more serious, and inherent in Lagrangian relaxation methods for unitcommitment. So at this point it is difficult to advocate the use of the algorithmfor market clearing. However, it is still a tool that can be used by ISOs forlocation-based market power studies as well as transmission capability assess-ment and expansion. Lastly, from an experimental economics viewpoint, thebest way to assess the performance of a market is to compare its efficiency tothat of the maximum social welfare solution. This algorithm can provide thissolution (the modifications for elastic demand are trivial) and help to makemarket efficiency comparisons.

ACKNOWLEDGEMENTSWe wish to thank Ray Zimmerman and Deqiang Gan for providing us with

their package MATPOWER [19] for the initial tests of the algorithm. Wewould also like to thank Csaba Mészáros, whose QP program [20] BPMPD weuse in several of our programs, and finally, John Zollweg, principal developerof MultiMATLAB.

REFERENCES1.

2.

3.

4.

5.

6.

7.

8.

9.

J.A. Muckstadt and R.C. Wilson. An application of mixed-integer programmingduality to scheduling thermal generating systems. IEEE Trans. Power ApparatusSyst., 87(12):1968–1978, 1968.J. A. Muckstadt and S.A. Koenig. An application of Lagrange relaxation toscheduling in power-generation systems. Oper. Res., 25(3):387–403, 1977.G.S. Lauer, D.P. Bertsekas, N.R. Sandell, and T.A. Posbergh. Solution of large–scale optimal unit commitment problems. IEEE Trans. Power Apparatus Syst.,101(1):79–85, 1982.C. Lemarechal and J. Zowe. A condensed introduction to bundle methods in non-smooth optimization, in Algorithms for Continuous Optimization, E. Spedicato,Ed., Kluwer Academic Pub., 1994.

and A new approach for solving extended unit commit-ment problem. IEEE Trans. Power Syst., 6(l):269–277, 1991.J. Batut and A. Renaud. Daily generation scheduling optimization with trans-mission constraints: a new class of algorithms. IEEE Trans. Power Syst.,7(3):982–989, 1992.A. Renaud. Daily generation management at Electricité de France: from planningtowards real time. IEEE Trans. Autom. Cont., 38(7):1080–1093, 1993.R. Baldick. The generalized unit commitment problem. IEEE Trans. Power Syst.,10(l):465–475, 1995.J.J. Shaw. A direct method for security-constrained unit commitment. IEEETrans. Power Syst., 10(3): 1329–1339, 1995.


10.

11.

12.

13.

14.

15.

16.

17.

18.

19.

20.

21.

22.

23.

24.

25.

26.

S.J. Wang, S.M. Shahidehpour, D.S. Kirschen, S. Mokhtari, and G.D. Irisarri.Short-term generation scheduling with transmission and environmental con-straints using an augmented Lagrangian relaxation. IEEE Trans. Power Syst.,10(3):1294–1301, 1995.K.H. Abdul-Rahman, S.M. Shahidehpour, M. Aganagic, and S. Mokhtari. Apractical resource scheduling with OPF constraints. IEEE Trans. Power Syst.,11(1):254–259, 1996.H. Ma and S.M. Shahidehpour. Unit commitment with transmission securityand voltage constraints. IEEE Trans. Power Syst., 14(2):757-764, 1999.X. Guan, R. Baldick, and W.H. Liu. Integrating power system scheduling andoptimal power flow. In Proc. 12th Power Systems Computation Conference, Dres-den, Germany, August 19–23, 1996, pp. 717–723.C.E. Murillo-Sánchez and R.J. Thomas. Thermal unit commitment includingoptimal AC power flow constraints. In Proc. 31st HICSS Conference, Kona,Hawaii, Jan. 6–9 1998.D.P. Bertsekas, G.S. Lauer, N.R. Sandell, and T.A. Posbergh. Optimal short-term scheduling of large-scale power systems. IEEE Trans. Autom. Cont.,28(1):1–11, 1983.G. Cohen. Auxiliary problem principle and decomposition of optimization prob-lems. Optim. Theory Appl., 32(3):277-305, 1980.G. Cohen and D.L. Zhu. Decomposition coordination methods in large scaleoptimization problems: the nondifferentiable case and the use of augmented La-grangians. In Advances in Large Scale Systems, Vol. 1; J.B. Cruz, Ed., JAI PressInc, 1984, pp. 203–266.B.A. Murtagh and M.A. Saunders. MINOS 5.5 user’s guide. Stanford UniversitySystems Optimization Laboratory Technical Report SOL 83-20R.R. Zimmerman and D. Gan. MATPOWER: A Matlab power system simulationpackage. http://www.pserc. Cornell.edu/matpower/.Cs. Mészáros. The Efficient Implementation of Interior point Methods for LinearProgramming and Their Applications. Ph.D. Thesis, Eötvös Loránd Universityof Sciences, 1996.O. Alsac and B. Stott. Optimal load flow with steady-state security. IEEE Trans.Power Apparatus Syst., 93(3):745–751, 1974.J. Zollweg. MultiMATLAB for NT Clusters. Cornell Theory Center SoftwareDocumentation, http://www. tc.Cornell. edu/UserDoc/Intel/Software/multimatlab/.A.E. Trefethen, V.S. Menon, C.C. Chang, G.J. Czajkowski, C. Myers, and L.N.Trefethen. MultiMatlab: MATLAB on Multiple Processors. Cornell UniversityComputer Science Technical Report 96TR239.W. Gropp, E. Lusk, and A. Skjellum. Using MPI: Portable parallel programmingwith the Message Passing Interface. MIT Press, 1994.A.I. Cohen and V.R. Sherkat. Optimization-based methods for operationsscheduling. Proc. IEEE, 75(12):1574–1591, 1987.G.B. Sheblé and G.N. Fahd. Unit commitment literature synopsis. IEEE Trans.Power Syst., 9(1):128–135, 1994.

Chapter 6

OPTIMAL SELF-COMMITMENT UNDERUNCERTAIN ENERGY AND RESERVE PRICES

R. Rajaraman and L. KirschLaurits R. Christensen Associates

F.L. AlvaradoUniversity of Wisconsin

C. ClarkElectric Power Research Institute

This paper describes and solves the problem of finding the optimal self-commit-ment policy in the presence of exogenous price uncertainty, inter-product substi-tution options (energy versus reserves sales), and different markets (real-timeversus day-ahead), while taking into consideration intertemporal effects. Thegenerator models consider minimum and maximum output levels for energy anddifferent kinds of reserves, ramping rate limits, minimum up- and down-times,incremental energy costs and start-up and shut-down costs. Finding the optimalmarket-responsive generator commitment and dispatch policy in response to ex-ogenous uncertain prices for energy and reserves is analogous to exercising asequence of financial options. The method can be used to develop bids for en-ergy and reserve services in competitive power markets. The method can alsobe used to determine the optimal policy of physically allocating generating andreserve output among different markets (e.g., hour-ahead versus day-ahead).

Unit commitment refers to the problem of deciding when to start andwhen to shut-down generators in anticipation of changing demand [1]. In tra-ditional utility systems, the problem of unit commitment was formulated andsolved as a multi-period optimization problem. In the traditional problemformulation, the anticipated demand was an input variable. The problem wassolved for multiple generators, generally owned by the same entity (a utility).The start-up, shut-down and operating costs of the generators were assumedknown. The standard way to analyze and solve this problem was by dynamicprogramming, and within this category of problems, the most popular solution

Abstract:

1. INTRODUCTION

method in recent years has been the use of Lagrangian relaxation [2,3]. Re-cently, a new method of decommitment has also been proposed [4,5].

Several things change in a deregulated market. Generators generally haveto self-commit their units optimally. Since in most power pools, no singlemerchant owns all the generating assets, the need to meet system load is re-placed with the need to optimize profits of the merchant’s generating plantsbased on the uncertain market prices at the locations where the generators arelocated. Of course, the forecasts of markets prices depend upon a number offactors, the most important of which include demand, system-wide generationavailability and cost characteristics, and transmission constraints.

We pose the problem of finding the profit-maximizing commitment pol-icy of a generating plant that has elected to self-commit in response to exoge-nous but uncertain energy and reserve price forecasts. Typically, one genera-tor’s output does not physically constrain the output of a different generator1,so this policy can be applied to each generator in the merchant’s portfolioseparately and independently. Therefore, for ease of exposition, we assumethe case of a single generator. Generator characteristics such as start-up andshut-down costs, minimum and maximum up- and down-times, ramping rates,etc., of this generator are assumed known. The variation of prices for energyand reserves in future time frames is known only statistically. In particular,the prices follow a stochastic rather than deterministic process. We model theprocess using a Markov chain. The method is applicable to multiple markets(e.g., day-ahead, hour-ahead) and multiple products (energy, reserves).

Other researchers have modeled the effect of energy price uncertainty ongenerator valuation. In [6], the author models the effect of the spark spread onshort-term generator valuation. In [7], the authors propose mean revertingprice processes and use financial options theory [9] to value a generatingplant. In both [6] and [7], however, the authors neglect the effect of realisticoperating constraints such as minimum start-up and shut-down times. In [8],the authors improve upon this work to more realistically include the effect ofoperating constraints to find the short-term value of a generating asset; yetthey neglect the effect of ramping rates. All these papers make assumptionsabout the risk-neutral price process in order to value the power plant.

This chapter focuses not on generator valuation, but on finding the gen-eral principles for generator self-commitment in the presence of exogenousprice uncertainty and market multiplicity. We derive the basic mathematicalprinciples from dynamic programming theory [10]. We consider energy andreserve markets, although the method can be extended to include additionalmarket choices, such as day-ahead versus real-time markets. Section 2 of this

Exceptions include multiple hydroelectric units connected in series and restrictions on aggre-gate emission levels from multiple generators within an area.


1

chapter defines the problem. Section 3 gives the dynamic programming solu-tion to the problem, and Sections 4 and 5 illustrate the features of the optimalcommitment policy using simple illustrative examples. Section 6 gives a moredetailed numerical result for a peaking generator, and Section 7 concludesthe paper. Appendix A is a technical section that solves the single-period op-timal generator dispatch problem given exogenous prices of energy and re-serves.

We begin by describing the exogenous inputs to the problem.Generator capability and cost characteristics. At any given time t,

generator G is assumed to be in state where is a member of a discrete setX={state 1, state 2, ..., state K} of possible states. Intertemporal constraintsare represented by state transition rules that specify the possible states that thegenerator can move to in time period t+1, given that the generator is in a state

at time t. Generally, there is a cost associated in moving between differentstates. In a simple representation, two states are sufficient: “in service” (or“up”) and “out of service” (or “down”). In general, however, many morestates may be needed to represent the various conditions of the start-up andshut-down process.

The degree to which a generator can participate in providing reserves de-pends on its ability to respond to the reserve needs in a timely manner. Forregulating and spinning reserves, the generator must be already be in service;the amount of MW of reserves that a generator can offer must be consistentwith its ramping rate. Generators that are already at a maximum in terms ofenergy provision are unable to also participate in the reserves market. Thus, toparticipate in the reserves markets, the generator cannot simultaneously sellall of its capability in the energy market.

The parameters that describe a generator include:Minimum and maximum output levelsRamping ratesMinimum up- and down-times for the generatorIncremental energy costs and no-load costsStart-up, shut-down, and banking costs.

Generator states and state transitions. Generators can be in any of anumber of several possible UP, DOWN, or transitional states. For example,for a generator with total capacity of 200 MW, and ramp rate of 100MW/hour, we could define two UP states: and The state wouldcover the operating range [0 MW, 100 MW], while the state would coverthe operating range [101 MW, 200 MW]. Likewise, minimum down times

Self-Commitment Under Uncertain Prices 95

2. THE PROBLEM


can be enforced by defining multiple DOWN states. Only certain transitionsamong these states are permissible. Furthermore, transition between statesgenerally involves a cost. For example, going from a cold shutdown to an on-line state will involve a start-up cost.

Generator dispatch constraints. A generator may have additional dis-patch constraints that restrict its operation. For example, the generator mayneed to be offline (in the “off” state) during certain periods for scheduled re-pairs. These restrictions are modeled as time-dependent constraints on genera-tor states.

Exogenous price forecast of energy. A discrete Markov process modelsthe exogenous price for energy. In each time period, a discrete price staterepresents a price range. The price at time is probabilistically related tothe price at time t via a price transition matrix. That is,

is a known quantity. One can think of a price forecast at any time tto be a baseline price point plus a random uncertainty around the baselineforecast.

Since the exogenous price forecast is an important input of the problem,we digress a little to discuss how one may obtain estimates of this input. Weconsider two ways in which price forecasts can be made:

One possible method is to use historical data. For example, to obtain aprice forecast for next week, one could use past week data and datafrom other weeks with similar load/weather patterns as that predictedfor next week. This would be a statistical data-mining problem.Another way to forecast prices would be to use numerous Monte Carloiterations of structural computer models (such as optimal power flowmodels and production cost models) to model the uncertainty ofprices.

The physical spot markets for energy include real-time, hour-ahead, day-ahead, and possibly week-ahead markets2. Each of these spot markets is adifferent market, e.g., energy prices for a particular hour could be different inday-ahead and real-time markets and could have very different characteristicsin terms of price volatility. A generator will often have a choice as to whichmarket to use to sell its energy.

Exogenous price forecast of reserves. Operating reserves are distin-guished by the speed with which they become available and the length of timethat they remain available. In the nomenclature of the Federal Energy Regula-tory Commission, the primary types of reserves (from fastest to slowest) areregulating, spinning, supplemental, and backup reserves. For example, regu-lating reserves need to be available for following moment to moment fluctua-

It is doubtful whether forward prices quoted month-ahead (or more) will influence an indi-vidual generator’s commitment and dispatch decisions.

2


tions in system demand and can generally be offered by generators on auto-matic generation control (AGC). As another example, to offer 10 MW ofspinning reserves, a generator must be online and must be capable of produc-ing 10 MW within 10 minutes. Supplemental reserves and backup reservesare slower reserve types. For generators who provide reserve services, thereare two types of reserve costs (see [11] for more details). These are reserveavailability costs, which are the costs of making reserves available even ifthey are not actually used, and reserve use costs, which are the costs incurredwhen the reserves are actually used. Generally, reserve use costs are compen-sated at the spot price of energy3. Reserve availability costs are the opportu-nity costs of generators, i.e., they include off-economic dispatch costs, andcosts of starting up or shutting down generators. In California, New York,New England, and the Pennsylvania-Jersey-Maryland (PJM) system, there iscurrently a competitive market-clearing process for setting the reserve avail-ability costs of some or all of the above reserve types. Reserve availabilityprices are modeled similarly to energy prices, i.e., as a discrete Markov chain.We allow, however, the reserve availability prices to be correlated with theenergy prices. One particularly simple way to model reserve uncertainty is toassume perfect correlation between energy prices and reserve availabilityprices.

Exogenous fuel price forecasts. Fuel prices can be modeled similarly toreserve availability prices. Since most practical commitment periods are lessthan a week and fuel prices typically show much less volatility than energyprices over this interval, it is a not a bad approximation to keep these inputsconstant. For ease of exposition, therefore, we assume fixed fuel pricesthroughout the paper. It is fairly straightforward to also include uncertainty infuel prices; see, for example, [8].

Start and end time periods. We will assume that the start time period isat time 1 and that the end time period is at time T. For most practical prob-lems, T will be between one day and one week.

Next, we define some notation; explicit functional dependencies are oftenomitted for clarity. Given generator state and a vector of energy and re-serve availability prices, at time t,

1. (or simply defines the commitment policy for time t, i.e., itsignifies a particular valid rule to move the generator from state attime t to a new state at time t+1.

3 In general, reserve use costs could also include the wear-and-tear costs of ramping up anddown to follow system demand. The current custom is that, in competitive markets for re-serve services, these costs are not directly compensated and must somehow be internalizedby the generators.


2.

3.

4.

(or simply defines the dispatch policy for time t, i.e., itrepresents a particular dispatch of energy and reserves for the genera-tor at time t, given that the generator is in state

denotes single-period profits at time i.e., the profits realizedby the dispatch

denotes the cost of transitioning (e.g., start-up or shut-downcosts) from state at time t to state at time t+1 due to thecommitment policy used in time period t.

The problem can now be posed as:[PROBLEM] Find the best commitment and dispatch policy (u*, d*)

that maximizes expected total profits over all possiblecommitment policies and all possible dispatch policies

where E denotes the expected value over the uncertainprice forecasts5.

Before we proceed further, it is useful to summarize the essential featuresof the problem:

Inter-product substitutability. Market participants have a choice be-tween sales in the energy versus sales in the reserve markets. Thesemarkets operate simultaneously (though the markets for energy and re-serves may clear sequentially, as they currently do in California).Moreover, market participants have a choice of offering their productsin different markets (e.g., day-ahead versus hour-ahead markets).Price uncertainty. The future prices of energy and of reserves at thelocation of the generator of interest are unknown but follow a knownrandom process. The general characteristics of the random process areestimated by the generator wishing to self-commit.Intertemporal ffects. Intertemporal constraints affect the generator’soperations. This may lead to situations when a market participant canelect to remain on during certain periods when operation will be at aloss in return for likely (but not certain) profits in later periods.

We are interested in finding both the optimal commitment and the optimaldispatch policy. We stress that the problem is complicated by the fact that atthe time the commitment decision is made, future prices are uncertain. Thenext section addresses this problem.

4 Current period profits do not include transition costs.5 This objective function assumes that the generator is risk-neutral. If the generator is risk-

averse, the objective function should reduce the expected outcome according to some meas-ure of risk. For example, the objective might be to maximize where “V” isthe variance of net profit and “a” is a risk aversion coefficient. As another example, the ob-jective function could be an exponential utility function with constant relative risk aversion[12]. Now the objective function would be multiplicative in nature, but we can take naturallogarithms to convert the objective function to the form shown in this chapter.

Self-Commitment Under Uncertain Prices

3. OPTIMAL COMMITMENT AND DISPATCHPOLICY

99

We now present the optimal commitment and dispatch policy. The opti-mal dispatch policy is fairly straightforward: given an exogenous price fore-cast for time period t, the generator takes its current state as given and dis-patches energy and reserves in an optimal manner for time period t, withoutregard to other time periods. Appendix A describes the single-period optimaldispatch. The profit-maximizing commitment decision for transitioning to thenext time period is more complicated, however, because actions taken nowaffect future time periods.

The backward dynamic programming (DP) method for solving this prob-lem starts at the final time period T and works backward using the followingsteps. The backward DP method [3,10] is as follows6:

Step 1. Let over all possiblecommitment policies and dispatch policies Let the optimal dispatchpolicy be denoted by and the optimal commitment policy bymust be computed for each possible state and each possible price level

Step 2. Let overall possible commitment policies and dispatch policies The expecta-tion E is taken over all possible price levels given that the price in timeT–l is is probabilistically related to via the Markov chain. Thestate at time T, is related to by the commitment policy Let the op-timal dispatch policy be and the optimal commitment be

must be computed for each possible state andeach possible price level

Step T. Let over all possi-ble commitment policies dispatch policies and price levels The ex-pectation E is taken over all possible price levels given that the price intime 1 is The state at time 2, is related to the previous state by thecommitment policy i.e., Let the optimal dispatch policy be

and the optimal commitment policy be must becomputed for each possible state and each price level

If the generator is in state and sees price level at time t=l, theprofit-maximizing schedules are represented by the commitment actions

and the dispatch decisionswhere for

t=1,2,...,T-l, and is related probabilistically to via the Markov chain,and the maximum expected profits are Actual profits and actual

6 For ease of exposition, we have ignored the boundary condition at time T+l.


schedules depend upon actual price levels encountered in the different timeperiods.

The algorithm for finding the optimal commitment policy is similar to theproblem of determining the value of an option using a tree approach [9].Therefore, we can characterize the problem of finding an optimal commitmentpolicy as a generalized tree approach that values and exercises a sequence ofcomplicated options in each time period. The options involve decisions suchas whether to commit or not, whether to ramp up or ramp down, whether toparticipate in the energy or reserves markets, etc.

4. ILLUSTRATIVE EXAMPLE 1

This section illustrates the concept. For simplicity we assume only oneproduct, energy, and one three-period market, i.e., T=3.

Suppose that the generator parameters are as shown in Table 1. For thisexample, the generator at time t can be in one of two states, “UP” or“DOWN," i.e., X={UP,DOWN}. There are no intertemporal constraints. Thegenerator can move from any state at time t to any state at time t+1.

A shut-down cost is incurred when the generator moves from an UP stateto the DOWN state. A start-up cost is incurred when the generator movesfrom the DOWN state to the UP state. All other transitions result in zerocosts.

In each time period, an exogenous price forecast may be described bytwo possibilities: is HIGH or is LOW, each having a specified probabilityof occurrence. The HIGH and LOW prices in each time period are allowed tovary, as Table 2 shows. Further assume that the prices at time t+1 dependprobabilistically upon the prices at time t. The probability of a HIGH-HIGHtransition is and the probability of a LOW-LOW transition is These areexogenous, with assumed values and


The backward DP algorithm finds the profit-maximizing commitment anddispatch policy. Tables 3(a)-(c) depict the solution. Columns in these tablescorrespond to time periods. The entry in Table 3(a) that corresponds to a“time period t” column and a “state” row represents the maximum total ex-pected profits for time periods t to T, given that the state at time t is and theprice is

Similarly, given state and price at time t, the corresponding entries inTables 3(b)-(e) show respectively:

1.2.

3.4.

The optimal dispatch for time t.The optimal commitment policy for time t, i.e., the next state to moveto at time t+1.The maximum profits obtained from the optimal dispatch at time t.The cost of the optimal commitment policy, or the cost to move fromthe current state to the new state at time t+1.

Table 3 illustrates the results of a standard Backward DP computation.These results are obtained, as is standard practice, by solving for the respec-tive entries in the table from right to left.


Suppose that, at time t=l, the price level is HIGH and the generator is UP.The maximum expected profits are $415.4 over the three time periods. Sincethe current price level is HIGH and the generator is UP, the optimal commit-ment policy is to stay UP at t=2 (from Table 3(c)), in spite of the possibility ofnet losses over the three periods. The actual profits and the commitment poli-cies at other times would, however, depend upon the actual price levels thatoccur in those time periods. For example, if the price level stays HIGH forboth t=2 and t=3, the optimal commitment policy is to stay UP at t=3, realiz-ing total profits of –5+500+50=$545. If, on the other hand, the price levelbecomes LOW for t=2, and LOW for t=3, the optimal commitment policy isto go DOWN at t=3, realizing profits of –5–10–22= –$37. In other words, thegenerator loses money under some price patterns, even with the optimal pol-icy. The expected profits are maximized, however.

The optimal schedules given by the backward DP are not static. Instead,they depend upon the exogenous prices in each time period. Thus the DPmethod does not merely give an optimal schedule. Rather, it gives an optimalscheduling policy corresponding to different conditions.


In this section, we describe the “optionality” features of the generatorself-commitment problem, and show that it has features analogous to financialoptions. We also make additional three points:

1.

2.

3.

Assuming a single average price forecast generally understates thevalue of the optionality, and could severely understate expected gen-erator profits.Running Monte Carlo methods without taking care to ensure that fu-ture prices are always uncertain (at the time the commitment is made)generally overstates the value of the optionality and overstates ex-pected generator profits.Not considering reserve products (and multiple markets) tends tolower expected generator profits, because these additional productsincrease generator optionality.

First, we consider the optionality due to price uncertainty. Assume a sin-gle time-period horizon and a single product – the energy service. Assumethat the generator for which we want to find the optimal commitment and dis-patch policy has no start-up or shut-down costs and no intertemporal con-straints. Assume that generator has a capacity of 100 MW, no minimum gen-eration constraint, and constant incremental costs of $30/MWh over thisrange.


Table 4 shows five different price forecast scenarios. Two possible pricestates, HIGH and LOW, each with a 50% probability of occurrence, representeach price forecast. The expected value of the prices for all price forecasts is$30/MWh. The optimal policy for the generator is to produce 100 MW when-ever the energy price exceeds its incremental costs and to produce 0 MWwhenever the energy price is below its incremental costs. For example, forforecast #3, the generator will produce 0 MW when the price is LOW (or$20/MWh), and will make no profits. When the price is HIGH ($40/MWh),the generator will produce 100 MW and make a profit of 100*(40-30) =$1000. Since both price scenarios are equally likely for this forecast, expectedprofits are 0*0.5+1000*0.5 = $500.

Table 4 shows that expected generator profits increase with increasingprice volatility8. This is analogous to the value of a financial option that in-creases in value when price volatility increases [9]. On the other hand, if oneuses an average price of $30/MWh to find a commitment policy for the gen-erators, then one will estimate that the generator will make no profit for anyprice forecast because the generator incremental costs will not exceed the ex-pected energy price. Therefore average price forecasts fail to calculate thevalue of optionality and understate generator profits.

Next, we examine a potential pitfall associated with Monte Carlo meth-ods. One possible way of finding the expected generator profits to capture theoptionality value could be to generate a large number of random price scenar-ios for the time interval [0,T] by Monte Carlo methods. Using the ensemble

7 Price volatility here is defined as the ratio of the standard deviation to the expected price,expressed in percent (and rounded).All other factors (e.g., expected energy price) remaining constant.

8


of all generated price scenarios one could then use a deterministic model tosolve for the optimal commitment and dispatch over this period for eachmember of the ensemble and then average over different Monte Carlo runs.This is, however, not always correct. To see this, consider the following ex-ample. Assume a two-period system, with each period having a 50% chanceof HIGH price ($35/MWh) and a 50% chance of having a LOW price($10/MWh), regardless of the previous period. The generator has the samecharacteristics as the above example, except that there is a minimum genera-tion limit of 90 MW, and an additional intertemporal constraint that, onceonline, the generator has to stay online for two consecutive periods. Theboundary condition is that the generator is offline initially and must be offlineat the end of two periods. It can be verified that the optimal policy is not tocommit the generators regardless of what the period 1 price is9. Therefore, theexpected profit under the optimal commitment policy is zero. On the otherhand, suppose that we first generate all the price scenarios (using a MonteCarlo method), and then run a deterministic optimal unit commitment on eachpossible price sequence. The four equally possible price sequences in the twoperiods are {HIGH, HIGH}, {HIGH, LOW}, {LOW, LOW}, and {LOW,HIGH}. If we make four deterministic unit commitment runs on these fourprice sequences, the deterministic unit commitment will only run the genera-tor at maximum output (100 MW) for both time periods when the price se-quence is {HIGH, HIGH}. The profit for this price sequence is $1000. For allother price sequences, the generator will not run, and the profit will be zero.Hence, expected profits using this method will be 0.25*1000+0.75*0 = $250,which clearly overstates the expected profits of zero under the true optimalcommitment policy. The reason for this is that in each Monte Carlo run, thegenerator “peeped ahead” and “knew” the future prices and therefore chosethe commitment accordingly. Models for commitment based on the traditionalapproach are likely to follow a variant of this deterministic optimizationmethod. This approach would result in overestimation of generator profits.

Monte Carlo methods are very efficient when one needs to simulate alarge number of different random outcomes and find the expected value (orsome other statistic) of some function based on these random outcomes. They

9 If period 1 price is HIGH, and the generator decided to commit, then the generator wouldproduce 100 MW in period 1 to make a profit of $500. There is a 50/50 chance, however,that the period 2 price is HIGH or LOW. If the period 2 price is HIGH, the generator’s two-period profit will be $1000. If period 2 had a LOW price, the generator would produce theminimum 90 MW and lose 90*30=$2700 in period 2 for a net two-period loss of $2200.Therefore, if the generator commits to be online when the period 1 price is HIGH, the ex-pected two-period payoff is 1000*0.5 – $2200*0.5 = ($600), for an expected loss of $600.


are much more complicated to implement10, and prohibitively expensive,when the value of a function at any given time t itself depends on what mayhappen in the future, as in optimal commitment policy problems that haveintertemporal constraints11. For example, using the finance analogy, MonteCarlo methods are used for European style options and for those other typesof options when one does not have to worry about when it is optimal to exer-cise the option. American style options, however, are much more difficult tosolve for using Monte Carlo methods [12, pp. 685].

Consider a final example to show the optionality value of multiple prod-ucts (or multiple markets). Assume a single time-period problem and a singlereserve product. Consider a generator whose incremental cost is $30/MWh,maximum capacity is 100 MW, minimum capacity is 0 MW, and maximumreserve capacity of 30 MW. Assume that the price of energy is $45/MWh andreserve availability costs of $20/MW/h. If the generator offers 100 MW ofenergy only, it will make profits of 100*(45–30) = $1500, on revenues of100*45=$4500. On the other hand, if it maximizes its profits12 and offers 30MW of reserve and 70 MW of energy, its profits are 30*20+70*(45–30)=$1650 on revenues of 30*20+70*45=$3750. (Shifting more ofthe generator output to reserves increases profits, even though total revenuesdecrease relative to the energy-only sale. This is typical.)

In summary, the more optionality that a generator has, the higher its ex-pected profits will be. Conversely, the more the operational constraints reducethis optionality (e.g., intertemporal constraints), the lesser will be its expectedprofits, all other factors being equal.


We now consider a slightly more realistic example.13 Assume a peakinggenerator with the characteristics illustrated in Table 5. The start-up and shut-down times for the generator are assumed to be zero. Figure 1 shows the base-

10 See reference [8] for the correct way to implement Monte Carlo methods for such problems.In [8], however, the Monte Carlo method becomes prohibitively expensive as the number ofgenerator states and price uncertainty states increase. See also Section 6 for how to useMonte Carlo methods once the optimal self-commitment policy is known.

11 The Monte Carlo method discussed in this section will work correctly on the problem de-scribed in Table 4, because the optimal self-commitment policy does not have intertemporalfeatures.

12 Appendix A shows how one may approach the problem when there are more than one re-serve type. In this example, while there is profit to be made on the sale of both energy andreserves, the generator sees a higher profit margin in reserves and so maximizes the sale ofreserves (30 MW). The remainder is offered as energy (100-30=70 MW).

13 The results of this section were derived using EPRI’s PROFITMAX model.


line energy price forecast. Figures 2 and 3 illustrate the anticipated baselineprices for 4 types of reserves: regulating, spinning, supplemental, andbackup.14 The total number of time periods is one week (168 hours).

14 Spinning reserves are defined to be the capability that can be offered in 10 minutes (ifonline). Supplemental reserves are the amount of MW available in 20 minutes, and backupreserves are the amount available in 60 minutes.


We model price uncertainty using a three-node Markov process. We as-sume that prices can be at one of three levels: HIGH, BASELINE, or LOW.The BASELINE prices are as illustrated. The HIGH price for energy and re-


serves is 115% of the corresponding baseline price, and the LOW price is85% of the baseline price. We assume that there is perfect correlation betweenenergy and reserve availability prices; e.g., when the energy price is HIGH, soare the reserve availability prices. For simplicity, we assume that the probabil-ity of transition between any one price state to any other price state is 1/3.That is, it is equally likely for the price to change states regardless of the pre-sent state of prices.

Using the backward DP methods described in Section 3, we derived theoptimal commitment and dispatch policy. The optimal commitment and dis-patch policy at any time t is a function of the state that the generator is in, anda function of the uncertain price forecast for future time periods as observed attime t. Future prices are always considered uncertain. We then applied theoptimal policy15 in numerous Monte Carlo runs to simulate different profitoutcomes. From the Monte Carlo prices, we then calculated the actual even-tual profits and generator outputs. Using these outcomes, we then illustrate theprobability distribution of different generator outputs: generator profits, opti-mal generator dispatch of energy and reserves, etc.

When we use the optimal policy found by the DP to simulate a number ofpossible states, we obtain a distribution of possible outcomes. Figure 4 illus-trates this distribution. There is no assurance that the distribution will beneatly “bell-shaped” as in this example. In other examples it is possible tohave distributions that are skewed or bimodal, particularly when we considerstart-up and shut-down costs.

The effects of the optimal commitment policy on the state transitions areshown in Figure 5. For a large number of scenarios, all transitions betweenstates that result from the optimal policy are recorded. For some times, thestate is unique (either UP or DOWN), but for other times the system couldend up in either state, depending on the price sequence. This is because it isnot known which state the generator will be at a future time t. Figure 5 showsthat, depending on the prices that are actually realized, the generator could bein any of the different states at a future time. The ball “size” represents theprobability of ending up in a particular state. The “thickness” of a transitionline indicates the likelihood that the particular transition will take place.

15 We stress that the optimal policy was an input (not an output) in the Monte-Carlo runs.


During the interval depicted in Figure 5, the optimal policy has the optionto take several transitions, depending on the actual price realized. Only hours93, 94, and 104 to 106 have a certain state (OFF in this case). For other peri-ods the relative probability of being in either state is represented by the size ofthe “ball” and the relative transition probability is represented by the thicknessof the transition “line.” During certain periods there is both an up transitionprobability and a down transition probability in the optimal policy.


For comparison, Figure 6 illustrates the optimal commitment policy whenthere is no uncertainty, i.e., the baseline price forecast is the perfect forecast.It can be seen that the effect of price uncertainty recognizes the possibilitythat the generator will either begin producing output (periods 104 through108) or will turn off (periods 94 through 104) during certain periods accord-ing to what prices are actually realized. That is, a quick-responding generatorhas the luxury of producing when the prices are high, and going offline whenthe prices are low. Thus high price volatility tends to be beneficial for the ex-pected profits of the peaking generator. When prices are higher than its in-cremental costs, the peaking generator will maximize its output (and increaseprofits), while when prices are below its incremental costs, the generator willshut down (and have zero profits). Since profits are bounded from below atzero, and monotonically increase as a function of price (above the generator’sincremental costs) the generator’s expected profits will increase.

Figure 7 illustrates this notion. It shows that when expected prices areheld constant among scenarios, expected profits increase as price uncertaintyincreases. This again shows that committing and dispatching a generator isanalogous to exercising a financial option. The option value generally in-creases when uncertainty increases.16 This is because a peaking generator isable to follow changes more readily than other plant types. Thus, price volatil-ity is beneficial to peaking units, a result that may be familiar to many read-ers. In Figure 11, we are able to precisely quantify this benefit.

16 An intermediate or cycling generator has less optionality features because inter-temporalconstraints affect its profits, and it has features similar to Asian options [9]. A baseload gen-erator has even less optionality features (excepting for the important question of which mar-ket to sell into), usually because its incremental costs are generally well below the marketprices and is analogous to forward contracts [9].


Figure 8 illustrates the expected sales of energy and reserves as a functionof time. Note the striking fact that the generator in question offers energy onlyduring certain periods, but derives income from making reserves availableduring many more periods.

7. CONCLUSIONS

The contribution of this chapter is to describe the multi-period multi-market uncertainty framework within which decisions for unit commitmentand dispatch will have to take place for many units that operate in a deregu-lated market. The chapter applies directly to the problem of optimal generatorself-commitment. It describes a method for finding the most profitable mar-ket-responsive commitment and dispatch policy that takes into full accountthe optionality available to a generator: reserve market opportunities, multiplemarkets, price uncertainty, and intertemporal constraints. The model usesbackward dynamic programming, and the algorithm in the model can bethought of as a generalized tree that values and exercises a sequence of com-plicated options. This algorithm can be used to obtain optimal power marketbids for energy and reserve services in markets that integrate both these needs.The method can also be used to profitably allocate output in different physicalforward markets, e.g., hour-ahead versus day-ahead.


ACKNOWLEDGMENTS

We thank EPRI PM&RM for support of this work, with special thanks toVictor Niemeyer of EPRI for his support and encouragement. In particular,the work on the PROFITMAX model developed by Christensen Associateswas sponsored by EPRI and initiated in 1997. We thank Blagoy Borissov ofChristensen Associates for his help in simulating some of the results in thispaper. We also thank Fritz Schulz for his help with the original implementa-tion of PROFITMAX.

REFERENCES

1.2.

3.

4.

5.

6.

A.J. Wood and B.F. Wollenberg. Power Generation Operation and Control, Wiley 1984.A. Merlin and P. Sandrin. A new method for unit commitment at Electricite De France.IEEE Trans. Power Apparatus Syst., PAS-102(5): 1983.D.P. Bertsekas et al. Optimal short-term scheduling of large-scale power systems. IEEETrans. Autom. Cont., AC-28(1): 1983.C.A. Li, R.B. Johnson and A.J. Svoboda. A New Unit Commitment Method. IEEE Trans..Power Syst., 12(1): 1997.C.L. Tseng, S.S. Oren, A.J. Svoboda, and R.B. Johnson. A unit decommitment method inpower system scheduling. Elec. Power Energy Syst., 19(6): 1997.M. Hsu. Spark spread options are hot! Electricity J., 11(2): 1998.


7.

8.

9.

S. Deng, B. Johnson, A. Sogomonian. “Exotic Electricity Options and the Valuation ofElectricity Generation and Transmission Assets.” Research Report #PSERC 98-13, 1998(also available in the limited access http://www.pserc.wisc.edu/index_publications.html).C.L. Tseng and G. Bartz. “Short Term Generation Asset Valuation.” Research Report#PSERC 98-20, 1998 (also available in the limited access website http://www.pserc.wisc.edu/index_publications.html).J.C. Hull. Options, Futures, and Other Derivatives, Prentice Hall, 1997.D.P. Bersekas. Dynamic Programming and Optimal Control, Volumes 1 and 2, AthenaScientific, June 1995.L. Kirsch and R. Rajaraman. Profiting from operating reserves. Electricity J., March: 1998.H.R. Varian. Microeconomic Analysis, W.W. Norton & Company, Edition, 1992.P. Wilmott. Derivatives: The Theory and Practice of Financial Engineering, John Wiley &Sons, 1998.

10.

11.12.13.

APPENDIX A

This appendix describes how a profit-maximizing generator would dis-patch energy and reserve availability services given exogenous market pricesfor a given hour, and given that it is committed to be online.

AssumptionsConsider the output choice faced by a generator that can offer, in any

given hour, energy and four reserve services. Assume that the energy price isPE, and that the availability prices for the four reserves are PR1, PR2, PR3,PR4, respectively. Suppose that the generator has maximum output level ME,and that, because of ramping limitations, the generator can provide the maxi-mum quantities of the reserve XR1 for reserve 1, XR2 for reserve 2, XR3 forreserve 3, and XR4 for reserve 4. Further suppose that the generator’s produc-tion cost function is:

where a, b, and c are constants and XE is the generator’s quantity of offeredenergy. What quantities of energy and reserves should the generator offer if itis maximizing profits?

The Optimization ProblemThe generators problem is to maximize profits17:

subject to the constraints that all energy and reserve quantities must respectmaximum limits:

17 When a generator offers reserves, there is a certain probability of these reserves being called.When reserves are called to produce energy, they will receive the market energy price. Onecan easily include this effect in the objective function (A-2).


For simplicity and without loss of generality, we ignore the constraintsthat all reserve quantities must be non-negative, and that the energy quantityhave a minimum limit. Although equations (A-2) and (A-3) make the reservesall appear to be mathematically identical, they are not because we assume(reasonably) that reserve prices have a particular order18:

Because production costs (A-l) depend only upon energy output, the gen-erator will prefer to sell Reserve 1 first and Reserve 4 last.

The SolutionThe Lagrangian for the optimization problem is:

The solution to the foregoing problem is:

Note that the shadow value of capacity only if energy and reservesuse the generator to its full capacity, and that only if XRj=Mj for j=1,2,3.

The solution to the above problem is:

18 Theoretically, the reserve availability prices must be highest for the “highest” quality reserve(regulation) and lowest for the “lowest” quality reserves (backup). The reasoning is that gen-erators that can offer “higher” quality reserves can always offer “lower” quality reserves, butnot necessarily vice-versa. Therefore, the availability prices for “higher” quality reservesmust be higher than the “lower” quality reserves. Because of market imperfections, however,this relationship is not always obeyed; e.g. see historical ancillary service prices from theCalifornia ISO website (http://www.caiso.com). Note that in equation (A-4), we do not nec-essarily assume that the highest quality reserves will have the highest price, i.e., we allowmarket imperfections.


If Reserve 1 is (optimally) offered in a positive quantity that is less thanits limit, then But if Reserve 1 is (optimally) offered to its limit, then

In general:if Reserve j is offered at all, then all reserves <j are at their limits;if Reserve j is offered in a positive quantity that is less than its limit,then and all reserves >j are not offered at all.

Chapter 7

A STOCHASTIC MODEL FOR A PRICE-BASEDUNIT COMMITMENT PROBLEM AND ITSAPPLICATION TO SHORT-TERMGENERATION ASSET VALUATION

Chung-Li TsengUniversity of Maryland

Abstract: In this paper, we model the unit commitment problem as a multi-stage stochas-tic programming problem under price and load uncertainties. We assume thatthere are hourly spot markets for both electricity and fuel consumed by thegenerators. In each time period, the operator needs to determine which unitsare to be scheduled so as to maximize the profit while meeting the demand.Assuming that the price and load uncertainties can be represented by a sce-nario tree, we develop a unit decommitment method using dynamic program-ming to solve this problem. When there is only one unit under consideration,we show that a scenario tree can be converted to a lattice that allows branchrecombination, which may greatly reduce the size of state space. This one-unitproblem can be used to value a generation asset over a short-term period. Inconclusion, we present our numerical results.

1. INTRODUCTION

Unit commitment is a problem to schedule generating units economi-cally to meet forecasted demand and operating constraints, such as spinningreserve requirements, over a short time horizon. The unit commitment deci-sion determines which units will be used in each time period. It is a mixed-integer programming problem and is in the class of NP-hard problems (e.g.[1]). Because of its problem size and the NP-hardness, the optimal solutionof the unit commitment problem is normally difficult to obtain. Many opti-mization methods have been proposed to solve the unit commitment prob-lem. These methods include the priority list method [2], the dynamic pro-gramming approaches (e.g. [3-5]), the branch-and-bound methods [6-8], and


the Lagrangian relaxation methods (e.g. [9-11]). Among them, the Lagran-gian relaxation methods are the most advanced and widely used approaches.

Although the unit commitment problem has been widely studied duringthe past decades, most of the approaches do not consider uncertainties. Thetraditional unit commitment problem aims to schedule generation units tomeet the forecasted demand. When the actual demand is not equal to theforecasted value, the discrepancy can be handled by system spinning reserveto some extent. In [12], we present a stochastic model for the unit commit-ment problem in which the demands are not known with certainty. We thenuse a scenario tree to capture the demand uncertainty and the apply Lagran-gian relaxation to decompose the problem into scenario subproblems usingprogressive hedging [13]. Carpentier et al. in [14] propose another decompo-sition scheme using the augmented Lagrangian method. In these two ap-proaches, the unit commitment decision is modeled as a multi-stage prob-lem. Carøe and Schultz develop a two-stage stochastic programming modelfor the unit commitment problem, in which the authors emphasize the plan-ning decision over the entire planning horizon rather than multi-stage im-plementation of the operating decision [15].

With the evolution of deregulation in the electricity industry and the in-troduction of spot markets for both electricity output and fuel input, incorpo-rating uncertainties to the unit commitment problem becomes a necessity forutilities or power generators. In this chapter, from the perspective of a firmowning generating units, we model the unit commitment problem as a multi-stage stochastic programming problem under price and load uncertainties.We assume that there are hourly spot markets for electricity and the fuelconsumed by the generators. Our research uses a scenario tree to representthe uncertainties as in [12]. However, we develop a new method using unitdecommitment to solve this problem.

Li et al. [16] and Tseng et al. [17] proposed independently the methodof unit decommitment for the traditional unit commitment problem. In [16],the authors propose a solution procedure, which initially turns on all avail-able units at all hours and then performs only decommitment. The authorsview their method as a Lagrangian relaxation-like method and take the mul-tipliers from economic dispatch instead of sub-gradients. On the other hand,in [17] the authors propose using unit decommitment as a post-processingtool for existing solution procedures for solving the unit commitment prob-lem. They consolidate these two approaches as a unified unit commitmentalgorithm [18], in which they also conduct extensive numerical tests. Theirresults show that the unit decommitment method on average obtains solu-tions almost as good as the Lagrangian relaxation approaches, but withmuch less CPU times.

In this chapter, we extend the unified approach presented in [18] tosolve the stochastic unit commitment problem. Assume that the price and

A Stochastic Model for a Price-Based Unit Commitment Problem 119

load uncertainties can be represented by a scenario tree. Initially, all generat-ing units are committed in all scenarios and in all time periods, to the bestextent. Each unit’s schedule (in all scenarios) is then tentatively improved inturn with other units’ schedules fixed. At each iteration, only one unit’s ten-tative schedule is selected and updated. The iteration proceeds until no im-provement can be made. Initial numerical tests seem to suggest that the unitdecommitment approach applied to the stochastic unit commitment problemretains the properties when applied to the traditional deterministic case, asreported in [18].

When there is only one generating unit under consideration, the problemcan be used to value a generation asset over a short-term period (e.g. [19]).This problem appraises the flexibility of a power plant’s real options, e.g.committing or decommitting a unit, in a competitive environment (e.g.[20]). We will show that in this special case we can convert a scenario treeto a lattice that allows branch recombination, which may greatly reduce thesize of state space.

In Section 2, we establish the mathematical model for the stochastic unitcommitment problem. Section 3 presents a special case in which the uncer-tainties of the problem are perfectly known. Through this deterministic spe-cial case, we derive the method of unit decommitment. Section 4 extends theunit decommitment method to the stochastic case. We then discuss a short-term generation asset valuation problem in Section 5. Section 6 gives nu-merical test results, and we present future directions in Section 7.

2. THE MATHEMATICAL MODEL

In the model development, we introduce the following standard nota-tion, with additional symbols introduced when necessary.

: index for the number of units: index for time

: zero-one decision variable indicating whether unit is up or down in

time period

: state variable indicating the length of time that unit has been up or

down in time period

: the minimum number of periods unit must remain on after it hasbeen turned on

: the minimum number of periods unit must remain off after it has

been turned off


2.1 Operational Constraints

The operational constraints to be discussed in this paper can be de-scribed in terms of state transition. Let be the set of unit commitment

state space for unit is composed of two subsets of the states: for

the on-line states, and for the off-line states.

: the number of periods required for the boiler of unit to cool down

: decision variable indicating the amount of power unit is generating

in time period

: minimum rated capacity of unit

: maximum rated capacity of unit

: reserve available from unit in time period

: system demand (MW) in time period

: system reserve (MW) in time period (assumed to be a function of

): amount of power transaction in time period Its value can be positive(power input) or negative (power output).

: electricity price ($/MWh) in time period

: fuel price ($/MMBtu) in time period

: the heat (MMBtu) required for unit to generate (MW) of power(assumed strictly convex, increasing, and smooth)

: start-up cost associated with turning on unit at state

: shut-down cost associated with turning off unit

The operational constraints include:Minimum up-time/down-time constraints, for and


2.2 A Multi-Stage Stochastic Programming Formulation

This research considers the problem from the perspective of a firm thatowns generating units. We assume that there are hourly spot markets forboth electricity and fuel consumed by the generators. Each generating unit isviewed as a cross-commodity instrument. That is, each generator buys fuelfrom the fuel market, converts it to electricity, and sells the electricity to theelectricity market. The uncertainties considered in this model are the pricesfor both electricity and fuel and the quantity of the electricity sold. Thequantity of electricity sold in the market , called load or demand in thispaper, may be contingent upon time (e.g. peak or non-peak hours) or may beprice-elastic. The firm may also involve option-type transactions, such thatthe amount of transactions may depend on the price of electricity. Forexample, a customer may tend to call an option of quantity (e.g. swing op-tions) when the price of electricity is high and vice versa. The firm wouldlike to maximize its expected profit while meeting its load and transactions.

In the proposed stochastic model, the timing of the event occurrence is

as follows. In time period , the uncertainties are re-

vealed. Based on the states of all units from the previous hour, , the

operator needs to schedule the units to achieve maximum profit while satis-fying the demand. The operator’s decision is not only based on the informa-tion obtained in the current period, but also the expectation of future return.Any commitment decision made in the current period will then become the

State transition constraints, for and

Equation (4) also implies the following relation between and

Capacity constraints for and

Initial conditions on for at


initial condition for decision-making in the following hour and will con-strain the availability or flexibility of the unit commitment in the subsequenthour(s). In addition, there are start-up and shut-down costs associated withthe commitment decision. In this paper, both costs are generalized in thefollowing function.

Let be the decision problem to be made in time pe-

riod , given the initial conditions and and the observed elec-

tricity and fuel prices The dynamic programming type recur-rence equations can be formulated as follows:

where denotes the expectation operator, and the subscript indicates thatthe expectation is based on the price information available at time . Thelast term in (8) with the expectation operator defines another stochastic pro-gram to be considered in the subsequent time period, which is also called arecourse function. Since the fist term on the right-hand side of (8) is a con-stant that can be pulled out from the maximization, can be decomposedto two terms as follows.

Equation (10b) is subject to the physical constraints (3) to (6) and the fol-lowing constraints.

Demand constraints for

Reserve constraints for


where and is the maximum reserve forunit i.

In (9), accounts for revenue, and represents cost minimization.The terminal condition of (9) is as follows

subject to (11) and (12) at time The optimal value representing themaximum expected profit that the operator can make over the periodcan be obtained from the last step of the recursive relation as

Finally, note that in the formulation the fuel price directly affects a

generator’s cost characteristics, and the electricity price is correlated to

the transaction amount and the load , which are uncertain per se. Allof these uncertainties influence unit commitments from different perspec-tives.

2.3 Economic Dispatch

The minimization problem on the right-hand side of (10b) representsscheduling generating units over a single period with generating costsminimized. The scheduling decisions include the determination of whatunits will be on (i.e. the commitment) and the generation levels for on-lineunits. Given a commitment, the economic dispatch problem is to allocateelectricity generation economically to on-line units while satisfying the de-mand and system reserve constraints. At time given state variable and

for each unit, let be the index set of on-line units attime (Note that in this chapter a variable with a tilde hat will denote a re-alization of the variable.) The economic dispatch problem at time denoted

by rced is defined below.


Note that rced stands for “reserve-constrained economic dispatch.” In [18]

the optimality condition of rced is interpreted as follows: the

units in are divided into two sets, one set of units with “cheap reserve”but “expensive generation” (relatively), and one is the counterpart. The unitswith cheap reserve but expensive generation are operated with the same

marginal costs (or as close as possible), which are the Lagrange multipli-ers corresponding to (15b). The units with expensive reserve but cheap gen-eration are operated such that their marginal costs for reserve equal (or areas close as possible) to the Lagrange multipliers corresponding to (15c).

Proposition 1. The solution of rced exists if and only if the

following conditions hold.

3. THE DETERMINISTIC CASE

In this section we discuss a special case with certainty. An algorithmwill be developed, and will be extended to handle stochastics in a later sec-

tion. Assume that the future prices for electricity and fuel as well

as the load and transaction , are fully and perfectly

known. Equation (8) reduces to the following deterministic price-based unitcommitment formulation.

subject to constraints (3) to (6), (11) to (12), and initial conditions on at

for The first term in (17b) is a constant, and the minimiza-tion term is a traditional unit commitment problem, which can be solvedusing the methods reviewed in the introduction of this paper, such as theLagrangian relaxation method. In this section, we focus on the unit decom-mitment method proposed in [17, 18] for solving the cost minimizationproblem in (17b). First we review the unit decommitment method.


3.1 Unit Decommitment

The unit decommitment method was proposed in [17] as a post-processing tool to improve solution quality for methods solving the tradi-tional unit commitment problem. Given and a feasi-

ble schedule consider the problem ofoptimally improving one unit’s schedule by decommitment. That is, in thetime periods when the unit under consideration is already off-line, the unitremains off-line. The unit may be turned off in some on-line periods only ifdoing so is cost-saving and would not cause infeasibility. While we are im-proving a unit’s schedule, say determining for unit by decommitment,

the commitments of units other than are kept fixed. However, their genera-tion levels in some time period may change in order to balance demand andreserve if unit is decommitted in the same time period. The decommitmentrule is summarized below.

Constrained by the decommitment rule, the optimal decommitment problemfor a unit, say is formulated below.

subject to additional constraints such as (3) to (6) and (11) to (12), wherecaptures the total cost changed from units other than if unit

were turned off in time period The exact value of can be ob-

tained by solving the dispatch problem twice, one with unit committed attime and the other without.

is a 0-1 integer programming problem, and can be solved using the

following dynamic programming recursive equation.

with boundary condition


Although the unit decommitment method starts with an initial feasiblesolution of the unit commitment problem, it can be used as a complete algo-rithm for solving the unit commitment problem. Initially, as many units aspossible are turned on in all hours without violating the minimum downtimeconstraints at the initial hours. This resultant unit commitment serves as aninitial commitment, to which the unit decommitment algorithm is then ap-plied. There are, however, minor modifications required in order to makethis approach work. First, the commitments in the early iterations tend to be

where can be interpreted as the minimum cost for op-

erating unit over a period starting from time to with initial state

The optimal solution of is obtained from the last step of the dynamic

programming algorithm as In this research, we call the solu-

tion of the tentative commitment of unit

In the following algorithm, superscript denotes the iteration of

the algorithm. Let be the total cost for unit associated with

the schedule and be the optimal objective

value of solved with respect to (Note the variables in

bold faces are vectors. For example,

The unit decommitment algorithm

Step 0:

Step 1:

Step 2:

Step 3:

Step 4:

Data: Prices for and a feasible solution aregiven.Set and evaluate for

Solve with respect to and obtain for

Select a unit such that If there is no such unit,

stop; otherwise update the commitment of unit in by

the tentative commitment obtained in The resultant unit

commitment is assigned to be

Perform dispatch on to obtain and evaluateforSet go to Step 1.


seriously overcommitted such that the first inequality in (16a), theminimum load condition, is violated. In other words, the dispatch phase inStep 3 of the unit decommitment algorithm may not be feasible. When theinfeasible situation occurs, the algorithm would dispatch the units to satisfythe optimality condition to the best extent, in the sense of matching theunits’ marginal costs for both fuel cost and reserve to the correspondingmultipliers and , respectively, as closely as possible (see [18]). Bydoing this, as the decommitment procedure proceeds, the commitment ob-tained eventually satisfies the minimum load condition, and the algorithmstarts to produce feasible schedules.

Instead of obtaining the exact value as in (20), may also be

approximated. In [18] the first order approximation of is derived.

where and are the multipliers associated with rcedWith (23) plugged into (19), it is shown in [18] that starting from an eco-nomically dispatched schedule performing either the unit de-

commitment step with respect to or the Lagrangian relaxation sub-

problem with respect to to any unit would result in the same (tenta-tive) commitment. However, these two approaches differ in the commitmentupdating (the unit decommitment updates one unit at a time, while the La-grangian relaxation approach updates all units at once) and the multiplierupdating (the unit decommitment performs economic dispatch, while theLagrangian relaxation approach uses sub-gradients). In [18], based on theirnumerical testing results on randomly generated instances, the authors reportthat the error between the solutions obtained by the unit decommitmentmethod and Lagrangian relaxation approach is within 0.2%, and the unitdecommitment method takes at least 50% less CPU time than the Lagran-gian relaxation approach on average. In the following section, we extendand apply the unit decommitment method to the stochastic unit commitmentproblem.

4. SOLVING THE MULTI-STAGE STOCHASTICMODEL USING UNIT DECOMMITMENT

The multi-stage stochastic model formulated in (8) appears to be intrac-table generally. A popular approach is to summarize the future realization ofuncertainties by a finite number of possible scenarios. We represent scenar-ios by the nodes of a tree such that given any node in the tree, there exists a


unique path leading to it from the starting node. A (directed) arc in a treeconnecting two nodes represents the evolution from one scenario to theother. Consider a scenario tree T(N, A), where N is the set of nodes, and

A is the set of arcs in the tree. Let where is an ordered set

of nodes in the tree corresponding to time period t, and de-

notes the set of the descendents for node at time t such that

Assume the scenario at node is . For each node

and a descendent there is a branching (conditional) prob-

ability associated with this arc denoted by . Given a scenario treeT(N, A) , the stochastic unit commitment formulated in (10) reduces to the

following equations for a node

subject to the demand constraints for

and the reserve constraints for

and (3) to (5) with an appropriate superscript in each variable to denote thenode in the tree. Next, we will show that we can extend the unit decommit-ment method to solve (25). Given a scenario tree T(N, A) , and a feasible

schedule for similar to the development in

Section 3.1, we discuss what the best strategy is to improve the generatingcost from unit with other units’ strategies fixed in all scenarios (cf. (21)).

with boundary condition (cf. (22))

and


for Again is subject to the decommitment

rule,

and the minimum up-time and down-time constraints between and

in (3), and the state transition constraint for defined in (4), for

We obtain the optimal solution of fromthe last step of the dynamic programming algorithm as

(Note contains only the start node.)

5. SHORT-TERM GENERATION ASSETVALUATION

In this section, we consider a special case in which there is only onegenerating unit (say unit Solving this problem can determine the expectedprofit for a generation asset over a short-term period (e.g. [19]). This prob-lem appraises the flexibility of a power plant’s real options, e.g. committingor decommitting a unit, in a competitive environment (e.g. [20]). We willshow that, in this special case, a scenario tree can be converted to a latticethat allows branch recombination, which may greatly reduce the size of statespaces.

Recall the general formulation of the stochastic unit commitment in (8)to (10). With employment of a scenario tree, these equations are reduced to(24) and (25). In (28), the unit subproblem (for unit using unit decommit-ment is presented. Although each unit subproblem (28) is solved independ-ently, the commitment status of units other than are also required, whichare implicitly accounted through That is, the value of depends

on the states of all other units. However, at each node to evaluate at

all possible unit commitments is virtually impossible. The approach sug-gested here is to incorporate a scenario tree such that for each node there

is one corresponding commitment state for each unit i to be determined,


as described in Section 4. In a tree there exists a unique path leading to anygiven node from the starting node, and the commitment of all units can betracked in each path. By doing this, the tree size is inevitably large, an expo-nential function of the number of times that the tree branches. For example,if we would like to design a scenario tree such that each node branches intotwo new nodes at each hour, for a 24-hour period the tree would contain

,more than 16 millions paths.When there is only one generating unit (say j) under consideration, the

situation changes because Equation (28) reduces to

6. NUMERICAL RESULTS

6.1 General Case with Multiple Units

The presented algorithm has been implemented in Fortran and applied toa test problem on a Pentium II PC (400MHz). In the test problem, 10 gener-ating units are considered. The heat requirement of each generator is mod-eled as a quadratic function.

It is now possible to evaluate all possible states at each node since there

are only states of If is evaluated at all possible we

would not rely on the unique path leading to to track the history of unit j’scommitment. Therefore, we can merge a scenario tree to a lattice that allowsbranch recombination without losing any information. The following sectiongives numerical results, along with an example of constructing a scenariolattice.

The start-up cost for is modeled as follows.

Table 1 summarizes the parameters of the generators.


Starting from a current electricity price at 25 ($/MWh), a scenario tree isused to represent the variability of the electricity prices over the period[1,24]. This 24-hour period is equally divided into 4 subperiods (1 to 6, 7 to12, 13 to 18, and 19 to 24). For the first and the fourth subperiods, there aretwo price scenarios indicating high (H) and low (L) price cases; for the sec-ond and the third subperiods, there are three price scenarios indicating high(H), medium (M) and low (L) price cases. Notations and i= 1,2,3,4 and

i=2,3 are used to represent the high, low, or medium price scenarios inthe i-th subperiod. Figure 1 illustrates the price tree, and Table 2 gives thedata of scenarios. Table 3 presents the conditional probabilities betweenscenarios. For simplicity, assume that hourly demand follows the followingrelation (including the effect of transaction

where can be considered as some base load, and represents some

“mean” prices for the electricity. When is deviated from , assume

the demand will also deviate from . The load deviation is proportional to

. This may be due to some options the firm has sold, which may be ex-

ercised at different price levels. Table 4 summarizes the data of , ,

and


We apply the proposed unit decommitment method to the test problemand obtain a near-optimal solution within 2.3 seconds of CPU time. Table 5reviews the algorithm performance. In all, we performed eight iterations.Initially, at iteration k=0, all units in all scenarios are turned on at all hours,to the most extent. The economic dispatch is then performed, and then theexpected profit of each unit is summarized under the column of Notethat at this point the system may be overcommitted, and the dispatch maynot exactly match the demand. In the first iteration (k=1), we perform sub-problem to each unit and obtain the tentative schedule. One unit m

whose tentative schedule can yield the most improvement on the expected

profit, is selected. This value is recorded under the column

of for each unit. In the first iteration, unit 1 yields the most improvementfor the expected profit. Its tentative commitment that completely shuts downthe unit in all periods replaces its original one. Economic dispatch is per-formed to the new unit commitment, and the expected profit of each unit isrecorded under The iteration goes on in a similar manner until no im-provement can be made. We can observe the following:

1.

2.

The expected total profit is strictly increasing as the iteration in-creases.Once a unit has been selected for improvement at some iteration, its(new) commitment remains “optimal” in the subsequent iterations.For example, we select unit 1 at the first iteration. The improvementthat can be made through decommitment (under the column of

remains 0 for unit 1 in the subsequent iterations. We can adopt thisobservation to improve the algorithm performance, because once aunit has been selected at some iteration, it can be exempted fromfurther consideration. Therefore, the number of iterations requiredby the algorithm is bounded by the number of units.


3. It can be verified that the following relation:

starts to hold when That is, after iteration 4, the algorithm hasproduced a feasible unit schedule. Therefore, solution feasibility ismaintained after the 4-th iteration.

Finally, to verify how good the solution obtained from the proposed methodis, the dual problem using Lagrangian relaxation has been established. Wecreated a simplistic version of dual optimization to obtain an estimate of the


dual optimum, which can serve as an upper bound of the primal optimum.Without much effort spent to fine-tune dual iterations, we obtain an ap-proximate dual optimum of $1,100,339. Compared with the primal optimumobtained from the proposed method of $1,089,819, the duality gap is within0.96%. This implies that the obtained solution is fairly close to the true op-timum.


6.2 Short-Term Generation Asset Valuation

Assume that the electricity and fuel prices follow some Ito processes. Asan example, consider the following two processes advocated in [21].

where and are two Wiener processes with instantaneous correlation

In the price models above, captures the seasonal average price. The

parameter captures reverting speed when the price deviates from andis the price volatility. We characterize the above commodity price models

by mean reversion and log normally distributed, seasonal prices.Let equivalently We

devised a two-dimensional price lattice to encompass both electricity and gasprices. Each price node in the plane is designed to branch out into a3×3 grid of 9 price nodes in the plane corresponding to the following timeperiod (see Figure 2). To form a lattice, the plane of each time pe-riod is divided into a predetermined grid such that branching is only allowedfrom grid nodes to grid nodes of the following time period. How to select thenine nodes to which to map, as well as their corresponding (conditional)probabilities, is discussed in [22]. Basically each branching must match (dis-crete-time) price movement characteristics (mean, variance, and correlation)implied by (36a) and (36b). We apply the branching process to each mappedgrid node and repeat the process until time period T is reached. Figure 2illustrates one such lattice. Note that the size of the lattice grows quadrati-cally (linearly in each variable) as time increases.

Furthermore, at each node we assume

Equation (37) essentially assumes that the operator would sell to the marketat the amount of electricity that optimizes profit, and that the market has in-finite capacity. Note that (37) ignores the reserve constraint (27), and thetransaction is set to zero.

Remember that Table 1 give the proposed method as applied to unit 1.Consider a 7-week (168-hour) period. Assume that the electricity prices andfuel prices both follow the processes described by (36a) and (36b). We ob-tain the parameters of the price processes are by fitting the historical pricedata series of Nymex natural gas prices and electricity prices from the Cali-fornia Power Exchange, taking the logarithm of these prices as our basic

and


data series. For gas we obtain and For electric-

ity we obtain and Refer to [22] for detailed val-

ues. At time 0 suppose prices and areobserved. We assume that the instantaneous correlation coefficient betweenelectricity and natural gas is

Figures 3 and 4 depict the relation between the expected profit of the

power plant and the length of planning horizon T and the volatility re-spectively. We see that the expected profit for the power plant increases as

either T or increases. The expected profit is extremely sensitive to theprice volatility. From Figure 4, we can estimate that a 1% increase in the

value of would result in roughly a 1% increase of the expected profit forthe power plant. An intuitive interpretation of these results is to view own-ing a power plant as holding a series of spark spread call options [20]. Thevalue of these options increases when (i) the number of options increases (asT increases) or (ii) the price volatility increases.


7. CONCLUSIONS AND FUTURE DIRECTIONS

In this chapter, we present and model a general formulation of a price-based unit commitment problem as a multi-stage stochastic programmingproblem under price and load uncertainties. We extend the method of unitdecommitment using dynamic programming to solve this problem. The unitdecommitment method is very efficient in computation and does not requirea fine-tuning process for algorithm parameters like in the Lagrangian relaxa-tion method. Preliminary numerical test results are promising. When we usea scenario tree to describe the evolution of uncertainties as presented in thisresearch, it can be fully and efficiently integrated with a dynamic program-ming approach such as the proposed unit decommitment method. The estab-lishment of scenario trees remains an open problem. Statistical methods canbe used to measure the correlation of random variables such as prices andloads at different hours. Also, the scenario trees seem to suffer the curse ofdimensionality. We demonstrate, using the example of short-term generationasset valuation, that under some problem structure the dimensionality cursecan be relieved. How to model the stochastic unit commitment problem suchthat it can provide a nice structure to cope with the dimensionality problem,and can still reflect real operational restrictions, seems to be a new directionof research.

REFERENCES

1. C.L. Tseng. On Power System Generation Unit Commitment Problems, Ph.D. Thesis,University of California at Berkeley, 1996.


2.

3 .

4.

5.

6.

7.

8.

9.

10.

11.

12.

13.

14.

15.

16.

17.

18.

19.

20.21.

22.

W.G. Chandler, P.L. Dandeno, A.F. Gilmn, and L.K. Kirchmayer. Short-range operationof a combined thermal and hydroelectric power system. AIEE Trans., 733: 1057-1065,1953.C.K. Pang and H.C. Chen. Optimal short-term thermal unit commitment. IEEE Trans.Power Apparatus Syst., 95(4): 1336-1342, 1976.C.K. Pang, G.B. Sheble, and F. Albuyeh. Evaluation of dynamic programming basedmethods and multiple area representation for thermal unit commitments. IEEE Trans.Power Apparatus Syst., 100(3): 1212-1218, 1981.A.J. Wood and B. F. Wollenberg. Power Generation, Operation and Control. NewYork: Wiley, 1984.A. Turgeon. Optimal unit commitment. IEEE Trans. Autom. Control, 23(2): 223-227,1977.A. Turgeon. Optimal scheduling of thermal generating units. IEEE Trans. Autom. Con-trol, 23(6): 1000-1005, 1978.T.S. Dillon. Integer programming approach to the problem of optimal unit commitmentwith probabilistic reserve determination. IEEE Trans. Power Apparatus Syst., 97(6):2154-2164, 1978.J.A. Muckstadt and S. A. Koenig. An application of Lagrangian relaxation to schedulingin power-generation systems. Oper. Res., 25(3): 387-403, 1977.D.P. Bertsekas, G.S. Lauer, N.R. Sandell Jr., and T.A. Posbergh. Optimal short-termscheduling of large-scale power systems. IEEE Trans. Autom. Control, 28(1): 1-11,1983.J.F. Bard. Short-term scheduling of thermal-electric generators using Lagrangian relaxa-tion. Oper. Res., 36(5): 756-766, 1998.S. Takriti, J.R. Birge, and E. Long. A stochastic model for the unit commitment prob-lem. IEEE Trans. Power Syst., 11(3): 1497-1508, 1996.R.T. Rockafellar and R.J.-B. Wets. Scenarios and policy aggregation in optimizationunder uncertainty. Math. Oper. Res., 16(1): 119-147, 1991.P. Carpentier, G. Cohen, J.-C. Culioli, and A. Renaud. Stochastic optimization of unitcommitment: a new decomposition framework. IEEE Trans. Power Syst., 11(2): 1067-1073, 1996.C.C. Carøe and R. Schultz. A Two-Stage Stochastic Program for Unit Commitment un-der Uncertainty in a Hydro-Thermal Power System. Working paper, University of Co-penhagen, Institute of Mathematics, Denmark, 1998.C.A. Li, R.B. Johnson, and A.J. Svoboda. A new unit commitment method. IEEE Trans.Power Syst., 12(1): 113-119, 1997.C.L. Tseng, S.S. Oren, A.J. Svoboda, and R.B. Johnson. A unit decommitment methodin power system scheduling. Elec. Power Energy Syst., 19(6): 357-365, 1997.C.L. Tseng, C.A. Li, and S.S. Oren. Solving the unit commitment problem by a unitdecommitment method. J. Optimization Theory Appl., 105(3), 2000 (in press).C.L. Tseng and G. Barz. “Short-term generation asset valuation.” In Proc. 32nd HawaiiInternational Conf. Syst. Sci. (CD-ROM), Maui, Hawaii, Jan 5-8, 1999.M. Hsu. Spark spread options are hot! Electricity J., 11(2): 28-39, 1998.G. Barz. Stochastic Financial Models for Electricity Derivatives. Ph.D. Thesis, StanfordUniversity, 1999.C.L. Tseng. Valuing the operational real options of a power plant using discrete-timeprice trees. Submitted to Elect. Power Energy Syst.

Chapter 8

PROBABILISTIC UNIT COMMITMENTUNDER A DEREGULATED MARKET

Jorge Valenzuela and Mainak MazumdarUniversity of Pittsburgh

Abstract: In this paper, we propose a new formulation of the unit commitment problemthat is suitable for the deregulated electricity market. Under these conditions, anelectric power generation company will have the option to buy or sell from apower pool in addition to producing electricity on its own. We express the unitcommitment problem as a stochastic optimization problem in which the objec-tive is to maximize expected profits and the decisions are required to meet thestandard operating constraints. Under the assumption of competitive market andprice-taking, we show that the unit commitment problem for a collection of Mgeneration units can be solved by considering each unit separately. The volatil-ity of the spot market price of electricity is represented by a stochastic model.We use probabilistic dynamic programming to solve the stochastic optimizationproblem pertaining to unit commitment. We show that for a market of 150 unitsthe proposed unit commitment can be accurately solved in a reasonable time byusing the normal, Edgeworth, or Monte Carlo approximation methods.

1. INTRODUCTION

In the short-term, typically considered to run from twenty-four hours toone week, the solution of the unit commitment problem (UCP) is used to as-sist decisions regarding generating unit operations [1]. In a regulated market,a power generating utility solves the UCP to obtain an optimal schedule of itsunits in order to have enough capacity to supply the electricity demanded byits customers. The optimal schedule is found by minimizing production costover a time interval while satisfying demand and operating constraints. Mini-mization of the production costs assures maximum profits because the powergenerating utility has no option but to supply the prevailing load reliably. Theprice of electricity over this period is predetermined and unchanging; there-fore, operating decisions have no effect on the firm’s revenues.

As various regions of the United States implement deregulation [2], thetraditional unit commitment problem continues to remain applicable for the


commitment decisions made by the Independent System Operator (ISO). TheISO resembles very much the operation of a power generating utility underregulation. The ISO manages the transmission grid, controls the dispatch ofgeneration, oversees the reliability of the system, and administers congestionprotocols [3,4,5]. The ISO is a non-profit organization. Its economic objectiveis to maximize social welfare, which is obtained by minimizing the costs ofreliably supplying the aggregate load. Under deregulation, the UCP for anelectric power producer will require a new formulation that includes the elec-tricity market in the model. The main difficulty here is that the spot price ofelectricity is no longer predetermined but set by open competition. Thus far,the hourly spot prices of electricity have shown evidence of being highlyvolatile. The unit commitment decisions are now harder and the modeling ofspot prices becomes very important in this new operating environment. Dif-ferent approaches can be found in the literature in this regard. Takriti et al. [6]have introduced a stochastic model for the UCP in which the uncertainty inthe load and prices of fuel and electricity are modeled using a set of possiblescenarios. The challenge here is to generate representative scenarios and as-sign them appropriate probabilities. Allen and Ilic [7] have proposed a sto-chastic model for the unit commitment of a single generator. They assumethat the hourly prices at which electricity is sold are uncorrelated and nor-mally distributed. In [8] Tseng uses Ito processes to model the prices of elec-tricity and fuel in the unit commitment formulation.

The purpose of this chapter is to present a new formulation to the UCPsuitable for an electric power producer in a deregulated market and considercomputationally efficient procedures to solve it. We express the UCP as a sto-chastic optimization problem in which the objective is to maximize expectedprofits and to meet operating constraints such as capacity limits and minimumup- and down-times. We show that when the spot market of electricity is in-cluded, the optimal solution of a UCP with M units can be found by solving Muncoupled sub-problems. We obtain a subproblem by replacing the values ofthe Lagrange multipliers by spot power prices. The volatility of the spot mar-ket price of electricity is taken into account by using a variation of the sto-chastic model proposed by Ryan and Mazumdar [9], The model, which is re-ferred to as the probabilistic production-costing model, incorporates the sto-chastic features of load and generator availabilities. It is often used to obtainapproximate estimates of production costs [10,11,12]. This model ignores theunit commitment constraints and assumes that a strict predetermined meritorder of loading prevails. This implies that a generator will be dispatched onlywhen the available unit immediately preceding it in the loading order is work-ing at its full capacity. We believe that this model provides a good approxima-tion to the operation of an electricity market, such as the California market, inwhich no centralized unit commitment decisions are taken. The model cap-

Probabilistic Unit Commitment 141

tures the fundamental stochastic characteristics of the system. At any moment,a power producer may not be fully aware of the exact characteristics of theunits comprising the market at that particular time. But it is likely to possessinformation about the steady state statistical characteristics of the units par-ticipating in the market. Ryan and Mazumdar’s probabilistic production cost-ing model can be used to provide a steady-state picture of the market.

We determine the hourly spot market price of electricity by the market-clearing prices. The market-clearing price can be shown to be the variablecost or bid of the last unit used to meet the aggregate load prevailing at a par-ticular hour. This unit is called the marginal unit. We determine the probabil-ity distribution of the hourly market-clearing price based on the stochasticprocess governing the marginal unit, which depends on the aggregate load andthe generating unit availabilities. We model the aggregate load as a Gauss-Markov stochastic process and use continuous-time Markov chains to modelthe generating unit availabilities [10,13]. We assume that the information isavailable on mean time to repair, mean time to failure, capacity, and variableoperating cost of each unit participating in the market required to characterizethese processes. We use probabilistic dynamic programming to solve the sto-chastic optimization problem pertaining to unit commitment. We also reportresults on the accuracy and computational efficiency of several analytical ap-proximations as compared to Monte Carlo simulation in estimating probabil-ity distributions of the spot market price for electric power.

2. FORMULATION

We consider the situation in which an electric power producer owns a setof M generating units and needs to determine an optimal commitment sched-ule of its units such that the profit over a short period of length T is maxi-mized. Revenues are obtained from fulfilling bilateral contracts and sellingelectric power, at spot market prices, to the power pool. It is assumed that theelectric power company is a price-taker. If at a particular hour the power sup-plier decides to switch on one of its generating units, it will be willing to takethe price that will prevail at this hour. We also assume that the power supplierhas no control over the market prices and the M generating units will remainavailable during the short time interval of interest.

In determining an optimal commitment schedule, there are two decisionvariables, denoted and The first variable denotes the amount of powerto be generated by unit at time and the latter is a control variable, whosevalue is “1” if the generating unit is committed at hour and “0” otherwise.The cost of the power produced by the generating unit depends on theamount of fuel consumed and is given by a known cost function


where is the amount of power generated. The start-up cost, which for ther-mal units depends on the prevailing temperature of the boilers, is given by aknown function The value of specifies the consecutive time thatthe unit has been on (+) or off (-) at the end of the hour In addition, a gener-ating unit must satisfy operating constraints. The power produced by a gener-ating unit must be within certain limits. When the generating unit is run-ning, it must produce an amount of power between and Ifthe generating unit is off, it must stay off for at least hours, and if it is on,it must stay on for at least hours.

2.1 Problem Formulation

The objective function is given by the sum over the hours in the interval[0,T] of the revenue minus the cost. The revenue is obtained from supplyingthe bilateral contracts and by selling to the power pool at a price of perMWH the surplus energy produced in each hour t. The cost in-cludes the cost of producing the energy, buying shortfalls (if needed) from thepower pool, and the start-up costs. Defining the supply amount stipulated un-der the bilateral contract by and by R ($/MWH) the price, the objec-tive function (maximum total profit) is given by:

A positive value of indicates that megawatts hour are bought fromthe power pool and a negative value indicates that megawatts hour are soldto the pool. Since the quantity is a constant, the optimization problem re-duces to:

subject to the following constraints(for and

Load:

Capacity limits:


Minimum down-time:

Minimum up time:

where

and unrestricted in sign

After substituting in the objective function the value of

obtained from Equation 4, we re-write Equation 3 as follows

which after removing constant terms is equivalent to:

subject to the operating constraints. Because the constraints (5) to (7) refer toindividual units only, the advantage of Equation 9 is that the objective func-tion is now separable by individual units. The optimal solution can be foundby solving M de-coupled subproblems. Thus, the subproblem for theunit

subject to operating constraints of the unit. Equation 10 is similar to thesubproblem obtained in the standard version of the UCP [14] excepting thatthe value of the Lagrange multipliers are now replaced by the spot marketprice of electricity.

2.2 Stochastic Formulation of the Subproblem

We next consider the spot market price of electricity, which is deter-mined by the market-clearing price, as a random variable. When the optimiza-tion subproblem is being solved for a particular unit, we assume that the mar-


ket, which includes the M-1 units owned by the power producer solving theproblem, consists of N generating units (N>>M). The generating unit forwhich the subproblem is solved is excluded from the market. We assume thatthe unit commitment decisions for any one unit have a negligible effect on thedetermination of the marginal unit of the market for a given hour.

To model the market-clearing price, we assume that the generators par-ticipating in the market are brought into operation in an economic merit orderof loading. The unit in the loading order has a capacity (MW), variableenergy cost ($/MWH), and a forced outage rate Under the assumption ofeconomic merit order of loading, the market-clearing price at a specific houris equal to the operating cost (S/MWH) of the last unit used to meet the loadprevailing at this hour. The last unit in the loading order is called the marginalunit and is denoted by The market-clearing price, is thus equal toThe values of and depend on the prevailing aggregate load and theoperating states of the generating units in the loading order.

We write the objective function of the subproblem for one of the M gen-erating units as follows:

subject to the operating constraints: capacity limits, minimum up-time, andminimum down-time.

2.3 Probabilistic Dynamic Programming Solution

The maximum profit over the period T (Equation 11) is a random variablebecause the hourly market-clearing price is a random variable. We assumethat at the time of the decision, hour zero, the marginal unit and the load forall the hours before hour zero are known. We denote the marginal unit at timezero by and solve the subproblem by maximizing the conditional expectedprofit over the period T. We express the objective function as:

This equation is subject to the same operating constraints described ear-lier. We use probabilistic dynamic programming to solve this optimizationproblem. We define the function by the following equation:


This function denotes the maximum profit at hour given that at this hourthe unit is determining the market-clearing price and the generator to bescheduled is in the operating state We also define the recursive function

to be the optimum expected profit from hour to hour T of operating thegenerator that is in state at time Thus, the expression for hour zero is

For hour the expression is given by the following recursive relation:

Setting the expected incoming profit at time T+1 to be zerowe obtain the boundary condition for the last stage

The initial conditions are given by the initial state of the generator andand the marginal unit at hour zero Consequently, the optimal schedule is

given by the solution of To solve the problem, the following condi-tional probabilities need to be computed:

Thus, the joint probability distribution of and and the marginalprobability distribution of are needed.

3. STOCHASTIC MODEL FOR THE MARKET-CLEARING PRICE

The stochastic model of the market-clearing price uses the production-costing model proposed by Ryan and Mazumdar [9]. This model has beenused in estimating the mean and variance of production cost [12] and mar-ginal cost [15] of a power generating system.


3.1 Stochastic Model of the Market

The assumptions of a market model with N generating units are:The generators are dispatched at each hour in a fixed, pre-assigned loadingorder, which depends only on the load and the availability of the generat-ing units. Operating constraints such as minimum up- and down-times,spinning reserve, and scheduled maintenance are not considered.The unit in the loading order has capacity (MW), variable energycost (S/MWH), mean time to failure mean time to repair andforced outage rate,After adjusting for the variations in the ambient temperature and periodic-ity, the load at time is assumed to follow a Gauss-Markov process[16,17] with and where and are as-sumed to be known. (An analysis given in [13] validates this assumption.)The operating state of each generating unit follows a two-state continu-ous-time Markov chain, with failure rate and repair rateThe forced outage rate equalsFor and are statistically independent for all values of and

marginal unit at time first note that

and that the events and are equivalent. Thus,

the following equality holds:

Thus, to obtain the probability mass function of the probability that

and needed for evaluating Equation 17, requires the following devel-opment. Writing

Each is independent of for all values of

1.

2.

3.

4.

5.

3.2 Probability Distribution of the Marginal Unit

To derive an analytical expression for the probability mass function of the

is greater than zero for all values of needs to be computed.

3.3 Bivariate Probability Distribution of the Marginal Unit

An analytical approximation for the bivariate probability mass function of


and observing that the events

are equivalent, we obtain the following equality:

Therefore, to compute the bivariate probability mass function of

and the probability that

the many values that the expression can take, which in the worst

case is (when Thus, the computational time increases exponentiallyas N increases and it would make an exact computational procedure prohibi-tive for large N. In our numerical examples, we have used three approxima-tion methods: the normal, Edgeworth, and Monte Carlo approximations. TheEdgeworth approximation [18] is known in the power system literature as themethod of cumulants. We also attempted the use of the large deviation orequivalently, the saddlepoint approximation method [19], but it turned out tobe prohibitively time-consuming for very large systems.

4. SOLUTION OF THE PROBABILISTIC UNITCOMMITMENT PROBLEM: A NUMERICALEXAMPLE

For our purpose, we assume that a complete description of the electricitymarket is given by the data concerning the N power generators that comprise

needs to be evaluated for all values of and

The computational effort in evaluating equations 18 and 20 depends on


the market, historical data of the aggregate load, and the hourly temperatureforecast for the day of trading. The description of the power generators in-cludes the order in which they will be loaded by the ISO, their capacities, en-ergy costs, mean times to failure, and mean times to repair. The data for theaggregate load gives the historically forecast ambient temperature and the cor-responding load for each hour in the region served by the marketplace. In thisexample, a data set that gave the actual ambient temperature and the corre-sponding load for each hour in a region covering the northeast United Statesduring the calendar years 1995 and 1996 was used. We chose the last day ofthis data set, September 20, 1996, as the trading day. Table 1 gives the actualtemperatures on this day, which were assumed to be the forecast temperatures.

Example: The market is described by the aggregate load and a power genera-tion system consisting of generators participating in the market. Using statisti-cal time series analysis on the data at hand, we found that an ARIMA

provided a very good description of the actual load ob-served. The model [13] used is as follows:

where is the average ambient temperature at hour and is defined as:

and


where is a Gaussian white noise process with mean zero and estimated

variance Table 2 provides the estimates of the least-square re-

gression coefficients, relating load to temperature.

We assume the market consists of 150 power generating units. This sys-tem was obtained by repeating ten times each unit of a 15-unit system, whichin its turn is a smaller version of the IEEE Reliability Test System (RTS) [20].The load data from [13] was also multiplied by a factor of ten. Defining asthe cumulative capacity of the first i units,

we assume that the variable cost of each unit is given by the function:

This assumption allows for the unit operating costs to increase in order of theposition of the units in the loading order. Table 3 gives the relevant character-istics of the units, in their loading order.

The problem is to schedule one of the generators of the power producerfor the next 23 hours given the information about the electricity market andthe known initial operating conditions for the generating unit. The characteris-tics of this generator were taken from [1], and they are reproduced in Table 4.We modified the fuel-cost function of the unit to be consistent with the rangeof the individual units’ energy costs. The objective is to maximize the ex-pected profit over this period. We assumed the generator to have been on foreight consecutive hours. As mentioned in Section 2.3, this generator is not


included in the set of generators that comprise the market. We also assumedthat the variable cost of the unit is currently determining the market-clearing price, which is $19.73 per MWH.

Table 5 summarizes the unit commitment solutions obtained using the dif-ferent algorithms. The optimal schedule produced by the Monte Carlo simula-tion (200,000 replicates used in estimating the probability distributions) is toturn the generating unit off during the first four hours. Then, the unit is turnedback on for the next nineteen hours. The Monte Carlo procedure estimatesthat the execution of this schedule will generate an expected profit of $37,509.The normal and Edgeworth approximation methods provide this schedule aswell. However, they estimate expected profits of $37,483 and $37,351, re-spectively. Details of these computations are given in [21].


5. DISCUSSION

In this chapter, we have proposed a new formulation of the unit commit-ment problem that is valid under deregulation. We have shown that, when weassume a competitive market and price-taking, the unit commitment problemcan be solved separately by each individual generating unit. The solutionmethod for the new formulation requires the computation of the probabilitydistribution of the spot market price of electricity. The power generation sys-tem of the marketplace has been modeled using a variation of the Ryan-Mazumdar model. This model takes into account the uncertainty on the loadand the generating unit availabilities. The probability distribution of the spotmarket price, which is determined by the market-clearing price, is based onthe probability distribution of the marginal unit.

The exact computation of the probability distribution is prohibitive forlarge systems. We evaluated three approximation methods. From the compu-tational experience, it appears that the proposed unit commitment can be ac-curately solved in a reasonable time by using the normal, Edgeworth, orMonte Carlo approximations.

ACKNOWLEDGMENTS

The authors are indebted to the editors and the two reviewers for theirvery helpful comments. This research was supported by the National ScienceFoundation under grant ECS-9632702.

REFERENCES

1. A. Wood and B. Wollenberg. Power Generation Operation and Control. New York:Wiley & Sons, Inc., 1996.


2.

3.

4.

5.

6.

7.

8.

9.

10.

11.

12.

13.

14.

15.

16.

17.

18.

19.

20.

21.

“Promoting Wholesale Competition Through Open Access Non-discriminatory Transmis-sion Services by Public Utilities.” FERC Order No. 888, 1996.F.D. Galiana and M.D. Ilic. A mathematical framework for the analysis and managementof power transactions under open access. IEEE Trans. Power Syst., 13: 681-687, 1998.“Introduction to the New California Power Market.” Draft/work document posted on theInternet, http://www.calpx.com, July 1997.“Zonal Market Clearing Prices: A Tutorial California Power Exchange.” Document postedon the Internet, http://www.calpx.com.S. Takriti, B. Krasenbrink, and L. Wu. “Incorporating Fuel Constraints and ElectricitySpot Prices into the Stochastic Unit Commitment Problem.” IBM Research Report, RC21066, Subject Area Computer Science/Mathematics, 1997.E.H. Allen and M.D. Ilic. “Stochastic Unit Commitment in a Deregulated Utility indus-try.” In Proc. North American Power Conference, Laramie, WY: 105-112, 1997.C. Tseng. “A Stochastic Model for the Price-based Unit Commitment Problem.” Paperpresented at the Workshop on Next Generation of Unit Commitment Models, Co-sponsored by DIMACS and EPRI, September 27-28, 1999.S.M. Ryan and M. Mazumdar. Effect of frequency and duration of generating unit outageson distribution of system production costs. IEEE Trans. Power Syst., 5: 191-197, 1990.M. Mazumdar and A. Kapoor. Stochastic models for power generation system productioncosts. Electric Power Syst. Res., 35: 93-100, 1995.S.M. Ryan and M. Mazumdar. Chronological influences on the variance of electric powerproduction costs. Oper. Res., 40: S284-S292, 1992.F. Shih, M. Mazumdar, and J.A. Bloom. Asymptotic mean and variance of electric powergeneration system production costs via recursive computation of the fundamental matrixof a Markov chain. Oper. Res., 47: 703-712, 1999.J. Valenzuela and M. Mazumdar. Statistical analysis of electric power production costs.Accepted for publication in IIE Trans.J.F. Bard. Short-term scheduling of thermal-electric generators using Lagrangian Relaxa-tion. Oper. Res., 36: 756-766, 1988.F. Shih and M. Mazumdar. An analytical formula for the mean and variance of marginalcosts for a power generation system. IEEE Trans. Power Syst., 13: 731-737, 1998.A.M. Breipohl, F.N. Lee, D. Zhai, and R. Adapa. A Gauss-Markov load model forapplication in risk evaluation and production simulation. IEEE Trans. Power Syst., 7:1493-1499, 1992.J. Valenzuela, M. Mazumdar, and A. Kapoor. Influence of temperature and load forecastuncertainty on estimates of power generation production costs. Accepted for publication inIEEE Trans. Power Syst.M. Mazumdar and Y. Wang. On the application of Esscher's approximation to computa-tion of generating system reliability and production costing indexes. IEEE Trans. PowerApparatus Syst., 104: 3029-3036, 1985.S. lyengar and M. Mazumdar. A saddle point approximation for certain multivariate tailprobabilities. SIAM J. Scient. Comput, 19: 1234-1244, 1998.APM Subcommittee. IEEE reliability test system. IEEE Trans. Power Apparatus Syst., 98:2047-2054, 1979.J. Valenzuela. Stochastic Optimization of Electric Power Generation in a DeregulatedMarket. Ph.D. Dissertation, School of Engin., University of Pittsburgh, 1999.

Chapter 9

SOLVING HARD MIXED-INTEGER PROGRAMSFOR ELECTRICITY GENERATION

Sebastian CeriaColumbia University and Dash Optimization, Inc.

Abstract: In this chapter, we describe the most recent advances in the solution of mixed-integer programming problems. The last ten years have seen enormous im-provements in the solution of the most difficult mixed-integer programs. Thetrend towards integration of modeling and optimization now makes it possible tosolve the hardest optimization problems arising from electricity generation, suchas the unit commitment problem. We report results with a leading softwarepackage that was used successfully to solve unit commitment problems in twoEuropean utility companies.

1. INTRODUCTION

Electricity generating companies and power systems face the daily prob-lem of deciding how best to meet the varying demand for electricity. In theshort-term, electric utilities need to optimize the scheduling of generatingunits so as to minimize the total fuel cost or to maximize the total profit over astudy period of typically a day, subject to a large number of constraints thatmust be satisfied. This problem, which is of crucial importance for electricgeneration companies, is referred to as the “unit commitment problem.” Thisproblem can be best solved by using state-of-the-art optimization technology.

Over the last twenty years, we have been witnesses to a revolution incomputational optimization. The availability of powerful computers and theimprovements in algorithmic development now make it possible to solveproblems that only a few years ago were thought to be unsolvable. Indeed,modern commercial linear and integer programming codes, such as CPLEX1,LINDO2, OSL3, and XPRESS-MP4 and research linear and integer program-

1 CPLEX is a trademark of ILOG, Inc.2 LINDO is a trademark of LINDO Systems, Inc3 OSL is a trademark of IBM Corporation4 XPRESS-MP is a trademark of Dash Associates, Ltd.


ming codes, such as MINTO [28, 34], ABACUS [24], MIPO [8], and bc-opt[9], are several orders of magnitude more powerful than the most efficientimplementations of linear and integer programming algorithms that wereavailable only ten years ago (when compared the present generation of com-puters).

In this chapter, we will focus on the solution of basic mixed-integer (bi-nary) programming problems of the form shown in Figure 1.

In Figure 1, is an A is a matrix of rows, is a vector of com-ponents, and is an The first variables are constrained to takingonly 0-1 values; thus, the problem in Figure 1 is called a mixed 0-1 program.

Traditionally, integer programming problems have been solved by usingone of two algorithms, branch-and-bound and cutting planes, both based onlinear programming technology (see [23] for a complete survey of the area).For the last ten years, several researchers have been studying and implement-ing a combination of the above, which is commonly referred to as branch-and-cut. The term was originally coined by Padberg and Rinaldi [31, 32] whowere the first researchers to demonstrate the computational efficiency of thealgorithm for the solution of traveling salesman problems.

But Padberg and Rinaldi’s branch-and-cut code goes beyond a simplecombination of branch-and-bound and cutting planes. It adds heuristics tospeed up the identification of feasible (almost optimal) solutions. It uses pre-processing to reduce the size of the problem and hence improve its solvability.Finally, Padberg and Rinaldi exploit the algebraic structure of the problem inorder to generate cutting planes. Hence, large-scale traveling salesman prob-lems, involving thousands of cities, which translates into millions of variablesand an exponential number of constraints, can now be solved efficiently in afew minutes. The success of their algorithm can be largely attributed to ex-ploiting the mathematical knowledge of the problem.

Researchers have solved other hard optimization problems using similartechniques. The caveat of the approach is that the algorithm has to be specifi-cally tailored for a particular problem, with the added difficulty that any

Solving Hard Mixed-Integer Programs for Electricity Generation 155

changes to the model cannot be easily accommodated. It is for this reason thatcommercial software companies have recently concentrated their efforts onthe development of programming environments that facilitate the develop-ment of such algorithms (see for example [11]). The main idea behind suchproducts is to break the separation that has existed for decades between mod-eling environments and optimization algorithms, thus making it easy for thedeveloper to prototype rapidly sophisticated algorithms for the solution of thehardest optimization problems. In Section 4 we briefly explore an applicationof this technology to the solution of unit commitment problems.

The resurgence of mixed-integer programming as a viable technology forsolving hard optimization problems is due to significant improvements in thequality of the basic optimizers. Section 2 lists some of these developmentsand includes several references. In Section 3, we study integrated tools formodeling and optimization and discuss this paradigm shift. The integration ofmodeling and optimization has been made possible by new software toolsavailable in the market and has recently been embraced by practitioners.

Finally, let us add that it is not the intent of this chapter to serve as a fullreference for the state-of-the-art in integer programming, nor to provide acomplete list of commercial or research software available for solving suchproblems. Interested readers should also consult references, such as [7] and[23]. Classic references for integer programming include [15], [29], and [35].

2. INTEGER PROGRAMMING: RECENT ADVANCES

Research in the algorithmic solution of mixed-integer programming prob-lems started in the early sixties with the development of two classes of meth-ods. The first one is that of cutting plane algorithms. In a cutting plane algo-rithm, valid inequalities that cut off the current solution of the linear pro-gramming relaxation are used to tighten the initial formulation until a feasibleinteger solution can be found (see Figure 2). Their origins can be traced as farback as the mid-1950s, with the landmark paper of Dantzig, Fulkerson, andJohnson [12, 13] where they show how cutting planes can be used to solve a48-city traveling salesman problem. Gomory [16, 17, 18], was the first one topropose cutting planes as a general solution procedure. Branch-and-bound isthe second class of methods, which are used to solve mixed-integer programs.The seminal papers of Land and Doig [25] for general mixed integer pro-grams and of Balas [1] for the case of pure 0-1 programs introduced the basicideas that would later become the foundation for the area. Branch-and-boundmethods are of the enumerative type, since they solve the problem by enu-merating a certain number, hopefully as small as possible, of feasible solu-tions. Branch-and-bound has been for long the prevalent way chosen by prac-titioners to solve mixed 0-1 programs.


In the last 10 to 15 years there have been several important innovations inthe practical solution of mixed-integer programs. The first is one of incorpo-rating combinatorial cutting planes, as opposed to general cutting planes fortightening the linear programming relaxation of certain classes of pure integerprogramming problems. The main drawback of these methods is that there hasto be an underlying combinatorial structure in order to generate the cuttingplanes. Crowder, Johnson, and Padberg [10] propose a method that circum-vents this difficulty and that applies to pure 0-1 programs. In order to obtaindeep cuts they generate facets of the knapsack polyhedra obtained by consid-ering each constraint in the constraint set separately. Van Roy and Wolsey[37, 38] use similar ideas for mixed 0-1 programs. In both cases, after the lin-ear programming relaxation had been tightened with the newly added cuts, theauthors apply branch-and-boun, and show that a significant reduction in thenumber of nodes in the branch-and-bound tree can be obtained.

Recently, researchers have explored other ways of tightening the linearprogramming relaxation of an integer program that involves reformulating theproblem into a higher dimensional space (one that uses more variables),where a more convenient formulation may give a tighter relaxation. Such pro-cedures are based on the work of Sherali and Adams [36]; Lovász and Schri-jver [27]; and Balas, Ceria, and Comuejols [2], [3], [4].

Cutting planes can also be used in conjunction with branch-and-bound asa method for strengthening the formulations at the nodes of the search tree(see Figure 3). Padberg and Rinaldi [31] were the first to propose such amethod – that they baptized branch-and-cut – for the traveling salesman prob-


lem and obtained impressive results with it. There are two important keys tothe success of their approach. First, the combinatorial cutting planes they gen-erate are deep cuts, and often facets, of the underlying integer polyhedron.Second, because of the combinatorial nature of their cutting planes, the cutsthey generate at one part of the tree can be easily made valid throughout theenumeration tree. There has recently been considerable interest in the devel-opment of a branch-and-cut method that can be applied to general mixed 0-1programs. Hoffman and Padberg [21, 22] obtained impressive results by us-ing the cutting planes of Crowder, Johnson, and Padberg in a branch-and-cutcontext for a class of sparse problems arising from applications.

Recent implementations of branch-and-cut have been proven it to be aneffective method for tackling some classes of mixed 0-1 programs, but there isstill an important need for a procedure that can solve general mixed 0-1 pro-grams, independently of any underlying combinatorial structure. In [2, 3, 4]we show that a branch-and-cut method based on lift-and-project cuttingplanes is able to find optimal solutions to some instances of mixed 0-1 pro-


grams that were previously unsolved, and it is faster than competing methodson other instances.

Most of these recent advances have been implemented in commercialsoftware. These improvements have been combined with preprocessing, heu-ristics, and fancy data structures. A modern MIP code starts by preprocessingthe problem, thus reducing the size of the coefficients and eliminating un-needed variables; then running a heuristic to improve the quality of thebounds; and finally applying a combination of cutting planes and branch-and-bound. It is now possible to solve problems that only a few years ago seemedunsolvable. Most commercial packages incorporate features that are particu-larly helpful when modeling some realistic problems in electricity generation.For example, XPRESS-MP allows the user to define semi-continuous vari-ables (variables that take either the value 0 or a value between a lower-bound(greater than 0) and an upper-bound). This feature can be used to model elec-tricity generation or consumption processes naturally.

We include computational results with XPRESS-MP5 are included in Fig-ure 5 (for linear programming) and Figure 6 (for mixed-integer program-

5 XPRESS-MP is one of several commercial LP/MIP packages commercially available.


ming). The code is compared with its earlier version that came on the marketonly one year before the current release. The significant improvements in so-lution times are due to improved preprocessing, faster LP engines, and theutilization of state-of-the-art cutting planes. The data sets are publicly avail-able through either NETLIB or MIPLIB.

3. INTEGRATED MODELING AND OPTIMIZATION

Traditionally, mathematical programmers that use modeling languages toexpress their problems have “artificially” separated modeling from optimiza-tion. Practitioners and researchers have to sacrifice either performance byconstantly switching between a modeling and optimization environments, orthey have to design specific algorithms and implementations that completelyignore modeling environments and work directly with an internal representa-tion of the problem. In the first case, knowledge about the structured algebraicmodel either missing and thus difficult to exploit. In the second, it is so em-bedded into the implementation that small alterations to the model may renderthe implementation completely useless.


There have been several attempts to integrate modeling environments andoptimization. The first such environment that has been implemented for thesolution of mixed-integer programming problems is EMOSL6 (Entity Model-ing Optimization Subroutine Library) [6, 14]. EMOSL is a combined model-ing and optimization subroutine library that overcomes the separation betweenmodeling and optimization by allowing the problem to be manipulated usingthe notation of the model language. By integrating modeler and optimizer, thescope and ease of algorithm development is greatly improved, without anyobservable degradation of performance. Figure 7 provides an illustration ofthe principle.

EMOSL was used in [11] for four applications: a block structure extrac-tion tool and the implementations of three MIP heuristics and cutting planestechniques. The result of their research effort is now a commercial productthat is also used in the electricity industry (see Section 4).

This does not mean that researchers have not been integrating modelingand optimization in the implementation of their mixed-integer programmingalgorithms. If this integration is done in the modeling environment then thedrawbacks in terms of speed are so great that these platforms cannot be con-sidered for serious application of efficient algorithms. On the other hand, ifresearchers use a low-level language, the original model becomes very hard tomaintain. For mixed-integer programming, the majority of current algorithmic

6 EMOSL is a trademark of Dash Associates Ltd.


implementations and development work uses conventional optimizer subrou-tine libraries (for example: ABACUS [24], bc-opt [9], MINTO [28, 34] andMIPO [8]). EMOSL is essentially a unification of an optimizer subroutinelibrary and a callable modeler. It provides all of the standard functions of anoptimizer library and many advanced features; we mention several later. Itprovides functions to parse a model and to regenerate it after changes havebeen made. The key feature is the new functionality to access the attributes ofthe variables and constraints of the problem, i.e., functions to inspect andchange the coefficients of the variables in constraints, the sense of constraints,the upper and lower bounds on variables, the type of variables (continuous,binary, integer), and functions to inspect the primal and dual solution valuesof the variables and constraints.

Other modeling languages, such as AMPL,7 GAMS,8 MPL,9 and OPL,10

are increasingly blurring the dividing line between modeler and optimizer. Forexample, MPL has a feature that allows access to entities in the modeling lan-guage from inside a C program, and OPL allows C/C++ model generation,effectively allowing the combination of model and algorithm.

7 AMPL is a trademark of AT&T.8 GAMS is a trademark of Gams Development Corp.9 MPL is a trademark of Maximal Software, Inc.10 OPL is a trademark of ILOG, Inc.


4. EXAMPLES IN ELECTRICITY GENERATION

There are several ways of formulating the UC problem. Most formula-tions make use of mixed-integer and nonlinear mixed-integer models. Unitcommitment problems are intrinsically difficult because in many cases operat-ing constraints are best represented by logical conditions, whereas in othercases they are represented by non-convex functions.

The UC problem has been tackled by using a variety of methods. Heuris-tic procedures were first studied in [20], while most of the current researchhas focused on a combination of dynamic programming and Lagrangian re-laxation [5, 19, 30, 33, 39].

In [26], Lekane and Gheury, from the Belgian electricity company Trac-tebel, consider a system including nuclear units, fossil-fired units, andpumped storage plants. Their model takes into account operating considera-tions such as the operating reserve constraint, the minimum operating powerand ramping rate constraints, the commitment constraints of the fossil-firedunits, the operation of the nuclear plants at constant output levels, and the en-ergy constraints associated with the pumped storage plants.

Their paper first presents a solution approach based on the Lagrangian re-laxation method and discusses the difficulties associated with discovering aprimal feasible solution. Then the authors describe a new solution system,which was investigated in the frame of the ESPRIT project called MEMIPS.The approach presented there is based on a column generation method thatintegrates dynamic programming to generate feasible power profiles for theresources and linear programming using EMOSL to find a set of profilesminimizing the total operating cost while respecting the system constraints.Heuristics are then applied to determine integer solutions. The system heavilyexploits the integration of modeler and optimizer available through EMOSL.By using EMOSL, the authors have the flexibility to change the model byadding new constraints without necessarily affecting the algorithm implemen-tation.

The approach presented in [26] is based on a reformulation of the UCproblem as a problem of determining the set of feasible thermal unit profileswhich minimizes the total operation cost over the studied period while re-specting the demand and reserve constraints as well as the pumped storageplant constraints. A feasible profile for a thermal unit is defined by bothpower and reserve profiles, which specify its hourly generation and reservecontribution over time. The principle of the proposed approach consists ingenerating a set of feasible thermal profiles and solving the MIP problem overthe specified set of profiles. Since a huge number of thermal profiles exist, anapproximate procedure is used to solve the MIP problem over a limited set ofprofiles. This procedure comprises four main steps, namely:


Generation of feasible thermal profiles: An initial set of feasible ther-mal profiles is generated by using the solutions to the sub-problems foundwhen solving the dual problem in the Lagrangian relaxation approach.Column Generation: The LP relaxation of the MIP that contains onlythese thermal profiles is solved using column generation.Relax and Fix: The resulting MIP problem is solved approximately byusing a relax and fix method.Solution of reduced MIP: The MIP problem obtained by fixing the bi-nary variables associated with the thermal units at the values found as aresult of the relax and fix heuristic is solved to optimality.

In this case, the advantages of using a framework that combines modelingand optimization for such an implementation are clear:

The combined nature of the modeling and optimization subroutine libraryfacilitates the integration.Access to variables and constraints in the modeler’s language allows forease of algorithm implementation.An implementation that uses both modeling and optimization constructs isextremely efficient since no overhead is incurred while communicatingwith the modeling system.The integrated modeling and optimization implementation (in EMOSL)results in easy-to-maintain code.

Another example solving unit commitment problems using mixed-integerprogramming comes from Power Optimisation, Ltd11. One version of the unitcommitment software, developed for Northern Ireland Electricity (NIE), hasbeen used every day since December 1996 to schedule the generating units inthe Northern Ireland power system. The users at NIE report that the schedulesproduced by the software are consistently of a very high quality. The softwareuses a novel multi-stage solution method, which drastically reduces the com-puter run-time required to find an excellent feasible schedule to just a fewminutes. A great advantage of using and MIP method is that it has proved tobe easy and quick to introduce new constraints and features into the unitcommitment software. Hence the software can easily be adapted to model thechanges in the plant and operating rules of the power system that occur fromtime to time.

More information on this application can be found at www.poweroptimisation.com. Theapplication was implemented using the XPRESS-MP software package.

11


5. CONCLUSIONS

In this chapter, we reviewed the latest developments in integer program-ming and the integration of modeling and optimization. We illustrated theprinciples in this integration through a several examples in the area ofelectricity generation problems. Even though mixed-integer programmingtechniques are not yet widely used in the electricity industry, we believe thatthe recent developments in theoretical and practical mixed-integerprogramming will enable practitioners to use these techniques moreeffectively. Undoubtedly, the biggest challenge will be the efficient solutionof large-scale instances of these problems.

REFERENCES

1.2.

3.

4.

5.

6.

7.

8.

9.

10.

11.

12.

13.

14.

15.

E. Balas. Disjunctive programming. Ann. Discrete Math., 5: 3-51, 1979.E. Balas, S. Ceria, and G. Cornuéjols. Mixed 0-1 programming by lift-and-project in abranch-and-cut framework. Manage. Sci., 42(9): 1229-1246, 1996.E. Balas, S. Ceria, and G. Cornuéjols. A lift-and-project cutting plane algorithm for mixed0-1 programs. Math. Prog., 58: 295-324, 1993a.E. Balas, S. Ceria, and G. Cornuéjols. “Solving Mixed 0-1 Programs by a Lift-and-ProjectMethod.” In Proc. Fourth Annual ACM-SIAM Symposium on Discrete Algorithms, 232-242, 1993b.J. Batut and R.P. Sandrin. “New Software for the Generation Rescheduling in the FutureEDF National Control Center.” In Proc. PSCC, Graz, Austria, 1990.Belvaux, G., Solving lot-sizing problems by branch & cut: Basic guide to use of mg-EMOSL. Working paper, CORE, Université catholique de Louvain, February 1998.A. Caprara and M. Fischetti. Branch-and-Cut Algorithms, in Annotated Bibliographies inCombinatorial Optimization. M. Dell’Amico, F. Maffioli, and S. Martello, eds. Chiches-ter: John Wiley & Sons, Chichester, 45-63, 1997.S. Ceria. “MIPO: A Mixed-Integer Program Optimizer.” Presented at: InternationalSymposium on Mathematical Programming, Lausanne, August 1996.C. Cordier, H. Marchand, R.S. Laundy, and L.A. Wolsey. “bc-opt: A branch-and-cut codefor mixed integer programs.” Discussion paper DP 9778, CORE, Universite catholique deLouvain, 1997.H. Crowder, E. Johnson, and M. Padberg. Solving large-scale zero-one linear program-ming problems. Op. Res., 31: 803-834, 1983.R. Daniel and J. Tebboth. A tightly integrated modelling and optimisation library: A newframework for rapid algorithm development. Ann. Op. Res., 0: 1-21, 1998.G.B. Dantzig, D.R. Fulkerson, and S.M. Johnson. Solution of a large-scale travelingsalesman problem. Op. Res., 2: 393-410, 1954.G.B. Dantzig, D.R. Fulkerson, and S.M. Johnson. On a linear programming, combinatorialapproach to the traveling salesman problem. Op. Res., 7: 58-66, 1959.Dash Associates. XPRESS-MP EMOSL Reference Manual, Release 10. Blisworth, Nort-hants NN7 3BX, UK: Blisworth House Church Lane, 1997.R.S. Garfinkel and G.L. Nemhauser. Merger Programming. New York: John Wiley &Sons, Inc., 1972.


16.

17.

18.

19.

20.

21.

22.

23.

24.

25.

26.

27.

28.

29.

30.

31.

32.

33.

34.

35.

36.

37.

R. Gormory. “An Algorithm for the Mixed-Integer Problem. Technical Report RM-2597,The Rand Corporation, 1960.R. Gormory. 1958. Outline of an algorithm for integer solutions to linear programs. Bull.Am. Math. Soc., 64: 275-278, 1958.R. Gormory. “Solving Linear Programming Problems in Integers. In CombinatorialAnalysis R.E. Bellman, M. Hall Jr., eds. Providence, RI: American Mathematical Society,211-216, 1960.X. Guan, P.B. Luh, and L. Zhang. Non-linear approximation method in Lagrangian relaxa-tion-based algorithms for hydrothermal scheduling. IEEE Trans. Power Syst., 10(2): 772-778, 1995.H.H. Happ, R.C. Johnson, and W.J. Wright. Large-scale unit commitment method andresults. IEEE Trans. PAS, PAS-90: 1373-1383, 1971.K.L. Hoffman and M. Padberg. Techniques for improving the LP-representation of zero-one linear programming problems. ORSA J. Computing, 3: 121-134, 1991.K.L. Hoffman and M. Padberg. Solving airline crew scheduling problems by branch-and-cut. Manage. Sci, 39: 657-682, 1993.E.L. Johnson, G.L. Nemhauser, and M.W.P. Savelsbergh. “Progress in Linear Program-ming Based Algorithms for Integer Programming: An Exposition.” School of Industrialand Systems Engineering, Georgia Institute of Technology, Atlanta, Georgia, August1999.M. Jünger and S. Thienel. Introduction to ABACUS – A branch-and-cut system. Op. Res.Lett., 22: 83-95, 1998.H.A. Land and A.G. Doig. An automatic method for solving discrete programming prob-lems. Econometrica 28: 497-520, 1960.T. Lekane and J. Ghuery. “Short Term Operation of an Electric Power System.” TractebelEnergy Engineering, Brussels, Belgium.L. Lovasz and A. Schrijver. Cones of matrices and set-functions and 0-1 optimization.SIAM J. Optimization, 1: 166-190, 1990.G.L. Nemhauser, M.W.P. Savelsbergh, and G.C. Sigismondi. MINTO, a Mixed INTegerOptimizer. Op. Res. Lett., 15: 47-58, 1993.G.L. Nemhauser and L.A. Wolsey. Integer and Combinatorial Optimization. New York:John Wiley & Sons, 1988.S.O. Orero and M.R. Irving. Large-scale unit commitment using a hybrid genetic algo-rithm. Elec. Power Energy Syst., 19(1): 45-55, 1997.M.W. Padberg and G. Rinaldi. A branch-and-cut algorithm for the resolution of large-scale symmetric traveling salesman problems. SIAM Rev., 33: 60-100, 1991.M.W. Padberg and G. Rinaldi. Optimization of a 532 city symmetric traveling salesmanproblem by branch-and-cut. Op. Res. Lett., 6: 1-7, 1987.C.K. Pang, G.B. Sheblé, and F. Albuyeh. Evaluation of dynamic programming basedmethods and multiple area representation for thermal unit commitments. IEEE Trans.PAS, PAS 100(3): 1212-1218, 1981.M.W.P. Savelsbergh and G.L. Nemhauser. Functional description of MINTO, a mixedinteger optimizer, version 2.3. School of Industrial and Systems Engineering, Georgia In-stitute of Technology, Atlanta, Georgia, November 1996.A. Schrijver. Theory of Linear and Integer Programming. Chicester: John Wiley & Sons,1986.H. Sherali and W. Adams. A hierarchy of relaxations between the continuous and convexhull representations for zero-one programming problems. SIAM J. Disc. Math., 3: 411-430, 1990.T.J. Van Roy and L.A. Wolsey. Solving mixed-integer programming problems usingautomatic reformulation. Op. Res., 35: 45-57, 1987.


T.J. Van Roy and L.A. Wolsey, Solving mixed 0-1 programs by automatic reformulation.Op. Res. 35: 145-163, 1987.S.J. Wand, S.M. Shahidehpour, D.S. Kirschen, S. Mokhtari, and G.D. Irisarri. Short-termgeneration scheduling with transmission and environmental constraints using an aug-mented Lagrangian relaxation. IEEE Trans. Power Syst., 10(3): 1294-1301, 1995.

38.

39.

Chapter 10

AN INTERIOR-POINT/CUTTING-PLANEALGORITHM TO SOLVE THE DUAL UNITCOMMITMENT PROBLEM – ON DUALVARIABLES, DUALITY GAP, AND COSTRECOVERY

Marcelino Madrigal and Victor H. QuintanaUniversity of Waterloo

Abstract: In this chapter, we use an interior-point/cutting-plane (IP/CP) method for non-differentiable optimization to solve the dual to a unit commitment (UC) prob-lem. The IP/CP method has two advantages over previous approaches, such asthe sub-gradient and bundle methods: first, it has proved to have better conver-gence characteristics in an actual implementation; and second, it does not sufferfrom the parameter-tuning drawback. The results of performance tests usingsystems with up to 104 units confirm the superiority of the IP/CP method overprevious approaches to solve the dual UC problem. We discuss issues that haveinfluenced whether or not UC models are used as the clearing mechanism inelectricity markets; these issues include duality gap, cost recovery, and the exis-tence of multiple solutions.

1. INTRODUCTION

Lagrangian relaxation (LR) has become one of the most practical and ac-cepted approaches to solve unit commitment (UC) problems of real-life di-mensions [1-3]. The key idea in LR-based approaches is to solve the dualproblem instead of the primal; the dual function has a separable structure, i.e.,in a per thermal-unit basis, which permits its easy evaluation and, at the sametime, provides a primal solution. The dual function is concave but not differ-entiable [16]; therefore, non-differentiable optimization techniques are re-quired to solve the dual problem. Pioneering work on LR-based UC solutionapproaches has used sub-gradient (SG) methods as the dual maximizationengine [2, 4]. Despite their bad convergence characteristics, they are still be-


ing used due to its simplicity and low per-iteration computer effort. Severalcutting-plane (CP) variants to solve non-differentiable optimization have beenemployed to solve the dual to unit commitment or other power schedulingproblems. For instance, in [5] a penalty-bundle (PB) method is used to solvethe UC problem. In [6], a CP with dynamically adjusted constraints is used tosolve the hydrothermal coordination problem. In [7], a reduced complexitybundle method is introduced to solve the dual of a power-scheduling problem.All these cutting-plane variants still have the disadvantage that parametersneed be carefully tuned; these parameters define a stabilization scheme thatprevents unboundedness in the maximization of the dual function and helpimprove convergence [8].

Interior-point/cutting-plane (IP/CP) methods have been used to solve non-differentiable problems in engineering applications, such as the lot sizing [9]and multi-commodity flow problems [10]. Recently, the authors have pro-posed the use of an IP/CP method to solve the UC problem [11]. This paperdescribes the use of such a non-differentiable optimization technique to solvethe dual to a unit commitment problem. IP/CP methods have two advantagesover previous approaches, such as the sub-gradient and penalty-bundle meth-ods: first, they have better convergence characteristics; and second, they donot suffer from the parameter-tuning drawback.

The first mechanism used by an electricity market to select power amongcompeting generators was a unit commitment model. In this Pool market-model, generators act as independent entities and the pool operator, based onthe costs submitted by generators and their physical limitations, decides theschedules for all generators and sets a market price. UC models may naturallyhave multiple solutions that, from the Pool point of view, are equally good (allminimize the cost) but, for generators, it means different schedules and, there-fore, different revenues that create a clear conflict of interest. LR algorithmsare still dependent on heuristics and are not able to identify or distinguishmultiple solutions [12-13].

The IP/CP method presented in this chapter has eliminated one of thedrawbacks (tuning in the dual maximization) of LR algorithms. We also pre-sent some other findings that relate duality gap and cost recovery when dualvariables set the market price. It has been shown that UC cost-minimizationmodels and artificial price definitions, derived from their solutions, do notnecessarily lead to lower prices for consumers [14]. For a simplified UCmodel, we show that, in the absence of duality gap, the dual variables are theminimum uniform market price that recovers participants’ cost.

In Sections 2 and 3, we describe the unit commitment problem and itsdual. Section 4 presents the solution of the dual problem by the IP/CP method.Section 5 discusses the issues of duality gap, stopping criterion, cost recovery,

Unit Commitment By Interior-Point/Cutting-Planes 169

and multiple solutions. Section 6 contains numerical results on the perform-ance of the IP/CP method, and we give some conclusions in Section 7.

2. THE UNIT COMMITMENT PROBLEM

The unit commitment problem consists on determining the generators thatneed to be committed and the production levels that are required for supplyingthe forecasted short-term (24 or, at most, 168 hours) demand and spinningreserve requirements (2-3); all of this at the minimum cost (1). The operationof the units is subject to several constraints (4), to be described. The primalunit commitment (PUC) is a very large, non-linear, mixed-integer problem;therefore, the non-convex optimization problem:

T is the set of time periods in the optimization horizon (subsequently | T |denotes the number of periods); I is the set of thermal units. The objectivefunction is comprised of production, and start-up cost, These costs arerepresented, respectively, using the classic [2] quadratic-convex and exponen-tial models:

The commitment state of unit at time is defined by the binary variable

(1: committed, 0: decommitted); represents the power outputof the unit. Equations (2) and (3) are the constraints to satisfy the system-widepower, and reserve, requirements. The reserve contribution of every

thermal plant is given by where is the maximum allow-able power output.

In (4), the vector contains the commitment states and production levels

of the particular unit that is, Each thermal

unit operation constraint set, contains: (i) minimum and maximum power


output constraints; (ii) ramp-constraints; and, (iii) minimum up- and down-time constraints [2]. In general, the sets contain non-linear and mixed-integer restrictions. The UC has been proven to be NP-hard [15].

3. THE DUAL UNIT COMMITMENT PROBLEM

The (Lagrangian) dual problem to PUC is

We obtain the dual objective function, from the Lagran-gian that is formed by adding to the objective function (1), through Lagrangemultipliers, the system-wide demand constraints (2), and the reserve con-straints (3). The dual variables of the power-balance equality constraints arenot necessarily constrained to be positive, but, since the objective function,see (5-6), always increases with power production, they will always take posi-tive values. After arranging some terms, the dual objective function has thefollowing separable structure:

where

and is the dual variable vector. The dual func-

tion (8), as proven in [16], is concave but not differentiable. The separablenature of the dual function is exploited in a two-stage solution process (La-grangian Relaxation) of the DUC problem.

3.1 The Lagrangian Relaxation Algorithm

The LR approach to solve the UC problem is outlined below:

Initialization.Obtain an initial dual vector and set Obtain a priority-list UCand afterwards perform a simplified economic dispatch for each time


The dual variables of the power demand constraints are used to initializethe respective Spinning reserve constraints’ dual variables are initial-ized to zero, i.e.,

Phase 1. DUC Maximization.1.

2.

3.

Dual objective function evaluation. From (8), evaluate the dual objectivefunction by solving the | I | individual unit commitment subprob-lems (9) using a forward dynamic programming approach. A primal vec-tor, (not necessarily feasible) is obtained after the subproblems aresolved.Convergence test. If a convergence criterion is satisfied, then go toPhase 2. Otherwise, continue.Dual Maximization. Find an improved dual solution vector, usinga non-differentiable optimization technique. Set go back to 1.

Phase 2. Feasibility search.Use a heuristic procedure to map the non primal-feasible solution, (ob-tained in Phase 1) to a feasible one, say Usually, feasibility is achievedwith little modifications on We follow the procedure described in [2]to perform such feasibility search.

The structure of the non-linear, mixed-integer subproblems (9) is such thatthey can be easily solved using a dynamic programming [2-3] or branch andbound techniques. Semi-definite programming has also been used to solve theUC subproblems [17].

4. AN INTERIOR-POINT/CUTTING-PLANE METHODTO SOLVE THE DUC PROBLEM

Classic methods for non-differentiable optimization, such as penalty-bundle and Kelley’s cutting plane methods [8], maximize a cutting plane ap-proximation of the objective function over a set of restrictions (stabilizationregion) that encloses the optimal solution and helps improve convergence byproperly setting parameters. Instead of maximizing a cutting planeapproximation over a stabilization region, IP/CP method takes the analyticcenter of a localization set as the new improved dual solution [9, 18]. Thelocalization set, is a convex closed region defined by


The components of are described as follows: (i) a cutting-plane ap-proximation of the UC dual function; (ii) the known dual variables’ lowerbounds (iii) a box constraint and (iv) a lower bound to thedual objective function value. Constraints (ii) and (iii) limit the localizationset from the “left” and “right,” respectively; and the constraints (i) and (iv)limit the localization set from “above” and “below,” respectively. The cutting-plane approximation of the dual function is constructed using a bundle of in-formation from all previous iterations of the maximization process. A bun-dle is a collection of: (i) dual vectors (ii) their corresponding

dual objective function values and (iii) the sub-gradients

A sub-gradient at iteration is readily available by com-puting the mismatches of the power and reserve constrains,

This vector always belongs to the sub-differential set

Therefore, it qualifies as a sub-gradient for the concave function (8) atAs pointed in [18], the selection of the box constraint, in (10), has a

limited influence on the convergence characteristics of the IP/CP method; inpractice, any large enough number based on knowledge of the problem can bechosen. For the UC problem, similar convergence characteristic is achievedwith any large value as presented in the results section. This value is ob-tained based on units’ cost coefficients. A lower bound of the dual objectivefunction is readily available from previous iterations, as

The localization set in (10) can be rewritten as

where


Note that has iteration-increasing size. The analytic center

of is defined as the point solving the potential problem

where defines a potential function whose maximum is achieved at

a point that is centered in the localization set. For example, a non-centeredpoint touching hyperplane has and the associated potential compo-

nent is taken as an updated dual solution vec-

tor, which is used to evaluate the dual function. If a stopping criterion is notsatisfied, the localization set (10) is updated to by adding a new cut,

and by replacing the dual-function lower

bound approximation by The analytic center of the updatedlocalization set is obtained and the process in repeated again. We depict thethird, fourth, and fifth iterations of an IP/CP method for a dual function inschematically in Figure 1. The dot inside each updated localization set(shaded region) represents the analytic center; the horizontal dotted line repre-sents the lower bound and the bold curved line represents the dual func-tion. This figure depicts classical behaviour of the IP/CP method; cuts gener-ated from the analytic center are deeper and the localization set rapidlyshrinks towards a single point corresponding to the optimal value.

Problem (14) can be efficiently solved using a primal, dual, or primal-dual interior-point method [19]. In [11], the authors use a semi-definite pro-

with and and

From


gramming formulation to solve the potential problem. In this chapter, we em-ploy a primal-dual logarithmic-barrier interior-point method to solve the po-tential problem (14) and describe it briefly in the following paragraphs.

We derive the primal-dual interior-point method from Kojima, Megiddo,and Mizuno’s primal-dual infeasible-interior-point algorithm [20]. The bar-rier-Lagrangian to (14) is given by

where is an iteration-decreasing barrier parameter The first-order necessary conditions for the optimality of (15) are

In these equations, the complementarity primal and dual

residuals are defined. From an initial strictly positive point, the

primal-dual interior-point method generates new points of the form

We compute the search directions by a one-iteration Newton’s method that isapplied to solve the optimality conditions (16)-(18). The x and y Newton’sdirections can be computed by solving the reduced system

The s -search direction is obtained from

The step length in (19) is computed as where


and is a safety factor that prevents variables touching zero. Abarrier parameter reduction scheme, derived from (16), is given by

The algorithm can be stopped when the optimality conditions (16)-(17)are satisfied up to an specified tolerance.

5. ON DUALITY GAP, COST RECOVERY, ANDMULTIPLE OPTIMAL SOLUTIONS

From well-known results on duality theory [16], we know that, for anyfeasible primal dual pair the dual objective value is always a lower

bound to the primal objective value Duality gap is defined asthe difference between the optimal primal and optimal dual objective functionvalues, i.e.,

If a feasible primal dual pair satisfies then the pair isoptimal, and there is no duality gap. It may happen, as in several non-convexprogramming problems, that duality gap exists; that is, the optimal primal anddual objective function values are not equal, and therefore,

5.1 Duality Gap And Stopping Criterion

At every iteration of the LR algorithm, a primal and dualfeasible solutions can be obtained. The dual solution is directly obtained fromthe dual maximization. We achieve a primal solution (not necessarily feasible)when the dual function is evaluated; a feasibility search (based on heuristics)can yield a feasible one. Let us define the complementarity and relative com-plementarity gaps as


Through the iterative process, the values CG or RCG can be used as op-timality indicators. The LR algorithm can be executed until DG is as small aspossible. If after a number of iterations it is found that CG = 0, then an opti-mal solution has been found and there is no duality gap. If CG cannot be re-duced below some value there are two possibilities: (i) a duality gap exists,and the solution at hand is optimal; or (ii) the LR approach fails to further im-prove the solution. Experience on solving UC problems [2, 15, 21] shows thatRCG can be reduced to “small” values, about 1-2%, especially for largeproblems. These experimental results can be explained by theoretical results[1, 16] that show that, as the number of separable components (generators) inthe dual function increases, the DG decreases.

5.2 Duality Gap and Cost Recovery: Do Dual VariablesRecover the Participants’ Cost ?

Using a simplified unit commitment problem, the authors have been ableto show that there is a direct relationship between duality gap and partici-pants’ cost recovery when dual variables are used to set the market price whenthe unit commitment is used as an optimization-based auctioning device [22].The simplified (static, 1-hour) UC problem considers cost functions that con-tain a linear and a constant start-up cost. The reserve constraint is dropped andthe operation of units is only constrained by the upper production limit; thatis,

Let be the uniform price set by a market to pay participants fortheir power output (optimal solution to (24)). The profit (revenue minusproduction cost) for each participant is given by


where has been introduced for convenience. The marketprice has to be the minimum possible that leads to the recovery of partici-

pants’ cost; i.e., the profits are positive The dual problem to(24) has the form of (7), and the dual function is given by

The dual function is piece-wise concave, and has slope-changing points atevery where as shown in Figure 2.

The optimal value of the dual function (26) can be found in a closed form;it just suffices to determine the point at which the dual-function slope be-comes zero or negative. Without loss of generality, let us assume that the unitsare ordered so that An optimal dual solution is given

by where is the smallest index such that In [22], the

conditions under which duality gap exists for this problem are derived. In theabsence of duality gap, if the optimal dual variable (or any of its multiple val-ues) is used to set the market price, then all the units will recover

their cost. Since for all scheduled units, then If a duality gapexists, there is a cost that is not recovered and its magnitude equals the gap.


Table 1 shows the data from an illustrative example, where we considerthree different demand conditions and 290 MW). In the firstcase, the dual problem has multiple dual solutions, but there is no duality gap;and, therefore, as Table 3 shows, all the units’ costs are recovered. For thiscase, there is only one optimal (global) primal solution. Notice that the profitof the last scheduled unit is zero; hence, there is no other smaller value (that isused as an uniform price) which can recover participants’ cost. This meansthat in the absence of duality gap, the optimal dual variables from the costminimization problem are also the minimum possible prices.

In the second case, the dual problem has a unique solu-tion. Since the optimal primal objective function is different from the optimaldual objective function, however, there is duality gap. The cost not recoveredis equal to the duality gap; see Table 3. In the last case, there is also a dualitygap of $3 and, therefore, the same quantity is not recovered. In this case, thereare two multiple primal solution; 40 MW can be supplied either by unit 4 or 5.

Some other results in [22] show that for systems with a larger number ofunits, the size of the duality gap dramatically decreases. This suggests thatsmall (or no) adjustments to the optimal dual variables need to be made inorder to use them to set the market price. Even though the relation betweenduality gap and cost recovery has not been proved for the general UC problem(1)-(4), the results in the next section suggest that the relation may hold.

Multiple solutions are more likely to exist for this type of combinatorialproblem, as happened in the last example. An approach that can identify allthe alternative optimal solutions is, obviously, the use of enumeration. This


would require the enumeration of possible unit commitment combina-tions; from these combinations, we take the ones that can satisfy the demandand solve them using a simple linear programming model. If different combi-nations have the same objective function, then there are multiple (combinato-rial) solutions. For a particular combination, multiple solutions (continuousmultiple solutions) may also exist. The identification of multiple solutions forthe PUC problem is a much more complex task.

6. RESULTS OF THE IP/CP FOR THE DUC PROBLEM

The LR procedure to solve the UC problem has been coded in C; test runshave been performed on a Pentium 200Mhz computer under the LINUX op-erative system. We solve the potential problem (14) using the primal-dual in-terior-point method, described in Subsection 4.1.1; it is also coded in C. TheNewton’s system (20) is solved using an efficient sparse-matrix processingand factorization techniques. We use the following four UC test systems: (i) a26-unit system [23]; (ii) a 32-unit system [15]; (iii) two larger systems with 76and 104 units, which are created by using the data from the systems in (i) and(ii). The data for the last two systems is selected in such a way that large, me-dium, and small units, as well as expensive, moderate, and cheap units coex-ist. All tests are performed over a | T |= 24 hour optimization horizon.

6.1 Comparative Performance

Table 4 exhibits the number of iterations and time required by each of themethods to solve the dual problem up to a point where the RCG is less than2% for the 26-units case, and less than 1% for the other cases. Notice that theIP/CP method requires far less iterations than the SG and PB methods, and itssolution time is competitive, especially for larger systems.

The results obtained with the SG and PB methods, shown in Table 4, areachieved after extensive individual parameters tuning for each system;whereas no parameter tuning is required for the IP/CP method. The box con-straint is easily set up and left unchanged in all the tests performed. Figure 3presents the evolution of the dual function for each method. The results showncorrespond to the 104-unit system; we observe a similar behaviour with theother test systems. In the first four to five iterations, the IP/CP method is ableto reduce the RCG to about 3%.


6.2 Effect of the Box Constraint

We established and left fixed box constraints in all test systems to themaximum incremental cost (evaluated at maximum power) of a system unit;

that is, It is likely that the incremental

cost at the solution point will never be larger than this value. If the incre-mental cost in the system at a particular time reaches this value, unfeasibilityis likely to occur since the most expensive unit is being put at maximum gen-eration. It has been observed that if the box constraint is set up to any otherlarge value, similar convergence occurs. This claim is supported by the resultsshown in Table 5, where the box constraint has been changed to 75%, 200%and even 300% of the base value Notice that for larger systems the set upof the box constraint has less effect; i.e., in the UC-104 system, only 10 moreiterations are required for convergence if the set up is 300% of


6.3 Duality Gap and Cost Recovery

As shown in Section 5.2 for a simplified UC problem, the magnitude ofthe duality gap is equal to the cost not recovered by the units when the opti-mal dual variables are used to set the market price. For the PUC (1)-(4) thesame relationship seems to hold as shown by the results presented in Table 6.For all the UC test cases, the dual variables to the power balance constraints(obtained in the last iteration of the LR algorithm) are used to set the marketprice The profit for each unit is and the

total non-recovered cost is As we can see, the

CG and NRC are very similar; this suggests that the relation between dualitygap and cost recovery may still hold for the UC problem. Small modificationscould be made to the dual variables in order to use them as the market price.

For instance, the revenues of participants whose profits are positive can beproportionally (to the total positive profits) adjusted (reduced) so that the re-duction is equal the duality gap and, therefore, is enough to recover all par-ticipants costs. This is equivalent to define a non-uniform market price; eachsupplier received a price computed from the dual variable, but scaled to pro-portionally recover the duality gap. Modification of prices to achieve desir-able revenue levels (revenue reconciliation) has been proposed for spot mar-kets [24].

It is worthwhile to point out that DG is a characteristic of the model and itsparameters; it cannot be reduced by the use of one or other algorithm used tosolve the problem. An algorithm does better than the other if it reduces CGcloser to DG. Changes in the model can lead to different DG, but now dualvariables have different meanings.


7. CONCLUSIONS

Interior-point methods are among the best alternatives to solve a wideclass of very large linear and even non-linear programming problems. Re-search on combinatorial and non-differentiable optimization has again putinterior-point methods as one of the promising approaches to most efficientlysolve such problems.

In this chapter, we proposed an interior-point/cutting-plane method tosolve the dual to UC problems. Comparisons to previously used approaches,such as sub-gradient and penalty-bundle methods, demonstrate the advantagesof the IP/CP method. The IP/CP method has better convergence characteris-tics and completely eliminates the need for parameter tuning.

The chapter also presents results that relate duality gap and cost recoveryif the dual variables are used as market prices. We showed, using a simplifiedUC model, that when duality gap does not exist, the dual variable is theminimum market price that can be used to set a uniform market price that re-covers participants’ costs. When duality gap exists, the optimal dual variablesdo not recover the costs; the un-recovered cost equals the magnitude of theduality gap. The same relation seems to hold for the general UC problem, asshown by the numerical results. This suggests that for large unit commitmentproblems, small (or none) modification can be made to the dual variables inorder to be used as market prices. More research into this problem and on theeffective detection of multiple optimal solutions may lead to better under-standing of the failures encountered in pioneering markets, as well as helpdesign better auctioning optimization models.

ACKNOWLEDGEMENTS

The first author gratefully acknowledges CONACyT and Instituto Tec-nológico de Morelia in México, for providing financial support to pursue hisPh.D. studies at University of Waterloo.

REFERENCES

D.P. Bertsekas, G.S. Lauer, N.R. Sandell, and T.A. Posberg. Optimal short-term schedul-ing of large scale power systems. IEEE Trans. Autom. Control, AC-28(1): 1-11, 1983.F. Zhuang and F.D. Galiana. Towards a more rigorous and practical unit commitment byLagrangian relaxation. IEEE Trans. Power Syst., 3(2): 763-773, 1988.X. Guang, P.B. Luh, and H. Yan. An optimization-based method for unit commitment.Electrical Power Energy Syst., 14(1): 9-17, 1992.

1.

2.

3.


J.A. Muckstadt and S.A. Koenig. An application of Lagrangian relaxation to schedulingpower-generation systems. Oper. Res., 25(3): 387-403, 1977.F. Pellegrino, A. Renaud, and T. Socroun. “Bundle and augmented Lagrangian methodsfor short-term unit commitment.” In Power Systems Computation Conference Proc.,pp. 730-739, 1996.N. Jiménez and A.J. Conejo. Short-term hydro-thermal coordination by Lagrangian relaxa-tion: solution to the dual problem. IEEE Trans. Power Syst., 14(1): 89-95, 1999.P.B. Luh, D. Zhang, and R.N. Tomastik. An algorithm for solving the dual problem ofhydrothermal scheduling. In IEEE-PES, Winter Meeting paper PE-333-PWRS-0-12-1997,New York, 1997.C. Lemarechal and J Zowe. “A Condensed Introduction to Bundle Methods in NonsmoothOptimization.” In Algorithms for Continuous Optimization, pp. 357-382, ed. E. Spedicato.Kluwer Academic Publishers, 1994.O. du Merle, J.L. Goffin, C. Trouiller, and J.P. Vial. A Lagrangian relaxation of the ca-pacitated multi-item lot sizing problem solved with an interior point cutting plane method.Technical report, Faculty of Management, McGill University, 1997.J.L. Goffin, J. Gondzio, R. Sarkissian and J.P. Vial. Solving nonlinear multi-commodityflow problems by the analytic center cutting plane method. Math. Prog., (76): 131-154,1996.M. Madrigal and V.H. Quintana. “An Interior-point/Cutting-plane Algorithm to SolveUnit Commitment Problems.” In IEEE-PES Power Industry Computer Applications Con-ference Proc., pages 179-185, Santa Clara, California, 1999. To appear in IEEE Trans.Power Syst.R.B. Johnson, S.S. Oren, and A.J. Svodoba. Equity and efficiency of unit commitment incompetitive electricity markets. Technical Report PWP-039, POWER-series, The Univer-sity of California Energy Institute, 1996.S. Dekrajangpetch, G.B. Sheble, and A.J. Conejo. Auction implementation problems usingLagrangian relaxation. IEEE Trans. Power Syst., 14(1): 82-88, 1999.J.M. Jacobs. Artificial power markets and unintended consequences. IEEE Trans. PowerSyst, 12(2): 968-972, 1997.C.L. Tseng. On Power Systems Generation Unit Commitment Problems. Ph.D. Thesis,University of California, Berkeley, 1996.D.P. Bertsekas. Nonlinear Programming. Athena Scientific, 1997.M. Madrigal and V.H. Quintana. “Semi-definite Programming Relaxations for {0,1}-Power Dispatch Problems.” In IEEE-PES, 1999 Summer Meeting Conference Proc., pp.697-702, Edmonton, Alberta, Canada, 1999.O. Bahn, J.L. Goffin, J.P. Vial, and O. Du Merle. Experimental behaviour of an interiorpoint cutting plane algorithm for convex programming: an application to geometric pro-gramming. Discrete Appl. Math, 49: 3-23, 1994.J.L. Goffin and J.P. Vial. Interior point methods for nondifferentiable optimization. Tech-nical reports, Faculty of Management, McGill University, 1997.M. Kojima, N. Megiddo, and S. Mizuno. A primal-dual infeasible-interior-point algorithmfor linear programming. Math. Prog, (61): 263-280, 1993.S. Sen and D.P. Kothari. Optimal thermal generating unit commitment: a review. Electri-cal Power Energy Syst., 20(7): 443-451, 1998.M. Madrigal and V.H. Quintana. “Using Optimization Models and Techniques to Imple-ment Electricity Auctions.” In IEEE-PES, 2000 Winter Meeting Conference Proc., Singa-pore, 2000.

4.

5.

6.

7.

8.

9.

10.

11.

12.

13.

14.

15.

16.17.

18.

19.

20.

21.

22.


23.

24.

S.J. Wang, S.M. Shahidehpour, D.S. Kirschen, S. Mokhtari, and G.D. Irisarri. Short-termgeneration scheduling with transmission and environmental constraints using augmentedLagrangian relaxation. IEEE Trans. Power Syst., 10(3): 1294-1301, 1994.F.C. Schweppe, M.C. Caramanis, R.D. Tabors and R.E. Bohn. Spot Pricing of Electricity.Kluwer Academic Publishers, 1987.

Chapter 11

BUILDING AND EVALUATINGGENCO BIDDING STRATEGIES ANDUNIT COMMITMENT SCHEDULESWITH GENETIC ALGORITHMS

Charles W. Richter Jr. and Gerald B. ShebléIowa State University

Abstract: Far from being an artifact of the past, the unit commitment (UC) algorithm isessential to making economical decisions in today’s competitive electricity in-dustry. Increasing competition; decreasing obligations-to-serve; and enhancedfutures, forwards, and spot market trading in electricity and other related mar-kets make the decision of which units to operate more complex than ever before.Decentralized auction markets currently being implemented in countries likeSpain use UC-type models, which should encourage researchers to continueworking on finding better and faster solution techniques. UC schedules may bedeveloped for a generation company, a system operator, etc. The need for manyflavors of UC algorithms, each considering different inputs and objective func-tions, is growing. Factors such as historical reliability of units should be consid-ered in designing the UC algorithm. Although a particular schedule may result inthe lowest cost, or highest profit, it may depend on generators that have varyingavailabilities. Traditionally, consumers had very reliable electricity whether theyneeded it or not. Given a choice in a market-based electricity system, many con-sumers might choose to pay for a slightly lower level of power availability if itwould result in sufficient savings. As the number of inputs and options grows inUC problems, the genetic algorithm (GA) becomes an important tool for search-ing the large solution space. GA times-to-solution often scale up linearly withthe number of units, or hours being considered. Another benefit of using the GAto generate UC schedules is that an entire population of schedules is developed,some of which may be well suited to situations that may arise quickly due to un-expected contingencies.

1. INTRODUCTION

Electric generators and energy service companies around the world arepresently embracing competition. Generation companies and energy servicecompanies can negotiate profitable electricity contract prices bilaterally or viaauction markets, which will likely play a vital role in price discovery. Auction


market bidding strategies will be important for participants’ profitability andsurvival. No longer guaranteed consumers nor a rate-of-return by a publicutilities commission (PUC), it is the responsibility of each generation com-pany (GENCO) to operate units profitably. Good bidding strategies must al-low the trader to negotiate profitable contracts, not only in the short-term, butalso in the mid- to long-term.

Providing a basic foundation for effective bidding strategies, the unitcommitment (UC) algorithm will remain one of the central tools in the com-petitive electricity industry. The business of GENCO is ultimately to generateelectricity. The changes brought about by deregulation (e.g., increases incompetition, decreases in the utility's obligations-to-serve, enhanced futures,forwards, and spot market trading in electricity and other related markets)make the decision of which units to operate more complex than ever before.

The need for many varieties of UC algorithms is growing. UC schedulesmay be developed and optimized for a particular generation company or for asystem operator. When choosing a UC schedule to the objective function mayconsider a host of factors, such as historical availability of units or transmis-sion lines, the credit risk of dealing with a particular consumer, etc. Althougha particular schedule may yield the lowest cost or greatest profit, considera-tion of historical generator availability may reduce expected profits or in-crease expected costs.

Historically, consumers were provided with (and paid for) very reliableelectricity regardless of their requirements. Given a choice in a market-basedelectricity system, many consumers would choose to pay for a slightly lowerlevel of power availability if it resulted in substantial savings. Allowing thetrade of electricity with varying reliability levels in a market environment willrequire improvements in allocation algorithms including UC algorithms.

As the number of choices and contract-types grow, the UC schedulesearch space becomes large. The genetic algorithm (GA) is an important toolin searching large solution spaces. GA times-to-solution often scale up line-arly with the number of units or the number of hours being considered. An-other strength that GAs have over other solution techniques is that entirepopulations of solutions are developed in parallel. While GAs work well forsearching large spaces, there is no proof that they are converging to the opti-mal solution. Because entire populations are being evolved during each run,complex fitness evaluations may require much computation.

If a trader is participating in a market, he wants to take as much profit ashe can. The auction market may be viewed as a game for which variousstrategies exist. How does the GENCO use its UC schedules and biddingstrategies in a manner that will result in long term profit? The procedure be-gins by developing a working model of the competition. Since generating unitcharacteristics don’t change fundamentally in the move from monopolistic

Using Genetic Algorithms in GENCO Strategies and Schedules 187

operation to competitive operation, information made public during regulatedoperation remains valid and can help develop a basic competitor model. Addi-tional information influencing the competitor is also public, e.g., price of fuel.These basic competitor models can be used to simulate future (e.g., for Thours into the future) trading. The models allow study of strategic behaviorthat may affect how close the price is to the price achieved under perfectcompetition. Any additional useful information should be considered in de-termining the best forward price and demand forecasts. A price-based UCalgorithm uses these forecasts when searching for the weekly schedule thatmaximizes expected profit, subject to some risk level. The schedule(s) shouldbe analyzed under various contingencies. Candidate UC schedules should beused as an input to the bidding strategy development process. Figure 1 showsan outline of this procedure in block diagram form.

This chapter gives a brief introduction to a basic market framework toexperimental economics. (The reader should be cautioned that this is a repre-sentative market and is not identical to existing markets.) We discuss issuesrelated to determining competitor models, and we present methods used tosimulate the various markets (spot and forward). Our work investigates themodels’ impact on GENCO operations and planning on daily and weekly op-erations. We highlight creation of bidding strategies for single and multi-period auctions and discuss how forwards and futures markets provide a


means to manage risk. We present a price-based unit commitment formula-tion, followed by a unit commitment genetic algorithm (UC-GA) formulation.Additional criteria (e.g., considering contingencies such as non-availability ofgenerators, network problems, unforeseen market disturbances) may highlightdifferences in UC schedules that have equivalent cost and profit. The authorsdiscuss each procedure to compare UC schedules with each other based ontheir expected monetary value under likely scenarios or contingencies. Adap-tive agents modelling buyers and sellers test the developed strategies by trad-ing electricity in a simulated deregulated electricity market.

2. MARKETS AND EXPERIMENTAL ECONOMICS

2.1 A Basic Market Framework

Many countries and some regions of the United States (e.g., California,Pennsylvania-Jersey-Maryland (PJM), New England, etc.) have deregulated.There does not yet appear to be a standardized final market structure that willbe adopted for all areas, but each market developed should be an improve-ment over those previously developed.

The framework described here is one in which electric energy is producedby GENCOs, sold to energy service companies (ESCOs), and delivered onwires owned by distribution companies (DISTCOs) and transmission compa-nies (TRANSCOs). An entity such as the North American Electric ReliabilityCouncil (NERC) sets the reliability standards. The contract prices are discov-ered in an auction. Buyers and sellers of electricity make bids and offers thatare matched subject to the approval of the independent contract administrator(ICA), who ensures that the contracts will result in a system operating safelywithin limits. The ICA submits information to an independent system opera-tor (ISO) for implementation. The ISO is responsible for physically control-ling the system to maintain its security and reliability. Participants are pro-vided with average contract price and volume, last accepted bid or offer, andwhether the participant’s previous bid was accepted or rejected.

Most of the market framework assumed here has been developed in pre-vious publications [1,2,3,4]. It allows for cash (spot and forward), futures, andplanning markets as shown in Figure 2. The spot market is a market wheresellers and buyers negotiate (either bilaterally or multilaterally through anexchange) a price for a certain number of MWh to be delivered in the nearfuture (e.g., 30 MWh – 10 MW from 1:00 p.m. to 4:00 p.m. tomorrow). Thebuyer and seller must arrange a transmission path for the electrons. The day-ahead market in California is an example of a spot market. The forward mar-ket allows trading in a manner similar to the spot market but is further into the


future than is the spot market. In both forward and spot markets, the buyerand seller intend to exchange the physical good (i.e., the electrical energy). Incontrast, the futures market is primarily financial, allowing traders to reduceuncertainty by locking in a price for a commodity in some future month. Theprovision for physical delivery exists, but since it is not normally intended,other entities (e.g., TRANSCOs, ICAs, and ISOs) need not be informed offutures trading. Buying a futures contract is akin to purchasing insurance. Itallows the traders to manage their risk by limiting potential losses or gains.To ensure sufficient interest for price discovery, futures contracts are gener-ally standardized such that it is not possible to tell one unit of the good fromanother (e.g., 1 MWh of electricity of a certain quality, voltage level, etc.).Although provisions for delivery exist, they are generally not convenient. Thetrader ultimately cancels his position in the futures market either with a gainor loss. The physical goods are then purchased on the spot market to meetdemand with the profit or loss having been locked-in via the futures contract.The planning market aids in financing long-term projects like transmissionlines and power plants.

2.2 Experimental Economics

Economic theories are often based on several assumptions that tend to betrue when looking at society as a whole, but the theories may not necessarilyring true for a particular agent or group of agents. The data used to substanti-ate these theories often comes from indicators observed in the real economy.It is hard to isolate the effect of a change in one input on one output in the realworld economy, because many things are changing simultaneously. Experi-mental economics seeks to advance the theories of economics through ex-


perimentation. Pioneers such as Charles Plott and Vernon Smith popularizedthe field through their work using the auction as an allocation mechanism. Ingeneral, researchers study agents in a laboratory setting where their behavioris monitored as they take part in experiments. Although initial experimentswere with humans, intelligent computerized agents are faster, cheaper, andmore consistent than human agents in the laboratory environment. Drawingfrom the field of complex adaptive systems/artificial intelligence, computer-ized agents can be given the ability to learn and to develop sensible GENCObidding strategies. (Figure 1 shows that simulating auction markets is one ofthe first steps to validate and fine-tune the forward price and demand curves.)The simulated markets (described later) are populated with computerizedtrading agents (seeded with GENCOs and ESCOs models) to obtain forecaststhat account for the strategic behaviors of the competitors.

3. DETERMINING THE COMPETITOR MODEL

If the auction simulations are going to produce any results that are to beused in actual GENCO operation, the models of the competing ESCOs andGENCOs need to be reasonably accurate. Developing competitor models isan involved process. Under the market framework assumed for this research,the results of the auctions are public information, similar to the Australianelectric power market. As indicated in Figure 3, if it is available, a database ofauctions from previous periods contains bidding information and can be intel-ligently mined to determine the general rules that the competitors are using.This information can be combined with any additional information knownabout the system or about the competitors to develop a fine-tuned model ofthe competing computerized agents in the simulated markets [5].

The competitor models populate computerized agents participating insimulated auctions for each period of interest. The competitor models mayrepresent a single aggregated GENCO competitor and a single aggregatedESCO, or many independent GENCOs and ESCOs. Important information(e.g., weather, unit outage information, status of transmission system, andtime of day) that will influence the bidding process is used to fine-tune thecompetitor model as shown in Figure 4.


4. SIMULATING AN HOURLY AUCTION MARKET

The procedure described here is loosely based on the auctions seen at theChicago Board of Trade. To prevent an infinite loop, we select a maximumnumber of cycles per round and maximum number of rounds per period. Buy-ers and sellers determine how much they would like to buy and sell. Theydetermine their bids and offers and submit them to the auctioneer. All transac-tions are subject to the approval of the ICA. Price discovery occurs whenthere are a sufficient number of buy bids and sell offers to allow a predeter-mined portion of the total participants to be satisfied with the resulting con-tract. Submission of the bids marks the beginning of the bidding cycle. When


the auctioneer reports the results of the auction to the market participants thecycle is complete. If, after the present cycle, the price has not been discov-ered, the auctioneer reports that price discovery did not occur and asks fornew bids and offers. Market rules (funnel rules) force subsequent cyclescloser to price discovery by requiring the buyers to increase their bids andsellers to decrease their offers. The cycles continue until price discovery oc-curs, or until the auctioneer decides to match whatever valid matches existand continue to the next round or hour of bidding. Figure 5 shows this proc-ess.

5. FORECASTING PRICES AND DEMANDS

For each auction market period, it is possible to determine a curve relat-ing the price to the quantity demanded and a curve relating the price to thequantity supplied. In the cost-based world of the past, the forward price curvefor an individual hourly market was simply a horizontal summing of GEN-


COs’ cost curves, which were public information. The forward price curve inthe competitive world will look fairly similar. In the price-based competitiveworld, however, GENCOs no longer have to reveal their true cost. This in-formation can be approximated from the forward and futures trading. Modelsfor generating unit output costs are relatively consistent from year-to-year; somuch of the information is publicly available from historical filings madewith regulators.

Some uncertainty comes with not knowing the consumer demand, whichis highly dependent on the weather. We can use neural networks and statisti-cal techniques to obtain demand forecasts. These price and demand forecastsare given to the unit commitment scheduler, which attempts to find theschedule of generating units that maximizes utility. For simplification, the UCalgorithm described in the next section uses a single expected price and quan-tity for each period. A UC schedule is typically developed for each hour ofthe following week. Each period the latest forecasts and market price aregiven to the algorithm and updated schedule is developed.

6. PRICE-BASED UNIT COMMITMENT

In an environment with vertically integrated monopolies, the schedulingof generating units to be on, off, or in stand-by/banking mode is done in amanner that minimizes costs. Consideration must be given to factors likevarying fuel costs, start-up and shut-down parameters/constraints of eachpower plant, and crew constraints. In order to determine the cost associatedwith a given schedule, an economic dispatch calculation (EDC), in which allunconstrained operating units are set so that their marginal costs are equal,must be performed for each hour under consideration. One possible way todetermine the optimal schedule is to do an exhaustive search. Exhaustivelyconsidering all possible ways that units can be switched on or off for a smallsystem can be done, but for a reasonably sized system the amount of time itwould take is too long. Solving the problem generally involves using methodslike Lagrangian relaxation, dynamic programming, genetic algorithms, orother methods using heuristic search techniques. Many references for the tra-ditional UC can be found in Sheblé and Fahd [6] and in Wood and Wollen-berg [7].

Without competition, demand forecasts of total consumer demand in theservice territory advised power system operators of the amount of power thatneeded to be generated. Under competition, bilateral spot and forward con-tracts make part of the total demand known a priori. Predicting the GENCO’sshare of the remaining demand may be difficult, however, since it is depend-ent on its prices compare to that of other suppliers, and the condition of the


transmission network, among other factors. Since the number of unitsswitched on or in banking mode affects the average cost of production, theUC schedule indirectly affects the price, making it an essential input to anysuccessful bidding strategy.

Utilities operating in regulated monopoly fashion have an obligation toserve their customers within a designated service territory. This translates intoa demand constraint that ensures all demand is met. For the UC problem, thismight mean switching on an additional unit to meet a remaining MW or two.Without an obligation to serve, the competitive GENCO can consider aschedule that produces less than the predicted demand. It can allow othersuppliers to provide that one or two MWs that might have increased averagecosts (they might not have secured that contract for which they would havehad to compete).

Demand forecasts and expected market prices are an important input to theprofit-based UC algorithm; they are used to determine the expected revenue,which affects the expected profit. If a GENCO comes up with two potentialUC schedules each having different expected costs and different expectedprofits, it should take the one that provides for the greatest expected benefit tocost ratio, which will not necessarily cost least. Since price and demand are soimportant in determining the optimal UC schedule, price prediction and de-mand forecasts become crucial.

Mathematically the traditional cost-based UC problem has been formu-lated as follows [8]:

Subject to the following constraints:

The cost based UC algorithm for the regulated monopolist has been wellresearched. Recently, research has included the use of markets in developingUC schedules. Takriti, Krasenbrink, and Wu [9] present a good descriptionand a stochastic solution of the UC problem that considers spot markets.Their research differs from that presented here in that it minimizes costsrather than maximizes profits. Here, we redefine the UC problem for theGENCO operating in the competitive environment by changing the demand


constraint from an equality to less than or equal (assume buyers are requiredto purchase reserves per contract) and changing the objective function fromcost minimization to profit maximization:

Subject to:

Where individual terms are defined as follows:(Capacity limits)(Ramp rate limits)

up-/down-time status of unit n at time period

power generation of unit n during time period tload level in time period tforecasted demand w/ reserves for period tforecasted price for period tsystem reserve requirements in time period tproduction cost of unit n in time period tstart-up cost for unit n, time period tshut-down cost for unit n, time period tmaintenance cost for unit n, time period tnumber of unitsnumber of time periodsgeneration low limit of unit ngeneration high limit of unit nmaximum contribution to reserve for unit n

Since a GENCO is not obligated to serve under competition, it maychoose to generate less than the total consumer demand. Therefore, maximiz-ing the profit is not the same as minimizing the cost. Choosing the amount ofdemand to serve allows the GENCO more flexibility in developing the UCschedules. Because prices fluctuate with supply and demand, engineers oftenassume that making the load curve level minimizes cost and should be a rea-sonable operating strategy. When maximizing profit, the GENCO may findthat during certain conditions greater profit may be possible under a non-levelload curve. If revenue increases more than the cost does, the profit will in-crease. Figure 6 shows a block diagram of the UC solution process.

EDC is an important part of UC. Formerly used to minimize costs, it isnecessary to redefine EDC for price-based operation. Where the cost-minimizing EDC ignored transition and fixed costs to adjust the power level


of the units until they each had the same incremental costour new EDC attempts to set equal to a pseudo price (i.e., pro-

duce until the marginal cost equal the price). This pseudo price accounts fortransition and fixed costs as shown in the following formula:

Transition costs include start-up, shut-down, and banking costs. Fixedcosts (present for each hour that the unit is on), are represented by the con-stant term in the typical quadratic cost curve approximation.

During each period, the fixed and transition costs are accounted for byadding an average value to the incremental cost. Note that this is one methodof allocating the transition and fixed costs, but there are many others thatcould be used. For instance, if GENCO A’s generators are able to produceelectricity far less expensively than the competing GENCOs during the night,but don’t have that advantage during the daytime, GENCO A could shiftsome of its daytime costs to be recovered through bids from the night-timeperiods.

7. A GENETIC-BASED UC ALGORITHM

7.1 The Basics of Genetic Algorithms

A genetic algorithm is a search algorithm often used in non-linear discreteoptimization problems. A population of data structures appropriate for theproblem solution is initialized randomly, evolves over time, and finds a suit-able answer (or answers) to the problem. The data structures often consist ofstrings of binary numbers, which are mapped onto the solution space forevaluation. Each solution (often termed a creature) is assigned a fitness - aheuristic measure of its quality. During the evolutionary process, those crea-tures having higher fitness are favored in the parent selection process and areallowed to procreate. The parent selection is essentially a random selectionwith a fitness bias. The parent selection method determines the type of fitnessbias. Following the parent selection process, the processes of crossover andmutation are utilized and new creatures are developed which ideally explore adifferent area of the solution space. These new creatures replace less fit crea-tures from the existing population.


7.2 GA Formulation for Price-Based UC

The algorithm presented here solves the UC problem for the profit-maximizing GENCO operating in the competitive environment [10]. VariousGA formulations for finding optimal cost-based UC schedules have been pro-posed by researchers [11, 12]. The profit or price-based algorithm presentedhere is a modification of a genetic-based UC algorithm for the cost-minimizing monopolist that was described by Maifeld and Sheblé [13]. Thefitness function now rewards schedules that maximize profit. The intelligentmutation operators are preserved in their original form. The schedule formatis the same. Figure 7 shows the algorithm.

The algorithm first reads in the contract demand and prices, the forecast ofremaining demand, and forecasted spot prices. During the initialization step, apopulation of UC schedules is randomly initialized (see Figure 8). For eachUC period of each member of the population, EDC is called to set the units’


generation levels. The cost of each schedule is determined from the paramet-ric generator data and the demand and price data read at the beginning of theprogram. Next, the fitness (i.e., the profit) of each schedule in the populationis calculated. “Done?” checks to see whether the algorithm has either cycledthrough for the maximum number of generations allowed, or whether otherstopping criteria have been met. If the algorithm is done, the results are writ-ten to a file; if it is not done, the algorithm proceeds to the reproduction proc-ess.

During reproduction, the algorithm creates new schedules. The firststep of reproduction is to select parents from the population. After selectingparents, children are created using two-point crossover as shown in Figure 9.Following crossover, standard mutation is applied. Standard mutation in-volves turning a randomly selected unit on or off within a given schedule.


An important feature of this UC-GA is that it spends as little time as pos-sible doing EDC. After standard mutation, EDC is called to update the profitonly for the mutated hour(s). An hourly profit number is maintained andstored during the reproduction process, which dramatically reduces theamount of time required to calculate the profit over what it would be if EDChad to work from scratch at each fitness evaluation. In addition to the stan-dard mutation, the algorithm uses two “intelligent” mutation operators thatwork by recognizing that, because of transition costs and minimum up-timeand down-time constraints, 101 or 010 combinations are undesirable. The first


of these operators purges these undesirable combinations by randomly chang-ing 1s to 0s or vice versa. The second of these intelligent mutation operatorspurges the undesirable combination by changing 1 to 0 or 0 to 1 based onwhich of these is more helpful to the profit objective.

7.3 Price-Based UC-GA Results

We tested the UC-GA on some small systems and the results were com-pared to the solutions found by exhaustive search. In all of the trials for whichknown optima existed, the GA was successful in locating the optimal UCschedule. Figure 10 shows the costs and average costs (without transitioncosts) of the 10 generators, as well as the hourly price and load forecasts forthe 48 hours. We chose the data so that the optimal solution was known a pri-ori. The dashed line in the load forecast represents the maximum output of the10 units. In addition to the information shown in the figure, the UC-GA pro-gram requires the start-up and shut-down costs, the minimum up and downtimes, and the cost to bank each generator. The generators are modelled witha quadratic cost curve (e.g., , where P is the power level ofthe unit). Though the GA took 730 seconds to find a population of solutionscontaining the best possible solution for a 10-unit, 48-hour case (see Figure11).

Prior to initiating the UC-GA, the control parameters shown in Table 1are specified, including the “Number of Units” and Number of Hours” to con-sider. “Popsize” is the size of the GA population. Increasing the populationsize increases the amount of space searched during each generation of theGA. This must be balanced with the consideration that execution time foreach generation of the algorithm varies approximately linearly with the sizeof the popsize. The number of “Generations” indicates the number of timesthe GA will undergo the selection and reproduction. “System reserve” is thepercentage of reserves that the buyer must maintain for each contract. “Chil-dren per generation” tells us how much of the population will be replacedeach generation. Changing this can affect the convergence rate. If there aremultiple optima, faster convergence may trap the GA in local sub-optimalsolution. “UC schedules to keep” indicates the number of evolved schedulesto write to file. There is also a “random number seed” that may be set be-tween 0 and 1.


The algorithm accurately calculates the cost of schedules in which mini-mum up- and down-time constraints appear to be violated by considering azero surrounded by ones to be a banked unit, and a one surrounded by zeros isignored (unit remains off) if it violates the minimum up constraint.

An advantage of using the GA is that its solution time scales up only line-arly with the number of hours and units, while dynamic programming quicklybecomes too computationally expensive to solve. The existence of additional


valid solutions, which may be only slightly sub-optimal in terms of profit, isanother main advantage of using the GA. It gives the system operator theflexibility to choose the best schedule from a group of schedules to accom-modate things like forced maintenance.

8. COMPARING AND SELECTING UC SCHEDULES

Even though a large percentage of the UC schedules encountered by thegenetic algorithm (or other search technique) may satisfy (within some smalltolerance) the primary objective function, they may not be equal. A set of UCschedules may initially have indistinguishable costs or profits, but when weconsider additional criteria, differences between the schedules may be re-vealed. A few examples of additional criteria might be:

Impact of units or transmission system availabilityAbility to respond to spot-market price fluctuationsSchedule’s profitability during network contingenciesAbility to accommodate maintenance activities

A unit that is unavailable likely reduces GENCO profitability. A unit maybe unavailable due to a unit outage (scheduled maintenance or forced), thetransmission line connecting it to the load may be congested, or other unfore-seen circumstances may exist that prevent the GENCO from selling electric-ity. The amount of time that a generating unit is forced out of the market isoften unpredictable and variable. If a unit undergoes a forced outage or istaken off-line for other reasons, costs such as shut-down (and subsequentstart-up) must be recovered. These unanticipated costs may have a large im-pact on the profitability of the UC schedule. Although reserves may mitigate


the consequences of a single unit outage, a possibility exists that many unitsmay be inoperable or at reduced capacity simultaneously, since independentunit outages may occur, as well as contingencies promoting system-wide dis-turbances. Therefore, the possibility that more than one unit is forced off-linein a given period of time must be considered. We can use historical availabil-ity of generating units and of the transmission system itself to differentiatebetween the UC schedules under consideration.

The spot market prices may undergo short-term unanticipated changesthat could be quite profitable for the GENCO having a UC schedule allowingthe amount of power to be increased or decreased easily (i.e., without turningon or off additional units).

A schedule’s performance under various contingencies may distinguish itfrom others. While searching for the optimal UC schedule, certain networkconditions, unit availabilities, load and price forecasts may have been as-sumed. Contingencies will impact some of the candidate schedules more thanothers.

The ability to schedule maintenance activities may be a characteristic ofschedules that distinguishes them from each other. Perhaps two schedulesresult in roughly the same amount of profit, but one of the schedules allowsfor preventive maintenance activities on some key units.

9. TESTING THE UC SCHEDULE IN SIMULATEDCOMPETITION

9.1 Intelligent Bidding Strategies

Once the tentative UC schedule(s) is developed, the GENCO should havebidding strategies that guide it in placing bids and in taking market positions.These strategies might be designed to limit risk, maximize profit, a combina-tion of both, or something entirely different.

Intelligent strategies can detect various market scenarios and respond ap-propriately (i.e., profitably). Because the number of scenarios that the agentmight encounter is extremely large, discrete, and non-linear, finding an opti-mal bidding strategy is a problem naturally suited for genetic algorithms. Ge-netic algorithms and genetic programming have been used to evolve biddingstrategies that maximize profit for individual hourly spot markets [2, 14].Representing the agent’s strategy in a easy-to-use and easy-to-evolve formatis sometimes the most difficult aspect of GA application. Fixed binary stringsmay be the simplest method of encoding a problem. Finite state ma-chines/automata offer a more powerful means of encoding the strategies, butmight be rather large to encode an independent response for each scenario


encountered. The authors use GP-Automata to circumvent this problem. Wetest the evolved bidding strategies are tested by repeatedly using them insimulated trades against competing strategies. Before we present the basics ofGP-Automata, here is a brief introduction of genetic programming.

9.2 The Basics of Genetic Programming

A sub-class of genetic algorithms, genetic programming (GP) is a newdiscipline attributed to John Koza [15, 16]. Although not for the electricitymarket, Andrews and Prager published research indicating that GP works forrepresenting simple double auction market strategies [19]. The evolving datastructures in GP are “parse trees” which allow complex relations to be de-scribed. Genetic programs (GPs) contain nodes and branches, with branchesconnecting the nodes. Nodes can be either operational nodes, having argu-ments and performing operations involving those arguments, or terminalnodes. Figure 12 provides examples of randomly generated GP-trees. The treeon the right side of the figure would return the average of five plus the aver-age buy bid from the previous round of bidding. The tree on the left wouldmultiply 10 by the absolute value of the high buy bid, and then (inefficiently)take the absolute value of the result.

The designer specifies the set of valid operators and terminals suitable tothe problem being investigated. For instance, in developing bidding strategies,suitable operators and terminals might be those described in Table 2. In de-signing GPs for the GP-Automata, it is desirable to give the trees an opportu-nity to return numbers in the range of competitive bids.


Valid GP trees are initialized randomly and then evolved in a standardgenetic algorithm (as described in the previous section) with the followingmodifications. The crossover of two parents involves randomly selecting anode from each parent and swapping the sub-trees rooted at those nodes. Mu-tation involves randomly selecting a node in the candidate child and throwingaway its sub-tree. In its place a new sub-tree is generated randomly.

9.3 The Basics of GP-Automata

GP-Automata are a combination of finite state automata and GP. Theywere first described as such by Ashlock [17] and were used by Ashlock andRichter [18]. The typical finite state automaton specifies an action and “nextstate” transition for a given input or inputs. With only one or two binary in-puts to work with, it can be fairly simple to develop a finite state diagram tocover the possible input/output relations. When the number of inputs is largethe task is much harder. The number of transitions needed to cover all possi-ble combinations of inputs grows exponentially (e.g., 10 inputs, each havingfive possible values would require transitions). This is where geneticprogramming comes in. The GP-trees perform bandwidth compression for theGP-Automata by selecting which inputs to consider and performing computa-tions involving these inputs.

The four-state GP-Automaton in Figure 12 begins by bidding the numberin the “Initial Action” field (in this case, 24). The “Initial State” tells us whichstate is used next (in this case, 2). Coupled with each of these states is a GP-


tree, termed a “Decider.” When executed, the decider returns a value in thevalid range of bidding. Following the decider evaluation, one of the followingtwo things will happen: (a) if the value is even after truncation, the actionlisted under “IF EVEN” is taken and the current state becomes the one listedunder the “IF EVEN” next state; or (b) if the returned value is odd after trun-cation, then the action and next state listed under “IF ODD” is used. The “Ac-tion” is the number listed in the action field of the automaton, with twoexceptions. The first exception is the “U” that indicates that the value re-turned by the decider should be taken directly as the action. The second ex-ception is a “*” indicating further computation is necessary – the GP-automaton immediately moves to the next state. This gives rise to the possi-bility of complex (multi-state) computation. To prevent infinite loops, after anexternally specified maximum number of *s have been processed, an action isselected at random from the valid actions.

A population of GP-Automata bidding strategies evolves in a GA. Fitnessis dependent on the goal of the strategies. Children/offspring are producedusing crossover and mutation. Crossover for the GP-Automata involves se-lecting (with a uniform probability) a crossover point ranging from zero to thenumber of states. Then parentl’s states from zero to the crossover point arecopied to childl and parent2’s states are copied to child2. Following thecrossover point, childl gets parent2’s state information and child2 gets par-entl’s state information (including the associated decider). Before replacingless fit members of the population, each child is subjected to mutation. Muta-tion may be standard GA mutation that selects a state or action at random andreplaces it with a valid entry. Other forms of mutation are acceptable as well.The goal is to introduce new combinations of genetic material into thepopulation.


9.4 Auction Bidding with the GP-Automata Strategies

While they are evolving in the genetic algorithm, each GP-Automaton’sstrategy competes against several other GP-Automata every generation (seeFigure 14). This helps to ensure that the resulting GP-Automata strategies willbe robust. Based on the competition, some fitness measure (e.g., the expectedamount of profit that results from using the strategies to win contracts) is as-signed to each GP-Automaton. Through the natural selection process, thepopulation evolves, finding strategies that are more likely to achieve goodfitness.

State information is supplied to the GP-Automata via the terminals. TheGP-trees use both the state information accessed by the terminals as well asconstants in the valid bidding range. Bids are taken from the action cell of theautomata, except in the cases where the action is listed as a * or a U, as de-scribed previously. The bids are submitted, along with the bids from the com-peting sellers and buyers, to the auctioneer for evaluation. The bids and offersare matched and a would-be price is reported, completing one cycle of theauction. The cycles continue until price discovery occurs or until some maxi-mum number of cycles (maxcycles) has passed. There is a maxcycles parame-ter, which is selected uniformly over a range to prevent the strategies from


falling in a local optima where the strategies only work well when the numberof cycles never changes over the trials in a given generation.

An evolved GP-Automaton contains trained rules, which may be quitecomplex. These rules may be used directly in a real auction just as they wereused in the simulated auction during evolution.

10. SUMMARY

The GENCO’s business is still one of generating electricity, and it mustultimately determine a profitable schedule to operate its generating units.Thus, the UC algorithm will continue to be an important tool in the evolvingindustry. While the ICA may minimize total costs when matching bids, theGENCO must maximize its profit. GENCOs must make decisions based onmarket projections. Customers and demand may no longer be guaranteed, butbilateral and multilateral forward contracts will ensure that the GENCOknows much of its load ahead of time. Accurate forecasts of the quantity de-manded and prices are crucial when solving the UC problem. If a GENCO’smarket projections are incorrect, the UC schedule may no longer be optimal.Flexible schedules and bidding strategies are important. With huge potentiallosses/profit at stake, UC schedules should be tested before use. Intelligentagents with evolvable strategies provide realistic competitive behavior andare thus ideal for robustness testing. GAs have demonstrated an ability tolearn and to build and adapt UC and bidding strategies for given scenarios.

REFERENCES

1.

2.

3.

4.

5.

6.

7.

J. Kumar and G. Shebté. “Framework for Energy Brokerage System with Reserve Marginand Transmission Losses.” In Proc. 1996 IEEE/PES Winter Meeting, 96 WM 190-9PWRS, NY: IEEE.C. Richter and G. Sheblé. “Genetic Algorithm Evolution of Utility Bidding Strategies forthe Competitive Marketplace.” In Proc. 1997 IEEE/PES Summer Meeting, PE-752-PWRS-1-05-1997. New York: IEEE.G. Sheblé. “Electric energy in a fully evolved marketplace.” Paper presented at the 1994North American Power Symposium, Kansas State University, KS, 1994.G. Sheblé. “Priced based operation in an auction market structure.” Paper presented at the1996 IEEE/PES Winter Meeting. Baltimore, MD, 1996.C. Richter. Profiting from Competition: Financial Tools for Competitive Electric Genera-tion Companies. Ph.D. dissertation, Iowa State University, Ames, IA, 1998.G. Sheblé and G. Fahd. Unit commitment literature synopsis. IEEE Trans. Power Syst.,9(1): 128-135, 1994.A. Wood and B. Wollenberg. Power Generation, Operation, and Control. New York:John Wiley & Sons, 1996.


8.

9.

10.

11.

12.

13.

14.

15.16.17.

18.

19.

G. Sheblé. Unit Commitment for Operations. Ph.D. Dissertation, Virginia PolytechnicInstitute and State University, 1985.S. Takriti, B. Krasenbrink, and L.S.-Y. Wu. “Incorporating Fuel Constraints and Electric-ity Spot Prices into the Stochastic Unit Commitment Problem,” IBM Research Report: RC21066, Mathematical Sciences Department, T.J. Watson Research Center, YorktownHeights, NY, 1997.C. Richter and G. Sheblé. “A Price-Based Unit Commitment GA for Uncertain Price andDemand Forecasts.” In Proc. 1998 North American Power Symposium, 1998.S. Kondragunta. Genetic algorithm unit commitment program, M.S. Thesis, Iowa StateUniversity, Ames, IA, 1997.S. A. Kazarlis, A. G. Bakirtzis, and V. Petridis. “A Genetic Algorithm Solution to the UnitCommitment Problem.” In Proc. 1995 IEEE/PES Winter Meeting 152-9 PWRS, NewYork: IEEE, 1995.T. Maifeld and G. Sheblé. Genetic-based unit commitment. IEEE Trans. Power Syst.,11(3): 1359, 1996.C. Richter, D. Ashlock, and G. Sheblé. “Effects of Tree Size and State Number on GP-Automata Bidding Strategies.” In Proc. 1998 Conference on Genetic Programming,Denver, CO: Morgan Kaufmann, 1998.J. Koza. Genetic Programming. Cambridge, Massachusetts: The MIT Press, 1992.J. Koza. Genetic Programming II. Cambridge, Massachusetts: The MIT Press,1994.D. Ashlock. “GP-automata for Dividing the Dollar.” Mathematics Department, Iowa StateUniversity, Ames, IA 1995.D. Ashlock and C. Richter. “The Effects of Splitting Populations on Bidding Strategies.”In Proc. 1997 Conference on Genetic Programming, Denver, CO: Morgan Kaufmann,1997.M. Andrews and R. Prager. “Genetic programming for the acquisition of double auctionmarket strategies.” In Advances in Genetic Programming, K. Kinnear Jr., ed. Cambridge,MA: The MIT Press, 1994.

Chapter 12

AN EQUIVALENCING TECHNIQUE FORSOLVING THE LARGE-SCALE THERMALUNIT COMMITMENT PROBLEM

Subir SenPower Grid Corporation of India, Ltd.

D.P. KothariIndian Institute of Technology at Delhi

Abstract: This chapter presents a new efficient solution approach for solving the unitcommitment schedule of thermal generation units of a realistic large scale powersystem. We base the approach on cardinality reduction by the generator equiva-lencing concept. This concept reduces the number of units in the large-scalepower system to the lowest possible number based on the units’ fuel/generationcost and other physical characteristics, such as minimum up and down time, etc.with units having similar (almost the same) characteristics form one group. Thereduced system consists of only each group of representative units and is firstsolved by the modified dynamic programming technique (one of the new solu-tion methods developed by the authors). Another option is to use any of the stan-dard unit commitment solution techniques. We obtain the overall solution to theoriginal unit commitment problem of the entire system by un-crunching thesolved reduced system based on certain rules. This chapter also presents test re-sults for real-life systems of up to 79 units and comparisons with results obtainedusing Lagrangian relaxation and truncated dynamic programming (DP-TC).

1. INTRODUCTION

One of the most important problems in operational scheduling of electricalpower generation is the unit commitment (UC) problem. It involves determin-ing the start-up and shut-down schedules of thermal units to be used to meetforecasted demand over a future short term (24-168 hour) period. The objectiveis to minimize total production cost while observing a large set of operatingconstraints. The unit commitment problem (UCP) is a complex mathematical


optimisation problem having both integer and continuous variables. One ob-tains the exact solution to the problem by complete enumeration, which cannotbe applied to realistic power systems due to its excessive computation time re-quirements [1,2]. In solving the unit commitment problem of a large system,the main cause of difficulty is the involvement of large number of units forcommitment. The problem cannot be solved easily if all units are involved inthe search for the optimal solution, since computational facilities could be ex-hausted. Research efforts have concentrated, therefore, on efficient, sub-optimalUC algorithms which can be applied to realistic power systems and have rea-sonable storage and computation time requirements. The basic UC methodsreported in the literature can broadly be classified in six categories [3]:

Priority listDynamic programmingLagrangian relaxationAugmented Lagrangian relaxationBranch-and-BoundBenders decomposition

Since improved UC schedules may save the electric utilities substantial re-sources per year in production costs, the search for closer to optimal commit-ment schedules continues. Recent efforts include application of simulated an-nealing, expert systems, Hopfield neural networks and genetic algorithms tosolve the UCP. References [4,5] give a survey of various approaches and theirmerits and demerits in this field. Some of these methods achieved a reductionof the computation requirement for large power systems. Researchers have yetto obtain an optimal solution to the problem for such systems.

There have been some past attempts in other areas such as coal modeling[6] to reduce a large-scale system to a smaller system. This chapter proposes anew, efficient solution approach to the UCP of a large-scale power system. Theapproach is based on cardinality reduction by generator equivalencing (hereaf-ter called “equivalencing”), which reduces the number of units in the large-scale power system to the lowest possible number according to their similarfuel/generation cost characteristics and minimum up- and down-time character-istics. We first solve the reduced system using a modified dynamic program-ming technique [7] and then obtain an overall solution to the original unitcommitment problem of the entire system by un-crunching the reduced solvedsystem and using certain rules. The Appendix explains the modified dynamicprogramming (MDP) technique.

An Equivalencing Method 213

2. NOTATION

N : number of thermal generation unitsT : total scheduling period

load demand, in MWpower generation by nth unit, in MWminimum generation capacity limit for n-th unit

maximum generation capacity limit for n-th unit

cost of power generation by nth unit, in Rs/hour

start-up and shut-down cost for nth unit, in Rs/hour

system reserve requirement in time period tup-/down-time status of nth unit

unit on; unit off

duration of unit n on and off, in hour

minimum up- and down-time for nth unit, in hour

number of units in group inumber of groups in the system

power output of the equivalent system

minimum and maximum generation limit of group g

3. DESCRIPTION OF THE UNIT COMMITMENTPROBLEM

The objective of the UC problem is to minimize the total production costover the scheduling horizon. The total production cost consists of: fuel costs,start-up costs, and shut-down costs.

We calculate fuel costs by using unit heat rate and fuel price information.We express the start-up cost as a function of the number of hours the unit hasbeen down (exponential when cooling and linear when banking). The shut-down cost is given by fixed amount for each unit per shut down.

The following must be satisfied during the optimization process:(a) system power balance (demand plus loss plus exports)(b) system reserve requirement(c) unit initial condition(d) unit high and low MW limits (economic, operating)


(e)(f)(g)(h)(i)(j)

unit minimum up timeunit minimum down timeunit status restrictions (must-run, fixed-MW, unavailable, available)unit rate limitsunit start up rampsplant crew constraints

Constraints (a) and (b) concern all the units of the system and are calledsystem, or coupling, constraints. For multi-area unit commitment, the systemconstraints must be modified to take into account the interchange schedules andthe tie-line limitations. In general, the system constraints must take into accountpossible transmission bottlenecks in the allocation of the demand and the re-serves to the generating units.

Constraints (c) through (i) concern individual units and are called localconstraints. Plant crew constraints can also be classified along with local con-straints, but they involve all the units in a plant.

4. THE EQUIVALENCING METHOD

The “equivalencing” method for solving large scale thermal unit commit-ment problem consider the particular reference to units generation cost (input-output characteristic), minimum up and down time, and ramp rate criterion. Alarge-scale power system consists of a large number of generation units, inwhich all the units are not of same size and similar characteristics. Yet there aremany clusters of units throughout the entire power system which have almostsimilar cost coefficients and other physical characteristics. Therefore, units inthe large scale system are re-grouped/ partitioned into various groups based ontheir fuel cost, having cost coefficients of almost similar value (maximum 1%variation), and other physical characteristics, such as same capacity, minimumup and down time, ramp rate, etc. so that identical/similar characteristic unitsform one group. Each such group is then represented by any one unit of thisgroup and is called a representative unit. Consequently, an equivalent smallersystem of units consisting only of representative units is generated. The basicconcept of this method is that the large-scale power system is represented by anequivalent smaller system with a smaller number of generation units, such thatthe unit commitment problem of the equivalent system would be easier to han-dle and solve than that of the original large size system. Once the solution ofthe unit commitment problem of the equivalent small system is obtained, thenthe solution to the original unit commitment problem can be determinedaccordingly by un-crunching the equivalent problem solution.


5. PROBLEM FORMULATION

The function to be minimized for the unit commitment problem can be ex-pressed in mathematical form as follows:

subject to the following major constraints:

i) demand constraint:

ii) capacity constraint:

iii) unit’s minimum up- and down-time constraints:

iv) unit’s generation capacity constraint:

Formulate the large-scale unit commitment problem by using the “equiva-lencing” method as given below:

v) Generate the equivalent system (based on the generator cost and other physi-cal characteristics) with the lowest possible number of representative unitsonly.

vi) Determine the number of groups in the system and the number of units in agroup, such that identical units form one group. The total number of unitsin the system would be


vii) Represent each group by one representative unit to form equivalent system.The total output of the equivalent system would be:

viii) Maximum and minimum capacity limit of each group (i.e., representativeunit) would be

ix) Minimum up and down time, ramp rate, etc. of the representative unit of agroup is set as any individual unit’s characteristics of that group.

5.1 Solution Technique

The large-scale unit commitment problem is solved by performing thecommitment (0-1 status of each unit) of the equivalent system using the modi-fied dynamic programming technique [7] developed by the authors. It can besolved, however, by any other standard unit commitment solution technique. If,during the solution of the equivalent system, the minimum output of a particu-lar representative unit of a group is less than the equivalent minimum run levelof that group, perform the sub-unit commitment of the individual units of thatparticular group. Once the reduced system is solved, the units in each group inthe equivalent system are treated according to the status of its representativeunit, based on the following strategy:

If the representative unit of group i at interval t is “OFF,” then all unitsof the group should be off.If the representative unit of group i at interval t is “ON” and operated atits equivalent minimum/maximum output, all the units in the groupshould be operated at its individual minimum/maximum output.If the representative unit of group i at interval t is “ON” and operated atsome percentage of its maximum/minimum output, carry out furthercommitment scheduling within the loop among the units in that group.

Finally, obtain the total cost over the commitment period using Equation (1).


5.2 The New Algorithm

Step 1:Step 2:

Step 3:Step 4:

Read input data, i.e., unit characteristics, load demand profile, etc.Categorize the units having similar/identical fuel costs, minimum upand down time, and ramp rate characteristics into different groups.Generate the equivalent smaller system of the representative units.Perform the commitment schedule of the equivalent system by modi-fied dynamic programming or by any standard technique with certainrules, i.e., if during the solution of the equivalent system, the minimumoutput of a particular representative unit of a group comes out less thanthe equivalent minimum run level of that group, perform the sub-unitcommitment of the individual units of the group.

Step 5: Generate the unit commitment schedule of the original system using therules as specified below:

if representative unit is “OFF,” all units in that group would be off.if representative unit is “ON,” and generate up to its equivalentmaximum or minimum capacity output, all units in that group wouldgenerate their minimum/ maximum output.if the representative unit is “ON” and generate a percentage of itsmaximum/minimum capacity output, schedule the units in thatgroup using modified dynamic programming to generate the optimalscheduling of the units in that group.

Step 6: Calculate the total cost for scheduling the normal way by Equation (1).Figure 1 presents the outline of the new algorithm for solving large scale

unit commitment problem.

6. TEST SYSTEMS AND RESULTS

The authors have tested the new “equivalencing” method for solving shortterm thermal unit commitment problem solution for two large power systemsconsisting of the 26 and 79 thermal units, respectively, with a particular dailyload demand profile [8,9]. We base the system spinning reserve on the capacityof the largest on-line unit. The 79-unit system represents Eastern Regional gridof the Indian power system. We implemented the new algorithm in FORTRAN77 code on a PC-486 computer.

In the equivalent system representation, 8 units represent the complete 26-unit system, and 19 units represent the 79-unit system. Therefore, we can seethat the size of the problem has been reduced drastically. The comparison of theproposed method with other traditional techniques like Lagrangian relaxationand truncated dynamic programming [10,11] are presented in Tables 1 and 2for 26-unit and 79-unit systems, respectively.


In the Lagrangian relaxation technique, form the Lagrangian dual by ap-pending the relaxed constraints to the primal objective. Find the minimum ofthe primal objective by maximizing the dual objective. Update the Lagrangianmultipliers using a sub-gradient method that drives the solution towards feasi-bility [12]. In the DP-TC technique, the ordering of the units are made based onan average full load cost of each unit, and the size of the search range of fourunits (16 combinations) has been considered.

7. DISCUSSION

The results given in Tables 1 and 2 clearly show that the new equivalencingmethod for solving the large scale unit commitment problem is able to providesolutions very close to the best solutions found by other approaches. The varia-tion in cost is within 0.05% and 0.19% respectively for the 26-unit systemcompared to truncated dynamic programming and the Lagrangian relaxationapproach, respectively. For the 79-unit system, the cost obtained using the new


method is 0.7% more compared to the DP-TC-based method while it is 1.27%less compared to Lagrangian relaxation-based method.

The solution time in the new approach, however, is smaller as compared toother techniques. This is one of the main requirements to solve the unit com-mitment problem for a large-scale power system. In addition, the computerspace requirement for the new method is comparatively less than other standardmethods of UCP solution. Also, the new method simplifies the unit commit-ment problem in terms of dimensionality of the problem. As a result, the newmethod of solving the unit commitment problem for large-scale systems turnsout to be a promising one.

We tested the algorithm for one sector of Indian power system, however, itis not presently implemented by Indian power utility. In fact, due to restrictionson the available generation capacity, presently India uses just merit orderscheduling. Further, the algorithm neglects to consider the transmission con-straint. If it must be considered, however, then the concept presented in thiswork would have to be applied to a cluster of units which are relatively close toeach other, and then the algorithm would not violate the transmission con-straint.

8. CONCLUSION

In this chapter, we developed the “Equivalencing” method based on cardi-nality reduction by generator equivalencing for estimating the short-term ther-mal unit commitment schedule for large scale power systems. In this method,we reduce a large-scale power system to an equivalent reasonably small-sizedpower system. We first solve the equivalent system by using a modified dy-namic programming approach (or it can be solved by any other standard unitcommitment solution technique). Finally, we return to the original system tocalculate the unit commitment schedule cost and determine the units’ finalstatus. This method simplifies the unit commitment problem in terms of dimen-sionality, and consequently, computer space as well as the computer time re-quired for solution can be reduced remarkably with an acceptable overall solu-tion of the large-scale power system. We have tested the model on a practicallarge and complex Indian power system. The results obtained from the abovemodel are highly impressive and encouraging for implementation of real-lifelarge scale power systems having large numbers of units. In short, the proposedunit commitment model yields a promising approach to solve the short-termthermal unit commitment problem and offers good performance which providesfast solutions for large-scale power systems.


ACKNOWLEDGEMENTS

The chapter’s presentation was greatly improved by the comments and sug-gestions of the editors and three anonymous referees. The authors also wish tothank B. Hobbs for suggestions, which improved many aspects of this chapter.

REFERENCES

1.

2.

3.

4.

5.

6.

7.

8.

9.

10.

11.

12.

A.J. Wood and B.F. Wollenberg. Power Generation Operation and Control. New York:John Wiley, 1996.I.J. Nagrath and D.P. Kothari. Power System Engineering. New Delhi: Tata McGraw-Hill,1994.S.A. Kazarils, A.G. Bakirtzis and V. Petridis. A genetic algorithm solution to the unit com-mitment problem. IEEE Trans. Power Syst., 11(1): 83-92, 1996.G.B. Sheble and G.N. Fahd. Unit commitment literature synopsis. IEEE Trans. Power Syst.,9(1): 128-135, 1994.S. Sen and D.P. Kothari. Optimal thermal generating unit commitment: A review. Elec.Power Energy Syst., 20(7): 443-451, 1998.S. Bullard and R.E. Wiggans. Intelligent data compression in a coal model. Oper. Res., 36:521-531, 1988.D.P. Kothari and S. Sen. Optimal thermal generating unit commitment – a novel approach.In Proc. International Seminar on Modelling & Simulation, Australia, 331-336, 1997.C. Wang and S.M. Shahidehpour. Effects of ramp-rate limits on unit commitment and eco-nomic dispatch. IEEE Trans. Power Syst., 8(3): 1341-1350, 1993.

S. Sen and D.P. Kothari. Evaluation of benefit of inter-area energy exchange of Indian powersystem based on multi area unit commitment approach. Elec. Machines Power Syst., 26(8):801-813, 1998.S.J. Wang, S.M. Shahidehpour, D.S. Kirschen, and G.D. Irisarri. Short-term generationscheduling with transmission and environmental constraints using an augmented Lagrangianrelaxation. IEEE Trans. Power Syst., 10(3): 1294-1301, 1995.C.K. Pang, G.B. Sheble and F. Albuyeh. Evaluation of dynamic programming-based meth-ods and multiple area representation for thermal unit commitments. IEEE Trans. PowerSyst. PAS-100, 3: 1212-1218, 1993.W.L. Peterson and S.R. Brammer. A capacity-based Lagrangian relaxation unit commitmentwith ramp rate constraints. IEEE Trans. Power Syst., 10(2): 1077-1084, 1995.

APPENDIX: Modified DP Technique for Solving UCP

Theory

Formation of Unit Commitment Schedule Table

Let the cost function of the n-th unit at a plant be [2]:


For minimization of cost, using well-known dynamic programming, a simplerecursive expression can be obtained as given below:

Where is the cost of generation at the n-th unit with as power dispatch.is the minimum cost of generating by the remaining (N-

1) units of the plant.Re-writing equation (10) as

where, For minimum of

or

where

From equations (11) and (12), obtain the composite cost function of units#1 and #2 for a demand of as shown below:

where


Equation (13) represents the most economical cost of the two units for totalgeneration allocation over two units with as generation allocation on the sec-ond unit.

In general form,

where

Equation (14) can also be expressed as

We obtain the critical value of generator loading/generation dispatch byequating (14) and (15) above, in which the combination of n-number of unitcommitment will be economical as compared to number of units combina-tion. Therefore, a “loading range” of operation can be obtained for unitsfor which the cost is minimum as compared to n-units combination. In thischapter, we use this principle to prepare a sequential economic order of unitsfor commitment and the economic “loading range” of operation for those com-binations.

Generation Limits Constraints Fixing

To satisfy the maximum and minimum generation limits during generationscheduling, we apply the following limits constraint fixing technique.

i) Solve unconstrained generation dispatch problem.

ii) If there is no limit violation, the solution is optimal, otherwise computeas follows:


for all generation limit upper-bound violation.

for all generation limit lower bound violation.

(iii)If ¨ fix all upper bound violations units to the upper limits, iffix all lower bound violations units to the lower limits, otherwise

fix both upper and lower bound violations to upper and lower limits re-spectively.

iv) Determine the new demand which is the original demand minus the sum offixed generation levels.

v) Set aside fixed generation level units for scheduling of new demand.

vi) Reschedule new demand among remaining committed units and return tostep (ii).

UP- and DOWN-Time Constraint Fixing

The purpose of this postprocessor algorithm (rules) is to detect the violationof the minimum up- and down-time constraint of units. Minimum up and downtimes are particularly difficult to model and cannot be incorporated directly inthe main program routine. So to detect a violation, we have established certainrules:

Check Constraint (up time/down time violated)Condition (unit on/off)Condition (unit on/off time < min. up/down time)

If a unit is committed, then de-committed, and the duration between on andoff state is less than its minimum up time, then the unit up time is violated.Therefore, the unit is charged as if it was on stand-by for those hours. In theoriginal unit commitment schedule, the stand-by hours are set to 1 (unit “on”)instead of 0 (unit “off”). The additional number of hours needed to satisfy theminimum up-time constraint are multiplied by the banking cost and then addedto the UC schedule cost.

On the other hand, if a unit is de-committed, then committed, and the dura-tion between off and on states is less than its minimum down time, then the unitdown time is violated. Therefore, the unit is charged as if it was on stand-by forthe additional number of hours needed to satisfy the constraint. The originalunit commitment schedule has those hours set to 1 instead of 0. UC schedule is“costed” as if it was banking for those hours.


Algorithm

Step 1:Step 2:

Step 3:

Step 4:

Step 5:

Step 6:Step 7:

Read number of units, unit parameters, hourly demands, etc.Select any unit as a first unit from the list of the total number of unitsand using equations (14) and (15), the composite cost function of thetwo units, taking all units one by one is formed.We find the critical loading value for the combination of two units byequating the cost function of the first unit with the composite costfunction of two units or equivalent two units taking sequentially. Theaccepted combination of two units is that having minimum critical“loading” valueRepeat Steps 1 and 2, taking all units sequentially as the first unit andfind the N-number of minimum critical loading are found from eachcombination. Among these, the maximum of minimum critical loading

is the best combination and sequential order of the first twounits. If is higher than the sum total of maximum capacity of twoor equivalent two units, then is reset to the sum of the maximumcapacity. For the equivalent two units combination case, the minimumloading will be the just slightly higher than the maximum loading ofthe previous combination.The sequence of the two units formed in Step 3 can be used as a basiccombination as the equivalent first unit to search the third most eco-nomic unit. Repeat the procedure in Steps 1 to 3 to find the best com-bination of three units and the corresponding third unit in sequence.The algorithm uses the combination of three units as basic combina-tion to search the next economic combination of four units. In thisway, for any addition of units in basic combination of j units, followthe procedure mentioned in Steps 1 to 3 until all units are consideredto form a sequential order/ combinations of units and a correspondingorder of combination and loading range of operation.Execute limits constraints fixing, up- and down-time violation rules.Based on the general unit commitment schedule table and Step 6, per-form the unit commitment schedule for the particular load profile.

Chapter 13

STRATEGIC UNIT COMMITMENT FOR GEN-ERATION IN DEREGULATED ELECTRICITYMARKETS

A. Baíllo, M. Ventosa, A. Ramos, M. RivierUniversidad Pontificia Comillas de Madrid.

A. CansecoIBERDROLA S.A.

Abstract: In this chapter we address some of the new short-term problems that are facedby a generation company in a deregulated electricity market, and we propose adecision procedure to address them. Additionally, we propose a strategic unitcommitment model, which deals with the weekly operation of the firm’s gener-ating facilities. In it we combine traditional cost-evaluation techniques and tech-nical constraints that grant a feasible schedule with new market-modeling equa-tions. We suggest strategic constraints that allow the accomplishment of thefirm’s medium-term objectives. We have formulated the model as a mixed-integer-programming problem and solved it by means of a commercial algo-rithm, instead of using the traditional Lagrangian relaxation approach. Results ofthe application of the method to a numerical example are presented. The proce-dure is a simplified version of one of several tools currently being used by aleading Spanish generation company, Iberdrola, for the weekly operation of itsgeneration assets in the Spanish wholesale electricity market.

1. INTRODUCTION

The electricity industry is in the midst of a profound restructuring processin an increasing number of countries. These changes are intended to bringabout competition in some of the electricity business activities, so as to pro-mote a higher level of efficiency in the provision of electric services. Al-though the details of the deregulated marketplace may vary from one case to


another, it is generally assumed that electricity should be traded in a similarfashion to other energy commodities.

Generation companies have traditionally been subject to regulatory poli-cies that guaranteed the full recovery of their costs. In the new framework,electricity generation is a deregulated activity and firms have to compete tosell the electric services provided by their facilities. Therefore, generationcompanies are now fully responsible not only for the efficient operation oftheir units, but also for selling their output.

The particular design of the marketplace where the electric services aretraded is of great importance. Two mechanisms coexist in recently deregu-lated electric industries throughout the world. The most common is a central-ized power exchange based on auctions where the price for each electric ser-vice during a certain time period is determined by the intersection of the ag-gregate supply and demand bid curves. Additionally, bilateral trading betweenbuyers and sellers is usually permitted.

The fact that generation companies’ revenues depend on the market forcesleads to a higher degree of uncertainty and risk. New procedures and toolsdevoted to the maximization of the firm’s profit, taking into account the dif-ferent market mechanisms available and keeping an upper bound on the de-gree of risk exposure, are needed. However, introducing in a model all thecomplexity of electricity trading does not necessarily result in a better under-standing of the environment. A gradual implementation of solutions for theopen issues should lead to a deeper and more solid knowledge of their impli-cations.

In this chapter, we address some of the new problems that are faced by ageneration company in the short term (one day to a week). Assuming that themajority of energy is traded in an energy exchange based on 24 day-aheadhourly uniform-price auctions, we outline a decision procedure. One of thetools incorporated in this procedure is a strategic unit-commitment model,which deals with the weekly operation of the firm’s generating facilities. Itincludes traditional cost evaluation techniques and takes into account techni-cal constraints that grant a feasible schedule. Its major contribution lies in theinclusion of a set of market-modeling equations intended to express the rela-tionship between the firm’s output and the revenue obtained in the successiveauctions. Additionally, we show the need to consider a set of strategic con-straints so as to direct the solution towards the firm’s medium-term objec-tives. We have formulated the model as a MIP optimization problem andsolved it by means of a commercial algorithm instead of using the traditionalLagrangian relaxation approach. We present results of the application of themethod to a numerical example.

Strategic Unit Commitment for Generation Companies 229

2. DECISION PROCEDURE

In the new competitive framework, a generation company not only has todetermine how to operate its generation facilities in the most efficient manner,but also must decide on the amount of each electric service that should besupplied, at which moment it should be produced, at what price it should besold, and with which units it should be provided. These new challenges re-quire decision procedures and tools specifically oriented to the maximizationof the firm’s profit and the hedging of its risk. The short-term decision proce-dure outlined in this section is represented in Figure 1.

2.1 Medium Term Guidelines

As in the past, generation firms have to decompose the problem of plan-ning the operation of their units into different time scopes to make it tractable.The traditional hierarchy used to classify the decision support tools into long-,medium-, and short-term models is still completely in force. The results ob-tained from a model with a longer time scope must affect all the inferior mod-els. Consequently, in a short-term decision procedure the generation firm’smedium-term goals play an important role.

In its medium term analysis (one month to a year) the generation firm stillhas to make traditional decisions related to the maintenance of the groups, theannual management of water reserves, or the fuel consumption of thermalplants. New issues include determining the expected equilibrium between


generation companies and estimating the prices that are likely to appear dur-ing the following months.

Two very important results of the medium term, which the firm must con-sider in the short term (one day to a week), are the water value assigned to thewater reserves and the medium-term market position that allows an equilib-rium with the rest of generation firms.

Firms must assign a certain value to their stored energy. Otherwise, short-term tools will interpret that producing with these reserves has no associatedcosts and the result will be that hydro units must permanently produce at theirmaximum capacity. Bushnell [1] analyzed the strategic management of hydroresources in a competitive environment and established a relationship be-tween the value of the available hydro energy and the marginal cost of ther-mal units. Scott and Read [2] used Dual Dynamic Programming methods tobuild up a weekly curve giving the optimal output for a certain water value.

Similarly, if a short-term model is not aware of the market position de-fended in the medium term by the firm, it will tend to follow blindly the short-term signals transmitted by the competitors through their supply curves. Wewill analyze in detail how the slope of the bid curve presented both by othergenerators and by the demand exerts an influence on the results given by ashort-term model that tries to maximize the short-term profit of a generationfirm. This influence must be limited and controlled so that the firm is able tokeep a steady pace towards its medium-term objectives, which include de-fending its market position. A firm may lose its market position by systemati-cally producing less than the market share it should have according to the costand size of its generating assets relative to those of its competitors. Severalapproaches have been proposed for approximating the medium-term equilib-rium that a number of generating firms should reach in a competitive electric-ity market. Some of these consider the generation firms as Cournot agentswhose decision variables are the quantities produced in each time period.Otero-Novas et al. [3] developed a simulation platform to evaluate the me-dium-term evolution of the Spanish electricity market. Ramos et al. [4] andVentosa et al. [5] successfully combined a detailed representation of the gen-eration operational costs with the Cournot-equilibrium conditions in a cost-minimization framework. Rivier et al. [6] used the complementarity problemapproach with satisfying results to determine the expected medium-term equi-librium reached in the recently deregulated Spanish electricity industry.Hobbs [7] and Wei and Smeers [8] extended the usage of the complementarityproblem to predict the outcome of an electricity market with significanttransmission constraints. An important feature of all these models is that theyreflect the objective of profit maximization of all generation firms so that themedium-term market equilibrium is accurately represented.


2.2 Forecasting Techniques

A major challenge for a generation company in the new framework is thedevelopment of forecasting techniques devoted to the estimation of the oppo-nents’ expected behavior. Information concerning the amount of each servicetraded at different prices is extremely relevant. These data can be exploited toestimate future competitors’ hourly offer curves. The expected level of de-mand is also a decisive variable, whereas the elasticity of the demand curve isa parameter whose importance is expected to grow as the agents gain experi-ence in the new regulatory scheme.

2.3 Short-term Generation Scheduling

Given a certain scenario for the competitors’ offers and the demand bidsfor the different electric services markets, the firm has to determine the en-ergy that it should offer as well as the capacity that should be reserved forancillary services. When the firm’s revenue is based on administrative andcentralized decisions, the amount of energy that each unit must produce andthe precise moments when this unit should start up and shut down are tackledwith weekly cost-minimization unit-commitment models. However, deregula-tion has shifted focus from obligation of supply and cost-minimization tocompetition and profit maximization. Therefore, the competitive environmentrequires unit-commitment models that take into account the expected priceseries for each of the electric services. Moreover, if the firm has a significantmarket share, the response of prices to the output of the generation firm mustbe considered [9].

Unit-commitment models provide a commitment schedule. However, theyusually give only approximate generation levels for the generators [10]. Oncethe commitment decisions are taken, the best hourly output of each individualgenerator can be determined with a daily model. This model will include amore detailed representation of the generating equipment, such as an accuratedescription of hydro generation resources.

2.4 Strategic bidding

The final stage of the generation firm’s decision procedure is the designof the hourly offer curves that must be submitted to the different auctions. Theresults of the previous short-term decision support tools include the firm’sexpected hourly optimal productions together with expected hourly prices.


However, the competitors’ sell offers as well as the demand-side buy bids areestimated with a certain degree of uncertainty, since the short-term volatilityof electricity prices is higher than those of other energy commodities. In thiscontext, instead of sticking to a single hourly quantity, the firm can submit anhourly offer curve for each of the electric services. The bigger and moreflexible its generation portfolio is, the wider the variety of hourly offer curvesthe firm can design. In this chapter, we assume that offer curves consist of aset of quantity-price pairs and that no additional information, such as fixed-costs, is submitted. Given a probability distribution for the last accepted bid(Figure 2), the generation firm can derive the offer curve that maximizes itsobjective function. This can be a combination of the expected short-termprofit and other targets such as market share goals.

3. MODEL DESCRIPTION

In the previous section, we suggested a short-term decision procedure fora generation firm participating in a day-ahead auction-based energy exchange.The unit-commitment model that we will develop henceforth, however, is asimplified part of the complex combination of tools that a generation firmshould use to face the short-term problems that will arise in the new competi-tive framework. We will only consider the day-ahead hourly energy market.Our aim is to gain insight into certain modeling features such as the manage-ment of hydro reserves or the influence of strategic constraints.

3.1 Objective Function

A major difference between the strategic unit commitment model and atraditional unit commitment model lies in the objective function. The aim of a


traditional model is the minimization of the overall system costs. In spite oftheir complexity, cost functions of thermal units have frequently been mod-eled as convex piecewise linear functions. In contrast, the strategic unit com-mitment model guides a generation firm to its maximum profit objective,which is a non-linear (and frequently non-convex) function of the firm’s en-ergy output.

3.2 Generation System Representation

Thermal units’ costs representation includes fuel costs (which, for sim-plicity, can be defined as a convex piecewise linear function of the unit’s out-put), start-up costs and shut-down costs. Thermal units’ most relevant con-straints are the minimum stable load, maximum output, and upwards anddownwards ramp limits.

Hydro units produce with nearly zero variable costs. Water reserves haveassociated opportunity costs, however, as they can be used to substitute ther-mal units. Therefore, we can define a cost function known as water value. Thewater value function gives the hydro energy that must be produced if the mar-ginal revenue of the firm exceeds a certain value. In the strategic unit-commitment model, we divide hydro reserves into several reserve levels andassign a different water value to each one of them. Depending on the marketcircumstances, the model will decide to use a certain amount of each one ofthese levels. To keep track of the contents of these reserve levels we will in-clude hourly reserve balance equations. Hydro units also have a minimum anda maximum power output.

The operation of pumped-storage power plants is subject to the same con-straints as regular hydro power plants, except that the water balance con-straints are modified to include the pumping mode of operation.

3.3 Market Representation

The strategies followed by the firm’s competitors are expressed by meansof their expected hourly offer curves (known also as hourly supply functions).Similarly, instead of using a fixed level of demand for each time period, en-ergy buyers submit hourly demand curves.


From the generation firm’s point of view, the competitors’ hourly offercurves and the hourly demand curve exert a very similar influence on thefirm’s profit (see Figure 3). If the firm increases its energy output, lowerprices will result. This is due to the combined effect of a decrease in the com-petitors’ output and an increase in the energy consumption. Therefore, to acertain extent, the firm is able to adjust its hourly revenue by varying its en-ergy output. In microeconomic theory, this is modeled by means of the resid-ual demand function. This gives the energy the firm is able to sell at eachprice. The firm should try to estimate this function for each hour.

Going a little further, a change in the firm’s production also modifies thecompetitors’ revenue. It must not be forgotten that a decrease in the firm’sproduction may or may not increase its profit, but it will surely benefit thecompetitors as they are able to produce more and at a higher price (Figure 4).

The objective function whose maximum is sought is the firm’s profit, de-fined as the difference between the obtained revenue and the incurred costs.Owing to the fact that the most powerful available solvers are those designed


for mixed integer linear problems, such as CPLEX and OSL, a linearizationprocedure for the firm’s revenue function is of great interest. An intuitivemethod is to divide the firm’s hourly revenue function into convex sectionsand approximate each one by a piecewise linear function. The slope obtainedfor each linear segment is the firm’s marginal revenue at the correspondingenergy output (Figure 5).

A group of consecutive segments with strictly decreasing marginal reve-nues defines a convex section in the revenue function. When we seek the op-timum we select a specific convex section by switching its binary variablefrom zero to one. Once we have chosen a convex section we fill its segmentswith continuous bounded variables. In other words, we obtain the hourlyrevenue by integrating the marginal-revenue function.

With this approach, we replace the set of constraints that represent thecompetitors’ generation facilities in a traditional unit commitment by a set ofhourly constraints, which define the firm’s hourly revenue as a function of itsenergy output. Hourly prices do not appear explicitly in our model.

3.4 Medium-term Guidelines

The results obtained from medium-term models add information to short-term decision-support tools. Good examples are the water value assigned tohydro reserves or the position that the firm must defend if it wants to maxi-mize its profit in the long run.

Our model decides the amount of hydro energy to be used depending onthe water value received from a hydrothermal co-ordination model. It alsodistributes this energy along the time scope of the model, trying to obtain auniform marginal revenue.

The total hourly energy production that the model suggests not only de-pends on the expected price and on the firm’s marginal production costs. Wealso consider the slope of the residual demand curve, which exerts a major


influence on the clearing price (Figure 6). If this curve is very steep and thefirm’s output is high (on-peak hours) the model will blindly tend to reduce thefirm’s production. This causes a rise of the energy price and an increase of thefirm’s profit. Another result is that competitors are able to produce more at ahigher price. Taking into account that the price of electricity usually behaveslike a mean-reverting process, if the firm gives up its position repeatedly dur-ing on-peak hours, competitors will increase their market shares and, in thelong run, prices will return to the original level.

An alternative is to define a set of hourly minimum-market-share con-straints. In this fashion, the short-term model maximises the firm’s short-termprofit while following the right medium-term strategy. We will analyze theinfluence of minimum-market-share constraints in the case study.

Many other constraints can be designed ad hoc to fulfill the firm’s re-quirements such as existing physical or financial contracts. In our model, theymust be expressed as linear equations. The strategic UC solution will be dif-ferent in each particular case. A great effort must be made to interpret cor-rectly the influence of each factor both in the short and long term.

4. MATHEMATICAL FORMULATION

4.1 Notation

In this section, we identify the symbols used in this chapter and classifythem according to their use. Table 1 shows the indices and sets considered,capitals being used for sets and lower-case for indices. Table 2 includes thedecision variables. Table 3 lists the auxiliary variables. Table 4 defines theinformation given to the model as fixed data.


4.2 Model formulation

We formulate the model as an MIP optimization problem. The objectivefunction to be maximized is the firm’s total profit for the scope of the model.We have classified operating constraints into thermal and hydro constraints.Additional market constraints model the behavior of the competitors and thedemand side in the day-ahead power market.


4.2.1 Objective Function

The objective function represents the firm’s profit defined as the differ-ence between the firm’s revenue and the firm’s operating costs for all loadlevels within the scope of the model:

4.2.2 Thermal Generation Constraints

Total thermal operating costs include fuel costs, O&M costs, start-upcosts and shut-down costs:

For each committed thermal unit, the maximum generation is less than themaximum available capacity, and the minimum generation is greater than theminimum stable load:

The hourly change in the output of each thermal unit is limited by theramp rates:

A logical relationship exists between the start-up, shut-down, and com-mitment variables:

Since the commitment decision variables are binary, both the start-up andthe shut-down decision variables can be continuous but must have upper andlower bounds:

4.2.3 Hydro Generation Constraints

The water reserves not used by the model have a value for the future:


Each unit has an upper and a lower limit for its power output:

The contents of the reservoirs depend on the energy produced or storedduring each time period and have upper and lower bounds:

4.2.4 Market Constraints

Each segment of the firm’s net hourly energy output is valued at a differ-ent marginal revenue. The sum of all the segments must equal the sum of thepower produced by thermal and hydro units minus the power consumed bypumped-storage units:

Each segment has an upper and a lower bound and the convex sectionsmust be chosen in order:


We calculate the total revenue by valuing the different segments of the netenergy output at their corresponding marginal revenues. In other words, therevenue is obtained by integrating the marginal revenue function:

The hourly price of energy is not obtained explicitly with this formula-tion. It must be calculated after the execution of the model. To do so, we sim-ply divide the firm’s hourly revenue by the firm’s hourly production.

4.2.5 Strategic Constraints

We define a set of hourly minimum-market-share constraints. In the nu-merical example, we investigate the influence of this constraint on hourlyprices and on the firm’s short-term benefit. In our formulation, we supposethat demand is perfectly inelastic. Consequently, the only variations of de-mand we allow are those introduced by pumping:

5. NUMERICAL EXAMPLE

The strategic unit commitment model has been implemented in GAMS[11]. A case study has been solved with the optimizer CPLEX 6.5.

5.1 Case Study

We include the results of the application of the model to a case study. Ouraim is to highlight the influence of the different modeling decisions previ-ously explained. Computational and convergence issues are secondary andshould be treated only when the researcher is sure that the model adequatelyrepresents the problem he is trying to solve. Consequently, we will only givethe essential information to define the case study. (The authors may be con-tacted for details if the reader wishes to reproduce the results.) The firm’sgenerating equipment is described in Table 5.


Hydro and reserves have been classified into the levels shown in Table 6according to the results of a hydrothermal coordination model.

We have estimated a piecewise linear residual demand function for eachhour. Figure 7 shows an example of a residual demand curve with its corre-sponding revenue function, formed by two convex sections. Each of thesesections has been divided into five segments. We have assigned a constantmarginal revenue to each segment.


In this example, we include a set of hourly minimum-market-share con-straints to obtain a generation schedule similar to the traditional one. If nostrategic constraints were used, the model would blindly follow all the short-term opportunities and the resulting operation would require extremely ineffi-cient dynamic performance of the generating units. The weekly problem isformed by 19446 equations, 24555 continuous variables and 4692 binaryvariables. It is solved in 89 seconds on a PC Pentium II 350 MHz 64 MB.

5.2 Results

In this section, we describe the results of the model when a minimum-market-share constraint of 29% is used. Table 7 states the differences betweenthe solution given as optimal by CPLEX and the first feasible solution.

The model decides the hourly power output for each ofthe firm’s generat-ing units. The hourly energy that each kind of unit should produce to achievethis hourly market share has been represented in Figure 8.


Each hourly energy output determines an hourly revenue by means of theestimated marginal revenues. Additionally, the production of that energyleads to an hourly cost. The difference gives the firm’s expected hourly profit.These three variables have been represented in Figure 9.

A subproduct of the problem is an estimation of the hourly price of en-ergy. We calculate it after the execution of the model, dividing the firm’shourly revenue by its hourly production. As we observe in Figure 10, due tothe differences among hourly revenue functions, the series of energy prices isonly partially correlated to the series of the firm’s energy outputs.

The model also decides the optimum strategic management of the existingwater reserves. The usage of hydro energy depends on the assigned water val-


ues. The model will use a certain amount of water reserves if their value islower than the maximum marginal revenue reached during the week. In thisexample, all reserve levels are used except for the one valued at 37.5 $/MWh.This indicates that the marginal revenue never reaches that value.

Similarly, the pumped-storage unit consumes energy when the value of itsreserve is times higher than the weekly minimum marginal revenue. Con-versely, this water will be released if the weekly maximum marginal revenuereaches the water value of the pumped-storage (Figure 11). In this case thefinal contents of the pumped-storage reservoir have been set equal to the ini-tial ones.

5.3 Influence of the Strategic Constraints

We now analyze the influence of introducing a minimum-market-shareconstraint. In our case study, the firm faces such a steep residual demandcurve that the incentive to withdraw production from the market is verystrong. Although this leads to high prices in the short term, the firm shouldadministrate its market power with care to guarantee a solid long-term marketposition.

We have solved the case study with ten levels of minimum market share,ranging from 26% to 30.5%. Figure 12 shows part of the price series obtainedfor three of these market-share levels.


As we can see, lower market shares produce higher prices. In this case,the rise of prices overcomes the market share reduction. Consequently, bywithdrawing, the firm achieves both higher revenues and lower costs (Figure13).

As stated by Viscusi et al. [12], one pricing strategy is for an incumbentfirm always to set price so as to maximize current profit. Typically, settingsuch a high price will cause the fringe to invest in capacity and expand.Therefore, this can be called myopic pricing. The polar opposite case is forthe incumbent to set price so as to prevent all fringe expansion (limit pricing).Myopic pricing gives higher profits today, while limit pricing gives higherprofits in the future. Pricing at a level to exclude from the market less effi-cient competitors is, of course, what competition is supposed to do. Pricing toexclude equally or more efficient competitors is known as predatory pricingand constitutes an intent to acquire the monopoly position.

To determine the position that must be defended in the market, a genera-tion firm can assign a cost to the deviations from its medium-term market-share objective. The firm must keep an eye on the, say, one-month moving


average of its market share. If its market share remains for more than onemonth below the objective, then its competitors may understand this as achange in the medium-term equilibrium conditions. Therefore, the firm canexpect prices to stay high for a month. After this period, prices will revert totheir medium-term mean. If this happens, the firm will be forced to suffer lowprices to recover the lost position. With this approach, each time the firm’sone-month market-share moving average lies below the medium-term objec-tive it accounts for a loss. On the other hand, the firm should increase its mar-ket share cautiously, as this can lead to a price war. The cost function lookslike the one sketched in Figure 14.

Depending on the short-term market conditions and on the accumulatedmarket-share, there will be weeks when the model will suggest producingabove or below the market-share objective. An additional consideration is thatthe cost of market-share deviations will probably be higher in on-peak hoursthan in off-peak hours.

6. CONCLUSION

In the new deregulated electric marketplace, generation companies haveto compete to sell the electric services provided by their facilities. They mustdevelop new procedures and tools devoted to maximize profit and hedgerisks.

In this chapter, we addressed several new short-term problems that arefaced by a generation company. Stemming from the medium-term objectivesof the firm, our aim has been to determine the optimal combination of offersand contracts for the supply of electric services through the different marketmechanisms available. We have tried to emphasize the importance of an ade-quate design and use of the decision-support tools.


In this context, we presented a strategic unit-commitment model specifi-cally devoted to the profit maximization of an electric generation firm. It in-corporates new market-modeling equations intended to express the relation-ship between the firm’s output and the obtained revenue. We have made aspecial effort to interpret the effect of these new equations on the managementof hydro reserves. Additionally, we demonstrated how the position that thefirm must defend in the market must guide the short-term operation towardsthe medium-term equilibrium.

The procedure is a simplified version of the one currently being used by aleading Spanish generation company, Iberdrola, for the weekly operation ofits generation assets in the Spanish wholesale electricity market.

REFERENCES

1.

2.

3.

4.

5.

6.

7.

8.

9.

10.

11.12.

J. Bushnell. Water and Power: Hydroelectric Resources in the Era of Competition in theWestern US. POWER Conference on Electricity Restructuring. University of CaliforniaEnergy Institute, 1998.T.J. Scott, and E.G. Read. Modeling hydro reservoir operation in a deregulated electricitymarket Int. Trans. Oper. Res., 3: 243-253, 1996.I. Otero-Novas, C. Meseguer, and J.J. Alba. A Simulation Model for a Competitive Gen-eration Market. IEEE Power Engr. Soc., Paper PE-380-PWRS-1-09-1998.A. Ramos, M. Ventosa, and M. Rivier. Modeling competition in electric energy marketsby equilibrium constraints. Utilities Policy, 7(4): 233-242, 1998.M. Ventosa, A. Ramos, and M. Rivier. “Modeling Profit Maximization in DeregulatedPower Markets by Equilibrium Constraints.” PSCC Conference, Norway, 1: 231-237,1999.M. Rivier, M. Ventosa, and A. Ramos. A generation operation planning model in deregu-lated electricity markets based on the complementarity problem. ICCP99 Conference,Wisconsin, 1999.B.F. Hobbs. “LCP Models of Nash – Cournot Competition in Bilateral and POOLCO–Based Power Markets.” In Proc. IEEE Winter Meeting, New York, 1999.J.Y. Wei and Y. Smeers. Spatial oligopolistic electricity models with Cournot generatorsand regulated transmission prices. Oper. Res., 47(1): 102-112, 1999.J. Garcia, J. Roman, J. Barquín, and A. Gonzalez. “Strategic Bidding in DeregulatedPower Systems.” PSCC Conference, Norway, 1: 258-264, 1999.R. Baldick. The generalized unit commitment problem. IEEE Trans. Power Syst., 10(1):465-475, 1995.A. Brooke, D. Kendrick, and A. Meeraus. GAMS A User’s Guide. Boyd and Fraser, 1992.W.K. Viscusi, J.M. Vernon, and J.E. Hamington. In Economics of Regulation and Anti-trust, ed, The MIT Press, 1998.

Chapter 14

OPTIMIZATION-BASED BIDDING STRATEGIESFOR DEREGULATED ELECTRIC POWERMARKETS

Xiaohong GuanHarvard University, on leave from Xian Jiaotong University, China

Ernan Ni and Peter B. LuhUniversity of Connecticut

Yu-Chi HoHarvard University

Abstract: Deregulation of the electric power industry worldwide raises many challengingissues. Aiming at these challenging issues and using California and New Eng-land power markets as background, this chapter focuses on the methodologiesfor integrated generation scheduling and bidding strategies for deregulated elec-tric power markets. We present a systematic bid selection method based on or-dinal optimization for obtaining “good enough” bidding strategies for generationsuppliers. A stochastic optimization method for integrated bidding and schedul-ing is developed with consideration of risk management, self-scheduling re-quirements, and the interaction between different markets.

The electric power industry worldwide is experiencing an unprecedentedrestructuring. In the United States, California was the first state to establish aderegulated power market starting in April 1998. Since then, almost everystate has or is deregulating its power industry [1-3]. This chapter summarizesthe methodologies developed by the authors and the results achieved so far indealing with some challenging issues on integrated resource bidding andscheduling in deregulated electric power markets.

In terms of the structure of resource allocation and scheduling, currentpower markets can be classified into two types: individual and pool. The Cali-

1. INTRODUCTION


fornia market belongs to the first kind. It includes a Power Exchange (PX)managing the “day-ahead” and “hour-ahead” energy markets, an IndependentSystem Operator (ISO) handling real-time balancing, reserve and other ancil-lary markets, and various energy and service suppliers and demanders. Thestructure and functions of ISO and PX have been described in [3-4]. In theday-ahead energy market, a power supplier submits to the PX piece-wise lin-ear and monotonically increasing power-price “supply bid curves” for eachgenerator or for a portfolio of generating units, one for each hour of the nextday. On the other hand, an energy service company submits to the PX anhourly power-price “demand bid curve” reflecting its forecasted demand. ThePX aggregates supply and demand bid curves to determine a “Market Clear-ing Price” (MCP) and “Market Clearing Quantity” (MCQ) as shown in Figure1. The power to be awarded to each bidder is then determined based on theindividual bid curves and the MCP. All the power awards will be compen-sated at the MCP. After the auction closes, each supply bidder aggregates itspower awards as its system demand, and performs a traditional unit commit-ment or hydrothermal scheduling to meet its obligations at minimum cost overthe bidding horizon. The ISO will check if the schedules submitted by suppli-ers can be implemented through the transmission grid by performing a powerflow calculation and modify the schedules by calling some must-run units asancillary service. In this case, the constrained MCPs will be re-determinedand would be different for different areas. As pointed out in [1], suppliers’bidding decisions are coupled with generation scheduling since generatorcharacteristics and how they will be used to meet the accepted bids in the fu-ture have to be considered before bids are submitted. Therefore, bidding deci-sions must consider the anticipated MCP, generation award and costs, andcompetitors’ decisions and other complicating factors, such as transmissionconstraints of the power grid.

The power market in the United States New England region is formedbased on the New England Power Pool and belongs to the second type [5-6].

Optimization-Based Bidding Strategies 251

Organizationally, the functions of PX and ISO are combined under the aus-pices of a single ISO. Unlike the separate and sequential energy and ancillaryservice markets in California, the energy bids are integrated with other servicebids such as reserve and AGC. In addition, since the ISO has all the systemoperational parameters of each generator, it clears the MCP and other marketprices by performing unit commitment or generation scheduling for the wholepower system in the market based on the power-price bid curves received.Although the capacity of each unit has to be bid, an energy supplier may“withhold” or self-schedule some capacity to meet some percentage of its ownload or to fulfill bilateral transactions with other market participators by bid-ding zero or negative prices. Another difference is that a bid curve in the NewEngland market is a staircase or piece-wise constant function rather than apiece-wise linear function.

Many challenging issues arise under the new competitive market struc-ture. Instead of centralized decision-making in a monopoly environment as inthe past, many parties with different goals are now involved and competing inthe market. The information available to a party may be limited, regulated,and received with time delay, and decisions made by one party may influencethe decision space and well-being of others. These difficulties are com-pounded by the underlying uncertainties inherent in the system, such as thedemand for electricity, fuel prices, outages of generators and transmissionlines, tactics by certain market participants, etc. Consequently, the market isfull of uncertainty and risk. The recent experience learned from the manymarkets has shown that MCPs are volatile and often out of the range of bid-ders’ expectation. How to handle MCP volatilities, how to manage uncertain-ties and risks, and how to allocate the generating capacity into different mar-kets have become extremely important under the new market environment.

Aiming at these challenging problems and using the California and NewEngland power markets as background, we focus on the methodologies foroptimizing bidding strategies. Since bidding problems are multi-person, gametheoretic problems generally associated with inherent uncertainties and com-putational difficulties, it is more desirable to ask which decision is better asopposed to seeking an optimal solution. In our research, we develop two bid-ding methods based on the structures and rules of two actual power markets:the United States California and New England markets. We first concentrateon a systematic bid selection method based on ordinal optimization to obtain“good enough bidding” strategies for generation suppliers. We then present astochastic optimization method for integrated bidding and scheduling withself-scheduling constraints. The risks in supply bidding are managed in a sys-tematic way. We also explore the interactions between energy and other ancil-lary service markets. Numerical testing shows that the algorithm is efficientfor daily bidding and scheduling.


2. LITERATURE REVIEW

Many approaches have been reported in the literature to address the struc-tures and mechanism of deregulated power markets. Prior to the deregulationin the United States, the market structure model discussed most is the “BritishModel” [2]. The California model is presented in [3]. The structure of theNew England market is described in [5] and its ISO’s energy and ancillaryservice dispatch problem is presented in [6]. Some recent studies on marketmechanism are primarily concerned with market analysis and market powerissues [7-8].

Game theory is a natural platform to model a gaming environment whereeach participant is determined to maximize its profit [2, 9-12]. Optimal bid-ding strategies to maximize a bidder’s profit based on the pool model of Eng-land and Wales were presented in [2] under the assumption that any particularbid has no effect on the MCP. For a market where a bid consists of start-upcost, variable price, and generator capacity, it was demonstrated that profit ismaximized by bidding each generator at its physical cost curve and maximumcapacity. This is done by showing that such a strategy is a “Nash equilibrium”for the market. The perfect conditions assumed in [2], however, may not betrue. Matrix games have been reported in [11] and [12]. Bidding strategies arediscretized, such as “bidding high,” “bidding low,” or “bidding medium,” andan ”equilibrium” of the “matrix bidding game” can be obtained. The strategicgaming behaviors and how the market structure affect the competition is ana-lyzed in [9]. It is shown that the strategic behavior on electric network mayproduce unexpected results from the traditional economic theory. Game the-ory is used in [13] to minimize the risk in bidding problems.

Various other methods for solving bid selection problems at different lev-els of the market have also been discussed. In [14], a bid clearing system inNew Zealand is presented. Detailed models are used, including network con-straints, reserve constraints, and ramp-rate constraints, and linear program-ming are used to solve the problem. Other approaches addressing various as-pects of generation and ancillary service bidding can be found in [15-16],where Lagrangian relaxation, and decision trees were used to analyze andsupport the bidding process. For example, a bidding strategy consideringrevenue adequacy was presented in [17] based on Lagrangian relaxation andan iterative bid adjustment process. However this process may not be avail-able for the current California PX market. A bidding method considering theuncertainties of other bidders, the ISO’s bid selection process and self-scheduling in New England power market is presented in [18]. The problem issolved within a simplified game theoretical framework. The exact “gaming”phenomenon among bidders however is not captured. Bidding behaviors un-der a simple auction market are studied in [19]. The results show that powersuppliers would tend to bid above their production costs to hedge against the

3. ORDINAL OPTIMIZATION-BASED BIDDINGSTRATEGIES


possibility of winning on the margin. The factors affecting bidding strategiesin Australia power market are analyzed in [7]. More recent work on biddingstrategies concentrates on the adaptive method, i.e., reacting to inputs, basedon genetic programming and finite state automata [31]. It should be noted thatLagrangian relaxation is a very successful price based method for hydrother-mal scheduling [20], and the framework is also very useful in dealing with thenew integrated bidding and scheduling issues [4].

Recently, an intelligent computational method – Ordinal Optimization(OO) has been developed to solve complicated optimization problems possi-bly with uncertainties [21-22]. Ordinal optimization is based on the followingtwo tenets. First, it is much easier to determine “order” than “value.” To de-termine whether A is larger or smaller than B is a simpler task than to deter-mine the value of (A-B) especially when uncertainties exist. Second, insteadof asking the “best for sure,” we seek the “good enough with high probabil-ity.” Softening the goal of optimization should make the problem easier. Abid selection strategy is developed to generate good enough bids based on theframework of ordinal optimization [23].

It can be seen from above that tools to support integrated bidding andscheduling process of deregulated power markets are far from satisfactory inview of the inherent complexity (multiple participants with their own objec-tives in a dynamic and uncertain environment) and the sizes of practical prob-lems (tens or hundreds of generators with various constraints). High qualityand computationally efficient approaches are critically needed to address thenew challenges.

Assume the bidding strategy is developed for an energy or generationsupply company E. Suppose there are I generating units in E. Supply bids canbe submitted for individual units or an aggregated bid can be submitted in the

PX market. The bidding objective for E is to select its bid curves

to maximize its profit over a time horizon T, i.e.,

Aggregated energy (generation)-price supply bid curveof E for hour t;

Generation and demand bids of other bidders unknown

where


to E for hour t;Generation cost for delivering generation award

Power generation award for E depending on the bid of Eand the bids of other bidders;Bidding time horizon (24 hours for the day-ahead mar-ket); and

T:

Market clearing price (MCP) determined by the aggre-gated supply bid curve and the aggregated demand bidcurve of all market participants as shown in Figure 1.The bid curves of other participants are assumed to befixed, and we are only interested in the influence of E’sown bids on the MCP, which will be modeled later.

According to the PX rules, if a supplier bidder awarded it will

be compensated by the dollar amount regardless of the origi-nal bid submitted by that supplier. Start-up costs should be accounted for inbid curves since there is no direct start-up compensation. The problem de-scribed in (1) is thus an optimization problem to determine the optimal supply

bid curves to maximize the profit subject to relevant operating

constraints, such as the minimum down-/up-time, ramp-rate constraints, etc.Since the PX rules require bid curves to be piece-wise linear and monotoni-cally increasing, searching the optimal bidding strategy is to determine thecorner points of the bid curves. Note that MCPs are determined by the bidssubmitted by all the bidders, and when submitting the bids, a bidder does notknow the bid curves submitted by others.

Once the PX determines the MCP, the generation award for individualunits are determined as the intersection from their bid curves or for the gen-eration supply company E from its aggregated bid curve with as thegiven price. Although E can submit bids for each individual unit, the PX viewtotal generation award to E as obligation. Given a total generation award, it isdesirable for E to schedule or to reallocate all generating units across the bid-ding horizon to deliver its total award at the minimal cost. This can be formu-lated as a traditional unit commitment or hydrothermal scheduling problemwith the total generation award as system demand as follows:

subject to


and other individual operating constraints such as minimum down-/up-timesdescribed in [23-26], whereC: Total generation cost over the entire bidding horizon;

Cost for generating for unit i at hour t;

Start-up or shut-down cost associated with the up-/down-state transitions for unit i to generate at hour t;

Power generation of unit i at hour t; and

Generation award for unit i at hour t according to unit’sindividual bid curve.

If we know the market-clearing price (MCP) and consequently know thegeneration award, we can re-write the profit calculation of (1) as

To select a good bidding strategy, it is necessary to identify two situa-tions: 1) the participant is small and has no significant influence on the MCP;2) the participant has market power and can influence the MCP. In the firstcase, it is desirable to find a nominal bid curve that would maximize its profitat any given MCP. For the second case, it is necessary to establish the rela-tionship between one’s bidding strategy and the MCP. One approach is to ob-tain the MCP by simulating market participants’ bidding strategies. By chang-ing one’s own bidding strategies, it is possible to establish the influence ofbidding strategies on the MCP. Another method is to establish an influencefunction by experience or by regression through one’s historical bids and theMCPs. Either way, assume the following influence functions:

where is the nominal MCP forecasted by the historical

such as fuel prices, etc. The nominal MCP forecasting using an ANNmodel has been reported in our recent work [30]. The influence function

is determined by the difference between one’s bid curve andthe nominal bid curve and can also be established by regression method basedon the historical MCPs and records of one’s own bids plus bidders’ experi-ence. The advantage the ordinal optimization method is that it has no restric-tion on any model used to describe bidding strategy influence on the MCP. In

MCPs the forecasted system demand and other forecasted factors


fact (5) can be replaced by a Monte Carlo simulation procedure or othergame-theoretic models.

The basic idea of the ordinal optimization method includes the followingsteps:

Generating a nominal bid curve by using Lagrangian relaxation methodfor hydrothermal scheduling;Generating N bid curves by perturbing the nominal bid curve and obtain-ing the MCPs associated with these bids;Generating a “good enough” select bid set by evaluating these N bids us-ing a very crude model with little computation effort, and ranking and se-lecting them based on ordinal optimization; andEvaluating the bids in the select bid set using an accurate model and solv-ing the computationally time-consuming hydrothermal scheduling prob-lem (2), and then selecting the best one.

The above four steps are briefly described next (see [23] for details).It should be noted that reflecting the transmission constraints of the power

grid in bids is very difficult and has not been considered in the above bid se-lection procedure. In fact, a unit can provide must-run service when thetransmission grid is limited by its transfer capability versus providing energyand other services. Resource allocation or asset allocation among differentenergy and service markets is an important issue that will be partially ad-dressed in the next section.

1.

2.

3.

4.

3.1 Generating the Nominal Bid Curve

A nominal bid curve should serve two purposes: 1) it is a basis in (5) todefine the relationship between bid curves and the MCP; and 2) it is an opti-mal bid curve if bidding strategy has no influence on the MCP. That is, if E isa “price-taker,” the nominal bid curve should maximize E’s profit for anyMCP determined by the market. To achieve this goal, let a set of MCPs be

given as bid prices and the optimal generation for an indi-

vidual unit to maximize its profit can be obtained based on (6):

subject to the individual operating constraints, where k is the point index on a

solved using the Lagrangian relaxation technique, where can be viewed

as the Lagrange multipliers given by the high level problem. The problem can

bid curve. For the given series of (6) is a subprob-

lem within the unit commitment or hydrothermal scheduling context when


thus be efficiently solved by dynamic programming as in our previous work

[24-25]. The results are the optimal generation levels for all units ateach hour. The nominal bid curves for individual units and the aggregated bidcurve for E are thus generated as

and

Based on the procedure where the nominal bid curve is created, we see that ifthe MCP determined by the market is equal to the generation awardwould maximize the profit of an individual unit as in (6).

3.2 Perturbing the Nominal Bid Curve

The nominal bid curve given above is perturbed to generate N bid curvesas

where is a perturbation function. A simple way to implement

is to keep the power generation of a bidding generation point the

same and uniformly sample in the neighborhood

so that always specifies a monotonically

increasing piece-wise linear bid curve as required by the PX. Based on a per-

turbed bid curve the corresponding estimated by using(5) can be obtained.

3.3 Selecting Good Bids

The N bid curves obtained in (7) can be evaluated and ranked by ordinaloptimization. The estimated profit of each set of bid curves is calculated as

T I

Note (9) is just a rough profit evaluation with the MCP given but without


solving scheduling problems. It is assumed that the generation award to a unitbased on its bid curves will also be delivered by that unit. This may not betrue since the generation award to a unit may not even satisfy the inter-temporal individual operating constraints. A generation company will sched-ule or re-allocate all its units to meet its total generation award with the mini-mal total generation cost for delivering its total obligation and satisfying everyindividual operating constraint of its units. The true profit can be obtainedby solving (2)-(4), and there may be significant error due to the above ap-proximation

where is error. The advantage of the ordinal optimization method is itscapability to separate the good from bad even with a very crude model,namely, the performance “order” is relatively immune to large approximationerrors. Even if the rough estimation is used to rank N bid curves, some goodenough bids will be kept within the select set with high probability.

The major task in applying ordinal optimization is to construct the se-lected subset S containing “good enough” bids with high probability, includ-ing the determination of its size The quantitative measure of the “good

enough” is the alignment probability defined as

where G is the “good enough” bid set and a is called the alignment level. In-tuitively the alignment probability is the probability of the event that there areat least a elements in the good enough set G matched in the select set S.

To select s good ones from the N perturbed bids generated by (7), theprofits are estimated by (9) and ranked. The top s bids are then selected as Sand its size s is determined by a regressed nonlinear equation to satisfy certainconfidence requirement ([27]). The value of s can be estimated by

where is the size of G and and are coefficients or parameters

obtained by nonlinear regression.Evaluating N bidding strategies using (9) is computationally efficient and

the ordinal optimization method can guarantee that good enough bids will beamong the s selected strategies. The result of [27] tells us how large s shouldbe. We apply a more accurate, but time-consuming, evaluation that requiressolving generation scheduling problems to evaluate the s selected bids. Foreach bid with the generation award, a traditional generation scheduling or unitcommitment problem described by (2) and (3) is solved to calculate E’s profitusing the Lagrangian relaxation-based algorithm in ([24-25]). The best bid isthen selected from by evaluating those bids in the subset S. Since is

much smaller than N, the ordinal optimization method is extremely efficient


in comparison with the brute force method of solving the N scheduling prob-lems.

3.4 Numerical Testing

We perform numerical testing is performed for the California day-aheadenergy market based on a 10-unit generation company and historical PX mar-ket clearing prices as the nominal MCPs. For testing purposes, we use thepublished MCPs for May 1, 1998, and January 4, 1999, on the California PXday-ahead energy market as the nominal MCPs in (5). The bidding influencefunction in (5) is simplified as a linear function just to dem-onstrate the effectiveness of the ordinal optimization method. The parametersfor ordinal optimization are selected as follows.

Search space size: N= 1000;Alignment probability:Good enough set: G = top 50 bids among N bids, i.e., g = 50;Alignment level:

To observe “order is relatively immune from error” claimed by the ordi-nal optimization method, the size of intersection of S and G, is listed

in Table 1. The ordinal optimization method tells us that at least 5 bids in Sshould be also be in G. It is observed from Table 1 that sizes of the intersec-tions of G and S are greater than 5 for all cases. Therefore, the good enoughbids will not “slip away” from select set S because of very crude approxima-tion and the ordinal optimization method is very effective. The testing resultsalso show that for the four cases tested the best bid is selected among the Nperturbed bids. The computational time is reduced from 11 hours to about 40minutes in comparison with the brute-and-force evaluation method. For de-tails see [14].

4. INTEGRATED BIDDING AND SCHEDULINGWITH RISK MANAGEMENT

This section presents an integrated bidding and scheduling problem formaximizing the profit of a power supplier while reducing bidding risks. Themethod is generic, but the problem considered here is for the New England


market. As mentioned in the introduction, the MCP may be volatile in view ofthe underlying uncertainties in the market. It is, therefore, important to dealwith the volatility and to manage the risks. Another issue is how to allocatethe generating capacity among different markets such as the energy and re-serve markets.

First we present a formulation of integrated bidding and scheduling. Weassume the “perfect market” where the MCP is not affected by any single bidand assume the MCP series to be a Markov chain. The risk management isexplicitly modeled, and the reserve market and the self-scheduling require-ments are also considered. The problem is formulated as a mixed-integer sto-chastic optimization problem with separable structure. To solve this problem,we develop a Lagrangian relaxation-based method to decompose the probleminto a number of subproblems, one for each unit. Then we apply stochasticdynamic programming to solve individual unit subproblems, and generate aset of generation strategies: how much power should each unit provide foreach possible market price value at what probability? We then create the bidcurves based on these generation strategies. Numerical testing base on an 11-unit system including a large pumped-storage unit shows the algorithm cangenerate good energy and reserve bid curves in 4-5 minutes on a P-III/600PC. The results also demonstrate how a pumped-storage unit affects the bid-ding of thermal units and how the risk management and the reserve marketaffect the bidding in the energy market.

4.1 Problem Formulation

Consider a utility company in New England with I generators of thermal,hydro, and pumped-storage units. The problem is to select bid curves for indi-vidual units to maximize the profit while reducing risks. The formulation in-volves market assumptions, the objective function to be minimized, and theconstraints to be satisfied.

In this section, the “perfect market” assumption is assumed, i.e., the MCPis not affected by a single market participant. Since there exist various time-dependent operating constraints such as minimum up-/down-times and ramprate limits, the bid curves and thus the MCPs between hours are coupled. Tomodel time dependence of the MCPs, the MCP series is assumed to be aMarkov chain. This means that the MCP at hour only depends on that ofthe previous The time independent MCP series can be consid-ered as a special case of the Markov chain. In practice, a “perfect market”may not exist, and the MCP is usually affected by the bids. This issue willalso be discussed in this section.

As mentioned in several previous sections, the bid curve for unit


i at time t is a power-price function indicating how much power the companywants the unit to generate at any given price. Based on the bid curve, if theMCP is then unit i will be awarded the amount of powerTherefore, to determine the bid curve for unit i is to decide a generation level

for any possible priceThe form of bid curves may be different in different markets. In the New

England market, up to 10 power blocks, each with a price, form a bid. Similarto the procedure of generating the nominal bid curve presented in section 3.1,

for each hour are given based on the MCP forecast-

ing. The transition probabilities between and are also knownand will not be affected by any particular bidder. The same is also assumedfor the reserve prices We can construct the bid curves by obtaining the

optimal generation for each given (k= 1, 2, ..., K), and

The utility may have its “own load” and the associated reserve re-quirement R(t) at hour t. We ignore the own load uncertainties since they areusually accurate to within 2% and are less significant than the MCP uncertain-ties. As a strategic decision for reducing the risks, the company may want tocover by itself, on an average, at least a certain percentage of its own load andreserve requirements. This “self-scheduling requirement” can be formulatedas the following constraints on expected and reserve price

and

where and are the self-scheduling coefficients. The algo-rithm would be just simpler for the cases without considering the self-scheduling requirements.

Currently there is no demand market in New England. Therefore, if the

MCP at hour t is and the reserve price is then

is the amount of the energy that the company has to buy (positive) from or sell

(negative) to the market at and the amount of reserve

the company has to buy (positive) or sell (negative) at The profit equals

the optimal reserve for each reserve price (j=1, 2, ..., K) for

each unit i and any hour t.


the revenue from the energy and reserve markets minus the operation costs.The objective is to maximize the expected profit or equivalently to minimizenegative expected profit, which requires the utility to sell power at hours withhigh prices and buy power at times with low prices. With known transition

probabilities of market prices, the MCP variances and reserve price

and reflecting the market uncertainties at hour t can be calculated. Ac-

cording to historical observation, the MCP may jump from its normal value of20 or 30 dollars to over 1000 dollars per megawatt hour. A company usuallyprefers to sell power when the market has large uncertainty and buy powerwhen the market has low uncertainty so as to avoid risks. Therefore, biddingrisks can be reduced by penalizing the product of price variances and the pur-chased amount of energy. Combining the above analysis, the objective to beminimized is a weighted sum of the negative profit (the first part) and the riskterms (the second part), i.e.,

where is a weight to balance the profit versus risks.The problem is also subject to individual unit constraints such as mini-

mum up-/down-time and ramp rate limits for thermal units, and reservoir dy-namics and volume limits for hydro and pumped-storage units. The con-straints for thermal and hydro units have been presented in [24, 25, 28], andthose for pumped-storage units were presented in [26] and [29].

4.2 Solution Methodology

The above integrated bidding and scheduling problem is a mixed-integerstochastic optimization problem. It is very difficult to obtain the optimal solu-tion in view of its computational complexity. One way to solve the problem isto perform scenario analysis. Since the number of scenarios may be very large,scenario analysis is inefficient. In view of the separable structure of the prob-lem, Lagrangian relaxation is a very efficient approach, and we apply it tosolve the problem.

By taking (14) and (15) as the demand and reserve requirements and re-laxing them using two sets of multipliers and respectively, atwo-level optimization is formed. The high level is to update the multipliersso as to maximize the dual. It is formulated as


where

and

Given multipliers and the low level problem consists of Isubproblems, one for each generator. The subproblem i is formulated as

min

subject to individual unit constraints.These two levels iterate until a convergence criterion is achieved. Based

on the subproblem solutions, the bid curves are constructed after the dual pro-cedure terminates. We briefly describe these steps next. For details, refer to[29].

4.2.1 Solving Subproblems

The above subproblem (20) is similar to that of a hydrothermal schedul-ing problem as in [24, 25, 28], where and can be interpreted as theenergy and reserve marginal costs of the system at hour t, respectively. How-ever, and are random variables depending on market prices and

The subproblem solution is a set of strategies rather than a generation

level for each hour. It establishes a relation of and the reserve

price with a probability to generate a certain quantity.

The subproblems for different type units are different. For example, thereare usually no fuel cost and start-up cost for hydro and pumped-storage unitsubproblems. In view of the combinatorial nature and the large number ofpossible MCP realizations, it is hard to solve the subproblem (20). Based onthe transition probabilities of a Markov chain, a stochastic dynamic program-ming (SDP) is developed, in which a stage corresponds to an hour, and a stateat hour t is associated with a pair of and (k, j= 1, 2, ..., K).

Constraints that couple the decisions at different hours, such as ramp rate andpond limit constraints, are relaxed by using an additional set of Lagrangian


multipliers so that the problems are stage-wise decomposable. We then obtainthe optimal generation level and reserve level for

each state by optimizing the stage-wise cost function, subject to feasible op-eration regions. Since the coefficients of the stage-wise cost function are de-termined by and depends on and and is gen-

erally an increasing function of The probability for each

state is calculated based on the transition probabilities of the Markov chain.The subproblem solution here is to provide a set of strategies: for each pair of

and each unit should provide amount of power

with probability We will use these strategies in constructing

bid curves.

4.2.2 Solving the Dual Problem

After solving the subproblems, the sub-gradient associated with (14) canbe obtained as

Then the multipliers can be iteratively updated by using a sub-gradient

or bundle method. The updating of is similar.

4.2.3 Generating and Selecting Bid Curves

After the dual procedure converges, the bid curves for each unit are con-structed based on the strategies obtained in the subproblem solutions. First,we calculate the amount of power to be submitted for each (k=1, 2, ...,

K) as the expectation of over for j=1, 2, ..., K. Since the

operation regions of a unit may be discontinuous, the result obtained is thenprojected into a feasible operation region to get the optimal power amount.This procedure is

Then we construct the bid curve for unit i at home t. As mentioned, the bidcurve required by the New England market is a staircase function. The size ofits power block equals and the price is as


The reserve bid curves can also be constructed similarly.In view of the existence of integer decision variables such as the unit

commitment decision, the maximum dual function does not mean the bestprimal solution. Therefore, the ordinal optimization described in Section 3 isapplied to sort the bid curves generated at the last several iterations. The bestone is selected as the final set of bid curves.

4.2.4 MCP Influenced by Bidding Strategies

In the above, we model the MCP series as a Markov chain under the “per-fect market.” As mentioned in section 3, however, the MCP may be influ-enced by the bids to be determined. To estimate the impact of bidding behav-iors on the MCP, a neural network model for predicting the MCP is underdevelopment. The idea is that in addition to the market information such asforecasted market demand and MCP lags [30], the aggregate bid curves arealso part of the inputs to the neural network, and the sensitivity of the MCPversus the bid curves can thus be analyzed. This model is to be integrated withthe above algorithm and the MCP distribution will be updated based on thebid curves generated at each iteration.

4.3 Self-scheduling, Market Interactions, and RiskManagement

Since plays the role of marginal energy cost and the marginal re-serve cost, their values determine the bid curves for each unit. From (18)-(20)and (22), one can see the impact of self-scheduling constraints and risk man-agement on bidding curves, and the interaction of the energy and reserve mar-kets.

The self-scheduling constraints couple the different units via and

and affect the marginal and as in (18) and (19). For exam-ple, if the self-scheduling requirement at hour t is high, will be large,causing large Consequently, the units have to bid the same amount ofenergy at a lower price to satisfy the self-scheduling constraints.

The tradeoff in allocating the limited generation capacity in the energyand reserve markets depends on the market prices and In the normalsituation, where is much higher than the profit largely depends onthe MCPs for energy. Occasionally, when the reserve price is relativelyhigh as observed in many markets, more capacity should be allocated to thereserve market. This may result in significant changes in the energy biddingstrategies, especially for pumped-storage units. A pumped-storage unit pro-


vides large reserves at either pumping or generating. In view of the reservoirdynamics and limits, the bidding strategies are coupled across hours. That is,decisions in one hour will affect results of other hours. Furthermore, thepumped-storage bidding strategies also affect thermal bidding strategies inview of self-scheduling constraints. Therefore, the pumped-storage units playa key role in both the energy and reserve markets.

The risk management affects bidding strategies through and re-

lated to the variances and If is high, uncertainty on is

large and is large. Then the optimal generation level ob-

tained in DP is large, resulting in large as in (22). As a result, the

units will bid the same amount of energy at low prices so that large amountsof energy could be sold on the market or buying large amounts of power atpotential high MCP could be avoided.

4.4 Numerical Testing

We perform numerical testing for the New England energy and reservemarkets based on a system with 10 thermal units and a 4-identical-generatorpumped-storage system with a large reservoir and the historical market prices.To investigate the influence of the pumped-storage units on thermal biddingstrategy, we test the pure thermal system. To investigate the influence of thereserve market on the bidding strategies, we also perform testing where thereserve market is ignored. The testing results are summarized below. For de-tailed analysis refer to [29].

The results show that the pumped-storage unit plays important roles inboth the energy and reserve markets and can significantly change the thermalbid curves. For example, they pump at hours with low MCPs, making thethermal units bid at a low price to satisfy the high self-scheduling require-ments and generate at hours with high MCPs, reducing the self-schedulingrequirement for the thermal units.

Risk management proves to be an effective way to avoid buying largeamounts of power from the market at potentially high MCPs and to reduce theprofit variances. In section 4.3, we explained why the bid price should be lowwhen the MCP variance is high. In this case, it is observed that the generationaward is large.

The significant influence of the reserve market on the bidding strategies isalso observed. Figure 2 shows the comparison of pumped-storage bid curvesfor hour eight on July 12, 1999, with and without considering the reservemarket. The reserve market is considered in Case 1, but ignored in Case 2.Figure 3 gives the MCPs and reserve prices. According to the market rules,


the reserve contribution of a pumped-storage generator is same as the pump-ing level or the one-line generation capacity minus the current generationlevel. In view of the high reserve price at hour eight, each generator of thepumped-storage system should either pump at its capacity or generate at itsminimum generation limit so as to provide maximum reserve for maximizingthe profit. Figure 2 depicts these two bidding strategies, and the bid curves aresignificantly different.

The CPU time of the algorithm is about four to five minutes on a 600 HzPentium III PC. Therefore, the algorithm is efficient for daily bidding andscheduling.


5. CONCLUSIONS AND ON-GOING WORK

In this research, we developed effective methods for dealing with the newchallenging problems in deregulated electric power markets. We created anordinal optimization based bidding strategy to select “good enough” bids for apower supplier. Numerical results show this method is efficient and can yieldgood bids. We developed a stochastic optimization approach for integratedbidding and scheduling with the consideration of risk management, self-scheduling requirements, and interaction between energy and reserve markets.Numerical testing results based on an 11-unit system of the New Englandmarket show that the algorithm is efficient for daily bidding and scheduling.The bidding strategies of pumped-storage units have significant influence onthe bidding strategies for thermal units and play important roles in both theenergy and reserve markets. On-going work includes simulation and forecast-ing of the market indicators such as the MCP, game theoretic modeling andanalysis of the MCP price spikes and bidding strategies, and integration offorecasting and bidding methods.

ACKNOWLEDGEMENTS

The research reported in this chapter is supported in part by EPRI/AROContract WO833-03, National Science Foundation under Grant ECS-9726577; National Outstanding Young Investigator Grant 6970025; and aKey Project Grant 59937150, National Science Foundation of China and 863Project of China. The authors would like to thank Dr. David Pepyne of Har-vard University for his valuable insight and comments.

REFERENCES

W. Dunn, M. Rossi, and B. Avaramovic. Impact of market restructuring on power systemsoperation. IEEE Comput. Appl. Power Engr., 8(1): 42-47,1995.G. Gross and D. J. Finlay. “Optimal Bidding Strategies in Competitive Electricity Mar-kets.” In Proc. Power Systems Computation Conference, Dresden, 1996, pp. 815-822.A. Farrokh and Z. Alaywan. California ISO formation and implementation. IEEE Comput.Appl. Power Engr., 12(4): 30-34,1999.X. Guan and P.B. Luh. Integrated resource scheduling and bidding in the deregulated elec-tric power market: New challenges. Special Issue, J. Discrete Event Dyn. Syst. 9(4): 331-350, 1999.P.A. Fedora. “Development of New England Power Pool’s Proposed Markets.” In Proc.

Hawaii International Conference on System Sciences, Maui, Hawaii. 1999.K.W. Chenug, P. Shamsollahi, and D. Sun. “Energy and Ancillary Service Dispatch forthe Interim ISO New England Electricity Market.” In Proc. International Conferenceon Power Industry Computer Applications, Santa Clara, California, 1999, pp. 47-53.

1.

2.

3.

4.

5.

6.


7.

8.

9.

W. Mielczarski, G. Michalik, and M. Wildjaja. “Bidding Strategies in Electricity Mar-kets.” In Proc. 21th International Conference on Power Industry Computer Applications,Santa Clara, California, 1999, pp. 71-76.T. J. Overbye, J.D. Weber, and K. J. Patten. “Analysis and Visualization of Market Powerin Electric Power Systems.” In Proc. Hawaii International Conference on SystemsScience, Maui, Hawaii, 1999.C.A. Berry, B.F. Hobbs, W.A. Meroney, R.P. O’Neill, and W.R. Stewart. “AnalyzingStrategic Bidding Behavior in Transmission Networks.” In IEEE Tutorial on Game The-ory Applications in Electric Power Markets, TP-136-0: 7-32, 1999.Owen G. Game Theory, Third Edition, Academic Press, 1995.V. Krishna and V.C. Ramesh. “Intelligent Agents for Negotiations in Market Games, PartI and Part II.” In Proc. International Conference on Power Industry Computer Appli-cations, Columbus, Ohio, 1997, pp. 388-399.R.W. Ferrero, S.M. Shahidehpour, and V C. Ramesh. Transaction analysis in deregulatedpower systems using game theory. IEEE Trans. Power Syst., 12(3): 1340-1347, 1997.H.Y. Yamin and S.M. Shahidepour. “Risk Management Using Game Theory byTransmission Constraint Unit Commitment in a Deregulated Power Market.” In IEEETutorial on Game Theory Applications in Electric Power Markets, TP-136-0: 50-60, 1999.T. Alvey, D. Goodwin, X. Ma, D. Streiffert, and D. Sun. A security-constrained bid-clearing system for the New Zealand wholesale electricity market. IEEE Trans. .PowerSyst., 13(3): 986-991, 1998.S. Dekrajangpetch, G.B. Sheble, and A.J. Conejo. Auction implementation problem usingLagrangian relaxation. IEEE Trans. Power Syst., 14(1): 82-88, 1999.G.B. Sheble. “Decision Analysis Tools for GENCO Dispatchers.” IEEE/PES SummerMeeting, San Diego, California, 1998, PE-231-PWRS-0-06-1998.C. Li, A. Svoboda, X. Guan, and H. Singh. Revenue adequate bidding strategies in com-petitive electricity markets. IEEE Trans. Power Syst., 14(2): 492-497, 1999.D. Zhang, Y. Wang, and P.B. Luh, “Optimization-Based Bidding Strategies in theDeregulated Market.” In Proc. International Conference on Power Industry ComputerApplications, Santa Clara, California, 1999, pp. 63-69.S. Hao. “A Study of Basic Bidding Strategy in Clearing Pricing Auctions.” In Proc.International Conference on Power Industry Computer Applications, San Clara, Califor-nia, 1999, pp. 55-60.A. Cohen and V. Sherkat. “Optimization-Based Methods for Operations Scheduling.” InProceedings of IEEE, 75(2): 1574-1591,1987.Y. C. Ho. “Soft Optimization for Hard Problems.” Web-distributed copy of a tutorial forordinal optimization, http://www.hrl.harvard.edu/people/faculty/ho/DEDS, 1997.Y.C. Ho. On the numerical solution of stochastic optimization problems. IEEE Trans.Autom. Cont, 42(5): 727-729, 1997.X. Guan, Y.C. Ho, and F. Lai. An ordinal optimization-based bidding strategy for electricpower suppliers in the daily energy market. Submitted, 1999.X. Guan, P.B. Luh, H. Yan, and J.A. Amalfi. An optimization-based method for unitcommitment. Int. J. Electric Power Energy Syst., 14(1): 9-17, 1992.X. Guan, E. Ni, R. Li, and P.B. Luh. An algorithm for scheduling hydrothermal powersystems with cascaded reservoirs and discrete hydro constraints. IEEE Trans. Power Syst.,12(4): 1775-1780, 1997.X. Guan, P.B. Luh, H. Yan and P.M. Rogan. Optimization-based scheduling of hydro-thermal power systems with pumped-storage units. IEEE Trans. Power Syst., 9(2): 1023-1031, 1994.T.W.E. Lau and Y.C. Ho. Universal alignment probabilities and subset selection for ordi-nal optimization. J. Optim. Theory Appl., 39(3): 455-490, 1997.E. Ni, X. Guan, and R. Li. Scheduling hydrothermal power systems with cascaded andhead-dependent reservoirs. IEEE Trans. Power Syst., 14(3): 1127-1132, 1999.

10.11.

12.

13.

14.

15.

16.

17.

18.

19.

20.

21.

22.

23.

24.

25.

26.

27.

28.


E. Ni and P.B. Luh. “Optimal Integrated Bidding and Scheduling for Hydrothermal PowerSystems with Risk Management and Self-Scheduling Requirements.” To appear in Proc.

World Congress on Intelligent and Control, Hefei, Anhui, China, 2000.F. Gao, X. Guan, X. Cao, and A. Papalexopoulos. “Forecasting Power Market ClearingPrice and Quantity Using a Neural Network Method.” To appear in Proc. 2000 IEEE/PESSummer Meeting.C. Richter, G. Sheble’, and D. Ashlock. Comprehensive bidding strategies with geneticprogramming/finite state automata. IEEE Trans. Power Syst., 14(4): 1207-1212, 1999.

29.

30.

31.

Chapter 15

DECENTRALIZED NODAL-PRICESELF-DISPATCH AND UNIT COMMITMENT

Francisco D. Galiana and Alexis L. MottoMcGill University

Antonio J. ConejoUniversity of Castilla-La-Mancha

Maurice HuneaultHydro-Quebec Research Institute (IREQ)

Abstract: This chapter sets forth a scheme for self-scheduling independent market partici-pants in a power pool. The approach, named DNSA for Decentralized Nodal-Price Self-Scheduling Auction, is proposed as an alternative to centralized Poolauctions and operation. DNSA exploits the intrinsic parallelism of the dual unitcommitment problem to decentralize the various scheduling and dispatch func-tions. Each competing participant (GENCO, DISTCO) maximizes its profit forany set of nodal prices by choosing its level of production or consumption.Similarly, the TRANSCO independently maximizes its merchandising surpluswithin the network security constraints. The price caller, a centralized entitywithout access to proprietary cost information, updates prices through an effec-tive Newton algorithm until the power balance at each bus is satisfied. DNSAdoes not assume a perfect market and accounts for the AC load flow model in-cluding transmission losses and line congestion, in addition to integer variables,ramping rates, start-up costs, and minimum up and down times. The conver-gence of DNSA hinges on the notions of profit optimality and the convexifyingmarket rule. We present several study cases to illustrate the characteristics ofDNSA. We conclude that to achieve fairness of treatment for all competing par-ticipants, they should be allowed to optimize their profit by self-scheduling.Therefore, to the extent possible, the next generation of unit commitment mod-els should include profit optimality.


1. INTRODUCTION

This chapter summarizes a number of research activities conducted atMcGill University on the decentralized self-operation of electricity marketparticipants, namely, GENCOs, DISTCOs, and TRANSCOs. A main goal ofthis research is to demonstrate that, under certain conditions, self-operationbased on maximizing individual profits yields the same electrical and eco-nomic operating point as centralized maximum welfare operation. Thisequivalence is known for simple cases of economic dispatch, but is less obvi-ous for more complex scheduling problems including unit commitment, in-tertemporal constraints, nonlinear transmission losses, flow congestion, andother network constraints. In particular, we will show that the equivalencebetween self- and centralized operation is also valid for conditions other thana perfect market.

The proof of equivalence is based on purely mathematical arguments un-der a main assumption, here named “profit optimality,” by which we meanthat all competing participants are free to maximize profit subject only tomarket prices. Without profit optimality, the centralized solution may requiresome individual participants to operate at less than maximum profit, an un-avoidable consequence of the difficulty of defining a measure of social wel-fare.1 Therefore, we argue that market rules can be considered unfair if theycan lead to solutions where some but not all participants are dispatched atmaximum profit. Such a possibility must be precluded by the market rulessince equality of treatment is of paramount concern among competing entities.Accordingly, in our view, profit optimality is not a theoretical issue but a de-sirable restriction of the next generation of unit commitment problems.

Under profit optimality, the dual and primal solutions of the mixed integerunit commitment problem are shown to be identical. This then leads to theproposed decentralized, nodal-price self-dispatch and scheduling auction,DNSA, the essence of which is to optimize the dual function while simultane-ously meeting the relaxed constraints of the primal problem. The DNSA com-prises some key innovations centered on the application of an iterative auc-tion. These are the introduction of a central price caller that updates andbroadcasts trial prices, the delegation of the self-scheduling tasks to the par-ticipants who then respond to the trial prices, the use of a fast Newton price-updating scheme, and the consideration of the full set of non-linear networkconstraints.

This chapter is organized as follows: after the nomenclature, we comparethe general features of centralized operation to those of decentralized self-operation, in particular under the proposed DNSA scheme. Next, the unit

1 See [1] for a compact summary on Arrow’s impossibility theorem. Kenneth J. Arrow was a1972 Nobel laureate in economic science.

Decentralized Nodal-Price Self-Dispatch and Commitment 273

commitment problem primal and dual forms are formulated and shown toyield identical solutions under the condition of profit optimality. The DNSAscheme is then described and compared to the classical centralized dual-basedunit commitment. DNSA requires two conditions to converge, the previouslydefined profit optimality, together with the “convexifying” rule imposed toensure rational participants’ behavior in response to nodal prices. We thendescribe the Newton-based price-updating component of DNSA. Finally, anumber of numerical results illustrate the characteristics of DNSA.

2. NOTATION

For quick reference, we classify below the main mathematical symbolsused throughout this chapter.

feasible space of demand ifeasible space of generation i

minimum down time of generator i

minimum up time of generator i

set of network nodesaverage real power injection at bus i during period

average real power of load i during period

average real power of generator i during period

profit of load i over the scheduling horizonprofit of generator i over the scheduling horizon

Benefit function of load i

ramp-down limit of generator i

ramp-up limit of generator i

shut-down cost of generator i occurring at the beginning of period

start-up cost of generator i occurring at the beginning of period

set of time intervals in scheduling horizon0/1 variable, which is equal to 1 if generator i is committed duringperiod or 0 otherwise0/1 variable, which is equal to 1 if load i is committed during period

or 0 otherwise0/1 variable, which is equal to 1 if generator i is started up at thebeginning of period or 0 otherwise


0/1 variable, which is equal to 1 if generator i is shut down at thebeginning of period or 0 otherwiseaverage voltage phase angle at node i during period

cumulative up/down time of generator i at the start of period

average nodal price of active power at node i during period

the vector over all time intervals for node i

the vector over all nodes for time

the vector all nodes and all time intervals

3. MOTIVATION FOR SELF-OPERATION

3.1 Current Practices

The emerging restructured power industry is guided towards two generictrading models: pool and bilateral markets [3,4].2 In the pool context, a centralagent collects the bids from sellers (GENCOs) and buyers (DISTCOs), deter-mining the winners and the amount of power each is required to sell or buy aswell as the market-clearing prices. Generally, this selection results from a cen-tralized constrained optimization. In contrast, the bilateral model allows par-ticipants to arrange contracts among themselves. Though the transmissionmonopoly may create unforeseeable challenges to the implementation of bi-lateral transactions, some authors argue that such contracts offer participantsmore freedom, thereby achieving greater decentralization in decision-making[5]. In view of their widespread use, the scope of this chapter is limited topure pool-operated markets.

Figure 1 illustrates the state of the art of today’s Pool operation. The mar-ket operates with three independent entities, GENCOs, DISTCOs, and theISO.3 Under this model, strictly speaking, there is no independent transmis-sion provider, as the network is under the authority of the ISO. The latter setsthe pool price (or nodal prices) by trying to maximize the overall social wel-fare, irrespective of the participants’ revenue or profit requirements. Clearly,the ISO has an important and often coercive decision-making authority, apractice inherited from the traditional centrally controlled utility industry.

2 The central buying entity for all suppliers of electricity, which in turn is the single agent forselling power to retail customers and their aggregators [2].

3 We use the term ISO to refer to the combined power exchange and system operator.


3.2 On Decentralizing Pool Operation

Reduced reliance on the decisions of a centralized scheduling and dis-patching entity is an alternative that some electricity market participants al-ready actively seek [6,7,8]. Centralized authority has been brought into ques-tion for both technical and philosophical reasons. For example, it is acceptedthat solutions to optimal power flow [9] and unit commitment [10] programscan be highly sensitive to small variations in the input parameters and to thealgorithm heuristics. Whereas the sensitivity of the objective function to pa-rameter variations is usually insignificant, that of the individual participants’responses can be more substantial, a potentially unfair result. Philosophically,it can also be argued that, in a competitive environment, the decision as towhether to trade or not and how much to trade must rest solely with eachtrader. Furthermore, such a choice should be guided by each participant’s pri-vate and confidential profitability expectations as well as on the market condi-tions. For this decision-making independence to function in the context of apower system, market prices alone must offer sufficient incentives to satisfyall network security and power balance requirements without the need forcentralized intervention.

One alternative to the centralized pool with the above characteristics isthe DNSA scheme put forward in this chapter. DNSA is an auction mecha-nism [11,12] that allows each independent participant to self-commit and dis-patch based on its own profit evaluation (see Figure 2).


It is instructive to compare the dual communication in DNSA betweeneach participant and the auctioneer – here the Price Caller – to the one-wayrelation between participants and the ISO in the centralized auction of Figure1. Under DNSA, convergence to equilibrium involves repeated interaction ofall participants, with the Price Caller updating trial prices and the independententities responding accordingly.

Note that the DNSA differs from other attempts to decentralize the au-thority of the ISO. For example, in the approach of Griffes [13], GENCOs andDISTCOs are allowed to resubmit new bids to the ISO if the previous sched-ule is not to their satisfaction. In that approach, however, for each new set ofbids the ISO has to repeat a fully centralized scheduling operation.

The DNSA, in a similar fashion as an iterative Lagrangian relaxation al-gorithm [14,15], exploits the decomposability of the unit commitment prob-lem in its dual form to achieve the goal of decentralization and self-scheduling. The next section analyzes the conditions under which dual andprimal methods yield the same solutions.

4. THE UNIT COMMITMENT PROBLEM

Traditionally, the unit commitment problem develops the on/off schedule ofgenerating units (loads) over an operating horizon. Once committed, a genera-tor (load) is synchronized to the grid and is ready to deliver (consume) power.The dispatch problem consists of determining the levels of production (con-sumption) of the committed generators (loads). In practice, however, thesetwo problems are solved concurrently and often referred to as the unit com-mitment problem.


4.1 Primal Problem

The primal unit commitment problem [16], abbreviated here as PUC, can beformulated as the maximization of the social welfare,

subject to the constraints,

The objective function (1) includes generation costs, and con-

sumer benefit functions, The total operation cost, in ($/h)incurred by Genco(i) can be written as,

where, is the fixed cost, is the variable cost, is the

start-up cost, and is the shut-down cost.4 Note that cost and benefitdata are bids.

Constraints (2) implement the real power balance equation at each net-work node, line power flow limits are enforced by (3), consumption limita-tions such as budget constraints are represented by (4), while generator con-straints are contained in (5). The symbols and represent the feasible

regions of the variables and respectively. The set in-

cludes initial conditions, (7), output limits, (8), minimum up/down times, (11)–(12), and ramp-up/down limits, (9)–(10), that is,

4 Refer to [17] for a formal discussion of cost and consumer benefit functions.5 A similar set of constraints could apply to


A few caveats must be made at this point. For the sake of clarity, wemade the decision to simplify some of the models. We have attempted, how-ever, to introduce the results in such a way that generalizations are apparent.Thus, the network optimization variables have been limited to the voltagephase angles, while still retaining the full nonlinear AC load flow. Reserveconstraints and multiple-generator (-loads) GENCOs (DISTCOs) have not yetbeen incorporated in DNSA.

The implementation of PUC for a typical power system involves a largenumber of 0/1 variables and numerous constraints. This problem is known tobe NP-hard and, unless drastic simplifications are made, seldom tractable.6

Furthermore, the complexity of the models that are solved increases undercompetition since profit-driven agents have no choice but to model their sys-tems with additional variables in order to achieve greater accuracy. In order toapply integer programming [18], in our simulations all variables are constantduring the discrete time intervals and all functions are approximated by dis-crete piecewise-linear mappings [19,20,21].

4.2 Dual Problem

In contrast to the primal, experience demonstrates that dual methods suchas Lagrangian Relaxation are quite successful at solving the unit commitmentproblem [22,23,15], notwithstanding the heuristic approximations that be-come necessary whenever the duality gap is non-zero.7

In the dual approach, relaxing the power balance constraints at every busand for every time period yields the Lagrangian function,

6 A problem is said to be NP-hard if the zero-one integer programming problem can be mappedto it in polynomial time.

7 In such cases, the primal solution is sub-optimal and can dispatch some participants at lessthan the maximum possible profit for the market prices.


from which the dual unit commitment problem (DUC) can be written as

By the weak duality theorem, the dual value is a lower bound to theprimal value defined by equation (1). Regardless of the structure of the objec-tive and constraints of the primal problem, the domain of the dual function

is convex and the function is concave over These elegant convexityproperties combined with the problem decomposability allow a solution of thedual problem consisting of several reduced sub-problems that can be solved inparallel as outlined next.

4.3 Self-Operation Sub-Problems

The three components in the right-hand-side of equation (18) can be in-terpreted as monetary objectives to be maximized by each type of entity forthe given prices.

4.3.1 The GENCO Profit Maximization Sub-Problem

Given any set of nodal prices GENCO(i) maximizes its profit while satis-fying its generation constraints, that is,

The arguments of this sub-problem are functions of the prices and are denotedby and The above-defined maximum profit is non-negative,

that is, This result stems from the fact that since a GENCO can


control its commitment, it will always turn itself off over the entire time hori-zon rather than operate at a loss.

4.3.2 The DISTCO Profit Maximization Sub-Problem

Given any set of nodal prices, DlSTCO(i) also maximizes its profitwhile satisfying its load constraints, that is,

The arguments of this sub-problem are functions of the prices and are denotedby and The maximum profit of any self-committing DISTCO

(with controllable 0/1 variable is also non-negative, that is,

4.3.3 The TRANSCO Merchandising Surplus Maximization Sub-Problem

Assuming that a single Transco operates the network, this sub-problem con-sists of maximizing the merchandising surplus over the planning horizon, sub-ject to the various network security constraints, Thus,

The argument of this sub-problem is a function of the prices and is denoted bya solution that is de facto security feasible, as the TRANSCO en-

forces security constraints on all network variables.We emphasize that in (24) the TRANSCO does not physically adjust the

voltage phase angles or the power injections, it merely computes the injec-tions that are then submitted to the Price Caller. What is perhaps surprising isthat giving the TRANSCO the freedom to maximize its merchandising sur-plus is consistent with the primal problem objective of maximum social wel-fare under the set of network constraints,

Even so, the abuse of monopoly power by a TRANSCO in the process ofmaximizing its merchandising surplus is a possibility that must be addressed.For example, manipulation of the line flow limits in could artificially cre-ate transmission congestion and nodal price differences, thereby increasingthe merchandising surplus. We believe that this practice can be virtually ruledout by making the TRANSCO data and response to prices public and open toscrutiny by all market participants. This is consistent with the rulings ofFERC [24,25]. Furthermore, because its revenues and/or profits are regulated,there is no incentive for a TRANSCO to create congestion artificially, unlessit were in an anti-competitive collusion with a GENCO or a DISTCO.


4.4 Profit Optimality

Let and be the main arguments of the pri-

mal problem (1) at its optimum. The corresponding profit of GENCO(i) is

Analogously, the profit of DISTCO(i) is

The TRANSCO surplus under the primal optimum conditions is given by

With these results, we can now define the notion of profit optimality.

DEFINITION 1 (Profit Optimality): A primal solution is said to be profit opti-mum if,

Profit optimality states that the primal profit of GENCO(i), is

the maximum achievable profit, A similar statement applies to the

profit of DISTCO(i) and to the merchandising surplus of the TRANSCO.

4.5 Equivalence between Primal and Dual

PROPOSITION: Assume that PUC and DUC are each unique. DUC andPUC yield identical solutions if and only if the solution of is

profit optimum.

PROOF:

Necessity: Let PUC and DUC have identical solutions, that is,Since maximizes the individual profits in DUC

for the prices then also maximizes the same profits for Thus, the

solution of PUC is profit optimum.


Sufficiency: Let the solution to PUC, be profit optimum. Weshow that PUC and DUC yield identical solutions. First calculate the dualfunction at the primal solution prices, From (18)–(24),

Next, applying the profit optimality conditions in (28), and then refining withequations (25)–(27),

Since the power balance at all nodes and times is guaranteed by the primalsolution, the second right-hand side term above disappears. Thus,

which is the value of the primal. This implies that at the primal prices, thedual function is maximized and the duality gap is nil. Making use of the gen-erally accepted assumption that the solutions of DUC and PUC are unique, itthen follows that these solutions are equal, that is, Q.E.D.

5. BASIS OF THE DECENTRALIZED NODAL-PRICESELF-SCHEDULING AUCTION (DNSA)

5.1 Profit Optimum Unit Commitment

We know (see Case C in Section 6) that the standard formulation of theunit commitment problem, (1)-(5), can yield solutions that are not profit opti-mum. Although mathematically feasible, such solutions may require someparticipants to operate at a profit below the maximum possible for the currentprices, a result that we contend is unfair in the context of competition. In fact,the current practice of “uplift” charges in Pool operation is a mechanism forcorrecting such inequities, albeit in a heuristic and sub-optimal manner. Topreclude the potential of unequal treatment of participants’ profits and to doso systematically, we assert that the next generation unit commitment formu-lation should include the condition of profit optimality. This is a subject of ourongoing research for both the centralized PUC and for DNSA.


5.2 Basic Steps of DNSA

DNSA differs from standard dual methods (SDM) in several respects:On the centralized nature of the method: all scheduling and price updating

is centralized in SDM, whereas in DNSA, only price updating is centralized.GENCOs, DISTCOs, and TRANSCO self-schedule.

On the requirements of price updating and commercial information: InSDM, as price updating is based on maximizing the central schedulerneeds full knowledge of private cost and benefit functions. In DNSA, as priceupdating is based on solving the Price Caller needs only a point-wise price-response by GENCOs and DISTCOs.10

On the requirements of network information: In SDM, all TRANSCOdata is required by a central scheduler to assure network security. In DNSA,the TRANSCO assures security. The Price Caller needs only a reduced set ofnetwork data and sensitivities.

On the completeness of network modeling: In SDM, network constraints,particularly those describing nonlinear behavior, have been often simplified orignored. In DNSA, most network constraints can be or have been included.

On the evolution of bids through the auction process: In SDM, bid func-tions are fixed throughout the solution process. In DNSA, response to pricesmay evolve during the price-updating trials subject to the “convexifying” rule,which is defined next.

8 Here9 The state-of-the-art methods for updating dual variables are the sub-gradient method [26], the

bundle method and the cutting plane method. In the current implementation of DNSA, aNewton approach is used to update prices.

10 The Price Caller requires no explicit bid functions. To prove that the self-scheduling schemeyields the same solution as PUC, however, we assume the existence of such functions. Thisimplies that GENCOs and DISTCOs behave according to some deliberate strategy.


5.3 Convergence of DNSA

We now address the question of equilibrium attainment, that is, if a com-petitive equilibrium is feasible, in the sense that PUC is profit optimum, howdoes the market attain it? The difficulty of this question is old and well knownto economists. Indeed, it was first contemplated by Leon Walras, one of theprogenitors of modern equilibrium theory [12]. In his conjecture, known asWalrasian tatônnement, Walras states that attaining equilibrium would in-volve a process by which a market groped toward equilibrium with the help ofa fictitious auctioneer who announced prices, then collected demands fromconsumers as to how much of each good they would wish to purchase at theannounced prices. If demand exceeded supply, the price was adjusted upward.If supply exceeded demand, the price was adjusted downward. Walras conjec-tured that this procedure would cause the market to eventually settle into equi-librium. This simple and intuitively plausible idea was widely accepted untilHerbert Scarf demonstrated, by his famous examples, the existence of an openset of economies having a unique equilibrium that was unstable, therefore un-attainable, under basic tatônnement [27]. In this study, we came to the conclu-sion that it was possible to construct an improved tatônnement procedure indecentralized electricity market. We conjecture that this procedure will con-verge, of course assuming the existence of a competitive equilibrium, by im-posing a restriction on the participants’ bidding here called the convexifyingrule. This constraint is not a theoretical limitation of DNSA but a practicalnecessity for attaining a competitive equilibrium in both decentralized as wellas centralized auctions. A consequence of the convexifying rule is that duringthe price-updating trials, participants are prevented from derailing the auctionprocess or rendering it chaotic.

DEFINITION 2 (Convexifying Market Rule): An electricity market is said tooperate under the convexifying market rule if, for each time period, the col-lection of trial price-response pairs of GENCO(i),

to the k-th auction iteration – can be extended to a monotone increasingincremental cost [monotone decreasing incremental benefit] function.11

11 A real-valued function of a real variable x is said to be monotone increasing if the value ofdecrease as increases; that is,


CONJECTURE (Convergence of DNSA): Extensive simulations and relatedexperience [28] indicates that DNSA converges well when PUC is profit opti-mum and the competing participants behave rationally according to the con-vexifying market rule.

5.4 Price Updating

In this subsection, a vector variable with the superscript k represents itsvalue at the k-th price-updating trial.

One important innovation of DNSA is the use by the Price Caller of aprice-updating algorithm that uses the first and second derivatives of the dualfunction. This Newton algorithm [29] solves the nodal power balance equa-tions, that is,

via the iterative procedure,

The second derivative matrix or Hessian is

The first two terms in the right-hand side of (35) are based on confidentialcompeting participant information and can only be estimated by the PriceCaller. Any estimation method may be used, among others, first differences,and regression using all the past price-response pairs. In the simulations ofthis chapter, each generator responds myopically so that the first two termsare diagonal matrices whose elements can be estimated from the participant’sbehavior during the price-updating trials. The convergence of Newton’smethod does not however hinge on knowing the exact Hessian, a good ap-proximation being sufficient. The third term in (35) is computed from publicnetwork data available to the Price Caller.

This Hessian term corresponds to the case with no active line flow constraints.Under congestion, the TRANSCO is required to disclose the Lagrange multi-pliers associated with the active line flow limits. These values may then beused by the Price Caller to refine the Hessian for faster convergence [30].


6. CASE STUDIES

To illustrate how DNSA works, we use a 5-bus system with network and gen-erator data shown in Table 1 and 2. The resistance, R, reactance, X, and totalline charging susceptance, Bcap, are per unit on a base of 100 MVA.

We named the generators and loads according to their location in the network.We chose identical generator costs because this case has been challenging fortraditional centralized UC algorithms that do not adequately represent thetransmission system. In this chapter, we report three simulations. In Cases Aand C, line flow limits are large enough to rule out congestion, while in CaseB some line capacities are reduced as indicated later to create congestion.

The planning horizon consists of three equal length periods. The load pat-terns over time of DISTCO(2) and DISTCO(5) appear in Table 3 and are as-sumed unaffected by nodal prices.

12Following the notation in (6), the variable cost of GENCO(i) is given by


CASE A (Primal solution is profit optimal): Figure 4 and Figure 5 showthat convergence to within is obtained in seven trials (or

Newton price-updating iterations), at which point the nodal prices have alsosteadied. Plainly, the auction is stopped successfully when the maximumnodal power imbalance (defined in footnote 8, page 12) is less than 1 MW.This is common practice in actual power system operation. The generatoroutput and profit profiles are reported in Table 4 and Table 5, respectively.All GENCOs make some profit over the planning horizon even thoughGENCO(l) and GENCO(3) must supply power at a loss during Period 3 ow-ing to their minimum up-time constraint, In contrast,

GENCO(4), with a lower minimum up time, elects to operateduring Period 1 and Period 2 only, de-committing during Period 3 when thereis no financial incentive because of the low demand and low nodal price.

Table 6 shows the merchandising surplus of the TRANSCO that, withoutcongestion, is due entirely to transmission losses.


CASE B (Effects of transmission congestion): Here, the data of Case A ismodified by setting the flow limit of line 1–5 to 35 (MW), producing a lightcongestion during Period 1 and a heavier one during Period 2.

As evidenced by Figure 6 and Figure 7, DNSA convergence is obtainedwithin 11 trials. Price updating exploits the Lagrange multipliers of the activeflow limits made available by the TRANSCO. Without this information, theDNSA still converges but at a slower rate (47 iterations).


The final generator output and profit profiles are shown in Tables 7 and 8,respectively. Comparing these results to those of Case A, we note a redistribu-tion of the generation profits and a substantial increase in the Transco mer-chandising surplus (Table 6 versus Table 9).

CASE C (Primal solution is profit sub-optimum): The same data as in caseA are used except that the minimum up-times of GENCO(l) and GENCO(3)are decreased to two hours

As the generating units can now switch off during Period 3, under theprofit maximizing trial iterations of DNSA the participants do not come to apower balancing agreement at this time period. The resulting cycling behaviorin prices and outputs during period 3 is shown in Figure 8 and Figure 9. Thisis a signal to the Price Caller that profit optimality is infeasible and that analternative price-updating scheme must be initiated, as discussed in the Con-clusions.

The primal unit commitment solution without profit optimality does how-ever exist as shown in Table 10,

Table 11, and Table 12.13 Comparing the last two columns of Table 12,we see that each generator primal profit, is positive but less than the

maximum possible, The unfairness of primal solutions that are

profit sub-optimal can be observed from the relative differences between pri-mal and maximum profits.

During those intervals where the DNSA and the primal solutions are theoretically equal, thenumerical results may differ slightly due to convergence tolerance.

13


7. CONCLUSIONS

As an alternative to centralized pool auctions and operation, we propose aDecentralized Nodal-Price Self-Scheduling Auction (DNSA) for self-commitment and dispatch of electricity market participants in a power pool.The scheme exploits the decomposability of the dual unit commitment prob-lem. Each competing GENCO (DISTCO) self-computes its level of produc-tion (consumption) and the 0/1 commitment by maximizing profit for a givenset of trial nodal prices. Similarly, the Transco independently maximizes itsmerchandising surplus while satisfying all network security constraints.


The Price Caller, a centralized entity without direct access to proprietarycost information, updates prices through an effective Newton algorithm untilthe power balance at each bus is satisfied. DNSA does not assume a perfectmarket and accounts for the AC load flow model including transmission lossesand line congestion, in addition to integer variables, ramping rates, start-upcosts, and minimum up and down times. The convergence of DNSA hingeson the notions of profit optimality and the convexifying market rule. The latterrule is imposed to ensure that the response to the Price Caller by market par-ticipants will not derail the price updating process. Interestingly, the convexi-fying rule, while preventing irrational and chaotic bidding, also offers thecompeting participants some latitude to refine their bids during the auction.

In study cases A and B, we illustrate the convergence characteristics ofDNSA with and without transmission congestion. In addition, we solve thetraditionally difficult case of generators with identical cost functions. In CaseC, we illustrate two important points as regards profit sub-optimal problems.First, the primal profits of individual participants are not the maximum possi-ble for the primal nodal prices. We contend that under free competition thisresult is unfair to some participants. Second, price cycling is a clear signal thatprofit optimality is infeasible.

To ensure fairness of treatment for all competing participants, the nextgeneration of unit commitment models should allow for self-scheduling, andshould therefore include profit optimality constraints. Ongoing research isconsidering various mathematical reformulations and computational solutionsof the profit optimal primal problem, as well as DNSA derivations that solvethis new problem in a decentralized manner [30]. We are looking at DNSAprice updating schemes that will enforce profit optimality through the trialiterations for those cases that are not naturally profit optimum, that is forwhich the primal unit commitment formulation is not already profit optimum.The crucial point here is to enforce profit optimality whenever possible whileachieving maximum social welfare.

REFERENCES

1.

2.

3.

4.

5.

K.J. Arrow. Social Choice and Individual Values, Second Edition. New Haven: Yale Uni-versity Press, 1963 ed. 1951).R. Green. “The Political Economy of the Pool.” In Power Systems Restructuring: Engi-neering and Economics. Kluwer Academic Press, 1998.M. Ilic, and F.D. Galiana. “Power Systems Operation: Old vs. New.” In Power SystemsRestructuring: Engineering and Economics. Kluwer Academic Press, 1998.M. Huneault, F.D. Galiana, and G. Gross. “A Review of Restructuring in the ElectricityBusiness.” In Proc. 13th PSCC Conf., Tronheim, Norway, 1999.F. Wu, and P. Varaiya. Coordinated multilateral trades for electric power networks: The-ory and implementation. Elec. Power Energy Syst., 21(2): 75-102, 1999.


6.

7.

8.

9.

10.

11.12.

13.

14.

15.

16.

17.18.19.

20.

21.

22.

23.

24.

25.

26.27.28.

29.

30.

C. Imparato. “Market-Making Attributes of Alternative ISO Structures.” Comments in aPanel Session, In Proc. IEEE PES Winter Meeting, 1999.Federal Energy Regulatory Commission (U.S.). “Regional Transmission Organizations:Rulemaking.” Docket No. RM99-2-000, Order No. 2000, December 1999.R. Wilson. “Activity Rules for the Power Exchange.” Report to the California Trust forPower Industry Restructuring, March 3, 1997.G. Gross, E. Bompard, P. Marannino, and G. Demartini. “The Uses and Misuses of Opti-mal Power Flow in Unbundled Electricity Markets.” In Proc. of the IEEE PES, SummerMeeting, Edmonton, Alberta, Canada, July 1999.R. Johnson, S.S. Oren, and A. Svoboda. “Equity and Efficiency of Unit Commitment inCompetitive Electricity Markets.” In Proc. POWER Conf. and paper PWP-039, Univ. ofCalifornia Energy Institute, March 15, 1996.C.W. Smith. Auctions: The Social Construction of Value. London: The Free Press, 1989.M.-E.L. Walras. Éléments d'Économie Politique Pure; ou la Théorie de la Richesse So-ciale. First edition, 1874. [English translation: Elements of Pure Economics, or; The The-ory of Social Wealth. Homewood, I11.: Published for the American Economic Associationand the Royal Economic Society, by R.D. Irwin, 1954.P.H. Griffes. “Iterative Bidding in the PX Market.” A report to the California Power Ex-change, http://www.calpx.com/news/publications/index.htm., January 2000.H. Everett. Generalized Lagrange multiplier method for solving problems of optimal allo-cations of resources.” Op. Res. 11: 399-417, 1963.F. Zhuang, and F.D. Galiana. Towards a more rigorous and practical unit commitment byLagrangian relaxation. IEEE Trans. Power Syst., 3(2): 763-773, 1988.A. Merlin, and P. Sandrin. A new method for unit commitment at Electricité de France.IEEE Trans. PAS -102, 5: 1218-1225, 1983.P.A. Samuelson. Foundations of Economic Analysis. Harvard University Press, 1947.L.A. Wolsey. Integer Programming. John Wiley & Sons, Inc., 1998.T.S. Dillon, K.W. Edwin, H.D. Kochs, and R.J. Tand. Integer programming approach tothe problem of optimal unit commitment with probabilistic reserve determination.” IEEETrans. PAS-97, 6:2154-2166, 1978.J.M. Arroyo and A.J. Conejo. Optimal response of a thermal unit to an electricity spotmarket. To appear in the IEEE Trans. Power Syst. Paper PE040PRS (10-99).A. Brooke, D. Kendrick, A. Meeraus, and R. Raman. GAMS: A User’s Guide. GAMSDevelopment Corp., 1998.J.A. Muckstadt, and S.A. Koenig. An application of Lagrangian relaxation to schedulingin power generation systems. Op. Res., 25(3): 387-403, 1977.D.P. Bertsekas, G.S. Lauer, N.R. Sandel, and T.A. Posberg. Optimal short-term schedul-ing of large-scale power systems. IEEE Trans. Autom. Cont., 28(1): 1-11, 1983.N. Nadira, and A.S. Cook. “On the Availability of Data Required by Optimal PowerFlows under Increased Competition,” IEEE PES, Tutorial Course – Optimal Power Flow:Solution Techniques, Requirements, and Challenges, 96 TP 111-0, 1996.Federal Energy Regulatory Commission (FERC–US). “Form 715: Annual TransmissionPlanning and Evaluation Report.” http://www.ferc.fed.us/electric/F715/, January 2000.B.T. Polyak. Introduction to Optimization. Optimization Software, 1987.H.E. Scarf. Some examples of global instability. Int. Econ. Rev., 1: 157-172, 1960.M. Huneault. “An Investigation of the Solution to the Optimal Power Flow Problem In-corporating Continuation Methods.” Doctoral thesis, McGill University, August 1988.J.M. Ortega and W.C. Rheinboldt. Iterative Solution of Nonlinear Equations in SeveralVariables. Academic Press, 1970.A.L. Motto. “Equilibrium of Electricity Market Under Competition.” Ph.D. thesis in pro-gress, McGill University.

Chapter 16

DECENTRALIZED UNIT COMMITMENT INCOMPETITIVE ENERGY MARKETS

Jinghai Xu and Richard D. ChristieUniversity of Washington

Abstract: In a competitive energy market, instead of, or in addition to, a centralized unitcommitment, individual generation owners will make independent unit com-mitment decisions. They will seek to maximize their profits against the predictedmarket clearing price. Their unit commitment strategy will be expressed in theirbids, so that they shut-down or start-up when the market price indicates such ac-tivity. In this chapter, we develop a unit commitment based price-taking (UCPT)bidding strategy with a simple price prediction mechanism and explore it using amarket simulator. Simulation results show that an individual generator hashigher profits with UCPT bidding than with simple price-taking bidding, andthat the cost of supplying price-inelastic loads achieved by the market is lowerwhen all generators use UCPT bidding. It appears that UCPT bidding gives re-sults similar to those from a Lagrangian relaxation unit commitment (LRUC),without a fix-up step, and it has problems with convergence and feasibility simi-lar to LRUC. We observe cyclic behavior in market prices with UCPT bidding,and we show that it depends on the price prediction mechanism. Alternativeprice prediction mechanisms can reduce cyclic behavior. Finally, we conceptu-ally explore potential strategic behavior and market power arising from unitcommitment constraints.

1. INTRODUCTION

The unit commitment problem – scheduling generator start-ups and shut-downs over a period of time to minimize the cost of serving expected loads -has been applied by the power industry and studied by researchers for dec-ades. Since unit commitment was typically performed for a set of generatorsall owned by one company—to meet load exclusively served by that com-pany—it was natural for the algorithm to assume that one central authoritycontrolled the status of every generator. This case is called centralized unitcommitment.

Deregulation has invalidated the assumption of centralized control. Anumber of different companies now own generators. Each company mustmake its own individual start-up and shut-down decisions, and cannot control


the decisions made by other companies. This case is called decentralized unitcommitment, because the commitment decision making is carried out in a de-centralized control structure.

Recent publications that consider deregulation and unit commitment dealmostly with centralized unit commitment. In some cases (such as the Englandand Wales Power Pool), the market structure requires generators to submit tocentralized unit commitment [1]. In other cases, researchers have assumed theexistence of a centralized unit commitment in decentralized markets [2, 3, 4].

Oren et al. [5], however, identify the problems inherent in the use of cen-tralized unit commitment. Specifically, they point out that due to the near-optimal nature of the solutions obtained by practical unit commitment algo-rithms, small changes in total cost can have large consequences for individualgenerators. When all generators are owned by one company, these differencesare not important. When generators are owned by different companies, thesedifferences are highly problematical.

The problems with centralized unit commitment have been recognized inpractice by various deregulated markets. California and the Nord Pool marketin Norway have no centralized unit commitment in the market process. ThePJM Interconnection has a voluntary centralized unit commitment, but allowsmarket participants to self-commit. Based on economic simulations the Cali-fornia tariff proposed an iterative energy market bidding scheme [13] to ac-count for start-up costs, but has so far decided not to implement it due to thecost of implementation and time constraints of the required communications.

Even if centralized unit commitment is required by, for example, connec-tion agreements, it seems unlikely that it can practically be enforced. If a gen-erator is required to run, and thinks that it should be shut down, it may sufferan operating problem of some kind that forces it off-line. Generators are wellknown to be the least reliable components of the power system and separatingintentional shut-downs from truly inadvertent ones is likely to be impractical.If a generator wants to run, but is required to shut down, it could, perhaps,claim restraint of trade, especially if its bid for the time period in questionshows that it is willing to run at minimum power for any price.

Note that we base this discussion of generator non-compliance on eco-nomic motivations over a relatively long period of time, and it does not ad-dress compliance during emergency conditions.

In [6], Li et al. introduce a market model that uses generator self-commitment to determine generator bids over a fixed time period, and thenthe usual market resolution process resolves the bids to determine marketprices. Then generator bids are redetermined for the same time period usingthe new prices. The concept of individual self-commitment to maximize prof-its given future market prices and the decentralized nature of the commitmentproblem are new. But the idea that bidders would have several opportunitiesto bid for the same time period is unworkable, as mentioned above.

Decentralized Unit Commitment 295

In the same journal, Huse et al. [7] introduce a method of computing gen-erator bids that includes self-commitment and generates one bid at a time.This is better-suited for most existing market resolution processes and for thegeneral desire in most markets to move bidding as close to operation as possi-ble. This technique, however, is introduced in the context of using a marketsimulation to solve a centralized unit commitment problem. Once marketclearing prices have been obtained for the time periods of interest, the biddingprocess is repeated for the same time periods, so an iteration loop is presentwhich is not present in real markets.

In this chapter, we apply the bidding strategy developed in [7] to a some-what more realistic market simulation. We then use the simulation to explorethe following questions: Is there an incentive for individual generators to payattention to unit commitment when bidding? What is the impact of unit self-commitment on the market, in particular, in comparison to a centralized unitcommitment?

Section 2 gives the mathematical notation of this model. Section 3 dis-cusses how generators can control their commitment through the form of theirbids. Section 4 describes several different bidding strategies that pay more orless attention to unit commitment. Section 5 illustrates the market simulationused, including the generation and loads. Section 6 gives results from the useof the different strategies, identifies an interesting cycling behavior in themarket, corrects it, and addresses the first question. Section 7 addresses thesecond question, and Section 8 discusses strategic bidding issues.

2. NOTATION

We used the following notation in our research:thermal time constant for the generator, in hourscold start cost, in $fixed start cost, in $

C(P) : cost function, in $/hour

modified cost function, in $/hourIC(P) :incremental cost function, in $/MWhM : unit self-commitment period

profit, in $/hourP : power output, in MW

high power output limit, in MWlow power output limit, in MW

p : market price, in $/MWhpredicted market price, in $/MWh

STC : start-up cost, in $


SDC: shut-down cost, in $/hourt: time that the generator was cooled, in hoursT: cyclic period of priceu : unit status, u = 1 if running, u = 0 if not runningw : price prediction weight

3. CONTROLLING COMMITMENT WITH BIDDING

In a Marshallian market where all bidders are paid the market clearingprice (MCP), the price-taking bidding strategy sets bids equal to incrementalcost for generators operating between minimum and maximum output power(segment CD in Figure 1a). What happens to the bid when these limits areapproached? For the upper power limit, the answer is easy. The generatorcannot produce more power no matter how much it is paid, so the bid curve issimply vertical (segment DE in Figure 1a). For the lower limit the answer isboth more difficult and more interesting.


If the generator is willing to stay on line at minimum power once MCPgoes below its incremental cost at its low output power limit atthen the curve simply extends downward, as shown by the segment BC inFigure la. When MCP reaches a level at which the generator is no longerwilling to operate, (the minimum acceptable price, or MAP), it will shutdown. The generator transitions from to zero output, as shown by the dot-ted line AB. Thus, by submitting the bid curve ABCDE, the generator informsthe market about its willingness to operate, and the market tells the generatorwhether to operate or shut down via the MCP.

It is certainly possible that the MAP for operation could be greater thanIn this case, the MAP supercedes the incremental cost. The bid curve for

this case is shown in Figure 1b.The transition from operating to shutdown poses feasibility problems. The

generator cannot actually operate below its minimum power but it isquite possible for a market solution to return an MCP equal to MAP, with ascheduled output between zero and (on section AB of Figure 1a). Resolu-tion of this infeasibility depends on the market type.

In an ex ante market, the generator has at least three possible ways to dealwith the problem. There is some time before the bid period starts in which thegenerator may be able to adjust its schedule by making bilateral contracts. Thegenerator also has an implicit contract with the ISO, since there must be ageneration adjustment mechanism of some kind to account for real time varia-tions in load and inadvertent generator outages. The generator could simplyshut down at the start of the hour and accept any costs charged for deviatingfrom its schedule. Finally, what is actually scheduled is the energy to be de-livered over the bid period. If metering is done on an hourly basis, the genera-tor could simply operate at minimum power for the amount of time requiredto deliver the scheduled energy, and then shut down. The time needed to re-duce generator power to zero may cause the generator to supply some energyto the power system while it shuts down even if it is scheduled for zero out-put, aiding this strategy. The generator will choose the lowest cost strategy,and it can factor any associated costs into its bids, as described later.

In an ex post market, a generator should simply shut down when assignedan output less than The real time nature of the ex post market will changebids in the next resolution period to account for changes in generator status.

Thus, if a minimum acceptable price (MAP) can be computed, generatorscan use this to modify their ideal price-taking bids, and the interaction of themodified bid with the market will signal the generator whether to stay on-lineor shut down in a given bid period.


4. BIDDING BEHAVIOR

Two bidding strategies with different approaches to finding the MAP arediscussed in this section. In developing these strategies, we make four as-sumptions: (1) generators are price-takers; (2) generators try to maximize itsprofit in the market; (3) each generator bids independently in the market; and(4) the generator’s incremental cost function is monotonically increasing.

4.1 Finding Minimum Acceptable Price

If a generator has no shut-down and start-up costs, the minimum accept-able price at which it will choose to operate is, therefore, the MCP at which itsprofit is zero. It is well known from microeconomic theory that this occurs atthe point where incremental cost equals average cost, which is also the mini-mum of average cost, i.e. [8]

The bid curve of the generator with this MAP is shown in Figure 2 (darkline). P* is the power output where average cost (AC) and incremental cost(IC) intersect.


4.2 Applying Power Output Limits

The power output limits of a generator, and modify the MAP.When P* is within the power output limits, i.e., the MAP isunaffected, as shown in Figure 3a.

With at the profit of the generator is

and is positive (see Figure 3b). Since the profit is zero at

which is the minimum value of average cost in the range andthe bid curve is shown in Figure 3b. Similarly, when the profit iszero at as shown in Figure 3c. In general,

4.3 Including Shut-down and Start-up Costs

The MAP is also modified by non-zero shut-down cost or start-up cost.Consider a non-zero shut-down cost. The MAP is the price that sets the gen-erator’s profit equal to the negative of shut-down cost (SDC), i.e.,

or

This is similar to equation (2) with a modified cost function:

Then the MAP is:

The MAP given by equation (8) may even be negative if SDC is highenough, giving the bid curve in Figure 4.


If a generator is shut-down in hour n-1, then the MAP for hour n mayhave to include the start-up cost STC. STC here can be treated as an extrafixed cost of operation. The modified cost function can be further revised to

and

If the generator is operating in hour n-1, then value of STC in hour n is zero.

4.4 Shut-Down Price-Taking Strategy

The MAP given by equation (10) controls commitment decisions basedon cost and income in hour n only. This strategy is referred to as shut-downprice-taking (SDPT).

The SDPT strategy is naive, since it considers only those costs incurred ina single period. A better approach to finding the MAP would include the ef-fect of the commitment decision on future profit.


4.5 Unit Commitment-Based Price-Taking Strategy

A bidding strategy that incorporates the future profitability of the genera-tor was first suggested in the unit commitment literature by Huse et al. [7].The future profitability here is evaluated by a unit self-commitment approach,so this strategy is referred to as unit commitment-based price-taking (UCPT).

In the UCPT strategy, when preparing the bid curve for hour n, a genera-tor first computes its future profits for the two cases of running and shuttingdown in hour n. Then the difference of the two projected future profits, or thelost profit due to shutting down, is treated as a extra cost and incorporated intothe MAP computation.

Suppose is the future profit if the generator is running in hour n, andis the future profit if the generator is shut down in hour n. The lost future

profit (which is referred to as in [7]) is given by:

Then the MAP for UCPT is computed by extending equation (10) as follows:

4.6 Unit Self-Commitment for UCPT

We find the future profit by solving a unit self commitment problem,formulated as:

Given a known operating state in hour n and predicted prices for hoursn+1 to n+M, find the commitment schedule for a generator that maximizes itsprofit in the interval n+1 to n+M, subject to generator output limits and mini-mum shut-down time constraints. Profits are income (which is market pricetimes power output) minus cost (fuel cost, and start-up cost) if operating, orthe negative of shut-down cost if shut down. The commitment does not con-sider minimum run time, ramp rate, or spinning reserve constraints, in order tosimplify the computation. Time varying start-up costs are considered.

In mathematical form,Given

Find and to maximize

s.t.


and minimum down-time constraints

Where

This problem is simple enough to be solved by forward dynamic program-ming. It resembles the single generator subproblem in Lagrangian relaxationunit commitment.

4.7 Price Prediction for UCPT

As indicated in equation (13), a future price profile is required when

computing future profits. This price profile has to be predicted by each indi-vidual generator. In this paper, we use two simple price prediction methodsbased on historical prices. In real life, various sophisticated price predictionalgorithms will be employed.

The first method is called the single period prediction (SPP). It assumesthat the market price is cyclic with period T, because the load profile used inthe example is also cyclic with period T. Then the estimated future marketprice is just the historical market price for the preceding cycle. Suppose thatthe historical prices before hour n are known. Then predicted future prices

starting from hour n are as follows (and are illustrated in Figure 5):


Note that is not computed because is unknown. The unit self-

commitment is therefore actually performed for m = l,...,T-1. Omitting thelast hour of a 168-hour week should have little effect on the result of futureprofit differential

If the historical price is not available (in the initial period), the predictedprice is set as:

As discussed later, the use of SPP leads to a cycling market price patternwith period 2T. To correct this, we use a weighted average price (WAP).Similar to SPP, a generator using WAP also assumes that the market pricesare cyclic with period T, but the predicted price here is a weighted combina-tion of the last two historical prices, i.e.,

Setting of the price prediction weight is discussed in the next section.

5. BID STRATEGIES IN MARKET SIMULATION

In this section, we test SDPT and UCPT in a simulated power market en-vironment. In addition, we address questions regarding the incentive for usingthe UCPT and the impact on the market efficiency.

5.1 Simulated Power Market

We constructed a 15-generator test case was constructed on a marketsimulator platform. There are no transmission constraints or costs. The gen-erator parameters, listed in Appendix 1, were chosen randomly from the 110-generator system published in [9]. In the market simulation, we include start-up and shut-down costs in the cost computation. The minimum shut downtime constraint of the generator is also enforced. We do not enforce other con-straints, such as ramp rate limits and minimum run time. We took a weeklyload (energy demand) shape from published market loads [12] and scaled it tothe generation in the example (Appendix 2). The same weekly load profilewas applied in every week of the simulation. The load bid for each hour isprice inelastic.


Market resolution proceeds hour by hour for ten weeks. In each hour,each generator submits a bid. The market price is decided by crossing the ag-gregated generator and load bids.

5.2 SDPT Results

When all generators use SDPT, the weekly average price is$13.220/MWh. The hourly price trajectory is shown in Figure 6. As expected,the prices for each week are identical.

5.3 UCPT Results

When all generators use UCPT with SPP price prediction with a one-week time horizon, we observe a weekly price oscillation with a period of twoweeks, presented in Figure 7. After a high average price week, price estimatesfor the coming week are also high. With high predicted future prices in theUCPT computation, lower MAPs result. With lower MAPs in supply bids,more generators stay on-line, which decreases the actual market prices andvice versa. We show weekly average prices in Figure 8.


When the price prediction horizon is extended to two weeks, we observe afour-week period of price oscillation (Figures 9, 10). This periodicity may bebecause near term predicted prices dominate the profit differential calculationand thus the value of MAP. The near term prices come from the initial hoursof the preceding period and are unaffected by recent market results.


To damp the weekly price oscillation, we replaced the SPP by WAP. Dif-ferent price weights, i.e. w = 0, 0.1... 1.0, in the WAP were tested. We findthe least price oscillation when w is 0.7, shown in Figure 11. The weekly av-erage prices in this case exhibit only small changes after five weeks as shownin Figure 12. In general, oscillations are well-damped for

Figure 13 shows the last four weeks of hourly prices from UCPT overlaidon each other. It is clear that the prices vary from week to week, indicatingthat there is not a clean convergence of the process.


Results show that WAP is better suited to predict future market prices. Inlater sections we used only WAP (with w = 0.7) in UCPT.

5.4 Addressing Questions

1. If all generators use SDPT, is one generator motivated to use UCPT?To answer this question, we constructed a test case with Generator 27 us-

ing UCTP and all other generators using SDPT. We chose Generator 27 be-cause it has the most status changes in the all SDPT case.

In the all SDPT case, Generator 27’s profit in the last four weeks is$5,367. When the generator changes its bidding strategy to UCPT, its profitincreases to $32,443. When Generator 27 uses UCPT in its bid, because of thelong-term profitability, lower MAPs are used in its bids. Lower MAPs allowit to run continuously, saving the start-up and shut-down costs.

Thus, Generator 27 is motivated to use UCPT by increased profits.2. If all generators use UCPT, is a generator motivated to use SDPT?

When all generators use UCPT, Generator 27’s profit for the last fourweeks is $15,807. If Generator 27 changes to SDPT while the rest of the gen-erators stay with UCPT, Generator 27’s profit falls to $5,614. Generator 27 ismotivated to use UCPT.

Profits of Generator 27 for the four scenarios are listed in Table 1.

3. Is the market better off with UCPT?With a price inelastic load, the total cost of generation is related to social

welfare. The best market, with the highest social welfare, has the lowest costof generation.

We compare generation costs of the four scenarios in previous subsec-tions. To avoid the initial unstable results, we use the data from the last fourweeks. A minimum total generation cost is observed in the scenario where allgenerators use UCPT. The maximum total cost is found when all generatorsbid with SDPT. Total generation costs in the last four weeks of different sce-narios are listed in Table 2.


When all generators use UCPT, the lowest average weekly price also oc-curs. This indicates that the price inelastic load also benefits from the sup-plier’s UCPT bidding.

The result with lower costs and lower average prices in the UCPT caseindicates that UCPT improves the efficiency of the specific market.

To broaden this observation, the 110-generator system and its associateddaily load curve from [9] are simulated. In this case, the market process isshortened to 10 days but the daily load curves are identical. The price predic-tion period is 23 hours. Other assumptions made in Section 5.1 still apply.

We test two cases. First, all generators use SDPT, then UCPT. In eachday of the SDPT case – except the first day when no start-up cost applies –the daily average price is $15.955/MWh, and the daily average cost and profitare $3,793,107 and $1,313,937 respectively. In the UCPT case, the result isstabilized after four days when using WAP with w = 0.7. Table 3 shows thedaily average prices, total cost and total profit for the UCPT case.

In the 110-generator system, after the initial day, the market efficiency ofUCPT is also higher than that of SDPT.


6. COMPARING UCPT WITH A LAGRANGIANRELAXATION SOLUTION

With the same 110-generator system and the same daily load curve usedin Section 5, we solve the unit commitment problem with a Lagrange Relaxa-tion Unit Commitment (LRUC) algorithm. The initial unit state for LRUC istaken from the UCPT solution for the last hour before the final day of thesimulation, allowing a reasonable comparison of LRUC results with the lastday of UCPT. We iterate LRUC 20 times and present the results in Table 4.The minimum generation cost was found in iteration 11. The total cost of theLRUC solution for the 11th iteration was only 0.02% less than UCPT. Theprofit was 0.47% lower in LRUC.

Note that the total daily cost of LRUC is lower than those reported in [7]and [9], because constraints such as minimum up-time and spinning reserveare not observed and the initial state differs.

LRUC and UCPT had very similar commitment patterns as shown in Ap-pendix 3. The LRUC commitment pattern is shown, with UCPT differenceshighlighted. Only three of 110 generators had different commitment patterns,two with different run time durations and one with a three-hour shut-down.


The differences in operating durations of generators 13 and 15 may be attrib-uted to the way LRUC deals with infeasible solutions, such as generatorpower outputs between zero and These are allowed by UCPT, as dis-cussed in Section 3, but not by LRUC.

Overall, comparison of LRUC and UCPT results gives some assurancethat centralized unit commitment solutions can be used as reasonable predic-tors of energy market prices, assuming no exercise of market power, and thusas benchmarks for market efficiency measures.

7. STRATEGIC BIDDING

We assumed in previous sections that generators are price takers. This as-sumption may not hold in reality. In a real world power pool, a generator willhave non-zero market power – the ability to affect the price of electricity. In-stead of bidding at its incremental cost, generators are likely to explore otherbidding strategies that maximize long term profit. This approach to bidding isoften called strategic bidding.

A common form of strategic bidding is to anticipate the response of otherbidders to one’s own bid. By examining the expected commitment behaviorof other generators, a strategic bidder can arrive at a computed profit differen-tial far different fromt that in a price-taking calculation, changing its MAP.

The technical constraints of unit commitment offer another opportunityfor strategic bidding. If competitors have generators with long time constants,such as long minimum shut-down times or long minimum run times, a strate-gic bidder with market power can attempt to drive market price down farenough to force the slow generators off-line at a strategic time, for example,just as load starts to increase, and then recoup the cost of this by exercisingmarket power while the slow units are off-line.

This problem is reminiscent of entry problems from economics. In gen-eral, slow units are cheap ones, and expensive units have short response times.The potential for entry (start-up) of an expensive unit should control the ef-fects of this sort of exercise of market power.

This discussion has only scratched the surface of unit commitment-awarestrategic bidding strategies. A lot of work can be done developing and evalu-ating such strategies, and developing methods to detect their employment,measure their effects on market efficiency and limit their negative effects.

8. CONCLUSIONS AND FUTURE WORK

We have explored two methods of including start-up costs and shut-downconstraints in market-based generation bidding. We found that prices depend


on the price prediction mechanism, and we showed that a simple weightedprice prediction methodology resulted in more stable market prices. Examplesshow that the Unit Commitment Price Taking (UCPT) strategy results inhigher profits for generators using it and in improved market efficiency.

A comparison between market based UCPT solutions and centralized La-grange relaxation unit commitment shows that the UCPT solution is veryclose to the LRUC solution, indicating that centralized unit commitment canbe used for price prediction in decentralized markets.

The area of strategic bidding, taking unit commitment considerations intoaccount, appears fruitful for future research. The possibility of a theoreticallink between the decentralized market-based unit commitment process withUCPT bidding, and centralized Lagrangian relaxation unit commitment, is anintriguing theoretical topic.

ACKNOWLEDGEMENTS

This work was funded by the Advanced Power Technologies Center at theUniversity of Washington, Chen-Ching Liu, Director. The authors are gratefulto Karl Seeley for discussions on economic issues.

REFERENCES

1.

2.

3.

4.

5.

6.

7.

8.9.

10.

D.P. Mendes and D.S. Kirschen, “Modelling of a competitive electricity power pool.” 32ndUniversities Power Engineering Conference, UPEC ’97, UK, 1: 387-390, 1997.G. Gross and D.J. Finlay. “Optimal Bidding Strategies in Competitive Electricity Markets.”In Proc. Twelfth Power Systems Computation Conference, PSCC, Dresden, 2: 815-823,1996.J.J. Ancona. A bid solicitation and selection method for developing a competitive spotpriced electric market. IEEE Trans. Power Syst., 12 (2): 743-748, 1997.G. Huang and Q. Zhao. “A Multi-objective Formulation for Competitive Power Market.”In Proc. 1998 Large Engineering Systems Conference on Power Engineering, LESCOPE98, Canada: 317-22, 1998.S.S. Oren, A.J. Svoboda, and R.B. Johnson. “Volatility of Unit Commitment in Competi-tive Electricity Markets.” In Proc. Thirtieth Hawaii International Conference on SystemSciences, 5: 594-601, 1997.C. Li, A.J. Svoboda, X. Guan, and H. Singh. Revenue adequate bidding strategies in com-petitive electricity markets. IEEE Trans. Power Syst., 14(2): 492-497, 1999.E.S. Huse, I. Wangensteen, and H.H. Faanes. Thermal power generation scheduling bysimulated competition. IEEE Trans. Power Syst., 14(2): 472-477, 1999.R.S. Pindyck and D.L. Rubinfeld. Microeconomics, Fourth Edition. Prentice Hall, 1998.S.O. Orero and M.R. Irving. Large scale unit commitment using a hybrid genetic algorithm.Elec. Power Energy Syst, 19(1): 45-55, 1997.A.J. Wood and B.F. Wollenberg. Power Generation, Operation, and Control, Second edi-tion. John Wiley & Sons, Inc., 1996.

Decentralized Unit Commitment

11.

12.13.

S. Takriti, J.R. Birge, and E. Long. A stochastic model for the unit commitment problem.IEEE Trans. Power Syst., 11(3): 1497-1508, 1996.California ISO Web Site, http://www.caiso.com.P.H. Griffes. “A Report to the California Power Exchange: Iterative Bidding in the PXMarket.” ANALYSIS GROUP/Economics, 1999, http://www.calpx.com/news/publications/index.htm.

APPENDIX 1. PARAMETERS OF 15 GENERATORS

MDT: minimum down timePUC: A, B, and C: coefficients of polynomial cost functionPmax / Pmin: maximum/minimum power outputSCC: cold start-up costSCF: fixed start-up costST: time constant of cold start-up costSDC: shut-down cost

313


APPENDIX 2. WEEKLY LOAD PROFILE

APPENDIX 3. UNIT COMMITMENT PATTERN FROMLRUC FOR 110 GENERATOR CASE

INDEX

ABACUS 154, 161AC transmission 9, 75, 78, 271, 278AMPL 72, 161ancillary services 6, 9, 16, 18, 20,

22, 26, 29, 33, 41, 76, 93, 95,96, 103, 107, 120, 122, 169,194, 251, 261, 266

arbitrage 33ARIMA 148artificial neural network 255, 265auctions 35, 185, 228Australia 190, 253autonomous agent models 190, 203,

253bc-opt 154, 161bid optimization 46, 231bidding 9, 16, 20, 21, 32, 34, 43, 55,

93, 185, 191, 207, 233, 256,283

bidding strategies 251, 252, 255,257, 259, 264, 267, 295, 296,304, 311

bids, multipart 33bilateral trading 17, 19, 26, 28, 29,

185, 188, 228, 274BPMPD 85branch-and-cut 154, 156, 157bundle method 77, 264California 17, 20, 22, 24, 32, 39, 46,

97, 135, 188, 249, 259, 294Chicago Board of Trade 191competitive market 15, 41, 53, 55,

94, 185, 227, 249, 274, 275,293

complementarity problem 230cost recovery 175, 176, 181Cournot model 230CPLEX 4, 6, 153, 235, 241, 243

cutting plane 154, 155, 167, 168,171

DC power flow approximation 10,75, 78, 79

decision trees 252demand-side 23, 26, 30, 44, 48, 56,

242demand-side bidding 53, 67, 276dispatch 114, 123, 193, 195, 231DISTCO 188, 271, 274, 276, 280distributed energy 10DSI-OPF 51duality gap 72, 76, 89, 134, 167,

175, 176, 181,278dynamic programming 2, 71, 75, 77,

81, 93, 125, 129, 162, 171,193, 212, 219, 221, 230, 260,303

dynamic programming, stochastic99,109,117,122,141,144,260, 263

economic efficiency 32Edgeworth approximation 147, 150emissions 10EMOS 40, 50EMOSL 160, 161, 162, 163EPRI ANN-STLF 51EPRI-DYNAMICS 51EPRI-OTS 51equivalencing method 211, 214ESCO 40, 44, 49, 188, 190experimental economics 91, 188,

189fairness 90, 294FERC 16, 96fuel price 123, 135fuel price forecasts 97game theory 252, 268


GAMS 161, 241GENCO 40, 44, 185, 188, 190, 193,

271, 274, 276, 279genetic programming 2, 185, 196,

204, 212, 253genetic programming automata 205hydropower 9, 41, 230, 233, 235,

237, 239, 244, 260, 262, 263Iberdrola 248India 217integer programming 2, 4, 125, 153,

155, 159, 171, 212, 235, 238,262

interior point method 167, 168, 171,174, 182

interruptible loads 9ISO 3, 9, 17, 18, 19, 23, 24, 25, 26,

31, 39, 43, 44, 46, 48, 140,188, 250, 274, 276

Ito processes 135Lagrange multipliers 2, 9, 60, 70,

77, 81, 143, 170, 176, 263,279, 285

Lagrangian relaxation 2, 4, 54, 57,70, 75, 78, 79, 84, 88, 89, 90,94, 118, 127, 133, 162, 167,170, 176, 193, 212, 219, 252,256, 258, 260, 262, 278, 293,303, 310

limits constraints fixing technique223

LINDO 153linear programming 6, 153, 158load recovery 54, 57, 62locational pricing 23, 25, 42, 46, 48,

271, 287, 290LSE 50maintenance 203market design 15, 18, 28, 31, 35market power 9, 18, 20, 28, 35, 227,

231,293,311

market simulation 9, 39, 187, 189,190, 203, 228, 250, 268, 293,304

Markov process 94, 95, 96, 97, 108,141, 146, 260, 262, 263, 265

Mathematical Programs with Equi-librium Constraints 9

MATLAB 85, 88MATPOWER 85MINOS 72, 85, 90MINTO 154, 161MIPO 154, 161modeling environments 160modeling languages 161Monte Carlo 96, 103, 104, 105, 106,

109, 141, 147, 150MPL 161MultiMATLAB 88, 90New England 16, 20, 22, 23, 24, 33,

97, 188, 250, 252, 260, 266New York 20, 21, 23, 24, 25, 42, 97New Zealand 42, 252Newton algorithm 285nominal group 7Nord Pool 294normal approximation 147, 150Northern Ireland Electricity 163numerical application 60, 85, 100,

106, 127, 130, 147, 162, 178,179, 200, 217, 241, 259, 266,286, 304

NYMEX 135OPF 42, 84, 85, 90OPL 161optimal self-commitment 93optimization 2options 103, 121, 129, 139Order 2000 18, 27Order 888 17, 19ordinal optimization 253, 258, 259OSL 153, 235

Index 319

parallel implementation 88penalty bundle method 168PJM 16, 18, 20, 21, 23, 24, 25, 26,

40, 42, 46, 97, 188POOLCO 16, 168, 274, 275, 282price 54, 67, 120, 143, 193, 244,

250, 254, 261, 296, 304, 306,307

price elasticity 9price forecasting 9, 45, 96, 103, 107,

187, 192, 197, 231, 260, 268,293, 303, 306, 311

price inversion 22price uncertainty 93, 98, 135, 143,

145price volatility 104, 111, 140, 251priority lists 2, 72, 212probabilistic production-costing

model 140, 145profit optimality 271, 281PROFITMAX 106programming 233pumped storage 48, 233, 237, 245,

260, 262, 263, 265, 266PX 17, 19, 20, 28, 39, 47, 49, 250,

259ramp rate constraints 76, 90, 93, 95,

170, 195, 214, 216, 239, 262,277

Rational Buyer Protocol 22reactive power 78, 79regression model 45, 258regulation 15, 17, 186reliability 9, 10, 33, 188, 202reserves: See ancillary servicesrevenue adequacy 252

risk management 47RTO 18, 27, 30, 31scenario analysis 262scenario tree 117, 127, 129schedule coordinators 3, 16, 40, 47settlement systems 21, 23simulated annealing 212simulation 43, 54Spain 248start-up 21, 55, 93, 95, 100, 107,

120, 142, 193, 195, 233, 238,239, 255, 263, 273, 299, 302

statistical data-mining 96, 190statistical methods 137stochastic dynamic programming:

See dynamic programming,stochastic

stochastic programming 118strategic behavior 65sub-gradient technique 71, 77, 127,

167, 172, 264Tractebel 162TRANSCO 188, 272, 280transmission 18, 20, 23, 27, 30, 32,

42, 202, 272, 277, 280, 283,285, 286, 288

transmission rights 25, 28UK 53, 54, 252, 294uncertainty 44, 251, 266unit decommitment method 117,

125variable duplication technique 79voltage limits 76, 78, 80Walrasian equilibrium 284XPRESS-MP 153, 158, 163zonal pricing 24

The Next Generation of Electric Power Unit Commitment Models

Documents

Transcript of The Next Generation of Electric Power Unit Commitment Models