Stochastic Combinatorial Optimization Approach to Biopharmaceutical Portfolio Management

Stochastic Combinatorial Optimization Approach to Biopharmaceutical PortfolioManagement

Edmund D. George and Suzanne S. Farid*

The AdVanced Centre for Biochemical Engineering, Department of Biochemical Engineering,UniVersity College London, Torrington Place, London WC1E 7JE, U.K.

Key strategic decisions in biopharmaceutical portfolio management include drug selection, activity scheduling,and third party involvement. Optimizing strategies is complicated by uncertainty, dependency relationshipsbetween decisions, and multiple objectives that may conflict. This paper presents the development of a stochasticcombinatorial multiobjective optimization framework designed to address these issues. The framework simulatesportfolio management strategies while harnessing Bayesian networks and evolutionary computation concertedlyto characterize the probabilistic structure of superior decisions and evolve strategies to multiobjective optimality.This formulation is applied to a case study entailing a portfolio of five therapeutic antibody projects.Optimization was driven by two objectives that conflicted here: maximizing profitability and maximizing theprobability of being profitable. Initial analysis of competing strategies along the Pareto optimal front indicatedthat strategies with clear differences in comprising decisions can compete with similar reward-risk profiles.Hence optimization yielded results that were not intuitive but instead suggested that flexibility between strategiescan exist in such large-scale problems. A cluster analysis was used to identify the prevalence of broad andsuperior building blocks along the Pareto front. In-house development of drugs generally emerged as a preferredconstituent of superior strategies which suggested a drive toward minimizing contracting fees, premiums,royalty charges, and losses in sales revenue to third parties; no budgetary constraints were imposed in thiscase study. It appeared that strategies for scheduling activities had the most overarching impact on performance.Strategies for portfolio structure appeared to have the greatest degree of flexibility relative to other strategiccomponents.

1. Introduction

Biopharmaceutical drug development is expensive, lengthy,risky, and complex. The literature has seen published figuresin excess of $800 million for developing a single drug,1,2 with6-10 years as a typical development time.3-6 Developmentaluncertainties complicate the development process by introducingthe possibility that it may be necessary to terminate developmentof a drug before a return can ever be realized. Market-relateduncertainties introduce the risk that a marketed product maynot meet the revenue expectations of the developer. Hence, amajor and ubiquitous problem confronting biopharmaceuticaldrug developers is how and when to best make and implementcritical business decisions so that important rewards such asprofitability are optimized. Yet more expensive, risky, andcomplex is the development of a portfolio of drug candidates.Within a company’s pipeline, valuing one drug at a time is notsufficient and drug developers must consider the entire portfoliounder technological and market uncertainties and resourceconstraints.7 Here the developer must also make decisions tobest construct the portfolio to optimize the management ofresources and reward related trade-offs that each drug develop-ment project may introduce.

Decisions made on the portfolio will include deciding thenumber of comprising projects subject to those that are mostpromising given resource constraints. Also included is thescheduling of critical development activities subject to aprioritized order of development based on considerations suchas the identification of the most promising projects andaccounting for strategic windows of opportunity in targeted drugmarkets. Portfolio selection problems pertaining to drug devel-

opment have been addressed in the literature and cover a varietyof methodological features, technical problems, and scenarios.For example, see Subramanian et al.,8,9 Blau et al.,10,11 Rogerset al.,12 and Jain and Grossmann.13

In addition to decisions made on portfolio structure, drugdevelopers must also address acquisition of or access toinfrastructure for capacity-related decisions in drug developmentand manufacturing endeavors. These will include importantstrategic decisions such as whether to integrate activities withinthe company, to outsource them to a contract researcher (CRO)or manufacturer (CMO), or to partner with a company that hascomplementary developmental, manufacturing, and marketingresources. For example, strategic outsourcing to a contractorhas become a vital component of the research and developmentprocess14 and plays an increasingly important role in theoperations of established and emerging pharmaceutical com-panies.15 There are contributions that address the issue ofcapacity planning in the pharmaceutical industry using optimi-zation-based approaches of which the majority pertains to drugdevelopers who have or plan to have integrated in-housedevelopment and manufacturing infrastructures. There are somewho develop frameworks for wider settings which consider thirdparty developers and manufacturers. These include Rogers andMaranas,12 Oh and Karimi,16 and Rajapakse et al.17,18

Approaches have been reported for the development offrameworks that incorporate both the problems of portfoliomanagement and manufacturing capacity planning simulta-neously, representing an important advancement. These presentmore challenging and realistic large-scale optimization problemsthat offer vast extensions to the lines of inquiry for either ofthe two problems in isolation. Such approaches include thoseby Levis and Papageorgiou,19 Maravelias and Grossman,20 andPapageorgiou et al.21

* To whom correspondence should be addressed. Tel.: +44 (0)207679 4415. Fax: +44 (0)20 7916 3943. E-mail: [email protected].

Ind. Eng. Chem. Res. 2008, 47, 8762–87748762

10.1021/ie8003144 CCC: $40.75 2008 American Chemical SocietyPublished on Web 10/28/2008

Given the relevance and importance of contributions to large-scale portfolio and capacity optimization problems in biophar-maceutical settings, especially in modern industry, this paperaddresses the development of a holistic framework for thesimultaneous stochastic, combinatorial, and multiobjectiveoptimization of biopharmaceutical research and developmentportfolio management alongside manufacturing capacity plan-ning decisions. More specifically, given a set of drug candidatesand their uncertain development, manufacturing, and commercialparameters, alongside an availability of external corporate bodiesfor development and manufacturing with their various uncertaintechnical characteristics, this work presents a novel method forfinding the optimal structure of the drug development portfolio,the development sequence for the selected drug candidates, theschedule of critical development activities, and the itinerary ofactivities at specific stages that should be integrated in-house,outsourced, or partnered to maximize important multiple objec-tives. The overall framework is a combination of a simulation-based evaluation framework based on work by George et al.22

and a bespoke estimation of distribution algorithm23 (EDA) toiteratively evolve a population of candidate strategies. It is theaim of this paper to investigate the performance of strategiesoptimized via the proposed framework and to ascertain anyimportant implications.

The organization of the remainder of this paper is nowdescribed. Section 2 details the complete model formulation forthe stochastic evaluative framework and for the EDA. Section3 presents a case specific to the production of monoclonalantibody drug candidates to exemplify the use of the framework.Section 4 describes and discusses the results pertaining to thecase presented. The final section discusses any final conclusions.

2. Model Formulation

The stochastic optimization process makes use of an evalu-ative framework to capture the impact of decisions made by

strategies in the modeled environment and to estimate theirstochastic properties (Figure 1). This is coupled with an EDAthat operates the optimization procedure through the machinelearning of instances of decision variables that are associatedwith superior strategy performance. Any interdependent rela-tionships between decision variables and their instances are datamined and are used to build a probabilistic model on which theformulation of new strategies is based. The quality of strategiesis thereby iteratively improved.

The model is designed using C++ and MS Excel. The entiresimulated environment is assembled in C++ as this containsthe most frequently utilized calculations and the C++ languageoffers significant savings in computational time. MS Excel isused as the main graphical user interface as it offers a convenientmeans for storing, extracting, and manipulating data. Theenvironment is interfaced with MS Excel via a dynamic linklibrary where it is represented as a user-defined function.Decision variables are entered into the function and definedparameters are outputted. Modules for controlling the flow ofdata in the simulation and optimization procedures are compiledin Visual Basic for Applications (VBA), which is readilyexecutable in the MS Excel environment.

2.1. Evaluative Framework. The framework for evaluatingeach candidate solution (Figure 1) models the performance ofthe strategy by mapping the consequences of decisions madewithin the simulated environment. Various facets of the drugdevelopment process and its wider commercial environment arecaptured within the modular structure of the evaluative frame-work. Each module is designed to carry out a set of relatedcalculations, and as illustrated, these modules are networkedand eventually result in the evaluation of a set of criteria. Thefollowing subsections will detail the contents of these modules.

2.1.1. Superstructure of a Candidate Solution. Considergeneration G(t) as a population of candidate solutions at a giveniteration of the optimization procedure, t, within the total number

Figure 1. Schematic of the framework used to evaluate populations of candidate solutions and iteratively evolve superior populations of strategies.

Ind. Eng. Chem. Res., Vol. 47, No. 22, 2008 8763

of iterations, tmax, that can potentially maximize the objectivesof the decision maker. Each candidate solution, g, in G(t) hasa generalized structure for representing its decisions. Figure 2demonstrates this generalized structure for a portfolio of fivedrugs, where the structure can be segregated into three subtypes:sequence, timing, and third party usage.

The strategy for the drug development sequence, Dg, codesthe drugs that are included in the portfolio and the order in whichthey are commercialized:

Dg ) {Dg,i)1, Dg,i)2, Dg,i)3, ..., Dg,i)I} ∀ g ∈ G(t), i ∈ I (1)

where Dg,i is the drug chosen as the ith drug in the drugdevelopment sequence by the gth candidate solution, and I isthe total number of drugs in the drug development portfolio. Itis clear that i must be any drug not already chosen as part ofDg.

The timing strategy for the ith drug in the developmentsequence according to the gth candidate solution, Tg,i, com-mences development of drug i according to when drug i - 1reaches the beginning of a particular stage of development.These stages can be instantiated as the following: targetidentification (ID), preclinical testing (PC), phase I clinical trials(PI), phase II clinical trials (PII), phase III clinical trials (PIII),FDA review (FDA), and market approval (MKT). Thus

Tg ) {Tg,i)1, Tg,i)2, Tg,i)3, ..., Tg,i)I-1} ∀ g ∈ G(t), i ∈ I (2)

Tg,i ) {ID, PC, PI, PII, PIII, FDA, MKT} ∀ g ∈ G(t), i ∈ I

(3)

where Tg is the set of timing strategies for all drugs in theportfolio.

The corporate strategy belonging to the gth solution for theith drug at the jth development activity, Cg,i,j, has three possibleinstances for each stage: in-house or otherwise referred to asan integrated activity (I), outsourced activity (C), and partneredactivity (P). Hence

Cg ) {Cg,i)1, Cg,i)2, Cg,i)3, ..., Cg,i)I} ∀ g ∈ G(t), i ∈ I (4)

Cg,i ) {Cg,i,j)1, Cg,i,j)2, Cg,i,j)3, ..., Cg,i,j)J}

∀ g ∈ G(t), i ∈ I, j ∈ J (5)

Cg,i,j ) {I, C, P} ∀ g ∈ G(t), i ∈ I, j ∈ J (6)

where J is the number of development activities. This formatallows the framework to formulate a plan of critical activitiesto complete either by themselves or by another corporate body.The development activities, Cg,i,j, are structured as follows: Cg,i,1,target identification; Cg,i,2, preclinical testing; Cg,i,3, phase Iclinical testing; Cg,i,4, phase II clinical testing; Cg,i,5, phase III

clinical testing; Cg,i,6, manufacturing for phase I clinical trials;Cg,i,7, manufacturing for phase II clinical trials; Cg,i,8, manufac-turing for phase III clinical trials; and Cg,i,9, commercialmanufacturing.

Structuring the strategies Dg and Tg is straightforward. Eachinstance within Dg can only refer to one drug, while anycombination of timing instances for each Tg,i in Tg is permissible.Structuring the corporate strategy requires the application ofsome rules to avoid the formulation of nonsensical strategies.The first set of stipulations restricts the company from breakingand resuming contracts with partners:

If Cg,i,1 )P then Cg,i,j )P for j) {2, 3, 4, 5, 6, 7, 8, 9} (7)

If Cg,i,2 )P then Cg,i,j )P for j) {3, 4, 5, 6, 7, 8, 9} (8)

If Cg,i,3 )P then Cg,i,j )P for j) {4, 5, 6, 7, 8, 9} (9)

If Cg,i,4 )P then Cg,i,j )P for j) {5, 7, 8, 9} (10)

If Cg,i,5 )P then Cg,i,j )P for j) {8, 9} (11)

If Cg,i,6 )P then Cg,i,j )P for j) {3, 4, 5, 7, 8, 9} (12)

If Cg,i,7 )P then Cg,i,j )P for j) {4, 5, 8, 9} (13)

If Cg,i,8 )P then Cg,i,j )P for j) {5, 9} (14)

This assumes that the partner is required for both clinicaltrial development and manufacturing. A second set ofstipulations prevents the breaking of outsourcing contractson clinical trial and clinical manufacturing activities:

If Cg,i,3 )C then Cg,i,j )C for j) {4, 5} (15)

If Cg,i,4 )C then Cg,i,j )C for j) {5} (16)

If Cg,i,6 )C then Cg,i,j )C for j) {7, 8} (17)

If Cg,i,7 )C then Cg,i,j )C for j) {8} (18)

It is assumed that contracting for clinical trial testing andcontracting for manufacturing will be completed by separatecompanies.

These stipulations have an added benefit of reducing thedecision space and computational time. Without them each drugwould have a total decision space for Cg,i of 39 solutions overits nine constituent decision variables; now this is reduced to207 solutions. To support computational efficiency, Cg,i isprogrammed as a single variable having 207 instances, ratherthan as nine Cg,i,j variables with each having three instances.This format means that Cg,i can be instantiated from a databaseof permissible combinations of each Cg,i,j, as opposed to havingan active procedure that validates and corrects the structure ofCg,i for each g in G(t) over each t in T.

Figure 2. Superstructure of a candidate strategy for the commercialization of a portfolio of five drugs.

8764 Ind. Eng. Chem. Res., Vol. 47, No. 22, 2008

2.1.2. Stochastic Variables. To simulate the uncertainenvironment, the model accounts for the stochastic nature of arange of uncertain variables. In this paper, each stochasticvariable is characterized by a triangular probability distributionbased on the specification of maximum, minimum, and mostlikely values. The types of variables included in the modelformulation are costs, commercial factors, and manufacturingcapabilities. Each variable bears relevance to a single elementof Cg,i, and a separate distribution must be defined for eachpossible instance of the relevant element. Thus, for a portfolioof five drugs, individual distributions of 99 stochastic variablesmust be defined. Depending on the structure of g, the relevantstochastic variables will be selected for sampling. Cost variablesdirectly influence decisions made within Cg,i. Commercial factorvariables impact the structure of Di and the instantiation of Cg,i,9,and hence the choice of drugs as well as third party involvementfor commercial manufacture. Manufacturing capability variableswill influence decisions made on the instances of Cg,i,j for 6 ej e 9, and thus the involvement of third parties in clinical andcommercial manufacturing.

2.1.3. Biomanufacturing Model. The framework used tomodel the manufacture of biopharmaceutical products is basedon work reported by George et al.,22 Farid et al.,24-26 and Limet al.27,28 Included in the model are the main process andancillary tasks involved in the manufacturing procedure. Themain input parameters to the model are the annual demand tobe met by the facility, the expected fermentation titer, the overallproduct yield, and the probability of achieving success for asingle batch fermentation. Equations for calculating the utiliza-tion of equipment, materials, utilities, and labor are alsoincluded. Cost calculations are supported by a database of unitcosts for equipment and materials used in the manufacturingprocess.

2.1.4. Dependencies. Within a set of decisions it is possiblethat at least one of these decisions can have an impact on theperformance of remaining decisions that are yet to be executed.In the real world, this can be a consequence of making decisionsthat affect the utilization of the same tangible or intangibleresource. The framework recognizes three such contexts ofdependency where this type of impact may occur: contractual,revenue, and manufacturing cost.

Contractual dependencies refer to the utilization of a thirdparty for manufacturing or research activities. Here, the premiseis that the longer the period for which a third party is used andthe greater the number of activities that it is involved in, themore favorable the rates it charges become. Additionally, foreach third party there is a minimum charge that it will notbreach. This clearly affects how the corporate relation strategyperforms in the model.

Revenue dependencies refer to the impact that constituentsof the company’s drug portfolio create when competing withinthe same market. When multiple drugs compete in the samemarket, each drug can suffer from reduced returns in comparisonto what it might have achieved if commercialized in the absenceof competing drugs, given the same commercial environment.It is also possible that multiple drugs can enhance the salesrevenue of each other if they have complementary applications.In the model, the impact of this dependency is realized throughDg and Tg. The configuration of Tg is important here because,in the absence of other dependencies affected by these strategies,it may be more beneficial to stagger the development ofcompeting drugs so that there are periods of reduced or nocompetition. Also, the performance of these strategies will

depend on the revenue-related penalties or benefits of havingcompeting or complementary drugs in the marketplace.

Capital dependencies are modeled here as affecting the capitalexpense required for manufacturing a drug, due to the sharingof resources for structurally similar drugs. Where such drugsare being manufactured, it may be possible to use some part ofthe same manufacturing facilities, hence reducing the overallcapital requirements. The impact of this dependency is relianton the capital savings that can be realized within a group ofdrugs. This dependency is affected by the strategies for thestructure of the drug portfolio, where the choice of structurallysimilar drugs supports reduced capital expense requirements.The strategy for the order in which drugs are commercializedis also important because drugs commercialized later in thepipeline may require less capital. Additionally, these strategiesmust reconcile the penalties and benefits of their impact oncapital and revenue dependencies.

2.1.5. Timeline. The timeline module contains calculationsfor producing the entire schedule of activities for the portfolio.The inputs for the calculations are Dg, Tg, and the time it takesto perform each activity in the process of drug developmentand commercialization. The timeline is used to instruct the profitand loss module when to account for the various elements onincome and expense.

Estimation of Distribution Algorithm. Estimation of dis-tribution algorithms29 (EDAs), are similar to a class ofoptimization algorithms known as genetic algorithms30 (GAs)and have been shown to be a class of promising approaches insolving combinatorial optimization problems.31 Like GAs, EDAsfunction by iteratively evolving a population of candidatesolutions to the problem until a termination criterion is satisfied.A main driver by which GAs are thought to achieve this isthrough the manipulation of building blocks.30,32 Building blocksare a central concept in GA theory, where superior buildingblocks are expected to be key components of superior solutions.They can be any set of instantiations of decision variablespresent in any set of candidate solutions. Additionally, a superiorbuilding block can be considered as a set of instances of decisionvariables that work together to support the performance of anyset of superior candidate solutions in the objective spaceconsidered. The instances of decision variables forming abuilding block may also exhibit a dependency relationship whereadded value in superiority is achieved by the holistic presenceof a set of instances rather than necessarily through individualincremental increases in performance contributed by eachconstituent of the building block. EDAs are designed torecognize superior building blocks explicitly by constructingprobabilistic models of the states of decision variables that arepresent in superior solutions. Examples of simple EDAs inliterature include the univariate marginal distribution algorithm33

(UMDA) and the compact genetic algorithm34 (cGA). Examplesof more complex EDAs include the Bayesian optimizationalgorithm23 (BOA) and the multiobjective hierarchical Bayesianoptimization algorithm35 (mohBOA).

In this paper, and like the BOA, the data structure used asthe framework for the probabilistic model is a Bayesian network.A Bayesian network is an annotated directed graph that encodesprobabilistic relationships, and their use here derives from thefact that artificial intelligence researchers have used them toencode expert knowledge.36 The topology of the network islearned directly from the structure of top-performing candidatesolutions in S(t) and is then randomly sampled via theconditional probabilities that it encodes to generate G(t+1). The


EDA used here (Figure 3) is largely based algorithmic conceptsfound in the mohBOA.

More detailed information regarding the selection of superiorstrategies using fast nondominated sorting and crowding dis-tance, clustering of the objective space using k-means clustering,and issues concerning practicalities of constructing Bayesiannetworks can be found in the Appendix A, Appendix B, andAppendix C, respectively.

3. Case Study Description

A hypothetical case was formulated to illustrate and examinethe ability of the framework to discover optimal strategies forperformance against multiple objectives in an uncertain environ-ment. In this case a biopharmaceutical company has 10monoclonal antibody drug candidates available for developmentbut can only choose five. It needs to know which drug candidatesshould be chosen, their order of their development, the timingschedule of development activities, and which corporate bodiesshould be assigned to each development activity. The optimiza-tion model is considerate of the commercial characteristics ofdrug candidates (Table 1), durations and costs associated withvarious stages of the drug development (Table 2), and technical

manufacturing characteristics for each corporate body (Table3). As seen in Table 1, there are three groups of indication thatthese candidates belong to. Annual demand figures and com-pound annual growth rates (CAGRs) are based on realisticfigures. The dependencies for revenue and capital expense aredetailed in Table 4. Specifications for contractual dependenciesand technical probabilities of success are displayed in Table 5.

An example of the complexity of the decision space concern-ing the five-drug portfolio is discussed. Overall, each strategyconsists of 54 decision variables: five for the selection of drugsfor the portfolio, four for timing the commencement ofdevelopment activities for the next drug in the pipeline, andnine assignments of corporate bodies to critical activities foreach of the five drugs. Hence, the choice of portfolio structurehas 10 × 9 × 8 × 7 × 6 possibilities that are combined with

Figure 3. Pseudocode for the EDA.

Table 1. Commercial Characteristics of Available Drug Candidates

drug candidate drug groupannual demand

(kg/year) CAGR (%)

A 1 Tr(120,250,380) Tr(1.00,1.01,1.02)B 1 Tr(100,200,300) Tr(1.00,1.01,1.02)C 1 Tr(80,150,230) Tr(1.00,1.01,1.02)D 2 Tr(50,100,150) Tr(1.00,1.01,1.02)E 2 Tr(100,200,300) Tr(1.01,1.03,1.04)F 2 Tr(80,150,230) Tr(1.01,1.03,1.04)G 2 Tr(50,100,150) Tr(1.01,1.03,1.04)H 3 Tr(80,150,230) Tr(1.02,1.05,1.06)I 3 Tr(50,100,150) Tr(1.02,1.05,1.06)J 3 Tr(100,200,300) Tr(1.02,1.05,1.06)

Table 2. Duration and Cost Information for Various Phases of theDrug Development Processa

phase of development duration (years) cost ($MM)

target identification 1 Tr(3,5,8)preclinical 2 Tr(20,35,50)phase I clinical trials 1 Tr(5,15,20)phase II clinical trials 2 Tr(15,25,35)phase III clinical trials 3 Tr(45,85,125)scale-up synthesis 1 Tr(3,5,8)formulation 1 Tr(5,10,15)commercial preparation 1 Tr(1,2,3)marketing 1 Tr(2,4,6)FDA review 1 Tr(2,4,6)market lifetime 10 -ramp time to peak market: I Tr(3,4,5) -ramp time to peak market: C Tr(3,4,5) -ramp time to peak market: P Tr(2,1,3) -decay time after market expiry: I Tr(2,1,3) -decay time after market expiry: C Tr(2,1,3) -decay time after market expiry: P Tr(3,4,5) -

a The cost of producing a drug for market is determined by thebiomanufacturing cost model.


5P5 possibilities for the order of development. There are 74

possibilities for the overall timing strategy. Additionally, thereare 2075 possibilities for assigning corporate bodies to criticalactivities during the developmental phases and commercial lifeof each drug. Overall the entire decision space has a total of∼3.31 × 1021 individual strategies. If each strategy took 1 s toevaluate, it would take 1.05 × 1014 years to enumerate allpossibilities. Hence, the combinatorial optimization algorithmhas been specifically developed to efficiently search this vastdecision space.

The settings used for the mechanics of the optimizationprocedure are now stated and are true for all generations. Themaximum number of generations tmax ) 17. The size of a givengeneration |G(t)| ) 1000. The number of superior strategiesselected from a given generation |S(t)| ) 500. The number ofclusters used in the objective space z ) 3.

A few comments should be considered. The number ofgenerations over which the populations are improved was testedthrough multiple evaluations and was indicated by the lack ofprogression in subsequent generations and the consistency ofresults across multiple tests. Because of the hierarchicalsuperstructure of a candidate solution, the total number ofdecisions is 14 instead of 54, which would have been the caseif decisions on internalizing or externalizing critical activitieswere not grouped into a single deicsion. Using 250 Monte Carlo

trials per candidate strategy was found to be adequate forpurposes of making comparisons of quality between candidatestrategies. To ensure that performance metrics are entirelydependent on the strategy as opposed to being assisted by thegeneration of an opportunistic set of random numbers, the sameset of random numbers is used across strategies. The pseudo-random number generator used here is the Mersenne Twisteralgorithm.37 To enhance the efficiency in estimating stochasticoutput properties, stratified sampling was used for each sto-chastic variable where boundaries of significance were setindividually. Also, a discount rate of 20% was used forcalculating values of the profitability indicator, net present value(NPV).

4. Results

The case study results are discussed in the following sections,focusing on analyzing competing strategies along the Paretofront generated and identifying trends along the frontier usingcluster analysis.

4.1. Pareto Front Progression. In this section the perfor-mance of strategies in the final population is examined in termsof their ability to generate profits, satisfy multiple objectives,and provide an acceptable risk profile. The analysis highlightsthe challenges that exist when pursuing multiple objectives inthe context considered.

The progression of the EDA in discovering the Pareto frontis demonstrated in Figure 4, where the mean positive NPV (netpresent value) is a measure of reward and p(NPV > 0) is ameasure of risk. As might be expected, random initializationof the population of strategies has resulted in a generally evendispersion of the mean positive NPV and p(NPV > 0)performance attributes over the objective space. Interestingly,what can immediately be seen is that it is possible to constructa portfolio of drugs with a significantly higher probability ofattaining a profit than any of its individual comprising drugs.Here drugs with a probability of reaching market that is lessthan 0.25 have been pooled in such a way that offers the decisionmaker a p(NPV > 0) value in excess of 0.70. Progression ofthe algorithm in the objective space appears to be considerablymore constrained in the p(NPV > 0) dimension than it is in themean positive NPV dimension. Progression of the evolutionaryprocess has been accompanied by some degree of successionat each generation. It is apparent that the strategies available inthe objective space have generally converged toward a clearlydefined nondominated frontier, the Pareto front. The Pareto frontis wide enough to be inclusive of p(NPV > 0) values that mightbe realistically sought after by a decision maker in this setting.It is also apparent that for this problem the machine learningmechanisms have greatly reduced the variability encoded in theconditional and probabilistic models of superior strategies. Itshould be noted that the ability of the algorithm to do this is

Table 3. Technical Information for Manufacture of Drug Candidates According to Clinical Phase and Corporate Body

corporate body development phase whole process yield (%) fermentation titer (g/L) batch success probability

I phase I Tr(0.25,0.35,0.50) Tr(0.20,0.30,0.50) Tr(0.80,0.60,1.00)C phase I Tr(0.45,0.55,0.85) Tr(0.80,1.00,1.50) Tr(0.70,0.90,1.00)P phase I Tr(0.40,0.50,0.75) Tr(0.60,0.75,1.20) Tr(0.65,0.85,1.00)I phase II Tr(0.40,0.50,0.75) Tr(0.30,0.40,0.60) Tr(0.65,0.85,1.00)C phase II Tr(0.45,0.60,0.90) Tr(1.20,1.50,2.30) Tr(0.75,0.95,1.00)P phase II Tr(0.40,0.50,0.75) Tr(0.80,1.00,1.50) Tr(0.70,0.90,1.00)I phase III Tr(0.40,0.50,0.75) Tr(0.80,1.00,1.50) Tr(0.70,0.90,1.00)C phase III Tr(0.55,0.75,1.00) Tr(1.50,2.00,3.00) Tr(0.75,1.00,1.00)P phase III Tr(0.45,0.60,0.90) Tr(1.20,1.50,2.30) Tr(0.75,0.95,1.00)I market Tr(0.40,0.50,0.75) Tr(0.80,1.00,1.50) Tr(0.70,0.90,1.00)C market Tr(0.55,0.70,1.00) Tr(1.50,2.00,3.00) Tr(0.75,1.00,1.00)P market Tr(0.45,0.60,0.90) Tr(1.20,1.50,2.30) Tr(0.75,0.95,1.00)

Table 4. Specification of Dependencies As Related to the Number ofDrugs from the Same Group within the Chosen Drug DevelopmentPortfolioa

no. of drugs % of full revenue % of full capital % of full COGS

1 100 100 1002 85 93 953 75 85 904 65 78 85

a COGS, cost of goods sold.

Table 5. Specification of Contractual Dependencies and StagewiseProbabilities of Successa

royalty rate(% of revenue) probability of success

development stage CRO CMO partner group 1 group 2 group 3

target identification 6 - 60 0.90 0.80 0.85preclinical trials 7 - 58 0.90 0.90 0.95phase I clinical trials 8 10 56 0.75 0.85 0.75phase II clinical trials 9 12 54 0.50 0.40 0.55phase III clinical trials 10 14 52 0.70 0.60 0.80market - 16 50 0.95 0.95 0.95

a A decrease in royalty rate applies per additional drug and ismeasured in percentage points. Royalty rates decrease by 1% with theCRO, and by 2% with the CMO or partner for each additional drugdevelopment project for which they are involved. A minimum royaltyrate applies of 3% with the CRO, 5% with the CMO, and 40% with thepartner. Groups 1, 2, and 3 refer to drug groups 1, 2, and 3.


dependent on whether a new set of building blocks is able toclearly distinguish itself as superior to any existing alternativesin the evolutionary process. At some stages, succession in someregions of the objective space is accompanied by deteriorationin other regions. This deterioration refers specifically to thenonpopulation of certain regions that were previously populated.In this particular case, it was considered appropriate that theconvergence of the algorithm was tracked visually, principallybecause the objective space has only two dimensions. Theoptimization procedure was considered to be converged whengeneral progression of the Pareto front was insignificant, leadingto an unfavorable loss of regions along the Pareto front. Somedeterioration was tolerated in regions where rational decisionmakers were unlikely to seek strategies. An example of such aregion would be where p(NPV > 0) is less than 0.20 or wherethe mean positive NPV is less than $200 million. Furtheranalysis indicated that progression beyond 17 generations forthis case study leads to a severe deterioration of the Pareto front.

The superior strategies selected from the 17th generation(Figure 5) show that a nondominated frontier exists in theapproximate region 0.18 < p(NPV > 0) < 0.75. The frontierindicates that a definite trade-off exists here between maximizingthe mean positive NPV to be generated by a particular strategyand maximizing the probability of attaining a positively valuedprofit, p(NPV > 0). At no point along the frontier can the

maximization of mean positive NPV and p(NPV > 0) bealigned. Hence, improving p(NPV > 0) means accepting adegradation in mean positive NPV. Also, this forces the decisionmaker into the process of selecting strategies that begins withdeciding upon minimum acceptable levels of profit and prob-ability of profitability. The region 0.50 < p(NPV > 0) < 0.65offers the least trade-off along the frontier when searching forstrategies that improve p(NPV > 0). In practical terms, not allof the frontier will be appealing to the decision maker as therewill be strategies that either offer probabilities of success ormean positive NPV values that are too low to be consideredfor further action.

4.2. Analysis of Competing Strategies. A close examinationof competing strategies along the frontier indicates that theycan have similar reward-risk characteristics while demonstrat-ing marked differences in key decisions relating either to theportfolio structure (drug selection), timing, or third partystrategies. Four strategies along the nondominated frontier havebeen selected to illustrate this and demonstrate how a decisionmaker might rationalize the choice between two strategies ofsimilar risk and reward as visualized in Figure 5.

S1 and S2 are strategies that generate the greatest meanpositive financial reward but also the greatest risk of beingunprofitable. Although these competing strategies have verysimilar third party strategies and portfolio structures, they differ

Figure 4. Progressive discovery of the Pareto front: (a) 1st generation, (b) 3rd generation, (c) 7th generation, (d) 10th generation, (e) 14th generation, and(f) 17th generation.


greatly in their timing strategies as illustrated in Tables 6 and7. Both strategies share the approach of partnering for themajority of activities for the first drug and then preferring touse a combination of in-house and outsourced development andmanufacturing for the remaining drugs. The only exception tothis is for drug 3 with S1, where partnering is used forcommercial manufacturing. The first two drugs are eitherdeveloped within 3 years of each other, as in S1, or simulta-neously, as in S2, and developing the remaining drugs oncethese drugs are close to the marketplace. This approach has theadvantage of dividing the risk and impact of failure betweentwo drug development projects, and it is clear that the expectedprofits from either of the first two drugs are used to fund thedevelopment of future projects. S1 staggers the developmentof its drugs with lengthy development intervals between eachdrug. Resultantly, the time it takes to complete development ofthe portfolio and subsequent market production is 35 years. S2has a shorter development time of 26 years because of theshorter intervals between the developments of its final threedrugs. Each strategy develops almost the same portfolio of drugs,with the difference being that for the fourth drug S1 develops

drug G and S2 develops drug B. Much like the strategy withthe first two drugs, S1 develops its third and fourth drugstogether and S2 develops its fourth and fifth drugs within 3years of each other. For S1 these two drugs are taken from thesame group, whereas with S2 this is not the case. S1 and S2generate similar performances in mean positive NPV, but astrong reason for selecting S2 exists. S2 has the additionaladvantage of being executed over a significantly shorter period.

Examining the differences between another pair of competingstrategies illustrates how marked differences in third partystrategies can also result in similar reward-risk characteristics.In contrast to S1 and S2, strategies S3 (Table 8) and S4 (Table9) have similar timing strategies but very different third partystrategies as well as portfolio structures. S3 utilizes a mixtureof in-house, outsourced, and partnered activities, whereas S4develops the entire portfolio in-house. The portfolio constructedby S3 consists of one low-demand drug, two medium-demanddrugs, and two high-demand drugs that are sourced from allthree groups. The first three drugs are developed first andsimultaneously with the remaining two drugs developed withlengthy intervals of 9 years in between. For the second and thirddrugs contractors are only used for clinical development, clinicalmanufacturing is kept in-house, and commercial manufacturingis conducted with a partner. The fourth and fifth drugs usecontracting more intensively for clinical development, butcontractors are also used for clinical manufacturing where theyare used during phase III trials. The total portfolio developmenttime plus time taken during the marketing phase for S3 is 28years. S4 develops drugs from all three groups, and its portfolioconsists of two low-demand drugs, two medium-demand drugs,and one high-demand drug. Similar to S3, the first three drugsare developed in close succession and S4 also staggers thedevelopment of its final two drugs in a similar fashion. Thetotal time for development and marketing completion of S4 is

Figure 5. Mean positive NPV versus p(NPV > 0). (s) Estimate of the nondominated frontier. Strategies chosen for further analysis are annotated as S1,S2, S3, and S4.

Table 6. Structure of the S1 Portfolio Development Strategya

i Di Ci,1 Ci,2 Ci,3 Ci,4 Ci,5 Ci,6 Ci,7 Ci,8 Ci,9 Ti

1 F I P P P P P P P P PI2 I I C C C C I I I C FDA3 J C C I I C I I I P FDA4 G I I I C C C C C I PII5 H C I I C C I I C C -

a I, in-house activity; C, outsourced activity; P, partnered activity;Ci,1, target identification; Ci,2, preclinical studies; Ci,3, phase I clinicaldevelopment; Ci,4, phase II clinical development; Ci,5, phase III clinicaldevelopment; Ci,6, phase I manufacturing; Ci,7, phase II manufacturing;Ci,8, phase III manufacturing; Ci,9, market. ID, target identification; PC,preclinical testing, PI/PII/PIII, phase I/II/III clinical trials; FDA, FDAreview; MKT, market approval.



1 F I I I P P I P P P PC2 I I C C C C I I I C MKT3 J I C I I C I I C I PI4 B I I I C C C C C I PI5 H C I I I C I I C C -




1 F I P P P P P P P P PC2 D C I I C C I I I P PC3 B C C I I C I I I P FDA4 H I I C C C I I C I FDA5 J C I I C C I I C C -



30 years, which is similar to that taken for S3. Comparing theirperformances, S3 generates a marginally superior mean positiveNPV than S4 and is executed over a slightly shorter period.Although marginal, the performance results suggest S3 is thesuperior strategy.

From sampling the strategies along the Pareto front, it isobservable that their positioning in the objective space and eventheir occurrence within the optimized population of strategiesis not intuitive. It is taken that this is due to the complexitythat is inherent in the model of the biopharmaceutical drugdevelopment pathway and the complexity that governs theformulation of superior strategies. This is an important under-score because it highlights to the decision maker the value ofaccounting for the concept of building blocks, which isconsiderably difficult to achieve without the use of advancedcomputational tools. Another important observation is thatstrategies with clear differences in either drug selection, timing,or third party strategies can compete with similar reward versusrisk profiles. Hence it is useful for the decision maker to identifya desirable region along the frontier and closely examine thedifferent options that can yield the desired return and acceptablerisk.

4.3. Cluster Analysis. In order to investigate if any discern-ible trends in strategy formulation exist among the populationof superior strategies, the objective space was decomposed intothree clusters. It was anticipated that an analysis of these clustersmight add useful insight when considering if any particularregions of the Pareto front give rise to the prevalence of certainbuilding blocks. As building blocks effectively compete witheach other for selection, it would be useful to discover if andwhy any particular building blocks are emphasized.

4.3.1. Portfolio Structure. Table 10 displays the mostprobable constituents of strategy for each cluster. This includesthe annual level of demand for the selected drug, the group thatthe drug belonged to, and the time after which developmentof the next drug would commence. The reader is reminded thatgroup 1 drugs have a low annual growth rate, while groups 2and 3 respectively have medium and high annual growth rates.Also, group 3 drugs have the greatest probability of achievingmarketing approval while group 2 drugs have the least. Cluster1 consists of strategies that perform highly in mean positiveNPV, such as S1 and S2, and cluster 3 consists of strategiesthat have high values of p(NPV > 0). Cluster 2 representsstrategies that balance both of these performance metrics moreevenly, such as S3 and S4. The clusters and their memberstrategies are derived by application of the k-means clusteringalgorithm. For cluster 1, the most probable portfolio consistsof one low-demand drug, one medium-demand drug, and threehigh-demand drugs from all three drug groups. Strategies inthis cluster tend to utilize the largest number of high-demanddrugs, which assists in explaining the larger mean positive NPVvalues generated by these strategies. Also, a larger portion ofselected drugs belong to drugs with high and medium prob-abilities of success. For cluster 3 at the opposite end of the Paretofront, the most probable portfolio is three medium-demand drugsand two high-demand drugs drawn from all three drug groups,which perhaps surprisingly highlights that cluster 3 strategiesaim to meet a similar level of market demand as cluster 1 drugs.As cluster 3 strategies have significantly lower mean positiveNPV values than cluster 1 strategies, this indicates that thereare other decisions outside of portfolio structure which lowerthis profitability measure but may contribute to a greater p(NPV> 0) value. With the exception of one drug, the portfolio optsfor drugs with high and medium probabilities of success, thereby

using more drugs of this type than the other clusters. This isintuitive because this cluster exhibits the greatest p(NPV > 0)values. Overall, there appear to be no clear trends in portfoliostructuring strategies. The presence of certain portfolio structur-ing decisions within certain clusters appears to align with theposition in the objective space relative to other clusters, whilethere are others which do not. The remaining constituents ofthe portfolio development strategy will be investigated todiscover if more overarching drivers of performance exist.

4.3.2. Timing Strategies. Decisions on timing are an im-portant constituent of the portfolio development strategy as theyare used to favorably organize cash flows. This is particularlyimportant when having to consider the probability that a projectwill succeed, the financial impact of failed projects, and theimpact of the discount factor when determining the meanpositive NPV. In cluster 1, Table 10 shows that the prevailingstrategy is to develop the first two drugs together coupled withlong intervals of either 8 or 9 years between the developmentof the remaining drugs. Because of the probabilities of successof available drugs, it would be rational to expect only onesuccessful drug within a portfolio of five projects. Also, it isobservable that, when considering the growth and impact ofthe discount factor over time, cluster 1 strategies must relyconsiderably on the production of a successful drug from thefirst two projects. This assists in reasoning the low p(NPV >0) values seen in this cluster. Remaining drugs have compara-tively little impact on cash flows because of the extent to whichtheir respective cash flows are discounted. This also providesan explanation for the superior mean positive NPV values, aswhen one successful drug emerges from within the first twoprojects its profits are not greatly discounted. Also, andimportantly, profits from this drug will mainly need to absorbthe expense of its own development and that of the otherconcurrent project. The cost of developing the remaining drugsis less restrictive as these are heavily discounted, although thisoccurs with a relatively modest probability. The time for



1 F I I I I I I I I I PC2 C I I I I I I I I I ID3 G I I I I I I I I I FDA4 J I I I I I I I I I FDA5 I I I I I I I I I I -


Table 10. Characteristics of Drug Selection and Timing Strategiesin Each Cluster

cluster characteristic drug 1 drug 2 drug 3 drug 4 drug 5

1 demand M H L H Hdrug group 2 1 2 3 3timing ID FDA PIII FDA -

2 demand M L H H Ldrug group 2 2 1 2 2timing PC PC MKT FDA -

3 demand M M H H Mdrug group 2 1 3 1 3timing ID FDA PII PII -

ID, target identification; PC, preclinical testing, PI/PII/PIII, phase I/II/III clinical trials; FDA, FDA review; MKT, market approval; H, high;M, medium; L, low.


development until the end of marketing for this strategy is 34years.

For cluster 2 it is shown that the first three drugs are mostlikely to be developed with short intervals of 1 year betweenthem and that the time for development and marketing comple-tion for this approach is 31 years. This strategy is a driver ofthe higher p(NPV > 0) values and the lower mean positive NPVvalues observed when making comparisons to cluster 1. Likecluster 1, the remaining drugs are likely to be developed withthe longest intervals between them and are anticipated to havemodest impacts on cash flow because of the magnitude of thediscount factor at these stages in portfolio development.Considering this, such strategies are reliant on at least onesuccessful drug emerging from the first three projects. Develop-ing the first three drugs in close succession in cluster 2 bears agreater likelihood that at least one successful project will emergefrom this group than from within the remaining two drugs. Whena successful drug emerges from this group, because of a morefavorable discount factor there is also a greater likelihood thatit can also cover the expense of other failed projects than whendeveloping two drugs in close succession. Having to effectivelyabsorb the cost of an increased number of projects whosedevelopment costs are less discounted contributes to lower meanpositive NPV values.

Strategies in cluster 3 exhibit a tendency to develop the firsttwo drugs together followed by a period of at least 9 years;then later drugs are developed with medium-length intervals of4 years in between them. The time for development andmarketing completion most likely in cluster 3 is 27 years. Thisstrategy relies on at least one successful drug being found intwo distinct groups, that is, a group of two drugs developedtogether at the outset and a group of three drugs developed inclose succession later in the timeline. It has already beenmentioned that the impact of developing two drugs togetherwhile positioning the remaining projects to subject them to farmore significant discounting results in relatively high meanpositive NPV values with relatively low probabilities of success.It follows that some aspects of the development of the remainingthree drugs contribute significantly to the high p(NPV > 0)values but low mean positive NPV values ultimately exhibitedby strategies in this cluster. The shorter portfolio developmenttime means that profits for each year are discounted by asignificantly smaller extent than seen with clusters 1 and 2.Strategies in this cluster also take advantage of the high annualmarket growth rate and high probability of success of group 3drugs and position them late in the pipeline of products tomaximize their potential for generating revenues that overcomethe magnitude of the discounting. This is important because ifeach drug is to contribute significantly to the revenue generatedby the portfolio, then any successful drugs must support up tofour failed projects, which helps to explain the smaller meanpositive NPV values. Also, it is more likely to see at least onesuccessful drug emerge from within the second group of threeprojects than from the first group of two. When this occurs,profits arise from these drugs which are heavily discounted andmust also be enough to cover the expenses of failed projectswhich are less heavily discounted. This further erodes thepotential magnitude for mean positive NPV. Overall, the trendwith timing strategies is that portfolio development times tendto become progressively shorter toward the right-hand side ofthe frontier and that strategies make use of grouping projectsmore closely together. Interestingly, the presence of the discountfactor appears to have a significant influence on strategyformulation. Hence it is anticipated that alternative settings for

the discount rate may result in a noticeably different set ofsuperior strategies.

4.3.3. Third Party Strategies. The probability of selectinga particular strategy for a particular drug position in a particularcluster of the objective space is addressed, and the top twostrategies in each circumstance are shown in Table 11. In allcases the most probable third party strategy for each drug is tointegrate all activities in-house. This is intuitive from theviewpoint of minimizing contracting fees and premiums as wellas avoiding the sharing of sales revenues and royalty charges.As seen in Table 11 and with strategies S1 through S4,alternative third party strategies are present in significantproportion within the population. This demonstrates that whensupported by appropriate strategic decisions in other areas theseare viable constituents of superior strategies. It is postulatedthat one possible reason for complete in-house strategies to bepresent with significant likelihoods is that such strategies mayserve as a more flexible building block than others for creatingsuperior portfolio development strategies. With the aim ofmanaging the impact of failure, it may seem counterintuitivefor a company to go the entire drug development process withoutthird party assistance. Because no budgetary constraint exists,in such scenarios there is no direct requirement to limit theimpact of failure. However, to some degree maximizing meanpositive NPV is likely to involve some mitigation of the impactof failure. As might be expected, it can be seen that strategiesin cluster 1 are most likely to choose to develop drugs entirely

Table 11. Selected Probabilities of Third Party Strategiesa

Di Ci,1 Ci,2 Ci,3 Ci,4 Ci,5 Ci,6 Ci,7 Ci,8 Ci,9 p

Cluster 1

drug1 I I I I I I I I I 0.64I I I I P I I P P 0.18

drug2 I I I I I I I I I 0.64I C C C C I I I C 0.12

drug3 I I I I I I I I I 0.64C I C C C C C C P 0.12

drug4 I I I I I I I I I 0.64I I I C C C C C I 0.10

drug5 I I I I I I I I I 0.63I I I C C I I I C 0.09

Cluster 2

drug1 I I I I I I I I I 0.38I P P P P P P P P 0.15

drug2 I I I I I I I I I 0.38C I I I I I I C P 0.15

drug3 I I I I I I I I I 0.38C C I I C I I I P 0.13

drug4 I I I I I I I I I 0.38I C I C C C C C C 0.15

drug5 I I I I I I I I I 0.38I C C C C C C C P 0.15

Cluster 3

drug1 I I I I I I I I I 0.54I I I I P I I P P 0.26

drug2 I I I I I I I I I 0.53I C C C C I I I C 0.16

drug3 I I I I I I I I I 0.53C I C C C I I I C 0.14

drug4 I I I I I I I I I 0.53I I I C C C C C I 0.15

drug5 I I I I I I I I I 0.53I I I I C C C C C 0.10

a For each Di in each cluster, the two most probable strategies havebeen displayed; I, in-house activity; C, outsourced activity; P, partneredactivity; Ci,1, target identification; Ci,2, preclinical studies; Ci,3, phase Iclinical development; Ci,4, phase II clinical development; Ci,5, phase IIIclinical development; Ci,6, phase I manufacturing; Ci,7, phase IImanufacturing; Ci,8, phase III manufacturing; Ci,9, market.


in-house as these offer the greatest potential for achieving thegreatest mean positive NPV values if the drug passes all stagesof testing and review. For the first drug an alternative strategy,albeit much less likely in this case, is to partner from thecommencement of phase III clinical trials or from preclinicaltesting. Partnering from phase III clinical trials allows for somesavings on the proportion of revenue that must be paid to thepartner. Partnering from preclinical testing onward allows forthe impact of failure to be shared, but a substantial share ofany sales revenues must be paid to the partner. Across allclusters strategies tend to utilize partners much less intensivelyfrom drug 1 onward, where partners are used almost exclusivelyfor commercial manufacturing and contractors are preferred. Theuse of contractors does not appear to be limited to a particularset or series of activities, and in most cases the use of contractorsis a supplement to integrated activities. Finally, although themodel shows that in-house activities are the most probable foroptimized strategies, the presence of budgetary constraints isexpected to impact such a decision. This will be explained in alater paper.38

5. Conclusions

The development of a stochastic multiobjective combinatorialoptimization framework has been presented that addresses threekey decisions simultaneously: portfolio management, schedulingof drug development and manufacturing, and the involvementof third parties for specific activities. Demonstrated within thiswork is the value carried by considering these critical strategicconsiderations within a unified framework that simulates andoptimizes all such decisions across the entire product portfolio.A case study was used to illustrate the capabilities of theframework and also highlighted that the scope of decisions thata drug developer may be confronted with can be vast andcomplex. Due to the complexity of this problem, a principlecontribution of this work is in demonstrating a formulation basedon techniques from artificial intelligence, in particular evolution-ary computation and machine learning, employed for an efficientsearch of the decision space and for effective traversing of theobjective space.

It is proposed that biopharmaceutical product developmentstrategies in the real world may be better analyzed whenconsidering the impact of decisions holistically rather than onlyindividually. One reason for this is the presence of dependenciesbetween decisions that may impact economic relationships.Another is that it has been demonstrated that an effectivestrategy for portfolio development can result in a p(NPV > 0)value that is significantly greater than the probability ofsuccessful development for any singular drug in the portfolio.Hence, by considering a portfolio of multiple drugs it is possibleto control its risk to some extent through careful strategicformulation, while this is not possible with a singular drug. Useof the model has highlighted that pursuit of mean positive NPVcan conflict with pursuit of high p(NPV > 0) values, althoughthis may change under different case study settings. The resultsof the case study lean toward suggesting the integration of allactivities in-house. This can conflict with common perspectivesin industry that accept such strategies with reluctance becauseof the uncertainty of the drug development process and theconsequential impact of failure. The added presence of budgetaryconstraints and a range of sizes for the portfolio would serveas factors that can further capture limitations in the real worldand are also capable of significantly influencing results seenhere; this is explored in a further paper.38 Furthermore, theoptimal set of solutions is expected to be sensitive to the relative

difference in manufacturing efficiencies assumed between in-house and external manufacturing.

Finally, the learning of Bayesian networks from superiorsolutions presented here has been shown to be effective andefficient in improving the population, and in discovering a denseand widespread Pareto front. Their effectiveness is presumedto be due to their ability to iteratively learn and exploit thestructure of the problem as noted by contributions in artificialintelligence literature. A noteworthy insight from using theframework is that use of machine learning has potential forfuture development in solving portfolio development andcapacity planning problems simultaneously.

Acknowledgment

The Advanced Centre for Biochemical Engineering (ACBE)is gratefully acknowledged. Support for the Innovative Manu-facturing Research Centre for Bioprocessing (IMRC) housedin the ACBE by the Engineering and Physical Sciences ResearchCouncil (EPSRC) is also gratefully acknowledged. Financialsupport in the form of an Engineering Doctorate (EngD)studentship for E.D.G. provided by BioPharm Services and theEPSRC is gratefully acknowledged.

Appendix A. Selection of Superior Candidate Solutions

To facilitate the selection of superior candidate solutions, fastdominated sorting and crowding distance algorithms39 are usedtogether. Both procedures are used to support the discovery ofa widespread and densely populated Pareto optimal front. Fastdominated sorting ranks each strategy according to its degreeof Pareto optimality in the objective space. A candidate solutionis Pareto optimal if there is no other solution that exists whichyields an improvement in any objective without yieldingdegradation in any other objective. To support the function ofthis sorting procedure, crowding distance is used to determinethe proximity of a candidate solution to others in the objectivespace. The crowding distance measure for a candidate solutionis the perimeter of the cuboid created by its two most proximalcandidate solutions in the objective space. The premise is thatcandidate solutions situated in less densely populated areas ofthe objective space are considered to be more unique and thuscan contain information not already discovered. Probabilisticmodels that are more inclusive of unique candidate solutionsfacilitate the use of a diverse base of information for theformulation of new superior candidates. Fast nondominatedsorting is used first to select candidate solutions in order of rankand add them to the set of superior performing candidatesolutions, S(t). If all the members sharing the same rank willnot fit into S(t), then candidate solutions are chosen in order ofthose having the greatest crowding distance. Algorithmicallysteering the optimization procedure toward greater inclusion ofcandidate solutions along the Pareto front has its advantageshere. First, the nature and severity of conflict between objectivescan be better understood with wider coverage of the objectivespace. Second, a more efficient traversing of the objective spacecan be achieved through accelerated progress at individualiterations due to probabilistic models that model nondominatedsolutions more closely.

Appendix B. k-Means Clustering

Once S(t) has been populated, it is clustered so that the propertiesof superior candidate solutions can be analyzed for region-specific properties in the objective space. The k-means clusteringalgorithm40 was selected for this purpose because its ease of


implementation. The advantage of clustering the objective spaceis that a probabilistic model can be built for each cluster, therebysupporting the generation of a widespread Pareto front. Thisalgorithm requires the number of clusters to be specified bythe decision maker.

Appendix C. Constructing the Bayesian Network

Bayesian networks41 are used here to characterize the proba-bilistic nature of relationships between decision variables andtheir instances occurring in the set of candidate solutions ofthe zth cluster, Kz, via machine learning. This data structure issubsequently sampled for the generation of G(t + 1). As theobjective space is partitioned into z clusters a Bayesian network,Bz, is constructed for each z. Each decision variable, δn, isrepresented by a node, δn,z, in the network Bz. Any relationshipsbetween each δn,z are represented by a directed edge drawnbetween them (Figure 6). The node from where the edge isdirected is a parent node, while the node to which the edge isdirected is its child node. That is, within each Bz the instanceof each child node is conditional upon the set of instances ofits parents. It is important to note that all Bz’s are acyclic,so a node cannot be directed to itself nor can it be part of astructure within Bz that eventually ends up being directed toitself (Figure 6).

In order to avoid this, this paper takes advantage of the natureof the problem for which this framework is developed and putsall δn,z into ascending order according to the time in thedevelopment schedule that each δn,z occurs. Hence, this sequencewill be

δ1,z )Di)1, δ2,z )Ci)1, δ3,z ) Ti)1, δ4,z )Di)2, δ5,z )Ci)2, δ6,z ) Ti)2, ..., δ3i,z ) Ti)I-1, δ3i-2,z )Di)I, δ3i-1,z )Ci)I

where Di is the decision variable representing the choice of theith drug, Ci is the decision variable for the choice of corporaterelations strategy for the ith drug, and Ti is the decision choicefor the length of time to wait before development of drug i -1 is commenced. The stipulation is imposed that no edge canbe directed from any δi to any other δi which occurs before itin this specified time order.

The procedure for constructing Bz from Kz is now described.A score-based approach is used with the Bayesian Dirichlet (BD)metric and the network is constructed with the intent tomaximize this score. The BD metric is a measure of how closelyBz models the data in Kz. The construction procedure beginswith an empty network and each possibility for adding the nextdirected edge in Bz is scored. The next directed edge to be addedwill be the one that maximizes the value of the BD metric. Thisprocess continues until no improvement in score can beachieved. Additionally, no restrictions are imposed on thecomplexity of the network that can be built so that no importanttopological details are lost.

The BD metric for Bz given Kz and background information,�, is denoted by p(Kz, Bz|�) and is defined as

p(Kz, Bz|�)) p(Bz|�)∏n)0

n-1

∏πδn,z

Γ(ε′(πδn,z))

Γ(ε′(πδn,z)+ ε(πδn,z

))×

∏δn,z,m

Γ(ε′(δn,z,m, πδn,z)+ ε(δn,z,m, πδn,z

))

Γ(ε′(δn,z,m, πδn,z))

(19)

where p(Bz|�) is the prior probability of Bz; δn,z,m is the nth nodeinstantiated to the mth value in Kz; the product over πδn,z runsover all instances of all parents of δn,z; the product over δn,z,m

runs over all instances of δn,z; ε(πδn,z) is the number of instancesin Kz where the parent nodes of δn,z, Πδn,z, are instantiated toπδn,z; ε(δn,z,m,πδn,z) is the number of instances in Kz that haveδn,z equal to δn,z,m and Πδn,z set to πδn,z; and Γ(x) is the Gammafunction where Γ(x) ) (x - 1)! When the set Πδn,z is empty,there is one instance of Πδn,z that is equal to 0, and the numberof instances is set to the number of members of Kz, |Kz|.Additionally

ε(πδn,z)) ∑

δn,z,m

ε(δn,z,m, πδn,z) (20)

where πδn,z,m is an instance of the parents of δn,z that is summedover all instances of δn,z. The numbers ε′(δn,z,m,πδn,z) and p(Bz|�)represent prior information about the problem that can beincorporated into the metric.

The extent to which the network being measured representsanother network relevant to the problem is measured by p(Bz|�).There are a number of methods available for calculating p(Bz|�).In this paper all networks are treated equally; thus p(Bz|�) is setto 1. There are also a variety of methods for setting the numbersε′(δn,z,m,πδn,z); for example, see Buntine,42 Heckerman et al.,36

and Yang and Chang.43 In particular, research by Yang andChang43 has demonstrated that, for a range of integer values ofε′(δn,z,m,πδn,z) between 1 and 10, and also for other methodsconsidered for scoring Bayesian network topology, the BDmetric with ε′(δn,z,m,πδn,z) ) 10 ranked as one of the best fordiscovering the true structure of the Bayesian network. Accord-ingly, ε′(δn,z,m,πδn,z) ) 10 is used here. Additionally, becausethe factorials in eq 20 can grow to unmanageably large numbers,especially within large sizes of Kz, its logarithmic equivalent isused.

Appendix D. Regenerating the Population

In generating G(t + 1) each δn,z is treated as a random variablethat can be instantiated to any of its possible instances subjectto probabilities encoded by Bz. For each decision variable allprobabilities for each possible δn,z,m conditional upon eachpossible πδn,z given Kz are computationally determined andreferenced when needed. The process for generating onecandidate solution for G(t+1) requires itarative sampling of Bz.

The joint distribution encoded by Bj can be written as

p(δz))∏n)1

N

p(δn,z|Πδn,z) (21)

where δj ) (δ1,j,δ2,j, ..., δN,z) is a vector of random variablesrepresenting the entire set of decision variables, and p(δn,z|Πδn,z)is the conditional probability of δn,z given its set of parentvariables Πδn,z. Intuitively, this conditional probability is definedas

p(δn,z|Πδn,z))

p(δn,z, Πδn,z)

p(Πδn,z)

(22)

Figure 6. Allowable and disallowable Bayesian network structures. (a) Anexample of an allowed Bayesian network structure consisting of three nodes.δ1,z has no parents and is the parent of δ2,z and δ3,z. δ2,z has one parent,δ1,z, and is the parent of δ3,z. δ3,z has two parents, δ1,z and δ2,z, and nochildren. Two Bayesian network structures that are disallowed in the model:(b) δ1,z has a node that is directed to itself; (c) δ1,z is part of a networkstructure that is later directed back to itself.


Each Kz is proportionally represented in the generation ofG(t + 1). As S(t) contains half the solutions present in G(t),the number of solutions generated using each Bz is 2|Kz|.

Literature Cited

(1) DiMasi, J.; Hansen, R. W.; Gabrowski, H. G. The price of innovation:new estimates of drug development costs. J. Health Econ. 2003, 22, 151–185.

(2) Boston Consulting Group. A ReVolution in R&D: How Genomicsand Genetics are Transforming the Biopharmaceutical Industry; BostonConsulting Group: Boston, MA, 2001.

(3) Foo, F.; Karri, S.; Titchener-Hooker, N.; Dunnill, P. Biopharma-ceutical process development: part 1, information from the first productgeneration. Biopharm Eur. 2001, 13, 58–64.

(4) Ashton, G. Growing pains for biopharmaceuticals. Nat. Biotechnol.2001, 10, 307–311.

(5) Werner, R. G. Economic aspects of commercial manufacture ofbiopharmaceuticals. J. Biotechnol. 2004, 113, 171–182.

(6) Reichert, J. M. Trends in development and approval for newtherapeutics in the United States. Nat. Drug DiscoVery 2003, 2, 695–702.

(7) Rogers, M. J.; Gupta, A.; Maranas, C. D. Real options based analysisof optimal pharmaceutical research and development portfolios. Ind. Eng.Chem. Res. 2002, 41, 6607–6620.

(8) Subramanian, D.; Penky, J. F.; Reklaitis, G. V. A simulationoptimization framework for research and development pipeline management.AIChE J. 2001, 47, 2226–2242.

(9) Subramanian, D.; Penky, J. F.; Reklaitis, G. V.; Blau, G. E.Simulation optimization framework for stochastic optmization of R&Dpipeline management. Process Syst. Eng. 2003, 49 (1), 96–112.

(10) Blau, G.; Mehta, B.; Bose, S.; Penky, J.; Sinclair, G.; Keunker,K.; Bunch, P. Risk management in development of new products in highlyregulated industries. Comput. Chem. Eng. 2000, 24, 2211–2222.

(11) Blau, G. E.; Penky, J. F.; Varma, V. A.; Bunch, P. R. Managing aportfolio of independent new product candidates in the pharmaceuticalindustry. J. Prod. InnoV. Manage. 2004, 21, 27–245.

(12) Rogers, M. J.; Maranas, C. D. Valuation and design of pharma-ceutical R&D licensing deals. AIChE J. 2005, 51 (1), 198–209.

(13) Jain, V.; Grossmann, I. E. Resource-constrained scheduling of testsin new product development. Ind. Eng. Chem. Res. 1999, 38, 3013–3026.

(14) Piachaud, B. S. Outsourcing in the pharmaceutical manufacturingprocess: an examination of the CRO experience. TechnoVation 2002, 22,81–90.

(15) Getz, K. The expanding outside clinical services contractormarketplace. Clin. Res. Regul. Affairs 1997, 14 (3 and 4), 243–254.

(16) Oh, H.; Karimi, I. A. Regulatory factors and capacity-expansionplanning in global chemical supply chains. Ind. Eng. Chem. Res. 2004, 43,3364–3380.

(17) Rakapakse, A.; Titchener-Hooker, N. J.; Farid, S. S. Modelling ofthe biopharmaceutical drug development pathway and portfolio management.Comput. Chem. Eng. 2005, 29, 1357–1368.

(18) Rakapakse, A.; Titchener-Hooker, N. J.; Farid, S. S. Integratedapproach to improving the value potential of biopharmaceutical R&Dportfolios while mitigating risk. J. Chem. Technol. Biochem. 2006, 81, 1705–1714.

(19) Levis, A. A.; Papageorgiou, L. G. A hierarchical solution approachfor multi-site capacity planning under uncertainty in the pharmaceuticalindustry. Comput. Chem. Eng. 2004, 28, 707–725.

(20) Maravelias, C. T.; Grossman, I. E. Simultaneous planning for newproduct development and batch manufacturing facilities. Ind. Eng. Chem.Res. 2001, 40, 6147–6164.

(21) Papageorgiou, L. G.; Rotstein, G. E.; Shah, N. Strategic supplychain optimization for the pharmaceutical industries. Ind. Eng. Chem. Res.2001, 40, 275–286.

(22) George, E. D.; Titchener-Hooker, N. J.; Farid, S. S. A multi-criteriadecision-making framework for the selection of strategies for acquiringbiopharmaceutical manufacturing capacity. Comput. Chem. Eng. 2007, 31(8), 889–901.

(23) Pelikan, M.; Goldberg, D.; Cantu-Paz, E. Linkage problem,Distribution estimation, and Bayesian networks. EVol. Comput. 2000, 8 (3),311–340.

(24) Farid, S. S.; Washbrook, J.; Titchener-Hooker, N. J. Decision-support tool for assessing biomanufacturing strategies under uncertainty:stainless steel versus disposable equipment for clinical trial materialpreparation. Biotechnol. Prog. 2005, 21 (2), 486–497.

(25) Farid, S. S.; Washbrook, J.; Titchener-Hooker, N. J. Combiningmultiple quantitative and qualitative goals when assessing biomanufacturingstrategies under uncertainty. Biotechnol. Prog. 2005, 21 (4), 1183–1191.

(26) Farid, S. S.; Washbrook, J.; Titchener-Hooker, N. J. Modellingbiopharmaceutical manufacture: Design and implementation of SimBiop-harma. Comput. Chem. Eng. 2007, 31 (9), 1141–1158.

(27) Lim, A. C.; Zhou, Y.; Washbrook, J.; Sinclair, A.; Fish, B.; Francis,R.; Titchener-Hooker, N. J.; Farid, S. S. Application of a decision supporttool to assess pooling strategies in perfusion culture processes underuncertainty. Biotechnol. Prog. 2005, 21 (4), 1231–1242.

(28) Lim, A. C.; Washbrook, J.; Titchener-Hooker, N. J.; Farid, S. S. Acomputer-aided approach to compare the production economics of fed-batchand perfusion culture under uncertainty. Biotechnol. Bioeng. 2005, 93 (4),687–697.

(29) Muhlenbein, H.; Paass, G. From recombination of genes to theestimation of distributions I: Binary Parameters Lecture Notes in ComputerScience, vol. 1411; Springer-Verlag: Berlin, Heidelberg, New York, 1996;pp 178-187.

(30) Holland, J. H. Adaptation in Natural and Artificial Systems;University of Michigan Press: Ann Arbor, 1975.

(31) Pelikan, M.; Goldberg, D.; Lobo, F. A survey of optimizations bybuilding and using probabilistic models. Comput. Optim. Appl. 2000, 21,5–20.

(32) Goldberg, D. E. Genetic Algorithms in Search, Optimization, andMachine Learning; Addison-Wesley: Reading, MA, 1989.

(33) Muhlenbein, H. The equation for response to selection and its usefor prediction. EVol. Comput. 1997, 5 (3), 303–346.

(34) Harik, G. R.; Lobo, F.; Goldberg, D. The compact genetic algorithm.IEEE Trans. EVol. Comput. 1999, 3 (4), 287–297.

(35) Pelikan, M.; Sastry, K.; Goldberg, D. E. Multiobjective hBOA,clustering, and scalability. Proceedings of the 2005 Conference on Geneticand EVolutionary Computation, Washington, DC; 2005; pp 663-670.

(36) Heckerman, D.; Geiger, D.; Chickering, D. M. Learning Bayesiannetworks: A combination of knowledge and statistical data. Mach. Learn.1995, 20, 197–243.

(37) Matsumoto, M.; Nishimura, T. Mersenne Twister: A 623 dimen-sionally equidistributed uniform pseudo-random number generator. ACMTrans. Model. Comput. Simul. 1998, 8 (1), 3–30.

(38) George, E. D.; Farid, S. S. Strategic biopharmaceutical portfoliodevelopment: an analysis of constraint-induced implications. Biotechnol.Prog. 2008, 24, 698–713.

(39) Deb, K.; Pratap, A.; Agarwal, S.; Meyarivan, T. A fast and elitistmultiobjective genetic algorithm: NSGA-II. IEEE Trans. EVol. Comput.2002, 6 (2), 182–197.

(40) MacQueen, J. B. Some methods for classification and analysis ofmultivariate observations. Proceeding of the 5th Symposium on Mathematics,Statistics, and Probability; University of California Press: Berkeley, 1967;pp 281-297.

(41) Neapolitan, R. E. Learning Bayesian Networks; Prentice Hall: UpperSaddle River, NJ, 2003.

(42) Buntine, W. L. Theory and refinement of Bayesian networks. InProceedings of SeVenth Conference on Uncertainty in Artificial Intelligence;Morgan Kauffmann: Los Angeles, CA, 1991; pp 52-60.

(43) Yang, S.; Chang, K. Comparison of score metrics for Bayesiannetwork learning. IEEE Trans. Systems, Man, Cybernet., Part A 2002, 32(3), 419–428.

ReceiVed for reView February 25, 2008ReVised manuscript receiVed June 25, 2008

Accepted August 15, 2008

IE8003144


Stochastic Combinatorial Optimization Approach to Biopharmaceutical Portfolio Management

Documents

Transcript of Stochastic Combinatorial Optimization Approach to Biopharmaceutical Portfolio Management