The use of intelligent Experimental

Designs for Optimal Automotive

Engine Calibration Online at Engine

Test Bench.∗

Thierry Dalon, Siemens VDO Automotive AG†

September 7, 2007

Control-unit calibration for modern internal combustion engines is cur-rently facing a con�ict caused by the additional e�ort needed to calibrateincreasingly complex engine data with a growing number of parameters, to-gether with extremely ambitious objectives regarding the period of time andthe resources needed for calibration, performance, consumption, and com-fort expected by the customer and emissions levels which are more and morestringent. To reduce costs we look for reducing testing time at test bench andhence use minimal number of measurements. That leads to Optimal Exper-imental Design approaches. Designing experiments often leads to trade-o�sbetween local and global search: local criteria encompass achieving best cal-ibration i.e. the optimization of a target (for example performance) undermany constraints (emissions, consumption), whereas global criteria tend toexplore the whole domain or improve model quality. We present here thecontext and methods investigated at Siemens VDO Automotive for optimalengine calibration online at the engine test bench. The approach will beillustrated on a practical industrial engine calibration example.

Keywords: Design of Experiments, Adaptive Online DOE, Online Optimization, EngineECU Calibration, Engine Test Bench Automation, Computer Experiments, SurrogateOptimization, Generalized Pattern Search, Mesh Adaptive Direct Search

∗This paper is a summary of the talk presented at ENBIS conference, 2007-09-26 in Dortmund†Siemensstr. 12, D-93055 Regensburg. thierry.dalon@siemens.com. www.siemensvdo.com

1

Contents

1 Introduction 2

1.1 DOE and model-based approach for engine calibration . . . . . . . . . . 31.2 O�ine versus Online Approach . . . . . . . . . . . . . . . . . . . . . . . 3

2 Theoretical background 4

2.1 Problematic Analogy with Computer Experiments . . . . . . . . . . . . . 42.2 Surrogate optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42.3 Generalized Pattern Search and Mesh Adaptive Direct Search Algorithms 42.4 Oracle principle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

3 Siemens VDO Tool for Online DOE and Online Optimization 6

3.1 Surrogate Management Framework . . . . . . . . . . . . . . . . . . . . . 63.2 Generalized Pattern Search for Online Optimization . . . . . . . . . . . . 73.3 Overview picture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73.4 Adjustment Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

4 Practical example: Full Load optimization for diesel engines 8

4.1 Diesel Full Load Optimization Problem Description . . . . . . . . . . . . 84.2 Practical application and results . . . . . . . . . . . . . . . . . . . . . . . 84.3 New challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

5 Conclusion 10

1 Introduction

In order to compete in today's global economy, automotive engine manufacturers are un-der pressure to provide engines that meet tougher emissions regulations and increasedcustomer expectations for comfort, performance and low fuel consumption. As a re-sult, the complexity of systems used to manage engine design has greatly increased. Toachieve the increased demands, engines are now designed with the most current technol-ogy, such as variable valve timing, exhaust gas recirculation systems, multiple injection,turbocharger controls, and advanced control strategies. As a consequence the numberof tuning parameters saved in the Electronic Engine Control Unit (ECU) has grownconsiderably and herewith the calibration task.Accordingly, methods applied to the speci�c task of engine tuning have evolved with

the complexity of the calibration tasks that are performed in the process. The goal ofthe calibration engineer is to �nd the best set of parameters possible to meet certainobjectives. Where this was once done manually by the calibration engineer, the increasedcomplexity has made this task almost impossible without the aid of computers andmathematical tools.One inevitable step of the engine calibration process occurs at the engine test bench.

The engine test bench allows to have full access to engine parameters in the engine

2

control unit and is also full equipped with measurement devices, so that you can gainmuch more information than in the real vehicle. For example you can play with engineparameters like ignition timing and quantity (for all injections available), fuel pressureand look at responses like exhaust temperature, smoke, emissions, cylinder pressure etc.Moreover it is possible at the engine test bench to run tests in a full automatic modethanks to an advanced automation system.

1.1 DOE and model-based approach for engine calibration

Under economical pressure and because of increasing engine management system com-plexity, Design of Experiments and Model-based approaches have become a standard inengine calibration process. We refer here to the plenary session talk by K. Röpke of sameENBIS 2007 conference or to the proceedings of IAV Design of Experiments in EngineDevelpment much frequented conferences ([17]). DOE and model-based methodologieshelp shortening the calibration process by reducing the required number of measurementsdrastically.

1.2 O�ine versus Online Approach

We can distinguish two di�erent approaches.The �rst common approach using DOE is the so-called o�ine approach. An experi-

mental design is built by experts at the desktop. This experiment plan is then measuredat the test bench. From measurements models are built and analyzed at the desktop tomake a new set of calibration parameters. See for example description of this Z-processof basic calibration using DOE in [16].The two steps - building the experiment and running it on the test bench - are com-

pletely uncorrelated. A major challenge at the test bench is that all measurement pointscan not be set as wished directly because of engine limits like cylinder pressure, tem-peratures or knock for gasoline engines. Some parameters combinations may lead toengine damages and at least will stop the automatic test on a hard security. There-fore the automation system needs special strategies to avoid exceeding engine operatingrange. That is why the resulting measured points frequently di�ers from the initialdesign because initial target points can not be reached safely.On the contrary the online approach makes use of the information gained at the engine

test bench to propose new interesting points to measure right-through at the engine testbench and not after an o�ine analysis at the desktop.Indeed we may need once the initial design has been run in the possible running

limits to re-adjust the experimental plan because limits violation occurred. This is theapproach presented for example in [10], [23],[12], [9] and [7] ; it is known under the termof Adaptive Online DOE.An other approach using a strongest interaction between the measurement process

and the experimental design makes use of the information gained by each measurementimmediately to de�ne the next interesting points to measure. One such approach is used

3

by BMW (see [14]); it is based on Arti�cial Intelligence theory and its so-called activelearning concept .At Siemens VDO we also have a strong online approach. The framework developed

to handle online DOE and online optimization is inspired from Computer Experiment(also called DACE ) literature.

2 Theoretical background

2.1 Problematic Analogy with Computer Experiments

In problems dealing with computer experiments the system is a huge computer simula-tion program for which each evaluation takes a very long time. System evaluation is inthis sense very expensive. (See for example [22] and [4]).Like for Computer Experiments in our case each system evaluation (i.e. measurement)

is expensive because it costs time at the Engine Test Bench.Moreover we don't dispose of any accurate enough engine model so to gain real infor-

mation on the engine behavior we need to make a measurement. We deal typically witha black-box system.

2.2 Surrogate optimization

An approach to deal expensive system is the so-called �surrogate� approach: the ex-pensive system will be iteratively modeled by cheaper models on which more expensive(in the sense of number of evaluations) optimization techniques can be applied. In thecomputer experiments literature these surrogate models are also called metamodels (see�rst day parallel session of same ENBIS 2007 conference). This iterative handling of thesurrogates is well described in the surrogate management framework (see [6]).

2.3 Generalized Pattern Search and Mesh Adaptive Direct

Search Algorithms

Because the problem we face handle with an expensive black-box noisy system, derivative-free algorithms are more suited to be used. One algorithm that has shown success inpractice is the so-called Generalized Pattern Search (GPS) and its recent extension theMesh Adaptive Direct Search (MADS). It is not the purpose of this paper to explaindeeply the theory of this kind of algorithm. For more details we refer the reader toM. Abramson's PhD. Thesis ([1]) for GPS algorithm description and Audet-Dennis-Ambramson publication ([5]) for MADS improvement.This algorithm is divided in two steps: one local step (called POLL) based on a mesh

around the incumbent optimum and one customizable step (called SEARCH) wherethe user can implement any strategy to speed up the optimization. Advantages ofthis method rely on the �exibility of the SEARCH step. A common approach for the

4

Figure 1: Oracle principle

SEARCH step makes use of surrogates to predict potential optima. (see for exampleprevious reference [1]).

2.4 Oracle principle

At SiemensVDO we handle online DOEs within a generic concept that comes also fromDACE literature ([22]) called oracle. After each new measurement an oracle is asked forwhat points he predicts as �interesting� based on the past measurement history. (See�gure 1)The criteria used by the oracle to predict interesting points can be di�erent according

to the objectives �xed, for example either from a more local or a more global perspective.We enumerate here a few that can be found in the literature:

• pure optimization oriented - like used in the GPS with standard surrogate man-agement

• model-based with trust region or con�dence criterion:

� model quality criterion (cf. [7])

� use model quality predicted decrease to update trust region size ([3])

� con�dence criterion (cf. [15])

5

• active learning and query criterion ([15], [9]and [8])

• design criterion (space �lling maximin criterion in [20] or transformed distancecriterion in [21]; D-optimality in [23])

• mixed optimization and design criteria:

� merit function ([20])

� balance local/ global search (BLGS, cf. [4])

• statistical criterion:

� mse, model uncertainty using DACE / kriging models ([6])

� expected improvement (see [19] and [4])

� in�ll sampling criteria ([18])

3 Siemens VDO Tool for Online DOE and Online

Optimization

At Siemens VDO we developed our own tool for handling Online DOE and Onlineoptimization tasks, called OnlineOptimizer. It is developed under MATLAB R©. We choseMATLAB

R©mainly for its �exibility and easy-to-code aspect. We describe here brie�y onwhich peaces this implementation is based.

3.1 Surrogate Management Framework

We developed a Surrogate Management Framework based on following structure:

• a model framework in which models are integrated in an object-oriented way withtwo main methods evaluate and calibrate. A library of models is available includingfor example Polynomial Response Surface Models (RSM), Neural Nets, RBF Nets(based on J. Orr toolbox [13]), statistical models like DACE models using DTUDACE Toolbox ([11]).

• a library of optimizer methods like : SQP, Evolutionary Algorithm, a modi�edBLGS algorithm (see [4])

• a framework to include the calibration optimization problem and its mirror surro-gate optimization problem

6

Figure 2: Structure tree

3.2 Generalized Pattern Search for Online Optimization

We use the implementation of the Audet-Dennis-Abramson GPS by M. Abramson inthe MATLAB R©NOMADm Toolbox ([2]).To speed up convergence of the POLL step we pass to the algorithm available deriva-

tive information in a crude form of a qualitative variables vs. responses e�ect matrixwith +1, -1 or 0 values for increasing, decreasing and unknown or non-monotonuoustrend respectively.

3.3 Overview picture

A brief picture of the OnlineOptimizer structure can be drawn as in �gure 2.If we just want to run a DOE without any optimization task behind we have a sepa-

rated light module (called adaptive online DOE) that does not use the GPS algorithmbut can call the oracle after the initial DOE has been run.

3.4 Adjustment Strategies

It is not the goal of this paper to explain in details how the target points are practicallyrun at the Engine Test Bench. Shortly we are currently using a step-by-step approach

7

strategy with stability criteria, an adaptive step width strategy depending on the currentdistance to the engine limits and an optimized screening start point. The de�nition oflimits to consider is also implemented in a �exible object-oriented way.

4 Practical example: Full Load optimization for

diesel engines

We present here a practical example for which previously described methods for theOnline Optimization are applied in a productive way. We take as example the Full LoadOptimization for Diesel Engine.

4.1 Diesel Full Load Optimization Problem Description

In the case of the Diesel Full Load Optimization we have a clear objective and optimiza-tion problem: we want to �nd out best parameter combination for injection timing andquantity for each injection available, fuel pressure and manifold pressure (if a turbo isavailable) in order to reach maximal performance (torque) under exhaust temperature,smoke and cylinder pressure limits. One variant of this problem is when the torquetarget is known and we want then to minimize the fuel consumption with the torquereaching a target value within a given tolerance. We perform this optimization overseveral engine speed points at full load (pedal wide open).

4.2 Practical application and results

In practice the experience has shown that using quadratic response surface models assurrogates is su�cient. A �xed POLL order re�ecting calibration engineer experienceis used. The targets for the optimization are known in advance, so that an Onlineoptimization approach using GPS algorithm is suitable.Figure 3 illustrates the results of an optimization run by varying 4 parameters.Some comments below:

• You can see intermediate iterates: these are coming from the stepwise adjustmentstrategy.

• The main role of the initial DOE is to serve as learning basis for the models. So wechose in practice not a full DOE over the whole feasible range but a much smallerstarting box. You can see the DOE starting box in black. The bigger red boxde�nes the feasible or exploratory bounds.

• The SEARCH step may have a globalizing e�ect on top of a convergence acceleratorrole. SEARCH iterates are marked in blue. You can see that models predicted anoptimum in a totally other direction (path running down to the left). In practisethis optimum was not much worse than the previous one and might have been abetter solution.

8

Figure 3: Example of a 4 parameter-run for Diesel Full Load Optimization

• We get thanks to the POLL step the local validation of the found optimum. (POLLiterates are marked in gray) You can see that �nal optimum (red circle) has POLLiterated all around to a prede�ned resolution. Optimum predicted by the SEARCHmodel step is readjusted by the POLL step. It is good so because models are onlyapproximations. So at the end we get the real validated optimum and not only themodel-based predicted one. Even if the models have a bad prediction, the POLLstep will �nd out a local optimum. Of course the better the model prediction is,the quicker the convergence of the algorithm.

4.3 New challenges

With the appearance of multiple injection, engine behavior is getting more and morenon linear so that simple quadratic response surface models may not be su�cient. Therewill be for the future the need for other type of models, more non-linear, well suited foronline application i.e. with quick learning capability and good prediction. It is not clearyet which model is the best for each application.

9

5 Conclusion

We presented in this paper theoretical fundaments to a new approach of online optimiza-tion and online design of experiments for engine calibration at the engine test bench.The theory comes from computer experiments literature which has a strong problem-atical analogy. It has been shown that online optimization and adaptive online DOEcan be seen from the same oracle conceptual view. The use of the Generalized PatternSearch algorithm was presented and illustrated on a practical example. Advantages ofthis online approach were pointed out. This is the basement for further investigationsand applications.

References

[1] M. A. Abramson. Pattern Search Algorithms for Mixed Variable General Con-strained Optimization Problems. PhD thesis, August, 2002. Also available as CAAMTechnical Report TR02-11.

[2] M. A. Abramson. Nonlinear Optimization with Mixed Variablesand Derivatives - Matlab. NOMADm software homepage, 2004.http://en.a�t.edu/enc/Faculty/MAbramson/NOMADm.html.

[3] N. Alexandrov, J. E. Dennis Jr., R. Lewis, and V. Torczon. A trust region frameworkfor managing the use of approximation models in optimization. In Springer Berlin/Heidelberg, editor, Structural and Multidisciplinary Optimization, volume 15, pages16�23. Februar 1998. Also as ICASE Report No. 97-50.

[4] C. Audet, J. E. Dennis, Jr., D. Moore, A. Booker, and P. Frank. A surrogate-model-based method for constrained optimization. AIAA, 4891, 2000.

[5] Charles Audet and J. E. Dennis, Jr. Mesh adaptive direct search algorithms forconstrained optimization. In SIAM J. Optim., volume 17, pages 188�217. 2006.

[6] A. Booker, J.E. Dennis, Jr., P. Frank, D. Sera�ni, V. Torczon, and M. Trosset. Arigorous framework for optimization of expensive functions by surrogates. StructuralOptimization, 17(1):1�13, February 1999.

[7] A. Buzy, M. Albertelli, P. Cotte, S. Grall, and J. Visconti. Online adaptive doesand global modeling for diesel engines and dpf calibration. In K. Röpke, editor,Proceedings of DOE Conference HdT-IAV. ExpertVerlag, 06 2005.

[8] D. A. Cohn, Z. Ghahramani, and M. I. Jordan. Active learning with statisticalmodels. In G. Tesauro, D. Touretzky, and T. Leen, editors, Advances in NeuralInformation Processing Systems, volume 7, pages 705�712. The MIT Press, 1995.

[9] F. Figueres and Y. Raynaud. Adaptive generation of doe for optimal neural net-work model / application on a gasoline engine. In AVL Development MethodologySymposium, 10 2005.

10

[10] K. Gschweitl, H. P�uegl, and T. Fortuna. Increasing the e�ciency of model-basedengine applications through the use of CAMEO online DoE toolbox. ATZ World-wide, 7-8/2001 Volume 103, 2001.

[11] S. N. Lophaven, H. B. Nielsen, and J. Søndergaard (DTU IMM). Designand Analysis of Computer Experiments. A Matlab Kriging Toolbox. Tech-nical University of Denmark, Informatics and Mathematical Modeling, 2002.http://www.imm.dtu.dk/̃ hbn/dace/.

[12] B. Neaves, J. Gruaz, A. Heindl, and H.-M. Koegeler. Method development and ap-plication of particulate �lter regeneration calibration. In AVL Development Method-ology Symposium, 10 2005.

[13] M. Orr. A matlab radial basis function toolbox, 1999.http://www.anc.ed.ac.uk/rbf/rbf.html.

[14] J. Polland, K. Knödler, T. Fleishhauer, A. Mitterer, S. Ullmann, and A. Zell. Model-based online optimisation of modern internal combustion engines. part 1. activelearning. MTZ Worldwide, 5/2003 Volume 64, 2003.

[15] Jan Polland. Modellgestützte und Evolutionäre Optimierungsverfahren für die Mo-torentwicklung. PhD thesis, University of Tübingen, Wilhelm-Schickard-Institutefor Computer Science, Department of Computer Architecture., 2002.

[16] K. Röpke, M. Knaak, B. Barzantny, U. Genc, and U. Plewnia. Future methods fora model-based engine development and calibration process. In AVL DevelopmentMethodology Symposium, October 2005.

[17] Karsten Röpke, editor. Design of Experiments (DoE) in Engine Development III,2007.

[18] Michael James Sasena. Flexibility and E�ciency Enhancements for ConstrainedGlobal Design Optimization with Kriging Approximations. PhD thesis, Universityof Michigan, 2002. PhD Thesis.

[19] M. Schonlau. Computer Experiments and Global Optimization. PhD thesis, Uni-versity of Waterloo, 1997.

[20] V. Torczon and M. Trosset. Using approximations to accelerate engineering designoptimization. AIAA, 4800, 1998.

[21] M. Trosset. Optimization on a limited budget. In Section on Physical and Engi-neering Sciences. American Statistical Association, 1998.

[22] M. Trosset and V. Torczon. Numerical optimization using computer experiments.Technical Report 97-38, ICASE NASA Langley Research Center, 1997.

11

[23] A. Zrim A. Gallacher Y. Raynaud, D. Lopes. T. Fortuna. Application of adaptiveonline doe techniques for engine ecu calibration. In IMechE Conference on Statisticsand Analytical Methods in Automotive Engineering., 2002.

Nomenclature

BLGS Balanced Local-Global Search

DACE Design and Analysis of Computer Experiments

DOE Design of Experiments

ECU Electronic Engine Control Unit

GPS Generalized Pattern Search

MADS Mesh Adaptive Direct Search

RBF Radial Basis Functions

SMF Surrogate Management Framework

SQP Sequential Quadratic Programming

12