IN DEGREE PROJECT ELECTRICAL ENGINEERING,SECOND CYCLE, 30 CREDITS

, STOCKHOLM SWEDEN 2018

Optimising energy consumption on straight roads using regression analysis

GABRIEL MASSO

KTH ROYAL INSTITUTE OF TECHNOLOGYSCHOOL OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE

2

Abstract

Cloud computation together with robotics has opened up possibilitiesto process large amount of data (big data) generated by the greatnumber of robotic systems. Todays vehicles are equipped withhundreds of sensors generating a lot of data that needs to beprocessed. The data can further be analysed and used to obtainmodels predicting the dynamics of the vehicles. It is thereforepossible to optimise the vehicle performance by studying thepredictive behaviour and finding the best combination of the vehicleparameters. In this thesis, the energy efficiency of an electric racingvehicle is studied on straight road whereafter an optimal velocityprofile is to be found. By using a multiple linear regression togetherwith regularization methods on previously recorded data, apredictive model managed to be obtained with an accuracy of 79.1 %.Having used this model in optimisation process, a velocity profilewas obtained which is shown that can enhance the efficiency of thesystem by 4.08%.

3

Sammanfattning

Molnprocessering tillsammans med robotteknik har öppnatmöjligheter för att bearbeta stora mängder data som genereras av detökande antalet robotsystem. Dagens fordon är utrustade medhundratals sensorer som genererar mycket data som behöverbearbetas. Sensordata kan vidare analyseras och användas för attmodellera fordonets dynamik. Det är därför möjligt att optimerafordonets prestanda genom att studera det prediktiva beteendet ochhitta den bästa kombinationen av fordonsparametrarna. I dennaavhandling studeras energieffektiviteten hos ett elbil på rak väg,varefter en optimal hastighetsprofil hittas. Genom att använda enmultipellinjär regression tillsammans med regleringsmetoder påtidigare insamlad data lyckades en prediktiv modell erhållas med ennoggrannhet av 79,1 %. Efter att ha använt denna modell ioptimeringsprocessen erhölls en hastighetsprofil som visas som kanförbättra systemets effektivitet med 4,08%.

Contents

1 Introduction 11.1 Research Question . . . . . . . . . . . . . . . . . . . . . . 21.2 Project Outline . . . . . . . . . . . . . . . . . . . . . . . . . 31.3 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . 4

2 System overview 6

3 Communication 83.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83.2 IoT Hub . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

3.2.1 ROS . . . . . . . . . . . . . . . . . . . . . . . . . . . 93.2.2 DynamoDB . . . . . . . . . . . . . . . . . . . . . . 9

3.3 SQLite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

4 Data mining 104.1 Collected sensor data . . . . . . . . . . . . . . . . . . . . . 114.2 Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . 114.3 Feature scaling . . . . . . . . . . . . . . . . . . . . . . . . . 114.4 Data filtering . . . . . . . . . . . . . . . . . . . . . . . . . . 12

5 Predictive modelling 145.1 Parameters used for modelling . . . . . . . . . . . . . . . 155.2 Multivariable linear regression . . . . . . . . . . . . . . . 15

5.2.1 Coefficient of determination . . . . . . . . . . . . . 165.2.2 Cross Fold Validation . . . . . . . . . . . . . . . . 16

5.3 Regularization . . . . . . . . . . . . . . . . . . . . . . . . . 185.3.1 Lasso . . . . . . . . . . . . . . . . . . . . . . . . . . 185.3.2 Ridge . . . . . . . . . . . . . . . . . . . . . . . . . . 195.3.3 Elastic Net . . . . . . . . . . . . . . . . . . . . . . . 20

4

CONTENTS 5

6 Optimisation 216.1 Objective function . . . . . . . . . . . . . . . . . . . . . . . 226.2 Sequential quadratic programming . . . . . . . . . . . . . 22

7 Results 237.1 Data processing . . . . . . . . . . . . . . . . . . . . . . . . 237.2 Predictive model . . . . . . . . . . . . . . . . . . . . . . . 24

7.2.1 Regression comparison . . . . . . . . . . . . . . . 307.3 Optimisation . . . . . . . . . . . . . . . . . . . . . . . . . . 307.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . 327.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . 327.6 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 32

Bibliography 33

Chapter 1

Introduction

Reducing the energy consumption of transportation systems hasbeen, and will continue to be, an important issue to address.Increasing energy prices along with environmental considerationsmotivates the research in creating more energy efficient systems.Electric vehicles are one of the key technologies for a sustainablesociety and recent studies show that an electric vehicle couldconsume up to 8 times less energy than a vehicle that runs on atraditional petroleum fuel. [1] We can therefore expect a lot moreelectric vehicles on the roads in the upcoming decades.

The theory behind making electric vehicles more energy efficient hasalready been present for several years. It is the hardware such assensors and on-board computers that there has been a lack of.Emerging technologies has helped create sensors, actuators andmicrocomputers that are cheap and easy to implement whichexplains the now increased use of the components in all types ofsystems [2]. This makes it easy for engineers to analyse the system’sbehavior in different scenarios whereafter optimisation of the systemparameters can be done. The implementation of the various sensorsin today’s vehicles have resulted in a huge generation of data.Onboard operating systems have become a common practice intoday’s vehicles, but a real time communication infrastructure needsto be established to transfer the data. By migrating the data from thevehicle to the cloud, calculations that require heavy computationalpower can be done without draining the limited energy in the vehicle.[3]

1

2 CHAPTER 1. INTRODUCTION

Focusing on racing cars, Tom Hyder, applications manager atMagneti Marelli Motorsport, describes the data acquisition and thetelemetry system for the logging of the race car performance asfundamental to further develop the vehicle performance.

The mountain of information that it [the vehicle] providesmust increasingly be processed by automatic means...To dothis successfully requires a team of well-organized and wellqualified people in the "back office". [4]

Focusing on electric vehicle racing, sensor data from the racing car isobtained using a communication infrastructure involving an onboardmicrocomputer connected to an IoT Hub in the cloud. The datacollected is then used for predictive modelling and optimisation ofthe vehicle parameters.

The major objective of this study is to examine an optimisationmethod by using predictive modelling using machine learningalgorithms. This has been done by comparing different regressionalgorithms and their predictive model accuracy on recorded data.Optimisation tools has been used on the model to obtain the optimalparameters that should be used when the vehicle is driving on astraight road section. The goal is for the optimisation method tooutput a speed profile that the vehicle should maintain in order tospend the least amount of energy while at the same time fulfilling atime criteria.

1.1 Research Question

• How can an efficient speed profile be obtained with a predictivemodel based on previously recorded performance data?

• Is it possible to improve the performance of the predictive modeland optimisation with data collected in near real time?

CHAPTER 1. INTRODUCTION 3

1.2 Project Outline

Chapter 2 presents an overview of the system and briefly describesthe different areas the thesis will focus on.

Chapter 3 introduces a full duplex communication infrastructureusing an IoT (Internet of Things) hub to send and receive sensor datavia the Robot Operating System (ROS).

Chapter 4 explains the data structure of the collected data and how itwill be filtered. The straight road sections must be extracted togetherwith the vehicle parameters during the travel on the straight roadsection.

Chapter 5 describes how the predictive model of the vehicledynamics is obtained with a linear regression model. A multivariablelinear regression (MLR) method is used to obtain a predictive model.

Chapter 6 continues with presenting an objective function that is to beminimized using nonlinear programming. The optimisation problemis solved using sequential quadratic programming.

Chapter 7 will present the results using the methods explained afterwhich a discussion follows.

4 CHAPTER 1. INTRODUCTION

1.3 Related Work

The first person who coined the term "Internet of Things" in 1999,Kevin Ashton, describes the importance of the human interactionwith components such as sensors, actuators and other "things"generating data. Using the information generated by us humans,"things" can help us track and find patterns in our environment thatgreatly improves our day to day life [5]. This has been shown to betrue in many applications ranging from agriculture [6] to stockportfolio optimization [7].

Communication

Using the cloud for storage and processing of the data is commonwhen discussing Wireless Sensor Networks. The development andincreased use of IoT devices has motivated the study of fast, reliableand cheap communication infrastructures[iot_wsn]. The functionsintegrated in many of the cloud services are useful to minimizeonboard computing on the IoT devices and increasing the ability tocommunicate with other IoTs [8]. Stream data processing in the cloudhas been studied in previous cases, however most of the IoTapplications are used for visualisation and processing of batch dataduring a given time period [9],[10]. A real time big data analyticsarchitecture to control the traffic in Intelligent Transportation Systems(ITS) has been proposed in [11]. However, a communicationinfrastructure between a Robot Operating System (ROS) and an IoThub for real time processing has to the best of my knowledge not yetbeen established and will be introduced in this project.

Optimising predictive models

Using the GPS as the primary data source to extract straight roadsections have been studied in previous cases such as in [12] wherethey proposed a point segmentation and grouping algorithm toproduce road maps using GPS traces. In this study, GPS traces will beused to extract the vehicle location at different times which will makeit possible to see if the vehicle is driving straight or not. Machinelearning have been used to forecast loads in electrical networks inmany studies [13]. Various techniques such as decision trees, artificialneural networks and multiple linear regression are common practices

CHAPTER 1. INTRODUCTION 5

to predict how much energy is used in different scenarios [14].Optimisation methods using regularization techniques has also beenstudied; however, batch data has been used in the majority of thestudies. [7]. Studying optimisation of a machine learning model thatis obtained in real time is therefore motivated.

Chapter 2

System overview

Figure 2.1: System overview

6

CHAPTER 2. SYSTEM OVERVIEW 7

An overview of the overall system studied in this thesis is presentedin figure 2.1. In order to conduct analysis on the vehicle in near realtime, a communication infrastructure needed to be established. Thiswill briefly be explained in section 3.1 where an introduction to anIoT Hub solution, in which the vehicle and a decentralised computerare connected, is made.

The thesis will continue to discuss the sensor data collected from thevehicle and how it is processed before the predictive modelling isstudied. The data mining is a vital part for any modelling of aphysical system which is why in section 4 the most important parts ofthe data mining is presented. When the collection and processing ofthe sensor data is done, it can be used to create predictive modelsusing regression and cross fold validation methods. This is done insection 5 where different methods for predictive modelling areintroduced whereafter a comparison between them will be done.

The final process in this project is the optimisation of the modelparameters. The objective function which will be minimised ispresented in chapter 6. The optimisation method used is sequentialquadratic programming which is explained in chapter 7 were theresults are presented. The main focus in this thesis will be on theanalysis and optimisation of the data collected.

Chapter 3

Communication

3.1 Overview

Figure 3.1: Communication overview

The sensor data is published through the ROS middleware on therobot to Amazon AWS IoT Core and stored in a DynamoDB table.The data can be accessed from any computer which can connect tothe internet and use the data for analysis by using AWS SDK forPython [15]. The communication can be overviewed in figure 3.1.

3.2 IoT Hub

AWS IoT Core is used as a distributed network for the robotic things(ROS topics connected to IoT Topics). The potential for the robot tocommunicate with other robots if wanted is made easy by the IoT Coreplatform [16].

8

CHAPTER 3. COMMUNICATION 9

3.2.1 ROS

Robot Operating System is the middleware that takes care of thecommunication between robots and other things such as in this casethe Internet. ROS is a flexible framework where tools, libraries andconventions are easily accessible which simplify the creation of arobust robotic platform. It uses a structure were robots can publishand subscribe to messages, e.g. sensor data, to any other robot whichdoes not necessarily have to use the same framework. [17]

In this project, ROS is installed on the vehicle and will publish thesensor data to the IoT Core.

3.2.2 DynamoDB

Sensor data is continuously sent from ROS to the IoT Core through aWiFi connection which is established by using an integrated networkadapter on the vehicle. The sensor data is stored in a DynamoDB tablewhich is a cloud based NoSQL database service that AWS offers to itscustomers.

3.3 SQLite

SQLite is a lightweight embedded SQL database engine [18]. If it isnot possible to use a full duplex communication between DynamoDBand the local computer, an SQLite database is used to store the filtereddata. SQLite is reliable and used in many high profile projects [19]which motivates the use of the database in this project. By connectingto the DynamoDB through a Python script using AWS SDK, the sensordata is downloaded and analysed locally. The data is processed andfiltered so that straight road section data is obtained and stored in theSQLite database. Predictive models can now be obtained by using thefiltered data in the SQLite database.

Chapter 4

Data mining

Figure 4.1: Process 1. Data mining overview

Data mining is a term describing the analysis of large observationaldata to find relationships which should be presented to the dataowner in an understandable way. The representation in this thesis is amodel in which response variables are related to the predictedvariable which will be specified in section 4.1. The dataset used for

10

CHAPTER 4. DATA MINING 11

analysis consists of the vehicle sensor data collected when it is activeon the racing track. In order to conclude an analysis on the vehicle, ahigh amount of performance data from it needs to be collected andstored. Once it is stored, it is of importance to filter out the data thatshould be used for the analysis and modelling of the vehicle.[20]

4.1 Collected sensor data

The collected sensor data from the vehicle are the following:

• Velocity

• GPS, longitude and latitude

• Battery status

• Time

4.2 Preprocessing

The option to sample is preferable when having small variations inthe variables that are being analysed in a larger dataset. In order tominimize large instantaneous variations, the data is smoothened byapplying a moving average to the data collected. By doing this, theeffects of noise or other disturbances in the data can be mitigated [21].A sequence of data aiNi=1 becomes a new sequence siN−n+1

i=1 . N is theamount of data points, n is the amount of data points in a sequence ofwhich the arithmetic mean is calculated and si:

si =1

n

i+n−1∑j=1

aj (4.1)

4.3 Feature scaling

Creating predictive models from data with different physicalproperties and large variations in dimensions can lead to biasedmodel parameters. Data normalization is used to resolve this. As thegoal is to seek for relationship in the variables, normalization makes it

12 CHAPTER 4. DATA MINING

possible for the variables to have the same range of values. Thismakes the relationship more robust and stabilises the convergence ofweight and biases as well as decreasing the computation time [22].The mathematical expression is:

Xscaled =(X −Xmin)

Xmax −Xmin

(4.2)

4.4 Data filtering

In order to extract the straight road sections in which the vehicledrives, a simple linear equation is applied to the collected GPS data.

y = kx+m (4.3)

A straight road is hence determined if the derivation of the equation isconstant.

y′= k (4.4)

By continuously feeding this equation with the GPS points during therace, the vehicle is traveling straight if the k value is the same for thefollowing GPS points in the dataset. The GPS data fulfilling this isthen stored in a dataframe together with the rest of the collected datacorresponding to the GPS points in which the equation was fulfilled,i.e. when the vehicle traveled on the straight road.

Data: GPS k values calculated from GPS data and stored indataframe

Result: Straight road segment datawhile not end of dataframe do

read k;if k is same as previous datapoint then

store in list with rest of data with same timestamp;look at next data point;

elsenot a straight line;

endend

Algorithm 1: Code to extract straight road sections

CHAPTER 4. DATA MINING 13

Furthermore, to avoid unnecessary data computation, redundantinformation such as when the vehicle is at standstill is disregarded.This is done by filtering out the data points in which the vehiclespeed sensor is equals zero.

Chapter 5

Predictive modelling

Figure 5.1: Predictive modeling overview

When the data has been collected, filtered and stored, a predictivemodel can be obtained using regression methods. By using thevelocity in different instances during the race as regressors, these canbe easily optimised and then sent back to the driver or the controlsystem as references.

14

CHAPTER 5. PREDICTIVE MODELLING 15

5.1 Parameters used for modelling

Variable selection is of vital importance in regression analysis. Herethe parameters used for the predictive modelling are:

• Velocity in which the vehicle enters the straight road section.

• Velocity in which the vehicle exits the straight road section.

• Length of straight roadsection.

• Mean velocity on the straight section.

The predicted variable is the energy consumed by the vehicle whiledriving on the straight road.

5.2 Multivariable linear regression

As we have multiple inputs to predict the energy consumption, amultiple linear regression method is used. This method is the mostpopular regression method used when having multiple predictivevariables [23] and the mathematical expression is as follows:

y = AX+ b (5.1)

Where X is a n x p matrix with the independent variables, A is a plength vector with the predictors coefficients to be estimated and b isa residual random variable. As explained in section 5.1, the predictivemodel is based on four variables which results in the following linearregression expression:

y = a1 ∗ x1 + a2 ∗ x2 + a3 ∗ x3 + a4 ∗ x4 + b (5.2)

where

x1 = Velocity inx2 = Velocity outx3 = Lengthx4 = Mean velocity

a1...a4 = Coefficient for respective variableb = Residual random variable

16 CHAPTER 5. PREDICTIVE MODELLING

The general expression for a predicted outcome for any given datapoint i is:

yi =

p∑j=1

aj ∗ xij (5.3)

The cost function to be minimized in linear regression is the ResidualSum of Squares (RSS). The RSS is the sum of the errors squared ofthe predicted outcome relative to the actual value. The mathematicalexpression is as follows:

Cost function(a) = RSS(a) =n∑

i=1

{yi− yi}2 =n∑

i=1

{yi−p∑

j=1

ajxij}2 (5.4)

The multiple linear regression estimates the coefficients by usingOrdinary Least Squares (OLS). Thus, the estimates are obtained by:

a = (XTX)−1XTy (5.5)

5.2.1 Coefficient of determination

The accuracy of the regression models is based on the root meansquared value together with the coefficient of determination, alsoknown as R squared. It takes the residual sum of squares (RSS) anddivides it with the total sum of squares (TSS), resulting in thefollowing mathematical expression:

R2 = 1− RSS

TSS(5.6)

where RSS is the same as in equation 5.4 and:

TSS =t∑

j=1

(yj − yj,mean)2 (5.7)

Equation 5.6 results in a value between 0 and 1 where 1 gives us aperfect model that predicts the outcome perfectly.

5.2.2 Cross Fold Validation

In order to further improve the validation of the dataset and obtainoptimal coefficients avoiding overfitting for the studied physical

CHAPTER 5. PREDICTIVE MODELLING 17

system, cross fold validation is performed. [24] The cross validation isperformed by splitting the data into multiple subsets in whichtraining of our model is performed on each of the subsets excludingone subset for validation of the trained model. The method is alsoknown as K-Fold cross validation.

K-Fold

The data is split in k different subsets. Each subset contains the sameamount of data. The data used for training consist of k-1 subsets,leaving the last one for validation. The algorithm is then iterated ofthe k folds whereafter the optimal coefficients are obtained by takingthe average of each of the folds[25].

18 CHAPTER 5. PREDICTIVE MODELLING

5.3 Regularization

Problems can often be ill-posed and mathematical models of aphysical system should have certain properties in order to be seen asacceptable. Goodness of a predictive model is often based on theprediction accuracy [26].The relationship between the outcome and the independent variablescan be nonlinear, that is why it is necessary to study the case forhigher powers of the data collected. Introducing polynomialregression studies the nonlinear relationship between the responsevariable and the independent variables but is defined as linear from astatistical estimation point of view. The reason for this is thah theparameters, or coefficients that are to be estimated from theregression, are linearly related to the estimated variable. The methodfor polynomial regression was introduced as early as 1805 by thefrench mathematician Joseph Giaz Gergonne. [27]

5.3.1 Lasso

LASSO is an abbreviation for "Least absolute shrinkage and selectionoperator" which uses `1 regularization to perform variable selectionand constrain the magnitude of the coefficients in the model. Lasso isa method to exploit sparsity in large datasets which is useful to obtainboth statistical and computational gains. The idea of Lasso based on`1 norm penalty was founded 1995 but was not a popular method tobe used in modelling problems. The reason for that may be that largedatasets were not as common then and the computational powerwere not as good as they are today. [28]. In this project, this methodhave been studied to find out if it results in a more accurate model ofthe vehicle dynamics. The cost function that is to be minimized usingLasso is expressed as follows:

Cost function(a) = RSS(a)+λ∗(sum of absolute values of coefficients)

Cost function(a) =n∑

i=1

{yi −p∑

j=1

ajxij}2 + λ

p∑j=1

|aj| (5.8)

Comparing with the regular linear expression (equation 5.4), it can benoted that the expression has been extended with the absolute value

CHAPTER 5. PREDICTIVE MODELLING 19

of the variable coefficients. In other words, Lasso regression adds apenalty equivalent to the absolute value of the magnitude of thevariable coefficients. This results in variable coefficients that do not"contribute" to the outcome and obtains a zero value hence the"selection" in the abbreviation. Depending on how large the penaltyis on the size of the coefficients, sparsity of the regression model canbe varied. The larger the penalty, more zero-valued coefficients areobtained which results in a more parsimonious model at the cost ofaccuracy in the predictions.

5.3.2 Ridge

Ridge regression is another regularization method which is based ontheories initially proposed by the former Soviet mathematicianAndrey Tikhonov in 1943[29]. It was further expounded by thestatistician Arthur E.Hoerl who presented a method for thisregression method, to be used in a finite dimensional case in 1970[30].The Ridge regression method performs `2 - regularization, i.e. adds aterm which corresponds to the sum of the squares of the variablecoefficients to the objective function represented in equation 5.4. Themathematical expression for Ridge Regression is as follows:

Cost function(a) = RSS(a) + λ ∗ (sum of squares of coefficients)

Cost function(a) =n∑

i=1

{yi −p∑

j=1

ajxij}2 + λ

p∑j=1

a2j (5.9)

The estimates are obtained by:

a = (XTX + λI)−1XTy (5.10)

In equation 5.5, λ is used to determine the emphasis on reducing thecoefficients. It can be seen as a weight which if increased, the residualsum of squares will increase and the variability of the coefficients willdecrease which results in increased accuracy of the predictions. Sothere is a tradeoff between the bias and the prediction accuracy of themodel parameters when using Ridge regression but studies have

20 CHAPTER 5. PREDICTIVE MODELLING

shown that the estimated parameters are more reliable when usingthis method for regularization. It also makes it possible to mitigatethe effects of multicollinearity of the coefficients which removes theissue of having a large variance in the predictions if unexpectedchanges in the independent variables occur. [31]

5.3.3 Elastic Net

Elastic net is a regularization method combining the advantages ofboth `1 and `2 regularization. The elastic net regression methodsimultaneously selects the variables automatically as well ascontinuously minimize the variable coefficients. As stated above, theLasso regression selects only one variable if a group of variables arehighly correlated with the outcome, hence vital information can belost when using the Lasso regression method. This is avoided whenusing the Elastic net which retains the variables as it can select groupsof correlated variables. [32]

Cost function(a) =n∑

i=1

{yi −p∑

j=1

ajxij}2 + λα

p∑j=1

|aj|+ λ(1− α)p∑

j=1

a2j

(5.11)It can be noted that equation 5.7 is a combination of equation 5.5(Ridge) and 5.6 (Lasso). The value of α are chosen by trial and errorand functions as the ratio of penalty for the `1 and `2 regularization.Choosing α = 1 results in a Lasso regression etc. λ have the samefunction as in Lasso and Ridge, determining how much emphasis theregularization should have on the penalties.

When using large datasets, selection of the variables are preferable toobtain a sparse model. However, drawbacks when using Lasso andRidge as stated above, have motivated studies improving the methodby combining `1 and `2 penalties such as the Elastic Net. Thesemethods will all be applied to the data at hand to evaluate theaccuracies.

Chapter 6

Optimisation

Figure 6.1: Optimisation overview

21

22 CHAPTER 6. OPTIMISATION

6.1 Objective function

The objective function to be minimized is the obtained model fromthe regressions which is done in section 5 that predicts the energyconsumption together with the time it takes for the vehicle to travelthe straight road section. As different regression methods has beentested, the objective function will adhere to the regression methodused. The mathematical expression is as follows and a similaroptimisation method have been used in [2]:

Predictive model + Time constraint:

We ∗ (a1 ∗ x1 + a2 ∗ x2 + a3 ∗ x3 + a4 ∗ x4 + b) +Wt ∗x3x4

(6.1)

6.2 Sequential quadratic programming

Sequential quadratic programming, or SQP, is a highly successfuloptimisation method to solve nonlinear problems and gainedpopularity in the 1970s but references to SQP can be found on a PhDthesis from Harvard University in 1963 [33]. The method involveslinear and quadratic programming. The nonlinear problem isformulated as:

minimizex

: f(x)

subject to : h(x) = c1

g(x) ≤ c2

Where f(x) is the nonlinear objective function 6.1 presented above.The problem to be solved have equality and inequality constraintsand is denoted by h(x) and g(x) respectively. In this project theequality constraint is that the length of the straight road section doesnot change, hence x3 will be constant during the optimisation. Theinequality constraints are such that the velocities should maintainreasonable values, hence not exceed values that are not possible todrive.

Chapter 7

Results

7.1 Data processing

Using the communication infrastructure introduced in chapter 3, thevehicle managed to send its sensor data through the internet usingWiFi. The vehicle corresponds to a ROS node which continouslypublish the sensor data to the IoT Hub as explained in chapter 3. Thesensor data obtained was collected at 5 Hz and stored directly in theDynamoDB table.

The raw sensor data collected on the Amazon DynamoDB is analyzedwhen 100 new datapoints have been collected. Straight road sectiondata is extracted by applying the filtering presented in chapter 4 andstored in another table in SQLite.

Smoothening the data resulted in the following plots for the vehicletrajectory:Plotting the raw sensor data together with the filtered data shows howthe vehicle have driven and where the straight road section are located.The algorithm takes the starting point where it has entered the straightroad section and the ending point where it has exited whereafter asimple line is plotted between. In figure 7.2a the vehicle has drivenone lap and in figure 7.2b shows the trajectory after 20 laps togetherwith the straight sections.

23

24 CHAPTER 7. RESULTS

(a) Before smoothening (b) After smoothening

Figure 7.1: Vehicle trajectory, one lap

(a) Trajectory, one lap (b) Trajectory, 20 laps

Figure 7.2: Vehicle trajectories

7.2 Predictive model

Predictive models are obtained by using the filtered data in theregression algorithms presented in section 5.The predicted variable is the energy the vehicle has consumed on thestraight road section and the independent variables are the onespresented in section 5.1 but repeated here for convenience.

y = a1 ∗ x1 + a2 ∗ x2 + a3 ∗ x3 + a4 ∗ x4 + b (7.1)

CHAPTER 7. RESULTS 25

Figure 7.3: Regression, 20 laps

where

y = Energy Consumptionx1 = Velocity inx2 = Velocity outx3 = Lengthx4 = Mean velocity

a1...a4 = Coefficient for respective variableb = Residual random variable

The collected data is for this conventional regression method splitinto two parts, one training set and one test set. 75 % of the data isused for training and 25 % of the data is used for validation. Ouralgorithm detected approximately 28 straight lines in figure 7.2. Thismeans that 21 of those are used in training our model and 7 are usedto validate it. Starting with multiple linear regression (MLR) whereonly the RSS is to be minimized, the regression is plotted in figure 7.3and numerical results presented together in table 7.1 together with KFold cross validation results.The low accuracies in table 7.1 can be explained by the low numberof samples in the validation set. it seems that K Cross Fold validationperformance is worse which may be a result of the small data set. Thiswill be further explored when more data points are collected.

26 CHAPTER 7. RESULTS

Method RMS Accuracy b a1 a2 a3 a4MLR 29.25 0.502 192.13 -10.24 -5.95 0.88 4.03K-Fold – 0.104 234.03 -11.69 -9.66 0.68 6.87

Table 7.1: MLR and K-Fold with 3 folds

Regularization

Lasso

Lasso regression is tried with different λ values:

λ RMS b a1 a2 a3 a4 Accuracy10−15 39.33 170 -9.8 -2.9 1.1 1.3 0.65810−10 39.33 170 -9.8 -2.9 1.1 1.3 0.65810−8 39.33 170 -9.8 -2.9 1.1 1.3 0.65810−5 39.33 170 -9.8 -2.9 1.1 1.3 0.6580.001 39.33 170 -9.8 -2.8 1.1 1.3 0.6580.01 39.33 170 -9.7 -2.7 1.1 1.1 0.6581 40.01 170 -8.2 -1.4 1 0 0.6465 52.44 200 -3.6 0 0.46 0 0.39310 67.29 240 0 0 0 0 020 67.29 240 0 0 0 0 0

Table 7.2: Lasso

It can be noted that the step from λ = 0.01 to λ = 1 the room meansquared error (RMS) increase is greater than for smaller λ. Higher λalso reduces the model complexity as can be seen when λ = 1 wherea3 = 0. It is hence shown to be a trade-off between model overfittingand accuracy. For this reason, λ = 0.01 are used for the predictivemodel.

CHAPTER 7. RESULTS 27

Ridge

Ridge regression is performed in a similar manner with λ = 0.01

being the optimal value in this case as well.

λ RMS b a1 a2 a3 a4 Accuracy10−15 39.33 170 -9.8 -2.9 1.1 1.3 0.65810−10 39.33 170 -9.8 -2.9 1.1 1.3 0.65810−8 39.33 170 -9.8 -2.9 1.1 1.3 0.65810−5 39.33 170 -9.8 -2.9 1.1 1.3 0.6580.001 39.33 170 -9.8 -2.8 1.1 1.3 0.6580.01 39.34 170 -9.6 -2.7 1.1 1.1 0.6581 45.61 230 -4.7 -2.1 0.62 -1.3 0.5415 57.51 250 -1.8 -1.1 0.22 -0.83 0.27010 61.51 250 -1 -0.64 0.12 -0.53 0.16420 64.12 240 -0.54 -0.35 0.066 -0.3 0.092

Table 7.3: Ridge

Elastic Net

Table 7.2 and 7.3 shows that Lasso and Ridge regularization yieldsoptimal solutions when λ = 0.01 why this value is used for the ElasticNet in equation 5.11:

Method RMS Accuracy b a1 a2 a3 a4Elastic Net 39.33 0.658 168 -9.80 -2.83 1.06 1.3

Table 7.4: Elastic Net with α = 0.5

It can be noted that the regularization methods all perform similarlywhen it comes to accuracy and the root mean squared error.Comparing the models obtained used the initial dataset where 20straights were extracted and a dataset with 100 new straights yieldsresults presented in 7.5. To be noted is that the new data appended tothe original data set consist of the same data with random Gaussiandistribution data added.λ = 0.01 is used for the regularizations as it is still the optimalsolution.

28 CHAPTER 7. RESULTS

Figure 7.4: MLR with new data

Method RMS Accuracy b a1 a2 a3 a4MLR 24.86 0.759 136.35 -7.45 -0.85 0.99 0.96K-Fold – 0.728 147.11 -7.74 -0.85 0.99 0.32Lasso 34.28 0.791 133.40 -7.32 -0.38 1.02 0.39Ridge 34.28 0.791 134.98 -7.71 -0.8 0.98 1.06Elastic Net 34.26 0.791 133.69 -7.67 -0.57 1.01 0.72

Table 7.5: Model results

An increase in accuracy can be observed with all the models. Theaccuracy and RMSE are the almost identical for the regularizationmethods as in the case with the original dataset. They differ in thecoefficients where Ridge regression are stating a higher correlationbetween the mean velocity and the energy consumption comparedwith Lasso and Elastic Net. Negative values on coefficients a1 and a2tells us that there a negative correlation between the energyconsumption and the velocities in and out of the straight road section.This is a natural result as the vehicle will spend less energy inaccelerating in the section if it comes out of the curve in higherspeeds. This is not as trivial for the exit velocity as the regressionhave pointed out by having a small coefficient for that variable. Thelength of the straight road section does seem to have a one to onerelationship with the energy consumption. As the mean velocity iswhat the optimisation should output, the model obtained with RidgeRegression is chosen.

CHAPTER 7. RESULTS 29

Higher orders of the independent variables are also studied to see ifthere are any nonlinear relationships between the variables and theenergy consumption. The independent variables are all squared andextended in the original dataset:

y = a1∗x1+a2∗x2+a3∗x3+a4∗x4+a5∗x21+a6∗x22+a7∗x23+a8∗x24+b (7.2)

Multiple linear regression and K Cross fold validation with three foldsare presented in table 7.6.

Method RMS Acc a1 a2 a3 a4 a5 a6 a7 a8 b

MLR 34.98 0.712 -22.40 -15.74 0.98 11.47 0.71 0.55 -0.0005 -0.27 214.31K Fold – 0.734 -22.89 -18.24 0.87 11.87 0.73 0.64 -0.0002 -0.29 235.80

Table 7.6: MLR and K Fold with higher powers

Regularization methods Lasso and Ridge are used to obtain modelswith higher powers and resulted in the following values forλ = 0.001, 0.01, 1.

Lasso:

λ RMS Accuracy a1 a2 a3 a4 a5 a6 a7 a8 b

0.001 35 0.745 -21 -16 0.94 7.4 0.66 0.56 -0.0003 -0.15 2500.01 35 0.743 -20 -12 0.94 1.9 0.59 0.44 0 0 2601 42 0.621 -6.3 0 0.71 0 0 0 0 0 180

Table 7.7: Lasso with higher powers

Ridge:

λ RMS Acc a1 a2 a3 a4 a5 a6 a7 a8 b

0.001 35 0.745 -21 -15 0.97 6.4 0.63 0.53 -0.0004 -0.12 2400.01 35 0.744 -17 -8.9 0.96 1.1 0.43 0.31 0.0001 -0.0009 2301 42 0.621 -3.7 -1 0.38 -0.18 -0.15 -0.024 0.0015 -0.01 200

Table 7.8: Ridge with higher powers

30 CHAPTER 7. RESULTS

7.2.1 Regression comparison

For the data used in this project, it can be concluded that using one ofthe regularization regression algorithms yields the highest accuracy.In table 7.5, an overview of the accuracy together with the coefficientestimates have been presented.

The regression results presented in tables, 7.6, 7.7 and 7.8 shows thatnonlinear relationship exists but does not further improve ouraccuracy. It can be seen how the Lasso selects variables when settingvalues of the coefficients to zero. The reason why more coefficientsare set to zero when λ increases is because the emphasis onminimizing the coefficients also increases. Regarding the vehicleparameters, it seems that both x1 and x2, i.e. the speeds in which thevehicle enters and exits the straight road have a quadraticrelationship with the power consumption. However, using themodels obtained with quadratic relationships results in loweraccuracy. This is why the model obtained with Ridge Regression intable 7.5 will be used for the optimisation process.

7.3 Optimisation

Models that is to be optimised:

We(a1 ∗ x1 + a2 ∗ x2 + a3 ∗ x3 + a4 ∗ x4 + b) +Wtx3x4︸︷︷︸

Time constraint

(7.3)

where

We = Weight on saving EnergyWt = Weight on time it takes to complete straight road sectiona1 = −7.71a2 = −0.8a3 = 0.98

a4 = 1.06

b = 134.98

Comparing with the mean speeds and times it has taken for thevehicle to complete the road section, the goal for the optimisation is

CHAPTER 7. RESULTS 31

to output a mean velocity under different time constraints.Optimising the velocities using SQP as presented in section 7 resultedin values presented in 7.10.

ValueLength 150 mV meanin 7.46 m/sV meanut 12.91 m/sVmean 15.69 m/sTime 9.9 sEnergy 241 kJ

Table 7.9: Mean values

We Wt Vmean m/s Time (s) Energy (kJ)0.01 0.99 40 3.75 256.540.1 0.9 35.69 4.20 251.960.2 0.8 23.79 6.30 239.350.3 0.7 18.17 8.26 233.400.4 0.6 14.57 10.30 229.580.5 0.5 11.90 12.61 226.750.6 0.4 9.71 15.44 223.430.7 0.3 7.79 19.27 222.390.8 0.2 5.95 25.22 220.440.9 0.1 3.67 37.83 218.39

Table 7.10: Energy optimisation

32 CHAPTER 7. RESULTS

7.4 Conclusion

Having studied relationships between variables in a dataset,predictive models have been obtained. Using the data available, amodel with the prediction accuracy of 79.1 % have been obtained andused to predict the outcome of a potential drive session on a straightroad section. The optimisation have output a speed reference that ifmaintained, the vehicle efficiency can be increased by 4.08%compared to mean values of driven parameters. It has been shownthat more data increased the accuracy of the model which is vital ifthe output of the optimisation is to be trusted and used in the race.The choice of independent variables were done based on thepracticality of optimising the velocities and send the output to thedriver of the vehicle or if autonomous, the control system as areference.

7.5 Discussion

As the data at hand were very limited, the model may be overfittingas it has only been tested on a specific length. However, the algorithmcan be used whenever the vehicle is active on a track and connected tothe IoT Hub through ROS.As the time criteria is changed throughout the race, different weightshave been tested so that a clear overview regarding time versus energyconsumed on the straight road is obtained. Further work may includepredictive modeling of the energy in the curves to have a optimisationmethod including all the road sections in the race.The procedure presented in this work can benefit the transportation oftrucks, taxis and other vehicles not necessarily driven on roads.

7.6 Acknowledgements

I want to dedicate this last paragraph to give thanks to my supervisorand counselor Azad Noorani whom have given me insights on thetopic throughout the project. I also want to show my gratitude to theone making this project possible for me to complete, Håkan Roos.Last but not least my examiner, Jonas Mårtensson, who has beensupporting and giving me guidance, thank you!

Bibliography

[1] A. C. R. Teixeira and J. R. Sodré, “Impacts of replacement ofengine powered vehicles by electric vehicles on energyconsumption and co 2 emissions”, Transportation Research PartD: Transport and Environment, vol. 59, pp. 375–384, 2018.

[2] S. Aradi, T. Becsi, and P. Gáspár, “A predictive optimizationmethod for energy-optimal speed profile generation for trains”,in Computational Intelligence and Informatics (CINTI), 2013 IEEE14th International Symposium on, IEEE, 2013, pp. 135–139.

[3] C. X. Mavromoustakis, G. Mastorakis, J. M. Batalla, andP. Chatzimisios, “Social-oriented mobile cloud offloadprocessing with delay constraints for efficient energyconservation”, in Communications (ICC), 2017 IEEE InternationalConference on, IEEE, 2017, pp. 1–7.

[4] J. Waldo, “Embedded computing and formula one racing”, IEEEPervasive Computing, vol. 4, no. 3, pp. 18–21, 2005.

[5] K. Ashton et al., “That ‘internet of things’ thing”, RFID journal,vol. 22, no. 7, pp. 97–114, 2009.

[6] R. K. Rajak, A. Pawar, M. Pendke, P. Shinde, S. Rathod, and A.Devare, “Crop recommendation system to maximize crop yieldusing machine learning technique”, Int Res J Eng Technol, vol. 4,no. 12, pp. 950–953, 2017.

[7] G.-Y. Ban, N. El Karoui, and A. E. Lim, “Machine learning andportfolio optimization”, Management Science, vol. 64, no. 3,pp. 1136–1154, 2016.

[8] A Kharel, D. Bhutia, S. Rai, and D. Ningombam, “Cloudrobotics using ROS”, in International Journal of ComputerApplications R©(IJCA)(0975–8887), National Conference Workshop

33

34 BIBLIOGRAPHY

on Bioinformatics and Computational Biology, NCWBCB, Citeseer,2014, pp. 18–21.

[9] Y. Yamato, Y. Fukumoto, and H. Kumazaki, “Analyzingmachine noise for real time maintenance”, in EighthInternational Conference on Graphic and Image Processing (ICGIP2016), International Society for Optics and Photonics,vol. 10225, 2017, p. 102252I.

[10] R. Ranjan, “Streaming big data processing in datacenter clouds”,IEEE Cloud Computing, vol. 1, no. 1, pp. 78–83, 2014.

[11] S. Amini, I. Gerostathopoulos, and C. Prehofer, “Big dataanalytics architecture for real-time traffic control”, in Models andTechnologies for Intelligent Transportation Systems (MT-ITS), 20175th IEEE International Conference on, IEEE, 2017, pp. 710–715.

[12] J. Qiu and R. Wang, “Automatic extraction of road networksfrom gps traces”, Photogrammetric Engineering & Remote Sensing,vol. 82, no. 8, pp. 593–604, 2016.

[13] J. Moon, K.-H. Kim, Y. Kim, and E. Hwang, “A short-termelectric load forecasting scheme using 2-stage predictiveanalytics”, in Big Data and Smart Computing (BigComp), 2018IEEE International Conference on, IEEE, 2018, pp. 219–226.

[14] G. K. Tso and K. K. Yau, “Predicting electricity energyconsumption: A comparison of regression analysis, decisiontree and neural networks”, Energy, vol. 32, no. 9, pp. 1761–1768,2007.

[15] AWS SDK for Python using boto3, https://aws.amazon.com/sdk-for-python/, Accessed: 2018-06-02.

[16] AWS IoT Core aws iot core, https://aws.amazon.com/iot-core/, Accessed: 2018-06-02.

[17] Robot Operating System about ros,http://www.ros.org/about-ros/, Accessed: 2018-06-01.

[18] SQLite as a Database Engine about sqlite, https://www.sqlite.org/about.html, Accessed: 2018-06-02.

[19] A. Zhang, J. Deng, H. Luan, and L. Guo, “Design andimplementation of wi-fi signals collection and storage in sqlite”,in Agro-Geoinformatics, 2017 6th International Conference on, IEEE,2017, pp. 1–4.

BIBLIOGRAPHY 35

[20] D. J. Hand, H. Mannila, and P. Smyth, Principles of data mining(adaptive computation and machine learning). MIT pressCambridge, MA, 2001.

[21] J. S. Simonoff, Smoothing methods in statistics. Springer Science &Business Media, 2012.

[22] J. Han, J. Pei, and M. Kamber, Data mining: concepts andtechniques. Elsevier, 2011.

[23] H. Konno and R. Yamamoto, “Choosing the best set of variablesin regression analysis using integer programming”, Journal ofGlobal Optimization, vol. 44, no. 2, pp. 273–282, 2009.

[24] T. Tušar, K. Gantar, V. Koblar, B. Ženko, and B. Filipic, “A studyof overfitting in optimization of a manufacturing quality controlprocedure”, Applied Soft Computing, vol. 59, pp. 77–87, 2017.

[25] Z Reitermanova, “Data splitting”, in WDS, vol. 10, 2010,pp. 31–36.

[26] B. Efron, T. Hastie, I. Johnstone, R. Tibshirani, et al., “Least angleregression”, The Annals of statistics, vol. 32, no. 2, pp. 407–499,2004.

[27] J. Gergonne, “The application of the method of least squares tothe interpolation of sequences”, Historia Mathematica, vol. 1,no. 4, pp. 439–447, 1974.

[28] R. Tibshirani, “Regression shrinkage and selection via thelasso”, Journal of the Royal Statistical Society. Series B(Methodological), pp. 267–288, 1996.

[29] A. N. Tikhonov, “On the stability of inverse problems”, in Dokl.Akad. Nauk SSSR, vol. 39, 1943, pp. 195–198.

[30] A. E. Hoerl and R. W. Kennard, “Ridge regression: Biasedestimation for nonorthogonal problems”, Technometrics, vol. 12,no. 1, pp. 55–67, 1970.

[31] R Muthukrishnan and R Rohini, “Lasso: A feature selectiontechnique in predictive modeling for machine learning”, inAdvances in Computer Applications (ICACA), IEEE InternationalConference on, IEEE, 2016, pp. 18–20.

[32] H. Zou and T. Hastie, “Regularization and variable selection viathe elastic net”, Journal of the Royal Statistical Society: Series B(Statistical Methodology), vol. 67, no. 2, pp. 301–320, 2005.

36 BIBLIOGRAPHY

[33] P. T. Boggs and J. W. Tolle, “Sequential quadratic programming”,Acta numerica, vol. 4, pp. 1–51, 1995.

TRITA TRITA-EECS-EX-2018:402

ISSN 1653-5146

www.kth.se