Abstract — Hybrid electric vehicles (HEV) are those equipped with two or more energy sources, usually, a fuel tank with its associated internal combustion engine (ICE) and an electrical storage system (ESS), typically a bank of batteries. In order to efficiently operate the system it is necessary to determine the instantaneous power split between the two sources when the vehicle performs a predetermined duty cycle. In this work, this problem is posed as an optimal control problem with constraints, specifically, as an inventory control problem and solved using dynamic programming (DP). Results obtained for the HEV being developed in the Applied Electronics Group, School of Engineering, National University of Río Cuarto are shown.

Keywords — Optimal Control with Constraints. Dynamic Programming. Supervisory Control of Hybrid Electric Vehicles.

I. INTRODUCTION

HEVs are those whose architecture includes two or more energy sources, usually a fuel tank and a bank of batteries. These energy sources are associated to energy converters such as an ICE and an electric motor respectively. These vehicles take advantage of the cleanliness and high efficiency of electrical traction and overcome its main drawback that is its low range. ESSs currently available have a low energy density. This is compensated in HEVs by the high energy density of fossil fuels, usually two or three orders higher than that of ESSs.

Energy storage elements and converters can be arranged following different topologies. Figure 1 shows a scheme of the so-called "series" configuration. In this configuration, an electric motor moves the wheels. An ESS that may consist of a bank of batteries and/or ultra capacitors feeds this motor. On the other hand, the ICE is fed by the fuel tank and drives an electric generator. This generator provides electric power to the traction motor when the power demanded by the driver exceeds that provided by the ESS. On the contrary, when the power provided by this generator exceeds that demanded by the driver, this excess is used to recharge the ESS.

Hybrid electric as well as purely electric powertrains have the advantage of "regenerative braking". This involves using the electric motors as generators during braking, transforming the mechanical energy into electrical energy. In this way the kinetic energy stored by the vehicle is recaptured by the ESS. The double arrows that connect the wheels to the ESS (see Fig. 1), represent this reverse energy flow.

For the same performance target, the ICE and ESS of a HEV can be of smaller size than those of a conventional or a pure electric vehicle. However, the whole system performance will also depend on how they interact. At first sight, it seems that the ESS should mainly perform velocity changes, taking advantage of the reversibility of the electrical path, whereas the ICE should supply the rest of the power. The nominal power of the latter should be such that it could be used most of the time near its optimal operation point. In this way, consumption as well as gas and sound emissions would be reduced.

HEVs need an electronic power manager that must determine at each instant the amount and direction of the flow in each path. This higher-level control is usually known as "supervisory control". Power electronics devices control each particular power converter according to the commands from the supervisory control.

The coordination between sources and physical and operational limitations of the many devices involved force trade-offs. Hence for an efficient operation of a hybrid powertrain it is necessary to optimize the supervisory control strategy. An HEV designed for city use is being developed by our research group. It is in city use where the advantages of HEVs are most noticeable, because of the frequent acceleration and deceleration. The purpose of this work is to contribute to the definition of an optimal strategy for the supervisory control of this vehicle.

This problem, as an optimal control problem, may be posed in different forms depending on the objective desired, the model considered, the control action and the constraints imposed. However, there are some common features to all approaches. Concerning the control objective, it is natural to consider minimizing fuel consumption while not degrading the vehicle dynamical response. Concerning the dynamical models, they unavoidably include combinations of linear and nonlinear, discrete and continuous, algebraic and dynamical systems. Moreover, they are subject to constraints not only on the control variables but also in the state variables as it is explained in Section II. Hence, most of the reported optimal supervisory control strategies use either intelligent control techniques such as rule-based, fuzzy logic and neural networks, or optimal control approaches. Within the latter, Delprat et al. (2001; 2004), Steinmauer and del Re (2001) and Daniels and Kumar (1999) use Pontryagin Maximum Principle optimality conditions. Zaremba et al. (2002), Brahma et al. (2000a;b) and Sciaretta et al., (2004) use a discrete approach and a dynamic programming algorithm.

In previous work (Pérez et al., 2004) we followed Brahma et al. (2000,a; b) and posed the problem as a shortest path problem. The drawback to this approach is the treatment of a constraint of integral form. This constraint arises from the need to preserve the batteries from depletion or overcharge, and/or if a "charge sustaining" operation of the vehicle is imposed. Brahma et al. (2000a;b) proposes a method using a penalization term added to the objective function. We applied this approach and proposed an alternative method based on checking this constraint as the algorithm proceeds. Both methods include heuristics and consequently give sub-optimal results.

In this paper, we have tried to overcome the use of heuristics by posing the problem as a deterministic inventory control problem. In this way we can find the optimum, up to the discretization step considered (Bertsekas, 2001). Section III refers to the algorithm used to find the solution.

This approach was applied to find the optimal power split between the bank of batteries and a hypothetical ICE and generator to be used in the HEV being developed. A brief example of some of the results obtained is included in Section IV.

We consider this work, one of the first steps towards the definition of a supervisory control strategy for our prototype. Finally, we include some conclusions and a prospective of our future work in Section V.

II. STATEMENT OF THE PROBLEM

A. Abstracted model for the vehicle

In order to solve the supervisory control problem, it will be enough considering an abstracted scheme for the system that represents the vehicle (Brahma et al., 2000a, 2000b; Rizzoni et al., 1999) where the intermediate devices of the powertrain such as rectifiers and converters, are replaced by the net power flow in each path. P_FT(t) will indicate the power flow at time t in the fuel tank/engine/generator path (which we shall call FTpath henceforth) and P_ESS(t) the power flow at time t in the ESS-path (Fig. 2). The energy losses that take place in the intermediate converters will be represented by an efficiency factor.

The following convention is also established: a positive power flow means power flowing away from the ESS. Consequently, during regenerative braking a negative flow will take place in the electrical path. Besides, the power flow from the fuel tank cannot be negative, as it cannot absorb any power.

The required vehicle velocity profile is considered a given function. The required power can be computed from this profile using a model of the vehicle longitudinal dynamics. Hence, it is also considered a known function that will be denoted by P_req(t). This is indeed not true for the real case, where the future velocity is not known but depends on the transit and road conditions, but we hope this approach will be able to be extended to real driving conditions by using a stochastic approach and/or short-term horizons.

B. Power balance

A balance equation can naturally be established, since the sum of the power from both sources has to be equal to the required power at all times:

P_FT(t) + P_ESS(t) = P_req(t). (1)

In the series configuration this addition takes place in the form of electrical power.

C. Energy consumption and efficiencies

Regarding the net energy consumed from each source in a time interval, we must take into account that not all the power delivered by the source can be actually used to supply the demand, since in every energy conversion process there are losses. Therefore, only a portion of the power delivered will reach the summing junction in Fig. 2. In our abstracted model, this portion will be represented by two functions η_FT and η_ESS that take values between 0 and 1, and will be referred to as the efficiencies associated to each path. Indeed, efficiencies depend on the power flows. We consider them known functions that in practice are experimentally determined for all possible power values. Hence, the net energy contained in the FT at time t can be computed as follows:

where E_FTo is the initial energy in the fuel tank.

To compute the net energy consumed from the ESS it has to be considered that during acceleration (P_ESS >0) the effect of losses is represented by dividing the delivered power by η_ESS as in the previous case. But during regenerative braking, because of losses, only a portion of the kinetic energy coming from the wheels will reach the ESS, and so the inverse situation has to be represented. Let us define

Then, the net energy contained in the ESS at time t is

where E_ESSo is the initial energy in the ESS.

D. Control objective

The control objective is to minimize fuel consumption in a time interval [0,T], T known. That is, to minimize the net energy consumed from the chemical source in the interval, i.e.:

E. Control action, state variable and state equation

Our purpose is to determine for each t in [0,T] the values of P_ESS and P_FT that minimize the objective. Using the balance equation (1), we can eliminate one of these functions. Hence, the problem can be posed in two alternative forms. Either P_FT or P_ESS can be taken as the control action or the independent variable, over which the minimization will be performed. The remaining one is considered the dependent variable and is obtained from the first. Physically, this implies that the supervisory control will be exerted either by the engine acceleration command or by the ESS power controller. The alternative device should provide the necessary remaining power to satisfy the demanded power Preq. Since both problems are similar only the case where P_FT(t) is the control action will be described.

From (1) and (4) we arrive at

This will be considered the state equation and E_ESS the state variable, with initial condition E_ESSo. Expression (6) is indeed a simple integral rather than a state equation in the usual sense, since the right hand side does not depend on the state variable but only on the control input. However, it is convenient to consider it a state equation in order to use DP algorithms for optimal control in a straightforward manner.

Note that considering that/is defined piecewise and that η_FT and η_ESS are non-linear functions, the state equation also results non linear.

Although it is not necessary to impose a final condition to the state, in order to simplify the presentation, we will set E_ESS(T) = E_ESSo. This represents a "charge sustaining operation" of the ESS, which may be a desired feature.

F. Constraints

Clearly, the power flows are physically limited, hence:

In addition, if the ESS is a bank of batteries, it has to be protected from depletion and overcharge. This implies that the net consumed energy from the ESS has to be maintained between proper limits at each instant t. Then,

In summary, there are constraints on the control action P_FT(t) and on the state variable E_ESS(t).

III. DYNAMIC PROGRAMMING SOLUTION

Because of the non-linearities of the dynamic system and the many constraints involved, we choose a discrete approach and a dynamic programming solution (Chiang, 1992). From this point of view the problem is similar to an inventory control problem (Bertsekas, 2001), where for the deterministic case, the demand is known, the cost function only considers the purchase inventory and there is the possibility of returning goods to stock.

A. Discrete formulation

Let us divide the interval [0, T] in N stages of length Δ.t. Let also P_FTk = P_FT(kΔt), k = 0, 1, ... N-l be the discrete control sequence that is being searched, i.e., the sequence of decisions on the power flow in the FT- path at each stage. Let E_ESSk represent the possible states of the system that satisfy the discrete state equation

The sequence P_reqk, is known and in the usual form, P_reqk=P_req(kΔt). Finally, let V_d be the discrete cost functional

B. Network and arc costs

In order to solve the problem, let us consider a network like the one shown in Fig. 3. The horizontal axis corresponds to time stages t_k, k = 0, 1, ..., N, and the vertical axis corresponds to the possible discrete values for the state variable E_ESSi, i = 0,1, ..., M, equally spaced between E_ESSmin and E_ESSmax. Then, each node corresponds to a possible state E_ESSik at time k. Each node E_ESSik, i = 0, 1, ..., M, at stage k is connected to each node E_ESSjk+1, j = 0, l, ..., M at the following stage k+1. In the algorithm used, all possible connections between nodes of successive stages have been considered, except for the initial and final stages, since they are fixed. Hence, a discrete state trajectory is formed by a sequence of nodes, one for each stage k, such that connects the initial and final states. In the network there are, in principle, M^N-1 possible trajectories. Because of the way in which the network is built, all of them satisfy (15).

Figure 3. Network considered in the supervisory control problem, exemplified for the case N=6, M=4.

Now, state trajectories are completely determined by P_FTk. according to the state equation (10). Considering that P_FTk is subject to the constraints (13) y (14), not all the trajectories of the network will be feasible. Let then S_k be the set of nodes of stage k, such that at least one feasible trajectory passes trough it.

As usually in DP, we call "arc" the segment that connects a state E_ESSik of stage k to another state E_ESSjk+1 of the following stage. Each arc has an associated "arc cost" that will be denoted by aijk and is the contribution to the total cost that is produced if the state changes from node E_ESSik to node E_ESSjk+1. For this problem, the arc cost is the energy consumed from the fuel tank when the energy contained in the ESS changes from E_ESSik to E_ESSjk+1 and is computed as follows:

where P_FTk^ij is the control input needed to make the state go bom E_{ESS ik} to E_{ESS jk+1}, according to Eq. (10).

C. Policy

To solve this problem using DP we also need a function that maps states to controls. This function is usually named "policy". In this approach, this function is obtained from (10) and can be formally expressed as:

This will allow computing, for each arc between two possible successive nodes E_{ESS ik} and E_{ESS jk+1}, the value for P_FTk^ij that drives from one to the other. From this value the corresponding arc cost is computed using (16).

It is worth noting that the formal expression (17) is a result of the discretization considered in (10). It is also the key point that allows to solve the problem by an algorithm which, even though it is of the same order of complexity than the one presented in the work of Brahma et al., (2000a;b), it does ensure that the constraint (9) is satisfied, without using any heuristics.

The expression (17) makes sense because of the form that the non-linear function f takes in the range of interest. Figure 4 shows the efficiency function η_ESS(P_ESS) used in this work. It was obtained by fitting two polynomials to experimental data. The corresponding graph for f is shown in Fig. 5. It can be seen that because of the form of η_ESS, f results strictly increasing and hence, setting f^-1(0) = 0, it is possible to recover P_FT for all pair of states  in the network.

D. DP Algorithm

Regarding dynamic programming algorithms, we refer to Bertsekas (2001) or Kwong and Rogers (2004), and only will say here that they consist essentially in looking over every node in the network and computing the minimum cost to go from each particular node to the final node (tail subproblem). The complexity is reduced using a recursive algorithm that usually goes backwards in time and computes all the tail subproblems of each stage using the solutions found for the tail subproblems of the following stage. The minimum cost trajectory is found at the last recursive step, after looking over all nodes in the network.

In the particular implementation made in this work, arc costs are computed as the recursion proceeds. For each node ik, the value P_FTk^ij capable of driving the state E_ESSik to the state E_ESSjk+1 is computed using (17) for all 0≤j≤M. Then, P_FTk^ij is checked to see if it satisfies constraints (13) and (14) and, if so, a_ij^k is computed using P_FTk^ij in Eq.(16). Otherwise, the corresponding arc is discarded from the network. Let S_k+1 be the set of the surviving j's. This is the set over which the minimum indicated in expression (18) below is taken. We include the algorithm excluding, for brevity, the special treatment of the initial and final stages and writing E instead of E_ESS for short.

Algorithm

IV. SIMULATION RESULTS

The above method has been applied to the case of the experimental HEV being developed in our group. This prototype is currently powered in a purely electric form by a bank of batteries and is equipped by a 32 kW electric motor for traction. In previous work, the longitudinal dynamics of this prototype has been modeled and validated through road tests (Pérez et al. 2002). This model allows the computation of the power demanded by the vehicle to follow a velocity cycle, including mechanical losses and losses from the electrical motor. The model has been linked to the DP algorithm to provide the values for P_req.

A hypothetical fuel converter system capable of delivering a maximum power of 40 kW has been considered. Its efficiency function, η_FT, was taken from Brahma et al. (2000b). The discretization step for the energy was ΔE_ESS = 0.0028kWh and the time discretization was Δt = 2.5seg, since it was observed that the results did not change substantially for finer grids. In addition, the solution is currently being checked by using an alternative approach based on "direct transcription" where the state variable need not be quantized.

The electric storage system is a bank of 20 Yuasa-Exide EV-5 batteries (in series), with nominal charge equal to 197 Amph each. The minimum charge needed for this system to work properly is 20% of the nominal charge. The voltage can be, in a first approximation considered constant, equal to its nominal value (U_nom = 120V). Then, E_ESSmin = 1.31kWh and E_ESSmax = 6.57kWh. However, this bank is excessively large for an HEV, since it has been sized for a pure electric vehicle. If the ESS has enough energy to perform the cycle and compensate for the losses, the solution of minimum consumption will be the trivial solution, P_FT(t) = 0. That is, no usage of the FT-path and full usage of the ESS.

Hence, to show illustrative results, we used for the simulation in Fig. 6, a hypothetical smaller ESS, with E_ESSmin = l.25kWh and E_ESSmax = 2.5kWh, and an initial state E_ESSo = 2.0kWh.

Figure 6-(a) shows the velocity profile corresponding to the European Normalized Driving Cycle that has to be limited to a maximum velocity of 60 km/h to fit the design constraints of this vehicle. Figure 6-(b) and (c) show in dashed line the power demanded by this vehicle to perform the velocity cycle in (a). The solid line shows in (b), the algorithm output, i.e., the optimal P_FT profile, and in (c) the complementary P_ESS profile. The graph in (d) is the corresponding ESS energy profile. It can be seen that it moves within the required bounds and that performs a charge sustaining cycle.

Regarding consumption, it is difficult to establish a comparison with other control strategies, since strategies that are defined by rules, often fail in following the required velocity profile and, therefore, the comparison is not relevant. Nevertheless we include in Table I consumptions obtained using some other strategies, just as reference values. The algorithm we proposed in Perez (2004) fails to render a result for an initial ESS energy initial value so low as the used in this case, so it is not included. The only strategy that is comparable with our results is that corresponding to the method proposed by Brahma et al. (2000,a). It consists of adding a term to the objective functional to penalize the use of energy from the ESS. This term includes a parameter that regulates this penalization. This parameter has to be determined by trial and error in such a way that the operation over the whole time interval resulted "charge sustaining". For the data of the example in Fig. 4, a value for this parameter was searched so that the final energy in the ESS resulted as close as possible to E_ESSo. Then, we run our algorithm with that final condition and so we obtained comparable results. Note that although the ESS energy consumption is almost equal, the FT consumption is lower for our algorithm . Finally, note that the optimal consumption shows a 27% reduction respect to the one that would have been obtained if only the FT-path had been used (as in the case of a conventional vehicle).

Table I. Consumption for different supervisory control approaches
 
* This strategy does not follow the velocity profile
** This strategy (Emadi et al., 2004) violates constraints (13) and (14).

Concerning the time consumed by this algorithm, it can be seen that its order of complexity is O(M²N). This is the same order than that of Brahma's algorithm. In this case, the execution time will depend mainly on the state discretization, while in the former depended on the discretization of the control function. In either case the choice is arbitrary. If the same power resolution than that of Brahma's algorithm is to be obtained, then M may result larger. However, the power resolution also depends on the choice of N. Hence, a trade-off may then be used to regulate the computation time. In the example of the figure, M = 450, N = 480 and so the whole network has 216000 nodes. This implies power steps greater than 4 kW. For these values, the "crude" main loop took about 30 min to run, using MATLAB on a standard PC with a 2.08 GH Athlon Processor. Clearly, this time can be reduced by translating the code to C language, interrupting internal loops when it is clear that constraints will be thereafter violated and replacing repetitive computations by look-up tables.

V. CONCLUSIONS AND FUTURE WORK

We think that this problem statement and the solution proposed have been successful for the off-line optimization of the power split between the two sources, without the need of heuristics used in previous works. Its main drawback is its high computational cost. We are now working on improving the algorithm to reduce this cost. In any case, the algorithm outputs provide a template to learn from, which can be used to develop rule based control laws.

In the presented approach, it seems that including stochastic features and additional controls would be conceptually simple, though, again, computationally heavy. Nevertheless, we think that considering short-term horizons reduces the computation time and thus, we will be able to extend this algorithm to on-line applications and to the problem of including an additional ESS such as a bank of ultra capacitors. This is an interesting point for hybrid traction, since ultra capacitors in contrast to batteries, can deliver high bursts of power, though for short periods of time. Therefore, they are usually included to improve the vehicle dynamical response.

Finally, we think that this algorithm may apply to other systems including sources, storage elements and consumers, of interest to our group. Examples of these are power electronic devices, stand-alone wind or solar stations and power networks including hydro electrical and nuclear power stations.

REFERENCES