Bilevel programming approach to demand response management with day-ahead tariff

Bilevel programming approach to demand response management with day-ahead tariff

Andra´s KOVA´ CS¹

Abstract This paper introduces a bilevel programming approach to electricity tariff optimization for the purpose of demand response management (DRM) in smart grids. In the multi-follower Stackelberg game model, the leader is the profit-maximizing electricity retailer, who must set a time-of-use variable energy tariff in the grid. Followers correspond to groups of prosumers (simultaneous produc- ers and consumers of the electricity. They response to the observed tariff, schedule controllable loads and determine the charging/discharging policy of their batteries to mini- mize the cost of electricity and to maximize the utility at the same time. A bilevel programming formulation of the problem is defined, and its fundamental properties are proven. The primal-dual reformulation is proposed in this paper to convert the bilevel optimization problem into a single-level quadratically constrained quadratic program (QCQP), and a successive linear programming (SLP) algorithm is applied to solve it. It is demonstrated in computational experiments that the proposed approach outperforms typical earlier methods based on the Karush–

Kuhn–Tucker (KKT) reformulation regarding both solution quality and computational efficiency on practically relevant problem sizes. Besides, it also offers more flexible mod- eling capabilities.

Keywords Demand response management (DRM), Smart grid, Game theory, Optimization, Bilevel programming

1 Introduction

A key to the stable operation of future electricity grid is realizing efficient demand response management (DRM).

With the increasing share of renewables in energy mix, the production is becoming less and less controllable. At the same time, electricity consumption is becoming more controllable due to new types of loads and storage (e.g., electric vehicles, home-level or small business energy management solutions) and various intelligent appliances at end consumers. As a result, a gradual shift from the traditional ‘‘supply follows demand’’ paradigm to a new

‘‘demand follows supply’’ approach can be observed. The critical success factor for efficient DRM is an appropriate electricity tariff that motivates consumers to schedule their loads and manage their batteries in such a way that it contributes to grid stability.

This paper studies the problem of optimizing the elec- tricity tariff offered by an electricity retailer to its cus- tomers in a game theoretical setting. A bilevel programming approach is introduced, where the retailer is the leader and the groups of end consumers act as multiple independent followers. The customers are modelled as

‘‘prosumers’’, i.e., simultaneous producers and consumers of electricity, who look for the best tradeoff between maximizing their utility and minimizing their cost of electricity. An effective and computationally efficient solution method is proposed. The bilevel program is firstly transformed into an equivalent single-level optimization problem using a primal-dual reformulation, and then solved using a successive linear programming (SLP) algorithm.

CrossCheck date: 28 April 2019

Received: 1 August 2018 / Accepted: 28 April 2019 / Published online: 14 November 2019

ÓThe Author(s) 2019

& Andra´s KOVA´ CS

akovacs@sztaki.hu

1 EPIC Center of Excellence in Production Informatics and Control, Institute for Computer Science and Control, Hungarian Academy of Sciences, Budapest, Hungary

After reviewing the related literature in Section2, a formal definition of the tariff optimization problem is given in Section3. The proposed approach is presented in detail in Section4. The approach is illustrated on a small-scale example and evaluated in thorough computational experi- ments in Section5. Finally, conclusions are drawn and possible directions for future research are suggested.

2 Literature review

2.1 Game-theoretic models for DRM

Game-theoretic models for DRM have received signifi- cant attention recently [1]. A fundamental classification of these models differentiates between ‘‘real-time’’ vs. ‘‘day- ahead’’ approaches. Real-time pricing (RTP) models often focus on the present time instant only, and ignore the interdependence between the present energy tariff and past or future consumption. Accordingly, these models con- centrate on load curtailment, but fail to capture deferrable loads appropriately. This limitation can be lifted by applying multi-period models. Still, most of the earlier contributions focus on the RTP scenario with a single time instant: a multi-leader, multi-follower Stackelberg game is defined for DRM among independent electricity providers and consumers in [2]. A closed-form analytical solution is derived, which can be obtained by a distributed algorithm.

The management of consumer-to-grid systems is modelled as a Stackelberg game in [3], with a central power station acting as the leader, and consumers as multiple followers.

Embedded into the Stackelberg game, the consumers play a generalized Nash game to establish their equilibrium strategies, and hence, to determine their response to the energy prices offered by the power station. A similar approach is applied to electric vehicle charging in [4].

Reference [5] investigates DRM on three levels of hierar- chy (the grid operator, multiple service providers, and the consumers) with RTP, and proposes a two-loop Stackel- berg game model. The existence of a unique Stackelberg equilibrium is proven by exploiting the strictly convex sub- problems of the individual players, and an iterative dis- tributed algorithm is proposed for reaching it. A Stackel- berg approach is investigated for DRM under load uncertainty in [6]. Again, an analytical solution could be derived.

Reference [7] studies a Stackelberg game for RTP over multiple time periods with a profit-maximizing retailer and a single end consumer who looks for the best tradeoff between electricity cost and comfort in the heat manage- ment of a building. The problem is formulated as a bilevel program, and then converted to and solved as a single-level mixed-integer linear program (MILP) using the Karush–

Kuhn–Tucker (KKT) conditions. The same paper shows that while RTP is vastly efficient for load shifting, it can cause excessive and imponderable payments for small consumers. Therefore, more predictable day-ahead pricing schemes are an attractive approach for households.

Despite this, the literature of ‘‘day-ahead’’ tariff opti- mization models is significantly scarcer. In [8], a Stackel- berg model is proposed for energy pricing and dispatch in a multi-period day-ahead setting in two coupled stages. The first stage addresses price setting subject to demand response from consumers who minimize their energy cost and maximize their utility by scheduling their controllable loads. In the second stage, the retailer establishes the operation strategy for its storage unit and its energy con- tracts by solving a robust optimization problem considering uncertain market prices. The authors transform this prob- lem into a single-level MILP by exploiting the KKT con- ditions and duality theory. In [9], a Stackelberg game is formulated and solved using an iterative heuristic approach. Two different games related to demand side management are studied in [10]: a Nash game between consumers equipped with batteries and a Stackelberg game between the utility provider and the consumers. A bilevel programming approach to the operation scheduling of a distribution network, with a cost-minimizing network operator as the leader and multiple profit-maximizing microgrids as followers, is considered in [11]. Again, KKT reformulation is applied to arrive at a single-level problem.

A sophisticated Stackelberg game model is presented in [12] to capture the interplay of a retailer (leader) and various types of distributed energy resources, including generators and consumers with different types of control- lable load as well (followers). Again, the problem is con- verted to a single-level MILP using the KKT reformulation. A similar problem, with power flow con- straints and a retailer who also oversees the operation of distributed generators and batteries, is studied in [13].

2.2 Related problems in energy management and DRM

The consumers’ (followers’) problem in the above Stackelberg games corresponds to an energy management problem for minimizing the cost and maximizing the utility.

Linear programming (LP) models limited to active power flow equations are commonly used in the literature for solving this problem [14]. More sophisticated, non-linear models allow capture reactive power and voltage magnitudes as well [15,16], or describe the behavior of the energy sys- tem components (battery, or heating, ventilation, and air conditioning in buildings) in a more realistic way [17].

Obviously, Stackelberg games and bilevel programming approach are not the only possible approaches to DRM

problems. Alternative methods are often based on statisti- cal models of the grid-level load response to the variation of energy prices [18]. An iterative solution procedure that alternates between the optimization problems of the con- sumers (minimizing the cost) and the grid (maximizing the load factor) is presented in [19] for smart building-to-grid systems, using a sophisticated thermal model of the buildings. In [20], the problem of dynamic pricing for DRM is formulated as a Markov decision process, and reinforcement learning is used to solve it. In [21], demand response is modeled by directly quantifying the delay-tol- erant demand and its dependence on price by linear, potential, exponential and logarithmic load functions.

Game-theoretic approaches to different, but related problems in energy management include [22], where a Stackelberg approach is proposed for achieving a fair curtailment of renewable energy generation. A Stackelberg game model is investigated in [23] with a central produc- tion unit (leader) who sets the electricity price to maximize its profit subject to the response from an electricity service provider (follower) that will accept load curtailment and distributed generation bids from various microgrids in view of the central producer’s price. A supply-demand game is investigated in a smart grid setting in [24], with generators and loads acting as multiple followers, and a data center server as the virtual leader; a deep transfer Q-learning algorithm is applied for finding the equilibrium. The opti- mal operation of multi-carrier energy systems is modelled as bilevel optimization problem in [25]. The upper level problem of minimizing the total energy cost and the lower level problem of minimizing the operation and dissatis- faction costs are solved through an iterative procedure.

2.3 Mathematical methodology

An introduction to bilevel programming approach, including basic modelling and solution techniques, is given in [26,27]. Approaches to transforming bilevel optimiza- tion problems into equivalent single-level models, includ- ing the optimal value or the KKT reformulation, are studied in [28]. A recent survey on bilevel programming for price setting problems is given in [29].

SLP has been applied in smart grid community, e.g., to the planning of generators’ investments and transmission network extensions [30], or to the tackling of non-linear phenomena in variants of the optimal power flow prob- lem [31]. At the same time, to the best of our knowledge, this paper is the first to apply SLP to tariff optimization for demand response.

2.4 Positioning of current contribution

This section surveys the literature on game theoretical models to electricity tariff optimization for DRM and related problems. The algorithmic techniques applied to Stackelberg tariff optimization problems are summarized in a tabular format in Table1. The survey shows that although some simpler formulations, all focusing on a single time period, adopt a solution that can be computed analytically in closed form, multi-period problems are computationally more challenging. This observation is also supported by a formal proof in [32], which states that multi-period models for DRM with controllable loads at the consumers are NP-hard. Accordingly, the vast majority of earlier contributions apply the KKT reformulation to arrive at a single-level MILP that can be solved using Table 1 Algorithmic techniques applied for solving Stackelberg tariff optimization models in literature

Paper Game model Time representation Solution approach

[2] Stackelberg game Single period Closed-form analytical solution*

[4,3] Generalized Nash embedded in Stackelberg Single period Iterative distributed algorithm using KKT & convex optimization [5] 3-level Stackelberg game Single period Iterative distributed algorithm

[6] Stackelberg game, stochastic Independent periods**

Closed-form analytical solution

[7] Stackelberg game Multi-period KKT for conversion to single level, MILP [8] Stackelberg game*** Multi-period KKT for conversion to single level, MILP

[9] Stackelberg game Multi-period Iterative algorithm

[10] Stackelberg game**** Multi-period Iterative algorithm

[11] Stackelberg game Multi-period KKT for conversion to single level, MILP [23] Stackelberg game Multi-period KKT for conversion to single level, MILP [13] Stackelberg game Multi-period KKT for conversion to single level, MILP

Note: * denotes that distributed algorithm for achieving the equilibrium is also presented; ** denotes that the multi-period problem can be reduced to multiple single-period problems; *** denotes that in a 2nd-stage problem, robust optimization is applied to implement the equilibrium under uncertainty; **** denotes that a Nash game among consumers is also described in the paper.

commercial solvers. However, at the price of considerable computational effort, and a number of papers mention that the solution approach is applicable mostly to small-scale problems [13]. Other approaches use customized heuristics for solving the problem.

The contribution of this paper is twofold. On one hand, it defines a generic game-theoretic model for DRM that slightly extends the above discussed models (e.g., it captures both distributed battery storage and controllable loads character- ized by a given utility function at the consumers). And the key properties (e.g., necessary conditions for feasibility, compu- tational complexity) can be formally proven. On the other hand, it proposes an efficient solution approach based on well- established mathematical programming techniques that first exploits duality for the followers’ model to convert it into a single-level quadratically constrained quadratic program (QCQP), and subsequently, applies an SLP approach to solve it. It is shown in computational experiments that the proposed approach outperforms earlier KKT-based methods regarding both solution quality and computational effort for practically relevant problem sizes.

This paper is a substantially extended version of the earlier conference paper [33]. In addition to a refined model that captures the profit-maximizing behavior of the retailer, the extensions are related to the core contributions of the present paper, i.e., the formal proofs of the funda- mental characteristics of the model and the thorough computational experimentation for assessing solution quality and computational efficiency.

3 Problem definition 3.1 System architecture

This paper investigates DRM as an interaction among an electricity retailer and various prosumers, i.e., clients who can both produce and consume electricity, in a smart grid. In order to ensure the tractability of the problem over a large popula- tion, prosumers are classified into prosumer groups (PGs), where each PG consists of prosumers with similar electricity consumption and production profiles as well as storage capabilities. The system architecture is displayed in Fig.1.

PGs are characterized by their uncontrollable production and consumption, controllable load requirements as well as their storage capabilities. The uncontrollable production C_i;t^þ and consumptionC_i;t of PG_i is fixed and given in the input for each time periodt¼1;2; :::;T. In addition, PG_i needs to schedule a (potentially zero) controllable load of Mi (Mi is the total controllable load during the horizon) over the time horizon, where a maximum of Li;t can be scheduled in each periodt. It is noted that time windows

can be defined for the controllable load by settingLi;t¼0 for the appropriate periodt. The preferences of PGion the timing of the controllable load are encoded in utility values Ui;t, whereUi;t captures the utility of scheduling a unit of controllable load in t. Hence, if PG_i decides for a con- trollable load of Li;t over time, this incurs a utility of P^T

t¼1

U_i;tL_i;tfor the PG. Similar models for controllable load are used frequently in the literature [8].

PGs can further optimize their energy management by the appropriate charging and discharging of their battery storage. The battery is characterized by its capacityBi, the maximum charge and discharge ratesR^þ_i andR_i , the initial battery state-of-charge (SoC) bi;0, and its cycle efficiency g_i. In order to safeguard from unexpected power outages, the prosumer wishes to retain a given, time-varying mini- mum SoCB_i;t in the battery.

Each individual PG schedules its controllable loadLi;tand determines its battery SoCbi;tover time to optimize its own objective, composed of maximizing the utility and mini- mizing the electricity cost with regard to the energy tariff set by the retailer. This PG model is generic enough to capture the behavior of diverse types of prosumers, ranging from households or offices with uncontrollable consumption only (and therefore, unresponsive to the energy tariff), via pro- sumers equipped with renewable energy generation and/or storage devices, owners of electric vehicles, to complex microgrid systems. It should be noted that various alternative approaches for modeling prosumer behavior have been subjected to extensive research recently. Questions of spe- cial interest include addressing individual prosumers or organizing them into PGs as well as using deterministic or probabilistic models. A richer, probabilistic approach to characterize the responsiveness of prosumers to the variation of the electricity tariff is presented in [34], together with a review of the recent literature on the benefits and drawbacks of different approaches.

...

Fig. 1 System architecture with a retailer and multiple PGs

The retailer employs the same time-of-use electricity tariff for all prosumers. The tariff is specified in the form of day-ahead electricity purchase pricesQ^þ_t and feed-in prices Q_t offered to PGs for periodst¼1;2; :::;T. It is assumed that the tariff is regulated by an a priori agreement between the retailer and the prosumers, which defines minimum, maximum, and maximum average electricity prices in the form of 0\QQ_t Q^þ_t Qand_T¹P^T

t¼1

Q^þ_t Q, where~ Q~is max. average electricity price for prosumers; Q is maxi- mum electricity price for prosumers;Qis minimum elec- tricity price for prosumers. Such an agreement is necessary to prevent the profit maximizing retailer from increasing purchase prices without any limit [7].

The focus of this paper is on the problem faced by the retailer, who has to cover the (potentially negative) net consumption of the ensemble of all prosumers from the electricity purchased or sold on the wholesale market. This paper assumes a time-variant dual pricing scheme on the wholesale market, given in the form of purchase pricesP^þ_t and feed-in pricesP_t . It is noted that the same model can be naturally applied to markets with uniform pricing (purchase prices equal to selling prices) by letting P^þ_t ¼P_t . It is assumed that the retailer appears as a price-taker on the market, without any ability to influence the market prices.

By offering an appropriate electricity tariff to its prison- ers, the retailer can initiate a demand response program that motivates the prosumers to purchase electricity in valley periods. When an ample amount of cheap energy is available on the market, the retailer can sell their surplus energy in peak periods. In this way, the retailer can contribute to the grid stability and maximize its profit at the same time. In this paper, the maximization of the retailer’s profit is addressed.

3.2 Stackelberg game model and its basic characteristics

The following communication protocol is implemented among various stakeholders: the retailer firstly announces the day-ahead electricity tariff to all prosumers. The pro- sumers observe this tariff and optimize their consumption profile, i.e., the amount of electricity purchased from or fed into the grid over time. Then, the parties implement their actions as planned. It is assumed that the retailer is aware of the decision model and the parameters of PGs. This leads to a ‘‘Stackelberg game’’ with the retailer as the ‘‘leader’’ and PGs as ‘‘multiple followers’’. The so-called ‘‘optimistic’’

assumption is adopted, i.e., if a follower has more than one optimal solutions according to its own objective, then it chooses its optimal solution that is the most favorable for the leader. The following additional assumptions are made, which guarantee the feasibility of the problem:

X^T

t¼1

Li;tM_i 8i ð1Þ

B_i;tBi 8i;t ð2Þ

bi;0Bi 8i ð3Þ

bi;0þtR^þ_i B_i;t 8i;t ð4Þ whereTis the number of time periods.

These assumptions require that bounds on the control- lable load allow schedule the required power over the horizon as for (1), the bounds on the battery SoC are consistent as for (2), the initial battery charge satisfies these bounds as for (3), and finally, that the charging rate of the battery allows satisfying the lower bounds on the SoC as for (4).

Lemma 1 (Existence of a solution) If assumptions (1)–

(4) hold, then the followers’ problem is feasible for any electricity tariff Q^þ_t andQ_t set by the leader.

Proof Setting the battery SoC to the required minimum, i.e.,bi;t¼maxb_i;t1;B_i;t

, and scheduling the controllable as early as the bounds allow, i.e., Li;t¼ min Li;t;Mi^t1P

u¼1

Li;u

result in a feasible solution for each follower i.

Lemma 2 (Independence of followers’ problems) The optimal demand response of an individual PG to a given energy tariff is independent of the response of other PGs.

Proof The objectives of the individual PG, i.e., its energy cost and utility, depend solely on the energy tariff and the consumption profile of the given PG. Moreover, the fea- sibility of a consumption profile is also independent of other PGs’ response, since the amount of electricity that can be purchased or sold on the market by the retailer to maintain the grid-level balance is unbounded.

It is emphasized that different PGs’ problems are still interconnected trough the retailer’s problem, but for any fixed decision of the retailer on the tariff, the PGs can optimize their behavior without considering the problems faced by fellow PGs. Hence, the problem can be modeled as a Stackelberg game with a single leader (the retailer) and multiple independent followers (the PGs). It is noted that when the optimal response of a follower is not unique, the response induced by the optimistic assumption (from the set of all optimal responses) can be dependent on other PGs’ response.

Lemma 3 (Computational complexity) The above defined bilevel energy tariff optimization problem is NP-hard.

Proof The simple multi-period energy tariff optimization problem (SMETOP) has been introduced as a minimal bilevel optimization model of energy tariff optimization for DRM, and it has been proven to be NP-hard in [32]. The problem investigated in this paper generalizes SMETOP in the sense that, in addition to all features captured by SMETOP, it also handles batteries and uncontrollable energy production and consumption at the PGs as well as bidirectional grid connections. This implies that the cur- rently investigated, generalized problem is NP-hard, too.

4 Solution approach 4.1 Overview

This section presents a bilevel programming formulation of the above Stackelberg game model, and proposes an efficient solution approach for that formulation. First, the models of an individual follower and the leader are for- mally defined. Then, the bilevel programming model received as a combination of the two parties’ problems is reformulated into a single-level QCQP, which is, in turn, solved using an SLP algorithm.

4.2 Prosumer groups’ (followers’) problem

The decision problem faced by an individual PG_i (the follower) is a parametric optimization problem, whose parameters encode the electricity tariff determined by the retailer (decision variables Q^þ_t and Q_t controlled by the leader). The problem can be captured by the following LP, where the symbolu^k_i;t on the right-hand side of the con- straints denotes the dual variables associated with the given constraint:

mingiðQ^þ;QÞ ¼X^T

t¼1

Q^þ_t x^þ_i;tQ_t x_i;tUi;tLi;t

ð5Þ

s.t.

C^þ_i;tC_i;tþx^þ_i;tx_i;tLi;t¼r_i;t^þr_i;t 8t;½u⁶_i;t ð6Þ g_ir_i;t^þr_i;t¼bi;tbi;t1 8t;½u⁷_i;t ð7Þ X^T

t¼1

Li;t¼Mi ½u⁸_i ð8Þ

Li;tLi;t 8t;½u⁹_i;t ð9Þ B_i;tbi;t 8t;½u¹⁰_i;t ð10Þ bi;tBi 8t;½u¹¹_i;t ð11Þ

r_i;t^þR^þ_i 8t;½u¹²_i;t ð12Þ r_i;tR_i 8t;½u¹³_i;t ð13Þ 0x^þ_i;t;x_i;t;r_i;t^þ;r_i;t;Li;t 8t ð14Þ wherex^þ_i;t is the electricity purchased;x_i;t is the electricity fed into the grid;r^þ_i;tis the electricity charged into battery;

r_i;tis the electricity discharged from battery;u^k_i;tis the dual variables.

The follower’s objective (5) is comprised of the total cost of energy, i.e., the cost of energy purchased minus the income from feeding energy into the grid, and the PG’s utility achieved by the timing of the controllable load.

Constraint (6) encodes that the energy balance at the PG is maintained. Equation (7) computes the battery state-of- charge based on the charge and discharge rates. Constraints (8) and (9) ensure that the amount and the timing of the controllable load satisfies the requirements. Finally, inequalities (10)–(14) define the allowed range of the bat- tery SoC, the charge and discharge rates as well as the electricity purchase and feed-in rates at the PG.

It is noted that all constraints in the followers’ model are linear, whereas the objective contains the leader’s variables as multipliers, making it a bilinear (quadratic) expression.

The models of different followers are linked only via the problem of the leader’s decision.

4.3 Retailer’s (leader’s) problem

The optimization problem faced by the retailer can be formulated as a bilevel program that contains the PGs’

problem as a nested sub-problem. This nested sub-problem, encoded as a constraint in the model, expresses that a part of the variables (decision variables x^þ_i;t and x_i;t, corre- sponding to the amount of electricity purchased from and fed into the grid) are controlled by the followers, according to their known decision model:

maxf ¼X^T

t¼1

X^N

i¼1

Q^þ_tx^þ_i;tQ_t x_i;t

P^þ_t y^þ_t þ P_t y_t

!

ð15Þ s.t.

y^þ_t y_t ¼X^N

i¼1

ðx^þ_i;tx_i;tÞ 8t ð16Þ

QQ_t Q^þ_t Q 8t ð17Þ

1 T

X^T

t¼1

Q^þ_t Q~ ð18Þ

x^þ_i;t x_i;t

!

2arg minf giðQ^þ;QÞ j ð6Þ ð14Þg 8i ð19Þ whereNis the number of PGs.

The leader’s objective (15) is to maximize its profit, calculated as its revenue from the prosumers, minus the cost of electricity purchased on the market, and plus the income from the electricity sold on the market. Equa- tion (16) encodes the grid-level energy balance. Inequali- ties (17) and (18) define the valid range of the energy tariff variables. Finally, constraint (19) states that the electricity purchase and feed-in values of prosumers are determined using the above optimization model.

4.4 Single-level QCQP reformulation

When the complexity of a bilevel optimization problem does not allow developing an analytical solution, which is apparently the case above, the two candidate solution approaches are the application of (meta-)heuritics directly to the bilevel problem, or the reformulation to a single- level problem. The considerable benefit of the latter tech- nique is that it allows the application of theoretically well- founded, potentially even exact mathematical program- ming approaches to solve the problem. For this reason, this paper adopts the reformulation approach and looks for a transformation of the bilevel problem (15)–(19) into a single-level mathematical program. The key to achieve this is the modeling of the optimality condition of the followers (19). By exploiting duality for the followers’ LP model it is noted that the followers’ LP model (5)–(14) contains the bilinear term Q^þ_t x^þ_i;tQ_t x_i;t in the expression of giðQ^þ;QÞ, i.e., a multiplication of the leader’s and the followers’ variables. With this, the model is still linear in the followers’ variables, and LP duality can be exploited for reformulating it. By exploiting duality for the follow- ers’ LP model, primal-dual reformulation of the followers’

problem can be applied: the optimality condition (19) is translated into the conjunction of followers’ primal con- straints (6)–(14), dual constraints, and an equality con- straint between the primal and the dual objectives. By duality, the ensemble of these constraints is satisfied if and only if the given instantiation of the variables is an optimal solution for the follower.

The complete single-level reformulated problem is shown below. It consists of the leader’s objective (15), the leader’s constraints (16)–(18), the followers’ primal con- straints (6)–(14), an equality relation between the follow- ers’ primal and dual objectives (20) as well as the followers’ dual constraint corresponding to the primal variables for the battery charge rater_i;t^þ(21), discharge rate r_i;t (22), SoC bi;t for t\T (23) and bi;T (24), electricity

purchase x^þ_i;t (25), electricity feed-in x_i;t (26), and control- lable loadLi;t (27).

maxf¼X^T

t¼1

X^N

i¼1

Q^þ_t x^þ_i;tQ_t x_i;t

P^þ_ty^þ_t þP_t y_t

!

s.t.

ð6Þ ð14Þ;ð16Þ ð18Þ X^T

t¼1

Q_t x_i;tQ^þ_t x^þ_i;tþUi;tLi;t

¼X^T

t¼1

C_i;tC_i;t^þ

u⁶_i;tþLi;tu⁹_i;tþR^þ_i u¹²_i;t

þR_i u¹²_i;tB_i;tu¹⁰_i;tþBiu¹¹_i;t 8i

ð20Þ u⁶_i;tþg_iu⁷_i;tþu¹²_i;t0 8i;t ð21Þ u⁶_i;tu⁷_i;tþu¹³_i;t0 8i;t ð22Þ

u⁷_i;tþu⁷_i;tþ1u¹⁰_i;t þu¹¹_i;t0 8i;t\T ð23Þ

u⁷_i;Tu¹⁰_i;Tþu¹¹_i;T0 8i ð24Þ u⁶_i;t Q^þ_t 8i;t ð25Þ u⁶_i;tQ_t 8i;t ð26Þ u⁶_i;tþu⁸_i þu⁹_i;tUi;t 8i;t ð27Þ u⁹_i;t; u¹⁰_i;t; u¹¹_i;t; u¹²_i;t; u¹³_i;t0 8i;t ð28Þ The primal-dual reformulation particularly suits the prob- lem in the scope, since the only occurrence of the leader’s variables,Q^þ_t andQ_t , in the followers’ problem is in their primal objective, and consequently, on the right hand side of the dual constraints. As a result, the only non-linear term in the single-level reformulation is the payment from the PGs to the retailer, contained both in the leader’s objective (15) and in the followers’ optimality constraint (20), which is a bilinear expression containing the multiplication of the followers’ and the leader’s variables. All other constraints are linear.

4.5 SLP solution method

Since the above QCQP is non-convex, no efficient exact algorithm can be expected for solving it, and accordingly, (meta-)heuristic approaches are of interest. Therefore, we propose an SLP heuristic solution approach, which shows good convergence properties especially on problems where most of the constraints are linear. SLP solves non-linear problems by iteratively constructing local LP approxima- tions of the original problem, and solving each approxi- mation using standard LP techniques [35, 36]. The

algorithm departs from an initial solutionX0, and in each iteration k, it builds a local linearization of the original problem around Xk, denoted as LPk. Then, the optimal solution of LP_k is sought subject to a given step bound, sXXks. If the optimal LP solution is feasible with a given tolerance, then it is accepted as the next solution X_kþ1 (possibly s is increased). Otherwise X_kþ1¼X_k and sis decreased.

The above SLP algorithm converges to a locally optimal solution of QCQP, which is potentially different from the global optimum. In order to reduce the risk of getting stuck in a local optimum, the SLP algorithm is embedded into a randomized restart procedure. It executes multiple SLP runs, using a random perturbation of the previous best solution as an initial solution in each run (orQ_t ¼Q^þ_t ¼Q in the first run). The implementation reported in this paper is based on the SLP package of Fico Xpress 7.8 by using its default SLP algorithm, with the number of SLP runs set to 10 in all computational experiments.

4.6 Discussion on possible extensions

While the above presented bilevel model captures the most important generic features of prosumers (production and consumption, controllable load, battery storage), it can be extended and refined in many different ways. The most relevant directions include the extension of the prosumer model with features for specific types of equipment that induce elastic load (e.g., high voltage alternating current (HVAC) in buildings, or refined battery storage models capturing state-dependent charging properties and los- ses [17]) as well as the extension of the retailer model with the generation or energy storage. The proposed solution method is directly applicable to the extended models as long as the prosumer model remains linear. The proposed reformulation still applies with binary variables in the retailer model (e.g., due to switchable generators). And commercial solvers offer algorithms to tackle the resulting mixed-integer QCQP though the computational efficiency of the approach which needs to be verified for the given application.

Below, we review two minor refinements of the baseline bilevel model (15)–(19) fixing specific issues that might be undesirable in some application scenarios. Firstly, the baseline model may trigger inappropriate end-of-horizon effects, namely, the followers sell all the energy stored in the batteries to maximize their revenue. This can be avoi- ded by subtracting a term that valuates the energy stored in the batteries at the end of the planning horizon from the followers objective (5) as follows:

Q^þ_T þQ_T

2 bi;T ð29Þ

Another example of a possible requirement that is not captured readily by the above model is that, among different optimal solutions that maximize the retailer’s profit, a solution with a smooth electricity purchase and/or sale over time is preferred. Unwanted oscillation of the energy purchased or sold on the wholesale market can be smoothed out by adding the following term to the retailer’s objective:

eX^T

t¼1

ðy^þ_t y_t Þ² ð30Þ

This quadratic term measures the squared deviation of the energy traded over time with a constant bias. Accordingly, adding it to the retailer’s objective with a small multipliere smooths unnecessary oscillations without affecting the payoffs of the players.

4.7 Discussion on KKT reformulation

As an alternative to the proposed solution, KKT refor- mulation and linearization can be applied to convert the proposed bilevel model (15)–(19) into a single-level MILP.

This approach is often considered to be the default choice for transforming bilevel problems into single-level ones.

Moreover, the resulting MILP, in theory, can be solved to exact optimality by commercial solvers.

Converting the bilevel model into a single-level MILP requires linearizing the KKT complementary slackness conditions using big-M constraints over additional binary variables as well as linearizing the quadratic term in the objective by expressing and substituting it from (20).

However, as it will be shown in the computational exper- iments, this approach is computationally challenging due to the high number of binary variables and big-M constraints.

In particular, linearizing the complementary slackness conditions requires introducing ca. 22 N T auxiliary binary variables into the model (one for each primal and dual variable, resulting in over 20000 additional binary variables for N¼20 and T¼48). Moreover, the corre- sponding big-M constraints are typically difficult to solve due to their weak LP relaxations. For further details on the KKT reformulation, the interested reader is referred to [26,28]. Finally, even minor modifications in the bilevel model can hinder the linearization of the KKT reformula- tion, as is the case with terms for the valuation of the remaining charge (29) or for smoothing (30).

In this paper, we use the KKT reformulation and the exact MILP solution approach in computational

experiments to assess the quality of the solutions found by the proposed SLP solution on small-size problems.

5 Experimental evaluation 5.1 Illustrative example

In this section, the proposed approach to DRM is demonstrated on a small-scale illustrative example, with three PGs and a one-day horizon (from 8:00, using hourly time units). The PGs correspond to different types of consumers as follows:

1) PG1 represents an intelligent energy-positive street lighting microgrid system called E?grid [37, 38].

Since the lighting system is controlled according to local traffic and environmental conditions, as captured by motion sensors and a local weather station, its consumption varies dynamically over time. The grid- connected system is also equipped with photovoltaic (PV) power generation and battery storage, which enables it to perform active energy management using an optimization approach that corresponds to the PG model adopted in this paper. Real-life data originates from a physical prototype with 191 luminaries and 151.2 m² of active PV surface area, and reflects the operation of the system on a sunny day in October.

The E?grid PG is a net producer (up to 15 kW) during the day, and a net consumer (up to 3.5 kW) during the night.

2) PG2 comprises owners of plug-in electric vehicles.

Data used in the example corresponds to three Nissan Leaf electric vehicles, with a 24 kWh battery pack in each vehicle, which has to be charged from a 50%

state to 100% state during the night. Individual vehicles are connected to the grid between 17:00 and 20:00 and disconnected between 6:00 and 8:00 in the morning. With the vehicle-to-grid (V2G) option ignored, this can be modeled as a controllable load of 36 kWh. It is assumed that the owners have a slight preference for charging the electric vehicles as early as possible, which is captured by utility values U_2;t linearly decreasing over time.

3) Finally, PG3 contains households with uncontrollable consumption only. This case study uses the data of 15 average Hungarian homes, with a peak consumption of 5.7 kW during the day, and a minimum consumption of 3.8 kW during the night. Since this PG has no controllable load or battery storage, it cannot partic- ipate actively in DRM, and its consumption appears only as a time-varying bias in the grid-level consumption.

The retailer aims to maximize its profit by offering an appropriate time-of-use electricity tariff to the PGs, respecting a priori contract that sets Q¼1 c/kWh, Q¼ 100 c/kWh, andQ~¼10 c/kWh. For the sake of simplicity, the wholesale market prices are assumed to vary in two steps: 12 c/kWh during the day (between 8:00 and 21:00) and 6 c/kWh during the night. The feed-in price on the wholesale market is a constant 3 c/kWh.

The system-level optimum for this example is deter- mined by the following characteristics: the overall grid is a net producer until 17:00 due to PV generation in the E?- grid microgrid, whereas it is a net consumer afterwards. In order to avoid losses stemming from dual pricing on the wholesale market, the retailer should motivate the PGs to anticipate load and charge batteries before 17:00. On the contrary, in the period after 17:00, it should encourage PGs to defer their load from the peak period lasting until 21:00 to the valley period afterwards.

This sample instance was solved using the proposed approach, applying formula (30) to eliminate the oscilla- tions of the energy flow that are visually disturbing.

Otherwise, it does not affect the payoffs of the players. The results displayed in Figs. 2, 3, 4, and 5 show that the proposed approach could indeed reach the above described system-level optimum. The diagrams compare the opti- mized consumption profile (red curve) to the baseline consumption (green curve) for the overall grid and for the individual PGs, where the baseline consumption is com- puted by scheduling the controllable loads to maximize utility (ignoring the electricity tariff) without using the batteries. The characteristic time periods are separated by dashed lines at 17:00 and 21:00. Finally, the optimized purchase tariff is also shown in the diagram of the overall grid: constant low prices (1 cent/kWh) are applied while the system is a net producer until 17:00, whereas high,

8 11 14 17 20

Time (hour)

23 2 5 7

Consumption (kW)

Baseline consumption; Optimized consumption; Purchase price -15

-10 -5 0 5 10 15 20 25

0 2 4 6 8 10 12 14 16 18

Purchase price (cent/kWh)

Fig. 2 Solution with optimized tariff: energy purchase price and overall consumption over time

slightly decreasing prices are used afterwards (15.74 cent/

kWh at 18:00, decreasing by 0.05 cent/kWh per hour).

On the level of individual PGs, the applied tariff moti- vated the E?grid PG to charge its battery while it is a net producer, to reach a fully charged state during 14:00–17:00, and to gradually discharge the battery in the rest of the peak period between 17:00 and 21:00. The controllable load of the EV PG was fully deferred to the valley period after 21:00. In that period, the slight decrease

of the purchase prices over time compensated the PG for its linearly decreasing utility function, Therefore, an arbitrary scheduling of the controllable load became optimal for this PG. There were no controllable variables for the household PG. This tariff and consumption profile are globally opti- mal for the retailer, since no further load can be moved outside the peak period between 17:00 and 21:00.

With 10 SLP runs within the randomized restart heuristic, the implementation of the proposed solution approach in Fico Xpress 7.8 could solve the above problem instance in 9.8 seconds on a computer with Intel i7 2.70 GHz CPU and 16 GB RAM.

5.2 Computational experiments

The evaluation of the proposed approach in computa- tional experiments focuses on two questions: the com- putational effort required by the proposed SLP solution;` the quality of the solutions found. Large problem instances are generated by the multiplication and random perturba- tion of the data used in the above illustrative example.

Table2 displays the average computation time in seconds over 10 instances for different combinations ofN(number of PGs) and T (number of time periods), which are achieved with the proposed model (15)–(19) and the algorithm using 10 SLP runs. The results show that the computation time increases moderately with problem size, and practically relevant problem sizes, e.g., with N¼20 andT ¼48, are tractable in a reasonable amount of time.

In applications where a different tradeoff between solution quality and computation time is looked for, the algorithm can be tuned by modifying the number of SLP runs.

In order to evaluate the quality of the solutions found by the proposed approach, they are compared to the exact optimal solutions of the MILP model received by applying KKT reformulation and linearization as discussed in Sec- tion 4.7. The results of the comparison are displayed in Table3, which displays aggregated results over 10 instances for each problem size. The column of Opt. (short for optimality) contains the ratio of instances that could be solved to prove the optimality using KKT, and the column of Time shows the average computation time required for

-20 0 20 40 60 80 100 120

-20 -16 -12 -8 -4 0 4 8

8 11 14 17 20

Time (hour)

23 2 5 7

Battery SoC (%)

Consumption (kW)

Baseline consumption; Optimized consumption; Battery SoC Fig. 3 Solution with optimized tariff: consumption and battery state of the E?grid lighting system PG over time

8 11 14 17 20

Time (hour)

23 2 5 7

Consumption (kW)

Baseline consumption; Optimized consumption Cumulated charge

0 2 4 6 8 10 12

0 5 10 15 20 25 30 35 40

Cumulated charge (kWh)

Fig. 4 Solution with optimized tariff: consumption and cumulated load of the electronic vehicle PG over time

8 11 14 17 20

Time (hour)

23 2 5 7

Consumption (kW) 3 4 5 6

Fig. 5 Solution with optimized tariff: consumption of the household PG over time

Table 2 Average computation time in seconds by problem size for proposed SLP solution

N T¼12 T¼24 T¼36 T¼48

5 0.55 1.46 2.22 16.04

10 1.10 73.38 57.97 228.37

15 1.67 155.74 51.59 283.56

20 3.63 257.31 98.82 345.06

this. The branch and bound search are aborted when the time limit of 600 s is hit, and the best integer solution found is recorded. The columns of Min., Avg., and Max display the minimum, the average, and the maximum gap between the SLP and the KKT solutions for the given problem size.

Finally, the column of Time contains the average compu- tation time for the SLP solution. The results show that the smallest instances, withN¼3, could be solved to prove the optimality using KKT with a single exception.

Although SLP is not an exact solution approach, in practice it also builds close-to-optimal solutions with an average gap of only 0.01%–0.1%. For larger instances N¼5, where KKT failes to find the optimal solution, SLP often constructs significantly better solutions, as indicated by negative gap values. Namely, SLP finds up to 40% better solutions than KKT with one or two orders of magnitude lower computation times.

Hence, it can be concluded that although KKT is an exact solution in theory, its applicability is limited to small problems, e.g., withN¼3. In contrast, the proposed pri- mal-dual reformulation couples with SLP scales much more favorably, and it computes high-quality solutions efficiently even for practically relevant problem sizes.

6 Conclusion

This paper introduces a bilevel programming approach to energy tariff optimization for DRM in smart grids. In the Stackelberg game model, the leader is a profit maximizing retailer, who sets the energy tariff offered to its prosumers and purchases electricity for them from the wholesale market. The prosumers, who act as multiple independent followers, opti- mize their controllable load and their battery charging schedule to maximize their utility and minimize their cost of

energy. A new solution is introduced, which exploits the pri- mal-dual reformulation of the followers’ problem to arrive at a single-level QCQP equivalent of the bilevel problem. It has been shown that the resulting QCQP can be solved efficiently using an SLP algorithm. In particular, it is illustrated in computational experiments that the proposed approach out- performs the technique based on the KKT reformulation, which is the dominant approach for solving similar problems in the literature. Hence, the main contributions of the paper are a bilevel programming formulation of the tariff optimization problem, formal proofs of some basic properties, and the application of new and efficient mathematical programming techniques to solve this problem.

The proposed model can be trivially extended to some more complex problems, e.g., with various types of con- trollable loads and storage devices for each PG, or switchable generators and energy storage at the retailer. A more important and challenging direction for future research is the investigation of richer, non-linear prosumer models that can capture more realistically, e.g., thermal processes of HVAC in buildings or charging properties of batteries. The extension to a stochastic variant, accounting for uncertainties in consumption, production, and spot market prices is also of interest.

Finally, it must be observed that while Stackelberg game models are becoming ubiquitous in the literature of DRM, a critical pre-condition of their practical applicability is that the leader should be able to identify the decision models and parameters. This is a challenging problem in application scenarios characterized by information asym- metry. A promising solution can be the application of inverse optimization, analogously to a case in inventory control [39]. With historical pairs of a follower’s input (i.e., energy tariff) and response (consumption), the inverse optimization approach looks for parameters which ensure that each response is optimal for the corresponding input.

Acknowledgements This research was supported by the GINOP grant (No. 2.3.2-15-2016-00002), the NKFIA grant 129178, and the Ja´nos Bolyai Research Fellowship.

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://

creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

References

[1] Esther BP, Kumar KS (2016) A survey on residential demand side management architecture, approaches, optimization models and methods. Renew Sustain Energy Rev 59:342–351 Table 3 Comparison of exact KKT and proposed SLP solution

Size KKT SLP

N T Opt.

(%) Time (s)

Gap (%) Time

Min. Avg. Max. (s)

3 12 100 0.51 0.00 0.00 0.00 0.43

3 24 100 4.43 0.00 0.09 0.98 1.11

3 36 100 55.00 0.00 0.01 0.06 1.29

3 48 90 240.64 -19.16 -1.93 0.05 2.24

5 12 60 304.01 -0.04 0.00 0.02 0.55

5 24 0 600.00 -6.99 -2.11 0.83 1.46

5 36 0 600.00 -40.92 -8.30 0.48 2.22

5 48 0 600.00 -28.13 -5.93 1.51 16.04

Note: negative gaps arise where SLP outperforms KKT within the time limit of 600 s

[2] Maharjan S, Zhu Q, Zhang Y et al (2016) Demand response management in the smart grid in a large population regime.

IEEE Trans Smart Grid 7(1):189–199

[3] Tushar W, Zhang JA, Smith DB et al (2014) Prioritizing con- sumers in smart grid: a game theoretic approach. IEEE Trans Smart Grid 5(3):1429–1438

[4] Tushar W, Saad W, Poor HV et al (2012) Economics of electric vehicle charging: a game theoretic approach. IEEE Trans Smart Grid 3(4):1767–1778

[5] Yu M, Hong SH (2017) Incentive-based demand response considering hierarchical electricity market: a Stackelberg game approach. Appl Energy 203:267–279

[6] Chen J, Yang B, Guan X (2012) Optimal demand response scheduling with Stackelberg game approach under load uncer- tainty for smart grid. In: Proceedings of IEEE third international conference on smart grid communications, Taiwan, China, 5–8 November 2012, pp 546–551

[7] Zugno M, Morales JM, Pinson P et al (2013) A bilevel model for electricity retailers’ participation in a demand response market environment. Energy Econ 36:182–197

[8] Wei W, Liu F, Mei S (2015) Energy pricing and dispatch for smart grid retailers under demand response and market price uncertainty. IEEE Trans Smart Grid 6(3):1364–1374

[9] Yu M, Hong SH (2016) Supply-demand balancing for power management in smart grid: a Stackelberg game approach. Appl Energy 164:702–710

[10] Soliman HM, Leon-Garcia A (2014) Game-theoretic demand- side management with storage devices for the future smart grid.

IEEE Trans Smart Grid 5(3):1475–1485

[11] Jalali M, Zare K, Seyedi H (2017) Strategic decision-making of distribution network operator with multi-microgrids considering demand response program. Energy 141:1059–1071

[12] Asimakopoulou GE, Vlachos AG, Hatziargyriou ND (2015) Hierarchical decision making for aggregated energy manage- ment of distributed resources. IEEE Trans Power Syst 30(6):3255–3264

[13] Nguyen DT, Nguyen HT, Le LB (2016) Dynamic pricing design for demand response integration in power distribution networks.

IEEE Trans Power Syst 31(5):3457–3472

[14] Gupta A, Saini RP, Sharma MP (2011) Modelling of hybrid energy system-part I: problem formulation and model devel- opment. Renew Energy 36(2):459–465

[15] Coffrin C, Van Hentenryck P (2014) A linear-programming approximation of AC power flows. INFORMS J Comput 26(4):718–734

[16] Scott P, Thie´baux S, van den Briel M et al (2013) Residential demand response under uncertainty. In: Proceedings of inter- national conference on principles and practice of constraint programming. LNCS, vol 8124. Springer, Berlin, pp 645–660 [17] Sharma I, Dong J, Malikopoulos AA et al (2016) A modeling

framework for optimal energy management in a residential building. J Energy Build 130:55–63

[18] Corradi O, Ochsenfeld H, Madsen H et al (2013) Controlling electricity consumption by forecasting its response to varying prices. IEEE Trans Power Syst 28(1):421–429

[19] Razmara M, Bharati GR, Shahbakhti M et al (2018) Bilevel optimization framework for smart building-to-grid systems.

IEEE Trans Smart Grid 9(2):582–593

[20] Lu R, Hong SH, Zhang X (2018) A dynamic pricing demand response algorithm for smart grid: reinforcement learning approach. Appl Energy 220:220–230

[21] Golmohamadi H, Keypour R (2018) Stochastic optimization for retailers with distributed wind generation considering demand response. J Mod Power Syst Clean Energy 6(4):733–748

[22] Andoni M, Robu V, Fru¨h W-G et al (2017) Game-theoretic modeling of curtailment rules and network investments with distributed generation. Appl Energy 201:174–187

[23] Asimakopoulou GE, Dimeas AL, Hatziargyriou ND (2013) Leader-follower strategies for energy management of multi- microgrids. IEEE Trans Smart Grid 4(4):1909–1916

[24] Zhang X, Bao T, Yu T et al (2017) Deep transfer Q-learning with virtual leader-follower for supply-demand Stackelberg game of smart grid. Energy 133:348–365

[25] Ni L, Liu W, Wen F et al (2018) Optimal operation of elec- tricity, natural gas and heat systems considering integrated demand responses and diversified storage devices. J Mod Power Syst Clean Energy 6(3):423–437

[26] Colson B, Marcotte P, Savard G (2005) Bilevel programming: a survey. 4OR: Quart J Oper Res 3:87–107

[27] Dempe S (2002) Foundations of bilevel programming. Kluwer, Dordrecht

[28] Dempe S, Zemkoho AB (2013) The bilevel programming problem: reformulations, constraint qualifications and optimal- ity conditions. Math Program 138(1):447–473

[29] Labbe´ M (2007) Bilevel programming and price setting prob- lems. Operations Research Proceedings 2006. Springer, Berlin, Heidelberg

[30] O¨ zdemir O¨, Munoz FD, Ho JL et al (2016) Economic analysis of transmission expansion planning with price-responsive demand and quadratic losses by successive LP. IEEE Trans Power Syst 31(2):1096–1107

[31] Castillo A, Lipka P, Watson JP et al (2016) A successive linear programming approach to solving the IV-ACOPF. IEEE Trans Power Syst 31(4):2752–2763

[32] Kova´cs A (2018) On the computational complexity of tariff optimization for demand response management. IEEE Trans Power Syst 33(3):3204–3206

[33] Kova´cs A (2016) Bilevel programming approach to optimizing a time-variant electricity tariff for demand response. In: Pro- ceedings of IEEE international conference on smart grid com- munications, Sydney, Australia, 6–9 November 2016, pp 674–679

[34] Valle´s M, Bello A, Reneses J et al (2018) Probabilistic char- acterization of electricity consumer responsiveness to economic incentives. Appl Energy 216:296–310

[35] Byrd RH, Gould NIM, Nocedal J et al (2003) An algorithm for nonlinear optimization using linear programming and equality constrained subproblems. Math Program Ser B 100(1):27–48 [36] Palacios-Gomez F, Lasdon L, Engquist M (1982) Nonlinear

optimization by successive linear programming. Manag Sci 28(10):1106–1120

[37] Csa´ji B, Kova´cs A, Va´ncza J (2014) Prediction and robust control of energy flow in renewable energy systems. In: Pro- ceedings of the 19th IFAC world congress, Cape Town, South Africa, 24–29 August 2014, pp 3663–3669

[38] Kova´cs A, Ba´tai R, Csa´ji B et al (2016) Intelligent control for energy-positive street lighting. Energy 114:40–51

[39] Egri P, Kis T, Kova´cs A et al (2014) An inverse economic lot- sizing approach to eliciting supplier cost parameters. Int J Prod Econ 149:80–88

Andra´s KOVA´ CSis a senior research fellow with the Institute for Computer Science and Control of the Hungarian Academy of Sciences. He received his Ph.D. degree in Technical Sciences from the Budapest University of Technology and Economics, Hungary, in 2006. His research interests include modeling and solving complex planning problems in diverse fields of engineering, using tools of operations research, artificial intelligence, and game theory.