A Greedy Agent-Based Resource Allocation in the Smart Electricity Markets

. The smart grid makes use of two-way streams of electricity and information to constitute an automated and distributed energy delivery network. Coming up with multi-agent systems for resource allocation, chieﬂy comprises the design of local capabilities of single agents, and therefore, the interaction and decision-making mechanisms that make them create the best or at least an acceptable power allocation. Due to the several issues in providing sustainable and affordable power energy, researchers try to think about creating a decentralized mechanism to be able to manage the entire transactions in retail electricity markets. As a result, this electricity infrastructure is predicted to develop into a market of markets, during which all the trading agents inﬂuence on each other and have role in toward an equilibrium one. In these markets, we are interested to minimize the buyers’ purchasing cost. Motivating this issue, we model the demand response problem in an evolutionary optimization framework and propose an evolutionary algorithm for handling the decentralized market-based resource allocation problem.


Introduction
Intelligent power networks, Smart Power Grids, are an enhancement of the present century and the result of experts' efforts to improve the power distribution mechanisms. Conventional power grids are mostly utilized to transmit the power from the central generators to an extensive variety of the end users. Conversely, the smart grid uses twoway streams of electricity and information to create an autonomous energy delivery network [1]. Toward this issue, multi-agent systems not only represent the consequent massive steps within the development of the next generation energy delivery network systems, but they also reveal new strategies to design highly flexible ones [2]. Coming up with multi-agent systems for distributed resource allocation, chiefly comprises the design of local capabilities of the trading agents and therefore the interaction and decision-making mechanisms that let them create the best or at least an admissible allocation. One of the challenges in this context is the Demand Response problem. Due to the several issues in providing sustainable and affordable power, creating a decentralized mechanism that be able to manage the entire transactions dynamically, is an interesting problem among researchers and practitioners [3]. Proposing a comprehensive approach considering the minimization of buyers purchasing cost, where the decision process of resource allocation is expressed as a demand response optimization under certain constraints, is one of the main contributions of this paper. In this respect, the "market-based resource allocation" mechanisms are motivated by economical features [4], in which the resources are allotted utilizing the competition between trading agents.
In order to model such mechanisms, one of the main approaches is to create decentralized markets among trading agents [5]. In this approach, instead of having a dealer, to control the whole system, the environment is divided into diverse non-overlapping regions, in which there is a dealer in each of them who is responsible for controlling all the inside transactions. Motivated this idea, a comprehensive mathematical model to formalize the decentralized demand response optimization problem in the smart power markets is proposed. Taking into account the fact that buyer agents are self-interested, they are concerned with minimizing their purchasing costs. In order to satisfy the buyers demands, agents named sellers, are responsible for producing their needed power. Therefore, finding an optimal permutation of buyers to purchase their required power in the electricity markets, necessitates using evolutionary computation techniques.
So far, however, there has been little discussion about using evolutionary computation techniques for the demand response optimization problem in the smart electricity markets. Finding an optimal permutation of buyers requires calculating all the permutations. Accordingly, this problem is similar to Quadratic Assignment Problem which is a NP-Hard one [6]. Evolutionary algorithms are typically used to provide good approximate solutions to problems that cannot be solved easily using other techniques. Many optimization problems fall into this category. It may be too computationally-intensive to find an exact solution but sometimes a near-optimal solution is sufficient. In these situations evolutionary techniques can be effective [7].
The rest of this paper is organized as follows. Section 2 contains a review of the related work. In Section 3, we presents the comprehensive description of the mathematical system model with the proposed solution approach. Decentralized greedy resource allocation algorithms along with the evolutionary optimization techniques, definitions and tips are members of the next section. Fifth section demonstrates experimental results. Finally, conclusion and future work are drawn in the last section.

Related Work
The initial idea of designing power markets is based on a central system which controls and politicizes all the transactions and exchanges. Many studies have indicated that the presence of only one system or a central manager is a waste of energy and money [5,8]. Furthermore, many studies have been conducted in various aspects of these markets, specially in using multi agent systems and information systems [5,9]. Lamparter et al. have designed an electricity market based on the intelligent agents' technologies. They have presented miscellaneous scenarios of software infrastructure based on the multi agent systems to create this type of markets. Moreover, there is a comprehensive literature review on electricity market designing and interdisciplinary processes required for achieving the decentralized markets [5].
Besides, some of the researches in the field of the energy optimization, including those reported in [10,11], have demonstrated optimal domestic energy management via linear programming, where power required by each device in a house is optimally met by the available resources. Similar works of power matching can be found in [10], in which the proposed market model is based on a centralized approach with only one manager. In this paper, HomChaudhuri et al. have used a centralized market which does not allow buyers or even sellers to have power negotiation with more than one individual. Furthermore, it is assumed that all the sellers' surplus powers are much more than the buyers' demand. Considering these parameters, the proposed market will not confront critical circumstances. In addition, behaving only in the benefit of buyers in line with the sellers' elementary price update mechanism, which depends only on the number of buyers which are desired to purchase power from them, are studied in [11]. The proposed technique optimizes the loss rate of power distribution. Considering their demands and surpluses, matching buyers to sellers is based on the linear programming. Nonetheless, if any of the buyers cannot satisfy its demand in one interval, then, rest of that will be transferred to the next power matching intervals. Finally, in their paper, the fitness is based only on the distance among the buyers and sellers.
This paper follows a case-study design, with in-depth analysis of demand response optimization problem in electricity markets. Proposing an evolutionary greedy power matching algorithm for this concept concerning buyers' profit, is the main contribution of this paper. Besides, proposing a greedy iterative price update rule for seller agents for updating their offered prices for each unit of price in the consecutive time intervals is another contribution.

System Model
Visualizing a city with the self-healing power grid necessitates considering the multiple issues that are significant to include. Some belong to the physical infrastructure, for example the generators, transfers and transmission lines. Hereinafter, we focus on the issues in the physical infrastructure which also involve decentralized market-based resource allocation. Consider a city which is composed of a unique main power source, namely macro-station, which is connected to the main power grid of the city, along with a network of buyers and sellers. It is assumed that the macro-station is too far from the main power grid of the city and also purchasing power from this station both costs a lot and incurs high power loss during the power transfer. Furthermore, considering the macro-station is always on-line and has infinite surplus power, will assure us of having a consistent and reliable power source in critical and unpredictable circumstances. Trying to reach the purpose of having a decentralized controlling mechanism of the power exchanges among the distributed buyers and sellers, has motivated us to partition our city into regions. Each of them not only contains a non-empty subset of buyers and sellers, but a power-dealer is also allocated in order to control the power exchanges among the agents. In addition, R k is denoted as the k th region. Subsequently, the buyers and sellers sets in the k th region are represented by R B k and R S k respectively. Finally, it is assumed that the system model is studied for consecutive independent intervals, t = 1, 2, . . . , T .
Analyzing essential parameters to obtain an optimal solution in power matching of buyers and sellers, represents the substantial challenges in demand response problem [12]. Therefore, a mathematical solution approach is applied toward this issue. That the markets behave only in the benefit of buyers is the markets' main perspective. Minimizing the purchasing cost in power negotiations is the buyers' main objective. In this approach, sellers announce power prices in consecutive intervals and buyers choose sellers considering prices along with other parameters.

Market Mechanism
In all the power markets, each unit of energy has a price for selling. Also, each unit in the electricity markets is in the range of (p l , p u ). p l and p u are constant and belong to minimum and maximum price of each unit of power respectively. Dealers broadcast this range prior to the start of power exchange to the sellers. Considering our market mechanism, sellers announce their own initial prices in the t th interval between (p l , p u ).
Some key factors are included such as financial parameters, general attractiveness of the market and agents' risk factors [1], which influence on both purposes of picking out an appropriate initial price and updating consecutive intervals' prices. Drawing out a unique parameter for all the sellers which elucidates malleability rate from the market, named δ for sellers, is essential for their mentioned purposes. In order to update the power prices in subsequent intervals, they try to alter their prices using "Iterative Price Updating" rule. For instance, if some of the sellers announce too high prices, this rule will behave in such a manner as none of them is able to have power negotiations in the next intervals. Hence, they will have to decrease their proposed prices regarding the others' prices.
In time interval t, the x t bisj ∈ Z * is represented as the number of power units which the i th buyer has purchased from the j th seller. Z * is represented as the non-negative integer set. Considering the way which this transfer is happening, there is a main transmission line in each of the regions, which all the buyers and sellers are connected to. Each of the transmission lines (τι k ) has a maximum capacity of power transferring which is denoted by f low τ ι k ∈ Z * . Limitation in power transfer using the distribution lines mentions that sellers cannot transfer the power more than the transmission lines' capacity [13].
∀τι k such that Along with the confined transmission line in each region, any power exchange between a buyer and a seller is accompanied by the power loss over the distribution lines [1]. This exchange is expressed by loss bisj . Both of the transmission lines' specification (i.e. maximum flow capacity and distance between the buyer and seller) have direct impact on the value of the loss bisj . For convenience, it is assumed that only the euclidean distance between a buyer and seller has impact on the power loss.

Agents' Specification
In order to explicate diverse types of the trading agents, which it is assumed to have N buyers and M sellers, following descriptions are presented as follows.
Buyers. Agents who need to satisfy their demands via the sellers are known as the buyers in which the buyer b i has a predetermined demand d t bi in interval t. Thereupon: Equation (2) demonstrates the buyers set. Equation (3) shows the demand set of the i th buyer during all the intervals. Furthermore, d t bi ∈ Z * will be the required power unit of the i th buyer in the t th interval. Intuitively, as (4) indicates, due to power loss assumption over the distribution lines, each buyer b i should buy more than its demand in order to fulfill its requirement completely.

Sellers.
Agents who have Micro-Grids (e.g. solar panels or wind turbines) [14], and aim on producing power for selling, are known as sellers. Then: Equation (5) shows the sellers set. Equation (6) demonstrates the set of surplus powers for the j th seller during all the intervals. Furthermore, the q t sj ∈ Z * is the number of the power unit that the j th seller has for selling in the t th interval. Equation (7) mentions that the sellers cannot sell more than their surplus powers in the j th interval.

Iterative Price Updating Rule
Designing a greedy and dominant rule for updating the sellers' prices, has encouraged us to propose an iterative price updating rule. ps t j ∈ Z * demonstrates the j th seller's surplus power price in the t th interval. Consecutively, the δs j explains the malleability rate of the j th seller from the market as mentioned in the previous section. Sellers learn from not only their previous prices, but also the other sellers' amount of power negotiations along with offered prices. Therefore, ps t represents the weighted average of all sellers' offered prices in t th interval, as they is formulated in (8).
x t bisj t = 1, 2, 3, ..., T Subsequently, after computing the ps t , dealers dispense these parameters to their regions' seller agents in order to let them commence with updating their next intervals' price.
ps t+1 j = ps t j − δs j × ps t j −ps t j = 1, 2, 3, ..., M ; t = 1, 2, 3, ..., T (9) If any of the agents announce high or low price in an interval, (9) permits them to alter their prices to get close to the weighted average of the power's unit price in the next intervals.

Solution Approach
In this approach, the markets' purpose is to operate only in the benefit of buyers along with optimizing their objective function (i.e. minimizing the purchasing cost). In this case, sellers announce their prices based on their own malleability rate. The ps 1 j is denoted as j th seller's initial price, which is within the certain range declared by the dealers. Then, buyers start the purchasing procedure regarding the determined sellers' prices and parameters of the market. As a result, the objective formula with the constraints are as follows: subject to:  x t bisj , d t bi , q t sj , p l , p u , ps 1 j , ps t j , ps t , flow τ ι k ∈ Z * ; loss bisj , δs j ∈ (0, 1)

Greedy Power Matching Algorithm
Selfish trading agents have caused us to focus on finding a near optimal solution for the demand response problem in retail power markets in a greedy approach. Each of the buyers attempts to purchase all of its required power from a seller who is close to it and has announced a low price. In line with this point of view, an Interest Table related to all buyers is required prior to start power exchanging in each interval.
Interest Table. Taking into account all the constraints related to the markets and agents, an interest table is created by buyers to demonstrate their desired value in having negotiation with sellers. For instance, in this approach, each buyer's satisfaction is in purchasing the power from the sellers, who have not only been close to the buyer, but also announced low prices. As an example, Fig. 1 illustrates a city, in which there are five buyers and sellers in the third region. The initial prices are distributed haphazardly between the (1000 ± 500)$ for sellers. where it shows the i th buyer's desired value of the j th seller, toward purchasing d t bi units of power in t th interval. Due to the power loss, it is obvious that a buyer hopes to purchase from a seller who has ability to fulfill its demand completely, with both lower price and power loss attributes. Table 1 demonstrates a sample interest table. Considering all the interest vectors gathered from buyers, dealers combine them to form an interest table, and then commence to run the Greedy Power Matching procedure. In each time interval, a random permutation of the buyers is created. Then, each buyer attempts to create its own interest vector. After that, it starts purchasing its required demand from the sellers who are in the first element of its interest vector. After running this procedure for all the buyers, if a buyer fails to satisfy its demand and also is unable to find a seller with available surplus power in its region, then, it will buy the rest from the other sellers in the neighbored regions using the region to region purchase sub-algorithm. Similarly in this sub-algorithm, one random permutation of unsatisfied buyers is created at the first and then, each of them creates its own interest vector and search for the appropriate sellers. Finally, if the buyer's demand still is not satisfied yet, it should buy from the macro-station (i.e. region to macro-station purchase process). As a result, finding the optimal sequence of buyers, which want to minimize their objectives, makes it necessary to calculate all the permutations in order to achieve the optimal one. In this regard, the problem is NP-hard and rapidly becomes computationally intractable with growing size of the problem. Thence, evolutionary computation techniques come to help in optimizing the objective and its relevant constraints.

Evolutionary Computation
Evolutionary Algorithms (EAs) are one of the most well-known meta-heuristic search mechanisms which utilized for the optimization problems [7]. Evolutionary algorithms form a subset of evolutionary computation, in which they generally only involve techniques and implementing mechanisms inspired by biological evolution such as reproduction, mutation, recombination, natural selection and survival of the fittest.
In the evolutionary algorithm, firstly, we generate a random population of feasible solutions for the problem. In this literature, each solution represents the matching tables of the buyers and sellers. Then, these randomly generated solutions try to reach the optimal solution of the problem by using natural parameters as mentioned before in consecutive generations. Considering the randomly generated population in the first step, we used the "decentralized evolutionary greedy power matching algorithm", as mentioned in the previous section. Candidate solutions in the optimization problem play the role of individuals in a population and the cost function. In this process, there are two main forces that form the basis of evolutionary systems [15]: • Selection acts as a force increasing quality.
• Recombination and Mutation create the necessary diversity and thereby facilitate the solutions' novelty.
In this evolutionary algorithm, we have used a simple tournament selection, with linear crossover and exchange mutation procedures. In addition, an efficient "constrainthandling" sub-algorithm is also presented for enhancing the reliability of feasible solutions [16]. Tournament Selection. Tournament selection is a method of choosing a solution from a set of solutions in the evolutionary computations. It runs a tournament between P randomly chosen solutions (i.e. tournament size=P ). Then, the winner (i.e. the one with the best fitness) will be chosen for using in the crossover procedure.
Linear Crossover. Linear crossover procedure gets two solutions and then combines them linearly with the probability of α, in order to produce two other solutions. These new solutions will be the ones in the offspring population. In the revised version of this procedure, we compare the fitness value of these newly generated solutions with their parents and then choose two best ones for using in the mutation procedure.

Exchange Mutation.
Mutation is an evolutionary operator used to preserve variety from one solution to the other one, where the solution may change entirely from the previous one. Hence, evolutionary algorithm can come to better solution by using mutation. Mutation parameters depending on the nature of the problem are different. We applied a mechanism, in which the mutation procedure exchanges amount of some negotiated power between two randomly selected buyers and sellers.

Constraint Handling Sub-Algorithm.
An efficient constraint-handling technique is a key element in design of competitive evolutionary algorithms to solve optimization problems [16]. We proposed a greedy constraint handling sub-algorithm, in which all solutions of the offspring population will be refined in order to be assure of their feasibility. Regarding the discussing literature, refining is checking the solution in order to find any probable errors (e.g. having "un-satisfied buyers" or "over-sold sellers")in the power matchings.

Experimental Evaluation
In this section, the setting of the data used to evaluate the model considering a community of agents and the results obtained from the greedy power matching algorithm are provided.

Experimental Setup
We begin by clarifying power markets', trading agents' and evolutionary algorithm settings as follows.
Power Markets. A city within a square of 10 km×10 km, in which it encompasses the macro-station which is far from the square, and K regions is simulated. A power market, which has a unique dealer is implemented in each of the regions. All of the dealers cooperate in order to decide on announcing an initial price range such as (1000 ± 500) $. Due to the problems in actual implementation, the sellers choose an initial price for themselves haphazardly. Having 24 consecutive time intervals, and also analogous to [13], where the surplus and required powers is different for the buyers and sellers, are concerned.
As mentioned previously, there is power loss in power transfer. This loss is calculated as follows: Equation (18) exhibits power loss between i th buyer and j th seller, regarding their distance. Owing to have a region to macro-station purchase possibility, (19) is also defined as the power transfer loss between i th buyer and the macro-station.
Trading Agents. N buyers and M sellers are settled randomly in K regions. Each s ∈ R S k and b ∈ R B k has a q ∈ Q and d ∈ D respectively. In addition, these parameters are supposed to be uniformly distributed among ((7, 316) MW) and ((8, 346) MW) for buyers and sellers respectively [13].
Evolutionary Algorithm Setting. We have used tournament selection operator (tournament size=2) along with linear crossover (crossover percentage=80) and exchange mutation (mutation probability percentage=20). Moreover, the generation and population sizes are equal to 400 and 100, respectively.

Experimental Results
In order to evaluate our proposed approach and algorithms, a community of agents is considered. Hereinafter, all the results' figures are drawn out from the 25 × 25 community (Fig. 1). Running evolutionary greedy algorithm necessitates randomly generating the first population. Besides, having infinite power production by sellers, informs that this greedy approach results in an efficient and optimal power matching. Unfortunately, this assumption is not practical. Hence, finding the best sequence of buyers in buying procedure is the main challenge in power matching. Finding the optimal solution under these conditions, needs computing all the possible permutation of the buyers. Figure 2(a) illustrates the fitness diagram obtained from running the evolutionary greedy power matching algorithm on the proposed solution approach. As pictured clearly, the algorithm outputs one solution as the best one in each generation. In the next generations, it tries to improve the initially randomly generated population by the evolutionary parameters to find the better solutions. The descending order of the Fig.  2(a) informs that the algorithm attempts to get as close as possible to the optimal permutation of the buyers. Furthermore, having eagerness to purchase the required powers from the sellers who have announced lower prices is another proof for its descending manner. Although coming to better solution in power matching problem is important, but finding the appropriate and near-optimal solution in a logical and polynomial time is also substantial. Therefore, increasing the population and generation sizes results in increasing the calculation time. As mentioned previously, based on the proposed solution approach and market assumptions, we have used the Integer Linear Programming (ILP) techniques toward solving the matching problem. Hence, due to the NP-Hardness nature of the discussing problem, it is not possible to solve or oven compare with the traditional linear programming techniques.
In order to analyze the robustness of the iterative price updating rule, in which sellers change their announced prices in each interval, we survey the obtained results based on multiple perspective. Figure 2(b) demonstrate all the sellers price changes in successive intervals. This price fluctuations is gathered from the best solution of the last generation. The sellers announced their initial price randomly. Then, in the next intervals, they altered their prices based on their both malleability rate from the market parameter and the weighted-median of other sellers offered prices. For instance, S9 and S24 have announced their initial prices close to each other. However, having difference in their malleability rate has incurred that they could not update their prices in the same order. Along with the malleability rates, the amount of negotiated power is also significant in price updating. Suppose there is a seller which sold all its produced power in a interval. Intuitively, it does not need to modify its next interval's power price rapidly if its malleability rate from the market does not very high. However, there may be a seller which could not sell as much as possible and therefore, it should change its next prices in order to increase its selling amount. Besides, there were some sellers that announced their initial price far from each other (e.g. S24 and S11). Nonetheless, they attempted to reach the price which the markets preferred. This issue also can be looked at from two perspectives. If a seller has sold all its surplus power at a specific price, probably it has announced a very low price. As a result, the market dealers forced it to change its next prices in order to prevent the market from crisis circumstances. Beside this point of view, the seller might be unable to sell its powers and therefore it has forced to decrease its next intervals' prices to be able to sell its powers. Finally, if an agent, announces an initial price approximately close to the weighted average of the power price, it is not vital to alter its next prices rapidly.

Conclusion and Future Work
In this paper, we have proposed a decentralized greedy resource allocation mechanism based on the competition, which manages the demand response problem in power markets. In this regard, a solution approach is proposed, in which electricity markets behave in the benefit of buyers. In this approach, sellers in each interval, based on the markets' available information and their relevant parameters such as malleability rate from the market, offer their own price for each unit of power. Then, buyers create their own interest vectors and start purchasing their required electricity from the sellers who are in the first element of their interest vector. In addition, in order to organize an equilibrium market, considering both of the sellers and buyers who have roles in advancing toward it, a greedy power matching algorithm is also proposed to allocate the energy produced by the sellers and the macro-station to the buyers. Finding a near-optimal power matching solution due to NP-Hardness nature of the matching problems, has motivated us to use evolutionary techniques in demand response optimization regarding the buyers objectives. As some future work, scrutinizing the possibility of having self-learning buyers in power markets along with considering on-line predictive algorithms, considering the markets in the benefit of sellers or in a comprehensive multi-objective manner can be considered.