Demand Side Management Energy Management System for Distributed Networks

This paper is focused on the development of a demand side management control method in a distributed network, aiming the creation of greater flexibility in demand and better ease the integration of renewable technologies. In particular, this work presents a novel multi-agent model-based predictive control method to manage distributed energy systems from the demand side, in presence of limited energy sources with fluctuating output and with energy storage in house-hold or car batteries. Specifically, here is presented a solution for thermal comfort which manages a limited shared energy resource via a demand side management perspective, using an integrated approach that includes an auction and a shifting load strategy. The control is applied individually to a set of Thermal Control Areas, demand units, where the objective is to minimize the energy usage and not exceed the limited and shared energy resource, while simultaneously indoor temperatures are maintained within a comfort frame. The developed solution is explained and applied to different scenarios wherein the results illustrate the benefits of the proposed approach.


Introduction
Nowadays buildings sector is responsible for 40% of the world's energy consumption and almost 50% greenhouse gas emissions. Buildings emit more gases than transports and industry sector, estimated in 31% and 28% respectively. By analyzing the energy profile of buildings, it is clear that most of the consumption is to heat/cool the spaces to provide indoor comfort [1]. The energy consumption in buildings sector has increased along the last 20 years. Consequently, data from 2009 [2], showed that in residences the space heating accounted for about 70% of the 68% of total final energy consumed in buildings.
This increasing energy consumption is mainly to fulfil the demand for thermal comfort, being presently the HVAC systems the principal energy end use in buildings [3]. By these facts, it is socially, environmentally and economically imperative to decrease the energy consumption by increasing the buildings efficiency. A viable choice to achieve the reduction of energy consumption in the building sector is the application of demand response (DR) mechanisms. Demand Response program, is an efficient load management strategy for customer side is a load allocation scheme from the demand side, that it is nowadays mostly used to encourage users to shift their energy usage to periods with low demand and, consequently, lower prices [4]. The DR potential it is not sufficiently explored, being the two key challenges to work with diverse heterogeneous loads and with the distributed nature of renewable sources. New technology advances in communication are providing solutions to overcome the current electricity demand requirements. This new development has accelerated devising various industrial programs for scheduling utilization of residential appliances [5]. This DR mechanism combined with Demand Side Management (DSM) methodologies will be relevant in the future distributed Smart Grids (SGs) [6], and provide solutions that allow buildings to be fully integrated and prepared to efficiently coexist in a dynamic and inconstant environment typically supported by renewable resources [7].
Currently, the grid balance is mainly made with the generation following the electricity demand. Nevertheless, being renewable resources mainly weather dependent, it is vital to provide flexibility to the grid, in order to respond to the resource variability. In future SGs, it is expected that the production will control the energy consumption. Therefore, buildings settings must be controlled and adaptable to the clean resources intermittency. With this approach, users will have a more active position instead of being merely spectators in the electricity grid systems. Several solutions are emerging to deal with the variability, flexibility and poor controllability of the green sources and consequently on the ability to maintain the balance between demand and supply. Remark that, DSM strategies must have into account the control of many kinds of appliances, for instance, HVAC systems, lighting or electric vehicles charging.
The methodology here presented seeks a solution to respond to this variability, implementing, with the technological and advanced environment that SGs will provide [8], pursuing new technologies and solutions that will allow simple home appliances to be entirely controllable. This active DSM [9] will manage the loads to obtain harmony in demand supply ratio. DSM can include a mixture of several approaches, load control manipulation models, pricing, with distinguished electricity tariffs along the day to encourage load management and other approaches that promote energy efficiency and conservation [10,11]. Therefore, a novel integrated control solution with DSM automated response is proposed based on Model Based Predictive Control (MPC) techniques. The MPC technique, in comparison with traditional HVAC systems used in the domestic sector, is able to save 16-41% of energy, and also it adds robustness, adjustment and flexibility to overall system [12]. Thus, due to its features, MPC is suitable for energy savings, energy management and optimization, in particular to control temperature set-points [13], see [14] for a review on MPC in HVAC control systems.
The MPC control has also evolved as a solution to act in distinguished distributed environments [15]. Distributed Model Predictive Control (DMPC) algorithms, offers the same features of MPC but have the advantage of supporting the distribution of sensing and control using local controllers/agents that cooperate by exchanging information to decide their control actions [16]. This is the reason why it was the chosen method to deal with this kind of system. These DMPC infrastructures are suitable to use in a Multi Agent System (MAS) framework where, in a distributed environment, several agents employing individually a MPC control strategy are able to interact and receive influence from neighbour subsystems, exchange predictions on their future state and incorporate this information into their local MPC problems [17]. An agent can be defined as a complex software entity or intelligent entities with three main characteristics: Pro-activeness (they react to external events and they are driven to their objectives), social ability (they can cooperate or compete between them) and autonomy (they can decide in order to archive their objectives) [18]. Remark that the SG vision can be based on this concept, a framework where distinguished identities cooperate to obtain a collective perception. Thus, the developed system considers each agent as an autonomous entity. Designated by Thermal Control Area (TCA) the entity is embedded in a distributed environment, where several TCA's are working individually to achieve their own goal but sharing their information among them to maintain some kind of coordination in face of a global objective.
The outline of the rest of the chapter is as follows. In Section 1.1 is presented the research question and hypothesis and Section 1.2 describe the innovative contributions. In Section 2 is presented the contribution of our work to the main theme of the conference "Technological Innovation for Cyber-Physical Systems", in Section 3 is made the scenario description and the developed dynamical models. In Section 4, the developed MPC control scheme is described, in Section 5 results from numerical simulations are depicted and comment, and finally Section 6 offers the concluding statements.

Research Question
With SGs, domestic customers will have an important role in the electric grid system. Final consumers will no longer be merely spectators; their contribution will be relevant due the new technological advances that allow fully manageable households appliances. Shifting their electricity consumption in time or, by changing their work conditions, these devices can be controllable, adjusting the demand to the desired intermittent source without decreasing the comfort of the residents. In distributed networks, aggregated to an intermittent source, an unlimited number of this kind of loads can exist, representing a control problem to achieve the network efficiencies, involving stakeholder's satisfaction.
Consequently, new distributed, coordinative and cooperative strategies are necessary to ensure that the control decisions of all identities present in the system, contribute for the global objective, and also as referred, is desirable any or negligible comfort impact on end users/occupiers. Considering the mentioned above, this work aims providing solutions to respond to the next questions: Q.1 How, in a distributed network, can the demand be adjusted to an intermittent source to maximize the energy efficiency? Q.2 How to improve energy efficiency using the domestic potential in a distributed network? Q.3 Which control methods should be applied in a distributed network with demand side management to obtain all the existing energy potential from intermittent energy sources? The adopted work hypothesis to address the research question is defined below: Using an integrated approach, that in a distributed environment, considers multiagent control scheme and an optimization MPC multi-objective approach with anticipative effect, capable to deal in a DSM perspective, with fluctuating energy sources, smart load control, thermal comfort and real-time price negotiation.

Research Contribution and Innovation
The novel proposed sequential multi-agent DMPC scheme for thermal house comfort and energy savings provides robustness, adjustment and flexibility to the global system. The sequence is built based on an energy bid where the highest biddings are placed first in the access order. After consume, each agent predicts its consumption profile and pass through the next, the information about the predicted available renewable energy. At each hour, the sequence is established and the energy price depends from the offered bid, amount and type (renewable or grid) of energy consumed. Through this DSM energy usage optimization scheme, the consumer has the flexibility to choose hourly between comfort or energy savings. The anticipated knowledge of the energy source value (by forecasts in renewable case or by quantifying in fuel case) allows the system to decide how to split and when and in what quantity the energy is spend to respect all the power and comfort constraints. Thus, the innovative cost function optimization scheme is also suitable to apply in environments where at the same time, the energy sources renewable or not, have strong limitations and must be rationed and the comfort issues must be also taken in to consideration, as for example, in remote areas or in boats/cruises where the rooms must be acclimatized and the fuel source is limited.
A shifting load algorithm for loads allocation is also established. The customer sets features of flexible loads and the algorithm fits them in the most favorable time interval gap.

Relationship to Cyber-Physical Systems
Cyber Physical Systems (CPS) are organized systems that are connected and related between them and have the capability to act, react and cooperate with multiple technological and biological agents (devices and humans). To allow this kind of features, CPS incorporates sensors, processors and actuators that provide safety and interoperable actions between components in real-time applications.
Advances in CPS are providing new solutions to control and manage buildings. Thus, buildings may have a fully integrated and embedded network of devices in their infrastructure to provide superior management solutions in the energy usage, namely in heating, cooling, lighting and elevators/transportation. Operating with several sensors and actuators planned to provide comfort, security, operational efficiency and intelligent performability, new buildings may be viewed as the CPS stereotype with a high level of interrelated systems that sense and process the data in the net. As mentioned, in future SGs household appliances are network connected and perfectly manageable in real time.
The developed framework considers a set of buildings, compose each one with several floors and each floor by several rooms, that consist in many distinct appliances with the capability to feel the surroundings (e.g., temperature, energy consumption) and actuators (e.g., air conditioning equipment) to influence the environment. Thus, building is perceived entirely as a cyber-physical energy system, where distinguished identities cooperate, sense, act and react to obtain a collective perception. Therefore, each agent is observed an autonomous entity, TCA that physically is a CPS that is embedded/integrated in a distributed environment, where several CPS are working individually to achieve their own goal but sharing their information among them to maintain some kind of coordination in face of a global objective.

Dynamical Models and Scenarios Description
The scenario here described intends to be a realistic solution to take advantage of SG's features. Smart meters, or commonly named advanced metering infrastructure (AMI), are applied in SG to allow the equipment monitoring and control. Because renewable energies are nowadays significantly expanded, electricity is being fed into both the medium-voltage and low-voltage grids, depending on changing external conditions (e.g., weather, time of day, etc.). These fluctuating energy resources can severely impair the stability of the distribution grid. One of the key challenges of a smart grid is therefore to quick balance out the energy supply and energy consumption in the distribution grid. Thus, the scenario here presented has in consideration this challenge, intend to provide a solution to it based on, and taking advantage of all the AMI and communication technologies that SG provide.
The conceptual scenario involves a set of buildings with an electricity source provided by their own renewable energy park and energy storage as presented in Figure 1. Henceforward, the term house will be applied to classify any type of structure for habitation (houses), office buildings or other kind of analogous constructions. The set = { 1 , 2 , … , } identifies the group of houses in consideration and the different spaces or divisions in each house are specified by NS sets given by and Ndi is the number of divisions for house i. Due the existing division diversity in houses, each area may vary in: construction materials, sun exposure, occupancy and indoor temperature set-points, thus, each division has its own energy needs to weatherize the space, and for this reason a TCA is considered. As mentioned, one TCA represents an autonomous thermal control entity within an environment where the actions and reactions are made in order to achieve a common goal. Therefore, depending from the desired infrastructure intent to be implemented, a set of buildings or a simple division may represent a TCA. Each TCA receives several external inputs; the outdoor temperature, available renewable power forecasts, the Market Operator (MO) kWh price, access order to the renewable resource and, when applied, the neighbour's indoor temperature forecasts. Figure 2 presents an example of the implemented TCA framework. Thus, in a scenario that privileges the renewable energy usage, the used demand side management approach allows the management of distributed loads, aiming the adjustment of the demand to the supply, providing thermal comfort, lower energy costs and lowering CO2.emissons. By using this active DSM control, the optimal control strategies for various appliances can be generated whilst maximum utilisation of energy supplied from intermittent systems is guaranteed.

Dynamical Models
The electro thermal-electrical modular approach used to develop the house models is described in detail in [19]. So, in this work are only showed the main equations. Firstly, a first order linear energy balance model is used to define the dominant dynamics of a generic division, where x  ℝ n is the state variable, indoor temperatures, vector containing all the divisions temperatures (ºC), u  ℝ m is the input vector containing all the heating and cooling power sources (W) to weatherize each division, and v  ℝ n includes all the disturbances (W), including load generated by occupants, solar radiation, any other heat/cooling sources or doors and windows aperture to recycle the indoor air, k is an integer number that denotes discrete time and A  ℝ n×n , B  ℝ n×m , are matrices.
Generically, using Euler discretization with a sampling time of , the discrete model space-state representation of (1) can be written, for the Ns TCAs and for each division (l) as, where Ns is number of TCA's, Ndi is number of divisions inside subsystem (i), is the indoor temperature in TCA/subsystem (i) inside division (l), is the used power to provide comfort in TCA/subsystem (i) inside division (l), ℎ is an element from the state matrix A that relates the state (indoor temperature) in division (m) from TCA/subsystem (h), with the state from division (l) in TCA (i) and is the thermal disturbance in TCA/subsystem (i) inside division (l) e(g. load generated by occupants, direct sunlight, electrical devices or doors and windows aperture to recycle the indoor air), and Toa, the temperature of outside air (ºC). Remark that the number of states variables in general model (1) . For a more complex scenario, Fig. 3 shows a distributed environment with five houses with different plans and with different zones that may thermally interact. Remark that as described in the sequel, adjacent areas can be doubly coupled, thermally and by the power constraint. In Fig.  3, ui represents the input (heat/cooling power) and yi is the output vector containing the temperatures/states inside the several divisions of house (i)

Model Predictive Control Cost Function
At each time step, each one of the agents must solve his MPC problem. The objectives are: minimize the energy consumption to heating and cooling; minimize the peak power consumption; maintain the zones within a desired temperature range and maintain the used power within the green available bounds. Feedback stability is provided by choosing a sufficiently long predictive horizon and proven by results presented in Section 5. Feasibility is achieved by the use of soft constraints in the optimization problem formulation as explained in the sequel. The generic linear convex optimization problem to be solved with a Matlab routine (fmincon) by each agent at each instant, assumes the following form: , , and subject to the following constraints, , ̅ , , ̅ ≥ 0.
In (4) is the number of divisions of house (i), represents the power control inputs from house (i) division (l), is the penalty on peak power consumption, is the penalty on the comfort constraint violation, Ψ i the penalty on the power constraint violation and HP is the length of the prediction horizon. In (7), ̅ and are the vectors of temperature violations that are above and below the desired comfort zone defined by ̅ and . In (8), the coupled power constraint, and are the power violations that are above or lower the maximum, , and minimum, , available green power for heating/cooling the house, with = − . Remark that, in each TCA (i), the power sum in all divisions cannot exceed .

Results
The results are presented with two different approaches. In Section 5.1 the energy split performance is based on a fixed sequential order, A1, A2 and A3, established from a previously done auction wherein the bids are daily made by each TCA, acting as demand side management agents and based on the energy daily price ( [20] allows the bidding value vary hourly and consequently, the agents order to access to the clean energy also varies). In Section 5.2 the results show the developed shifting and loads allocation. All the presented results were obtained with an optimization Matlab routine.

DMPC for Thermal House Comfort with Sequential Access Auction
In the first approach three houses are considered, two of them thermally interacting (with a thermal resistance between them of R12=30ºC/kW and the third is isolated. As mentioned, agents can also have distinct penalties on power and temperature constraints violations, they can hourly privilege comfort or cost according to consumer choice. Therefore, to explore the concept two scenarios are here presented. In the first scenario (S1) the penalty values of the parameters related with consumption were increased. The comfort issues are less important, with the agents mainly concerned with lower consumptions and in satisfy the power constraint. With this variation, the soft power constraint was transformed in a hard constraint. The second scenario (S2) is focused in maintaining the indoor comfort, all agents want to respect the established temperature gap regardless the required consumption. To accomplish this goal, the temperature penalty was significantly increased, and the consumption parameters decreased. The thermal disturbances forecasts and the indoor temperature with its constraints, as well the power profile for the TCA, A1, A2 and A3 are pictured above.   Due the scarcity of green resource and the obligation to respect the power limits, A3 was the one that presented lower consumptions, Fig. 10(a), and consequently minor costs, on the other hand, the indoor temperature was the most penalized with the highest deviation from the chosen comfort range. With this parameterization, the consumption profile always preserved inside bounds, for all TCA's, and can be seen that any or negligible red resource was consumed.   As can be seen in Fig. 7(a),8(a) and 9(a) that all the indoor temperatures are mostly maintained inside the comfort gap. As consequence, the power constraints, Fig. 7(b), 8(b) and 9(b), are violated and the red resource consumption increased significantly.
Being now the comfort a priority it's quite clear that the controller tries to accomplished the pre-defined comfort range leading to a consumption and cost surplus.  Comparing the economic differences between scenarios, the S2 has shown to be the most economical however, this decreased consumption led to a higher indoor temperature deviation from the established boundaries. On the other hand, S1 was the most expensive, all the necessary resources were consumed, leading to higher costs, in order to respect the comfort limits. The possibility of obtaining comfort in detriment of the cost may be important in various situations. For example: To acclimatize rooms with children or areas in laboratories and/or hospitals. Remark that, despite this comfort preference, the cost function (in due proportion given by the parameters) also minimizes all the other terms.

Implemented Shifting and Loads Allocation Scheme
In buildings, DSM is based on an effective reduction of the energy needs by changing the shape and amplitude consumer's load diagrams. So, DSM can involve a combination of several strategies; pricing, load management curves and energy conservation are implemented for a more energy efficient use. Load shifting is considered a common practice in the management of electricity supply and demand, where the peak energy use is shifted to less busy periods. Properly done, load shifting helps meeting the goals of improving energy efficiency and reducing emissions, smoothing the daily peaks and valleys of energy use and optimizing existing generation assets. With new technological advances, DR programs may shift loads by controlling the function of air conditioners, refrigerators, water heaters, heat pumps and other similar electric loads at maximum demand times. The work here presented is distinct because provides an integrative solution which is able to, in a distributed network with multiple TCAs, adjust the demand to an intermittent limited energy source, using load shift and maintaining the indoor comfort [21]. With this data, all the possible loads schedule combinations (PLSCS) are establish (see Fig. 19 e.g.). At each time step, it's verified if inside the predictive horizon, any PLSCS exceeds the maximum available green energy. The sequences that are at any instant above the limit are removed, and the remaining are the feasible load schedule combinations (FLSCS). The FLSCS are tested in the minimization problem (4) as maximum available green resource for comfort (8). The hypothesis that provided less consumption is chosen. Once one sequence is started, all the others that are different until the current step time are eliminated until the final load sequence is chosen, FLSeq. The results here presented show only the shifting loads procedure for one house represented by one division with thermal disturbance. A distributed scenario, algorithm and details about the implemented scheme can be seen in [21]. The house thermal characteristics and cost function parameterization are showed in Table 3, and the Loads that can be daily shifted have the characteristics present in Table 4. In order to minimize the energy costs by consuming only green resource, the implemented algorithm chooses the gaps that fit properly in the maximum available green energy.   In Fig. 20 are the total energy costs of the FLSCS shown in Table 5, and can be seen that the chosen sequence, third hypothesis, is the less expensive.  The periods between 7-10h and 17-20h are extremely demanding, all green energy is consumed by the shifted loads, with no remaining one for comfort proposes. Although, Fig. 18 shows that in that periods the algorithm choose to not use the red resource and, taking advantage of the prediction horizon, pre-heat or pre-cool the spaces when only renewable resource is available.

Conclusions
In this paper, a distributed MPC control technique was presented in order to provide thermal house comfort. The solution obtained solves the problem of control of multiple subsystems dynamically coupled subject to a coupled constraint. Each subsystem solves its own problem by involving its own and adjacent rooms state predictions and also the shared constraints. Changing the penalty values, the consumer can choose in each division between indoor comfort and lower costs. It could be observed through the simulations and results analysis that suitable dynamic performances were obtained. Also, the approach shows that distributed predictive control is able to provide house comfort within a DSM policy, based in a price auction and the rescheduling of appliance loads. It is a valid methodology to achieve reduction in consumption and price. The method is more effective with wider periods where the loads are allowed to slide and consequently allocate the most favorable zone.