Building Multi-occupancy Analysis and Visualization Through Data Intensive Processing

. A novel Building Multi-Occupancy Analysis & Visualization through Data Intensive Processing techniques is going to be presented in this paper. Building occupancy monitoring plays an important role in increasing energy efficiency and provides useful semantic information about the usage of different spaces and building performance generally. In this paper the occupancy extraction subsystem is constituted by a collection of depth image cameras and a multi-sensorial cloud (utilizing big data from various sensor types) in order to extract the occupancy per space. Furthermore, a number of novel visual analytics techniques allow the end-users to process big data in different temporal resolutions in a compact and comprehensive way taking into account properties of human cognition and perception, assisting them to detect patterns that may be difficult to be detected otherwise. The proposed building occupancy analysis system has been tested and applied to various spaces of CERTH premises with different characteristics in a real-life testbed environment.


Introduction
Knowing the true occupancy, the presence or the actual number of occupants of a building at any given time is fundamental for the effective management of various building operation functions ranging from security concerns to energy savings targets, especially in complex buildings with different internal kind of use [1]- [4].The accurate definition of occupancy is the amount of people per building's spaces at any given time.Furthermore the influence that the occupants' actions have in the indoor environment [5], including those related to their business processes can also be added to the definition.Occupant's locations within the building varies throughout the day, therefore it is difficult to characterize the number of people that occupy a particular space and for what duration because human behavior is considered stochastic in nature [6].Due to the random nature of individuals' behavior and challenges accessing accurate data, current studies include the creation of deterministic schedules where a standard workday profile is the same for the whole workweek and both weekend days have the same profile [7].
There are numerous techniques to detect space occupancy and even track their movements, which can be found in the literature.These techniques range from user surveys, interviews or walkthrough inspections [8]- [13] to a more or less complex deployment of sensors within the area of study [1], [2], [4], [14], [15].The sensors used to this purpose are of various kinds and in general present lack of accuracy.In most cases, a combination of different sensors types is preferred to achieve better results [4].Measurement of occupancy is more commonly undertaken in residential environments rather than offices or commercial buildings [14].The most commonly used are:  CO 2 (Carbon dioxide) sensors are often deployed in commercial buildings to obtain CO 2 data that are used to automatically modulate rates of outdoor air supply.Furthermore, CO 2 sensors show very slow response to the change of the occupancy [16].Sometimes, more than 15 minutes is necessary so as to indicate a change to the occupancy of the space [1].An additional drawback is that the CO 2 measurements are highly influenced by the ventilation system and the open doors windows, etc.;  Passive infrared (PIR) sensors are commonly used for non-individualized occupancy detection.PIR sensors suffer from two main limitations: i) they only give information about whether a room is occupied or not providing no indication about the exact number of people and ii) they often do not detect stationary occupants, leading in false negative signals [14].To overcome these limitations, they are often coupled with other sensors;  Video Cameras: Video imaging typically uses small cameras mounted overhead of a doorway and video analytics to count and differentiate between people entering and exiting a building or space.The video analytics creates two lines, similar to the infrared using two beams and detects motion as to when the line is broken.It is not unusual to find the people counting capability as an add-on module of a video surveillance system.However, video cameras can heavily raise privacy and ethical issues and are sensitive to lighting levels.Video cameras if improperly installed and configured show substantial errors [1];  RFID System uses wireless radio communication technology to determine the location of occupants who carry special tags.Depending on the layout of the receivers the zones can overlap and detect occupants going from one zone to another while they are not moving [17].
It is difficult to determine the actual number of occupants in a predefined space and their patterns of movement with current sensing techniques [14], even more, low-cost and non-intrusive environmental sensors to measure occupancy in commercial buildings are not fully explored [14].In this paper the results of new occupancy extraction system [18] by means of depth cameras are presented; the developed system offers data anonymity and privacy preserving.Depth-image cameras (such as Microsoft Kinect) are used to extract occupancy (exact number of people, location and track) within a space or zone through the analysis of the depth-images collected.They are usually installed near ceiling to cover the examined area as much as possible but can also be installed on entrances/exits if zoning isn't required.They are more suitable for closed spaces but can also be used for open spaces and they need proper calibration in order to provide proper results.The equipment needed (number of cameras, cable extensions etc.) and the topology used depends on the current application, space layout and limitations (e.g.wooden separators between offices), cameras limitations (maximum distance, depth image limitations), overlapping FOV (Field of View) and number/location of entrances (which should be inside FOV).For example, 2-3 cameras would be adequate for a one-door closed space of 50m 2 given that there are not many obstacles.Real-time depth-image analysis is demanding as it requires high computational power in order to detect the presence of an occupant and extract his/her location.Moreover, occupancy detection based on depth-image cameras is sensitive to changes in image background and requires time consuming setup and calibration for defining the monitored area and tracking the detected occupants in the monitored space.On the contrary, depth-image cameras provide quite high accuracy with relatively low cost and have the capability to exhibit totally transparent properties (i.e.no colour images are recorded for human detection).Also, they are not sensitive to the lighting levels of the environment, although they will not work in direct sunlight conditions since they use infrared (IR) radiation.
Real-time estimation of the number of occupants in a building's space is a challenging task.A main challenge is to determine the method of processing the input received by the multiple sensor types.There are two main occupancy extraction approaches from a sensor fusion model: the rule-based approach and the probabilistic model approach.
The use of a rule-based system results in logical inference from sensor data [21].According to this approach, a set of rules for a set of installed sensors are defined and applied.Rules are defined by a domain expert and knowledge about sensor characteristics is required.The set of applied rules usually depends on the combination of sensors that are used.Two studies where this method is applied for occupancy detection is [22] and [23].
The probabilistic model approach views occupancy extraction as a classification problem.A probabilistic model is created by training a selected classifier and the target is to infer the occupancy class based on the input from the various sensors.Different models have been examined such as support vector machines, neural networks, hidden Markov models, agent-based models, decision trees etc. [24], [25].A training phase must be performed in advance in order to learn the parameters and be able to start the occupancy extraction process.On the contrary, a training phase is not mandatory for the rule-based approach.
The rest of the paper is organized as follows.Section 2 briefly presents the system used to extract the building measurements (data extraction), while the data analysis process is presented in Section 3. Finally the conclusions are drawn in Section 4.

Occupancy Extraction Approach
The subsystem utilized for the occupancy extraction is constituted by a collection of depth image cameras and a multi-sensorial cloud (various types of simple sensors) in order to extract the occupancy per space.The system is able to monitor multi-space environments and it has been built based on a client-server architecture.
The proposed occupancy extraction system is a real-time system since it is able to detect and track people and visualize the results in real-time and in high accuracy and details.Since depth image cameras are utilized in order to extract the occupancy of the building, issues like camera calibration, overlapping areas, error propagation, etc. have to be dealt with [18].Furthermore the depth image cameras provide only depth information in order to take into account all legal and ethical issues regarding individual privacy and provide anonymity.Finally depth information is immutable to luminance and shadow changes [18].
As aforementioned various types of simple sensors are also utilized in order to detect human activities and collect occupancy data which have been analysed by S. Zikos et al. [30] using a Conditional Random Field approach.Double-Beam Sensors [27] are established in specific locations, where a semantic event may occur.These locations are the doors of the building, as well as the doors of all building spaces.Moreover two Pressure Mats Sensors [28] are placed next to each other, separated by a small space in order to detect movement direction and PIR Motion Sensors [29] which were already installed through the Alarm system installation.When movement is detected, an activation event is sent by the sensor and after a specified period (configured to a few seconds) of no movement detection, a deactivation event is sent.Finally CO 2 Sensors are established in some spaces which measure the CO 2 concentration of the air and can be very useful when combined with other sensors mentioned above, since it can provide information on occupancy density.

Building Installation
The proposed occupancy extraction system can operate in any type of building.An indicative example of the physical installation of each sensor in CERTH premises is depicted in Figure 1.The test bed consists of eight (8) main areas with different usage (offices, corridors, rest area, meeting room and kitchen).The most remarkable spaces of the building are the Developers' office which is 56.7 m 2 , the corridor and the rest area which are 81.5 m 2 , the Meeting Room (26.4 m 2 ) while the kitchen is 33.7 m 2 .The majority of the sensors for occupancy extraction have been installed at the developers' office, a characteristic area of the building, since it is a core element for testing the real-time occupancy extraction system.In total six (6) Kinect Cameras were used as depth image sensors to provide occupancy information to a sub-space level, while three (3) Pressure Mats (x2), eight (8) Active Infrared Beams (x2) per door, ten (10) PIR sensors and three (3) CO 2 sensors covered the area at a space level.
The number and the type of the sensor cloud installed in this building are shown in Table 1.

Data Acquisition
The data acquisition is performed utilizing the sensor cloud and the system that has been described in Section 2.1.The VGA resolution of the infrared depth cameras, i.e. the pixel size, determines the point scaling of the depth data on the XY plane (perpendicular to camera axis).Since each depth image contains a constant 320 x 240 pixels the point density will decrease with increasing distance of the object surface from the sensor.Considering the point density as the number of points per unit area, while the number of points remains constant the area is proportional to the square distance from the sensor.Therefore, the point density is inversely proportional to the square distance from the sensor.The depth resolution is determined by the number of bits per pixel used to store the disparity measurements.The specific cameras disparity measurements are stored as 11-bit integers.Therefore, a disparity image contains 2048 levels of disparity.Since depth is inversely proportional to disparity the resolution of depth is also inversely related to the levels of disparity.That is, the depth resolution is not constant and decreases with increasing distance to the sensor.For instance, at a range of 2 meters one level of disparity corresponds to 1 cm depth resolution, whereas at 5 meters one disparity level corresponds to about 7 cm depth resolution.Furthermore, they have an angular field of view of 57 o horizontally and 43 o vertically.
The depth cameras monitor all the area under interest, detecting, tracking and extracting the occupancy during the whole monitoring period.The specific cameras capture depth images and extract real-time occupancy information at a frame of about 20 fps.The occupancy information extracted by the depth cameras carries not only the occupancy of the areas under interest, but also the detailed occupancy trajectories in it.The experiments show that the data extracted and stored by the system for a single normal working day are approximately comprised by > 120.000 events/measurements.
The data acquired by the system are of different kind and they are provided at different time instances depending on the sensor type and are stored in a central NoSQL database.

Data Analysis
The assessment of the building performance towards occupant's comfort and energy savings has been set as a main concern nowadays by using the required and equivalent software.Building Occupancy Extraction and more specifically occupants' trajectories are really important for building performance, occupants' work efficiency, building usage and is directly related to occupants' comfort, therefore occupancy statistics are depicted per space, as well as the number of transitions from one space to another.Based on these meaningful information, one can extract Key Performance Indicators (KPIs) related to the building occupancy.
The basic building occupancy related KPIs are: (a) average work efficiency and (b) average building usage.The average work efficiency KPI provides a measurement of the work efficiency and it is defined as: 100 * ancyHours totalOccup ityHours totalActiv avgWE  (1) where totalActivityHours is the overall hours that the occupants are involved in any activity in a specific space and totalOccupancyHours is the overall hours that the building is occupied.The average building usage KPI provides the usage of the building and it is defined as: where totalOccupancyHours is the overall hours that the building is occupied, Time is the overall is the duration (in years) of the monitoring activity and N spaces is the total number of building spaces.
Finally, the basic KPIs related to occupant's comfort are: (a) average overcrowding factor, (b) average Predicted Mean Vote (PMV), and (c) average Predicted Dissatisfied (PPD) [31].The average overcrowding factor is defined as: where N spaces is the number of spaces in the building under interest, SP is the set of all spaces, cap i is the capacity of space i, Time is the duration (in hours) of the monitoring activity, occupancyHours i represents the hours that the space i is occupied, oc-cHours i represents the hours that the space i is occupied by occupant j, and N occ is the number of occupants at space i.The average Predicted Mean Vote (PMV) KPI [31] is a thermal comfort model, which is defined as:  L where M is the rate of metabolic rate (W/m 2 ), q met,heat = M-w is the metabolic heat loss, the difference between the metabolic generation converted to work (e.g., lifting, running), w is the external work (W/m 2 ), f cl is ratio of clothed surface area to DuBois surface area (Acl/AD), h c is the convection heat transfer coefficient (Btu/h m 2 o C), t cl is the average surface temperature of clothing ( o C), t a is the air temperature ( o C), t r is the mean radiant temperature ( o C), p a is the vapour pressure of air [kPa].Since, all these parameters are not available (e.g. the ratio of clothed surface area of a human), some default values that have been used: The average Predicted Percentage Dissatisfied (PPD) KPI [31] predicts the percentage of occupants that will be dissatisfied with the thermal conditions, which is defined as: All the above mentioned KPIs are calculated during the measurement extraction procedure for a 4 month testing period of CERTH's premises.The collected data are over 1.15 billion of information, which is analyzed as shown in Table 2.In order to handle and process the big amount of data, which can be a very difficult and time consuming task due to size and diverse data types, a visual analytic intuitive user friendly application has been developed which presents a set of visualization techniques that facilitate users to perceive readily the data extracted.The data visualization application that was developed uses a coarse-to-fine approach to visualize information.Coarse-to-fine approaches are becoming more widespread as statistical problems grow into larger and significant domains.The coarse-to-fine approach minimizes the loss of accuracy, while executes the process at successively finer granularities.In accordance with the coarse-to-fine approach, the user can observe the key performance indicators for the building, which was described above in a kiviat diagram (Figure 2).The space occupancy data can be displayed per day, week or month in relation to user preferences, thus an extension of the Clock Map was utilized proposed in [19], with the addition of a 3 rd dimension encoded in the radius using concentric cycles.Figure 3 illustrates the space occupancy in Clock-view form, where each building space is represented by different colour.More specifically, orange colour indicates the kitchen; the dark green denotes the researcher's office and the developer's office is represented by light green colour.Respectively the radius of each building space denotes the portion of the total building occupancy and occupants per building space, where the intensity of the colour denotes the relation of the space occupancy with the corresponding completeness space occupancy.
Except from the Clock Map, occupancy heat maps were developed to further detect human presence and their trajectories.The system tracks occupants' movements and turns this information into heat maps as shown in Figure 5. Heat maps are a popular and intuitive visualization technique for encoding quantitative values derived from gaze points or fixations at corresponding image or surface locations.Heat maps enabled us to gain additional insights into temporal and spatial patterns present in the data.More specifically, the occupants' trajectories through the space are depicted in Figure 5 where the colours on the floor correspond to foot traffic during a particular time period.Pink areas are hotspots that lots of occupants walked through, while the small splotch of blue in some regions indicates lower traffic congestion.Moreover, an alternative heat map view is depicted in Figure 6.It is two-dimensional graphical representation of building occupancy data where the different values are shown as colours.The intuitive nature of the colour scale, as it relates to the temperature minimizes the amount of learning necessary to understand it.From experience we know that red is warmer than orange, yellow or light green.As it is observed the Developer's office (intense red colour) is the most congested space of CERTH premises while distinct spaces such as the meeting room, director's office and long corridor are areas with the least occupants (intense green), since they are not permanently occupied.This can be easily also observed in Kiviat Diagram (Figure 4).The distribution of the occupants among the different spaces in the CERTH pilot building is shown in Figure 8 for a typical week.All spaces are occupied during the working days, while during the weekends only partial part of the building (e.g.developers, kitchen) is used for a small portion of time (Figure 7).
Finally, all experimental results have been performed on an Intel I7 (8 cores with 3.5 GHz) workstation with 16 GB RAM under Windows 7 without any particular code optimization.All visualizations can be generated in near real time, e.g. less than a second, for 4-month data.

Conclusions
To respond to the need for improved detection of building occupants resolved in space and time, we developed a universal building occupancy big data analysis sys- tem in which information collected by a network of multiple and low cost privacy preventing depth sensors and various types of sensors including, but not limited to, CO 2 , double beam sensors, pressure mats sensors and PIR Motion sensors for detecting occupants' movement, trajectories and direction.Finally it must be pointed out that along with the building occupancy extraction system, visual analytics techniques have been developed in order to visualize the collected big data through a large number of visualizations, allowing the user to evaluate the performance of a building from a building occupancy point of view using an intuitive graphical user interface.Also the proposed system is designed in such a way that can deal with the big data produced by the sensor network over a long period of time.
In future, further experimentation in other buildings is planned covering a larger time period, in order to better explore the potential outcomes.Furthermore, it is planned to extend the system in order to combine information from various other building elements except from occupancy, such as environmental conditions and energy consumption of equipment devices, HVACs or lights.This will allow a much more assiduous evaluation of the building performance.

Figure 1 .
Figure 1.Physical configuration of sensors installed in CERTH premises

Figure 2 .Figure 3 .
Figure 2. Kiviat Diagram: Top level visualization displaying key performance indicators for the building

Figure 7 .
Figure 7. Occupancy extraction per space for a typical week

Table 2 .
Events produced for a 4 month testing period of time