Enabling Energy Eﬀiciency in Manufacturing Environments Through Deep Learning Approaches: Lessons Learned

,


Introduction
Today's manufacturing is moving towards an upgrade of the currently available manufacturing practices to a more efficient and intelligent level [13].This upgrade incorporates advances of various fields, particularly the field of artificial intelligence, which helps in different facets of a manufacturing company such as machines, processes, facilities, software and staff [1].Thus, sensory data will be then collected across the manufacturing company.Exchange of information and instructions in near real-time between smart machines and smart products will be a remarkable vision of manufacturing industries in the coming future [13].Under this setting, effective usage of data should consider different aspects of improving quality, reducing costs and energy simultaneously [8].Reducing the consumption of energy is referred to as "energy efficiency" from an engineering point of view [4].To achieve energy efficiency in manufacturing, researchers such as May et al. [8] considered the challenges against efficient energy usage in manufacturing.Based on the studies of [8] [13], artificial intelligence (AI) has the potential to enable energy efficiency in manufacturing.However, as observed in the literature research, only a few papers show AI as a technology for reducing energy consumption in manufacturing.In this paper, we look at the methods of Deep Learning (DL), as DL is one of the subfields of AI.DL has been considered in this study because of its potential to support near real-time decision making in manufacturing by handling large data from different sensory components as well as their complexities.In addition, the high-volume modelling capability of DL allows automatic processing of large raw data instead of "handcrafted" data.This aspect of DL is a powerful advantage over conventional machine learning (ML) methods because the performance of ML models is limited by their ability to process high dimensional and varied input data in their raw form [6]. Therefore, we investigate DL through three use cases for the assistance of energy-efficient processes in manufacturing.
Section 2 describes the challenges for achieving energy-efficient manufacturing as well as DL as a technology for enhancing efficiency in terms of energy.Section 3 introduces three case studies in the context of energy-efficient manufacturing.In each use case, DL contributes to an energy-efficient process.Numerical analysis of DL-models performance is not the focus of this article.Instead, in section 4, practical lessons learned during the implementation and deployment of DL models in manufacturing processes are described.Section 5 provides the conclusion.

Challenges for enabling energy-efficient manufacturing
Modern manufacturing has been affected by growing energy prices, environmental regulations and customer demands to meet sustainable products [9].Above all, industries face several barriers while implementing energy efficiency in manufacturing.Firstly, companies produce manufacturing goods concerning quality and time efficiency.On the one hand, energy efficiency is mostly not considered due to the possible multiplicity of process configurations as manufacturers alone cannot monitor numerous, high-dependency parameters and optimize them regarding energy efficiency.On the other hand, most up-to-date datadriven methods are developed without considering how these methods can be applied in practice [5].Consequently, manufacturers fail the implementation of pragmatic approaches to improve their processes energetically, qualitatively and economically; so that, changes in process parameters will not negatively impact their product quality.
Secondly, the energy consumption of manufacturing machines is not static but dynamic.In some cases, it also depends on the variation in the quality of input raw materials.Moreover, machinery often uses different interfaces to their internal meters, which becomes an inconvenience while interpreting the different production data and later while linking them to the energy-related data.Thirdly, some manufacturers do not know how to treat their available production data and how to interpret them in order to efficiently improve their processes and products [5].Particularly, there are not many software and modelling systems to analyze data in a simple manner yet [5].In this regard, manufacturers require tools or settings that provide them with operational information of machines and data of energy consumption, which can then be used to evaluate and control energy-related performances [9].
Based on the above challenges, a better understanding of processes and settings in manufacturing as well as a better usage of available process data with less hand-engineering are required.Moreover, the use of tools or technologies that provide exact information about manufacturing operations, energy consumption and the ones that at least do not harm the quality of the end product are preferred.In this paper, we provide three relevant industrial use cases where DL is applied as a method that meets the above observations.

Energy efficiency in manufacturing with deep learning
In many energy-intensive manufacturing processes, there is no available model which supports energy efficiency when considering the fluctuations of input raw materials or other process-varying inputs such as quality features.A quality feature defines both the quality of the process and of the good, taking into account the energy efficiency of a manufacturing process.The following energy-intensive use cases were considered in this research because their raw materials (or natural resources) have inconsistent properties, i.e., the quality of raw materials themselves vary in their composition over time, which affects the process behavior.Additionally, manufacturing processes may change with regard to maintenance and operator skills, which also affects the data associated with the process.Here, DL approaches can be used as an advanced predicting tool once they have been trained on large data of a particular process.Therefore, the integration of DL forecasting models enables energy efficiency in processing.
In the first case study, according to results of Bouktif et al. [3], a data-driven model based on a Long Short Term Memory (LSTM) architecture to predict the energy consumption in food processing is implemented.In the second case study, inspired by the experimental results of the Single Shot Detection (SSD)-based model [7] on standard datasets, image processing for identifying raw materials in a waste processing facility is applied.And in the last study, an approach of Yan et al. [12] for predictive maintenance in the animal feed industry is investigated by using an autoencoder (AE).

Use case 1: Energy efficiency in food processing
The industry of food processing involves energy-intensive processes.To ensure a high-energy efficiency and an optimal setting of parameters in a certain french fries processing line [2], parameters like temperature and steam pressure have to get adjusted continuously with fluctuations in the raw materials used.In this case study, streams of the raw material get scanned and measured, which generates a complex time-series of statistical histograms according to a rawmaterial criterion like shape or humidity.The generation of a data-based model for food processing in relation to the efficiency of energy and final product quality must include the quality of input raw materials and variations of the process settings.
In order to handle these complex histograms of process data, an LSTMbased data-driven model is applied.LSTM addresses sequences of varying-length statistics and captures long-term process dependencies on different time-scales data of food processing.The time-series production data and different settings of parameters in a food process have been collected for six months; so that in this case study, an LSTM-based approach learns features from these data histograms.Thereby, it projects the relationship between sensor data and quality features of the food process onto the model.Fig. 1 shows a process flowchart for implementing LSTM in order to increase energy efficiency in a french fries processing line.The model provides online forecasts from both the energy consumption and the final food quality.The forecasting is visualized and assists the plant operator, who is able to react in time to variations in process behavior rather than waiting for the end product inspection.This additional information leads to quicker reactions, reduces end product rejections caused during quality inspections and effectively enables energy efficiency in food processing.

Use case 2: Energy efficiency in waste processing
Waste management is one of the major global challenges of today.The collected municipal, commercial and industrial waste is first sorted in a waste processing facility.The sorted solid waste material is then pressed into tight bales in a certain baling line [11], which are ready to be delivered for reuse or further treatment.The length of the final bale and the pressure to be applied to the waste materials depends on the type of the input raw waste to be processed in a baling line.Therefore, prior to pressing, the streams of waste material, which are transported by a conveyor belt, must be visually identified.Thereby, material-dependent parameters of a baler press must be continuously adjusted.Otherwise, a poor quality bale is rejected, causing additional logistical, financial and energy costs.Hence, a key performance indicator for a waste baling line is proper identification of the input raw materials and the energy consumed during the process of baling.In this case study, a robust identification of input materials in a waste processing line to enhance energy efficiency is considered.Fig. 2 illustrates how the integration of an SSD-model helps to automatically adjust the parameters of a baler press in a waste baling line.The SSD-model forecasting assists both, automatic adjustment of process parameters and an energy-efficient baling process.

Use case 3: Energy efficiency in feed processing
Different machinery is used for processing animal feed.Hammermill machine is used to shred and make natural raw materials such as maize, wheat to smaller pieces [10].The process of converting the grains (e.g., maize) to smaller particles is an essential step in the animal feed production because it affects the extent to which the animal's body can absorb the feed nutrients.After analyzing the energy consumption in a feed processing plant, it was found that the energy consumed during the shredding process of a particular compound feed product caused by hammermill failures is more compared to energy waste related to raw material or due to operator behavior.An automated Supervisory Control And Data Acquisition (SCADA) system shows the warnings and alarms related to failure and maintenance of this equipment.However, no further information has been gained for forecasting the occurrence of failures based on SCADA warnings and alarms.A predictive maintenance system offers to predict the occurrence of next possible sensory failure by using the SCADA information, which can be integrated into the maintenance schedule.The input data from the SCADA system contain the followings; the frequency of warnings and failures, their importance (risk), type of raw material, which is being shredded in the hammermill and speed of hammermill.The data from sensors has been collected for 1.5 years in the form of daily logs.Because of the existence of failure codes, high-dimensional data behaviour and their interdependence, the data has a complex behaviour; so that, an AEbased model for predictive maintenance is proposed here.Fig. 3 illustrates the model proposal based on an AE-model.An AE-model gives suggestions of the anomalous behaviors of a hammermill process based on complex high-level representations of sensory data.The results of AE are presented in the form of clustering spaces of failure types which shows the similarities among failures.As a result, predictions of next failures are added into the maintenance schedule, which in turn helps to adjust maintenance and production plan and therefore, enable an energy-efficient process.

Discussions and findings
The three case studies revealed the following findings.Application of DL approaches provide means for better analysis and comprehension of a manufacturing process and its settings, which in the case studies was exploited to automatically provide suggestions to enhance energy efficiency.However, still there are challenges associated with adaptation, implementation and deployment of DL models for energy-efficient manufacturing processes.Challenges to the development of energy-efficient manufacturing in terms of data and learning transferability are discussed here.
Challenges of annotation in datasets.As DL models become larger and more complex, they require training datasets that are bigger than those required by other ML techniques.Majority of the available manufacturing datasets are not labeled, or if labeled, there exist noisy labels which must be manually removed.Moreover, it is not easy to define labels by hand i.e. for images of waste materials.Sometimes, hand-labeling becomes very complex to differentiate among classes, i.e. when waste materials to be identified are located very close to each other or when they are overlapping and only a small portion of the material is visible to the DL model (in the input image).
Dependence on the variability of input data.A common presumption in DL is that the performance of DL algorithms has high dependencies only on the scale and quality of raw datasets.So far, DL has shown feasibility in our case studies when DL is applied to only an input data type such as images and to welldefined tasks.However, the DL-models of our use cases show that the DL still has difficulties in identifying objects when the variability within the data class is high.Specifically, material classes that appear in many colours and shapes are still tough for the deep model to classify correctly, although significant volumes of data are used.
Dependence on hardware performance.By selecting a deep model, a certain dependence on the used hardware platform is considered.The depth of the model architecture and the available data impact the DL-model performance significantly.Both of these factors demand the usage of high-performance hardware such as GPUs due to computationally intensive processes during the training of deep models.
Model transferability.Additionally, the generation of DL-models and maintenance of them require more data, which for a novel or changed-domain manufacturing processes are not always readily available.Transferability of the entirely or even parts of deep models used in our case studies to similar instances within the models are shown to be possible.

Conclusion
This paper presented an overview of the potentials of DL to achieve energy efficiency in manufacturing processes.DL does not only offer a new point of view into the manufacturing operations but also supports near real-time measurement and energy saving during manufacturing.In our case studies, the integration of DL played a significant role in production planning and therefore, in the energy efficiency of these processes.Moving towards an energy-efficient production planning requires the inclusion of energy efficiency within the goals of production design, input quality control and maintenance at all levels, together with time cost and flexibility.The reduction of the machinery idle-times through energyefficient process planning with the combination of better orders organisation, predictive maintenance and quality controls lead to a better prediction of the workflow and therefore to better assistance in energy-efficient manufacturing.

Fig. 1 .
Fig. 1.Application of a LSTM-based data-driven model in food processing

Fig. 2 .
Fig. 2. Application of a SSD-model in waste processing

Fig. 3 .
Fig. 3. Application of an AE-model in feed processing