Dynamic Search Tree Growth Algorithm for Global Optimization

. This paper presents dynamic version of the tree growth algorithm. Tree growth algorithm is a novel optimization approach that belongs to the group of swarm intelligence metaheuristics. Only few papers addressed this method so far. This algorithm simulates the competition between the trees for resources such as food and light. The dynamic version of the tree growth algorithm introduces dynamical adjustment of exploitation and exploration search parameters. The efficiency and robustness of the proposed method were tested on a well-known set of standard global unconstrained benchmarks. Besides numerical results obtained by dynamic tree growth algorithm, in the experimental part of this paper, we have also shown comparative analysis with the original tree growth algorithm, as well as comparison with other methods, which were tested on the same benchmark set. Since many problems from the domains of industrial and service systems can be modeled as global optimization tasks, dynamic tree growth algorithm shows great potential in this area and can be further adapted for tackling many real-world unconstrained and constrained optimization challenges.


Motivation
In the todays era of digitalization, Internet of Things (IoT) and Cloud Computing, the industry and service sectors are transforming by introducing many innovations, where the most emphasized features of these innovations are so-called "exponential technologies".This transformation is usually being expressed with the terms Industry 4.0, Smart Manufacturing and Economy 4.0.
In order to implement and to establish innovations in industry and service sectors, many problems need to be addressed.Most of such problems can be mathematically modelled and can be categorized as NP hard problems.For example, in the domains of Industry 4.0, cloud computing and IoT, many scheduling problems exist, that are characterized with NP hardness [1].Also, the implementation of the wireless sensor networks (WSN) is very important in conducting innovations of industry and service sectors.Many problems, such is the problem of localization, from this domain belong to the NP hard group [2].
Another area which belongs to the group of NP hard problems is data clustering, along with data mining.Modern industry often relies on large amount of data.Large datasets are stored on the clusters, and focus of the algorithms is partitioning any dataset into an optimal number of groups through one run of optimization [3].Modern industry also heavily depends on telecommunication networks.In order to keep the cost of the network reasonable, it is not possible to directly connect all communication nodes.Routing process is responsible for selection of the path between source and destination, by optimizing the objectives together with constraints.Any routing problem has the main objective to maximize the network performance [4].
For solving NP hard problems, it is not enough to use classic deterministic algorithms, since the results cannot be generated in a reasonable amount of time.For such purposes, it is better to employ stochastic methods like metaheuristics.Metaheuristic algorithms can be roughly divided into those who are inspired by the nature, and those who are not nature inspired.Nature-inspired metaheuristics can be further divided into evolutionary algorithms (EA) and swarm intelligence (SI).
Swarm intelligence is population-based, stochastic and iterative search methods that try to improve the population of candidate solutions in a predetermined number of iterations.These algorithms simulate the groups of natural organisms such as flock of birds and fish, groups of fireflies and bats, etc.One of the first swarm intelligence approaches was particle swarm optimization (PSO) that was developed in 1990s [5].After the PSO, many other swarm intelligence algorithms emerged that were adapted for solving benchmark, as well as real-world NP hard optimization tasks.Some of the examples include artificial bee colony (ABC) [6], firefly algorithm (FA) [7], fireworks algorithm (FWA) [8], monarch butterfly optimization (MBO) [9], and many others [10].
According to the literature review, many swarm intelligence algorithms were successfully applied to the domains that are crucial for innovations of industry and service sectors.The PSO metaheuristics obtained satisfying results for solving job-shop scheduling problem in the era of industry 4.0 [1], as well as in some problems from the domain of IoT [11] and cloud computing [12], MBO algorithm can be applied to localization of unknown sensor nodes in some wireless sensor network topologies [13], etc.
Besides mentioned, swarm intelligence algorithms were also successfully applied to the domains of neural networks training [14], and image processing [15].In recent years, the adaptations of swarm intelligence methods for image classification with convolutional neural networks (CNN) emerged [16].These implementations may be crucial for digitalization.
Since the role of metaheuristics will be important in solving NP hard problems that will have to be tackled in the process of transformation of industry and service sectors, and due to the fact that many of such problems can be formulated as bound-constrained and constrained problems global optimization problems, our motivation behind this work is to improve recently proposed tree growth algorithm (TGA) [17] for global optimization assignments.
In this paper, we propose dynamic version of the TGA swarm intelligence metaheuristics for solving global optimization problems.The approach was tested on a set of standard global benchmarks, and with only few modifications, enhanced TGA can be successfully applied to NP hard problems from the domain of innovations of industry and service sectors.

Research Question
According to the literature survey [18], [17], we concluded that the TGA can be successfully applied to solving range types of NP hard problems.The research question, which is the stepping stone of this paper, can be formulated as: How to design and implement an efficient and robust TGA algorithm that will be capable of solving bound constrained and constrained global optimization NP hard problems and that will outperform other similar approaches tested for the same problem formulations?
In order to address the research question, we have formulated the following hypothesis: TGA metaheuristics, as novel and promising method, can be further improved for solving global optimization NP hard problems.Once improved, this method can be easily adapted for solving different kinds of problems that innovations in industry and service systems face.

Related Work
TGA algorithm is novel approach that was developed in 2017 by Armin Cheraghalipour and Mostafa Hajiaghaei-Keshteli [17].The algorithm was tested on standard global optimization benchmarks and on some engineering (constrained) benchmarks [17].According to the literature review, only two papers that deal with this specific method exist in the literature.
On the other hand, since many NP hard problems that are crucial for innovations in industry and service systems were successfully tackled by employing swarm intelligence methods [3], [11], [12], and according to our analysis of the TGA metaheuristics, we concluded that the TGA can be further improved, and in the future research can be adapted for solving various kinds of problems from the domain innovations of industry and service systems.

Contributions
In this paper, we show an implementation of the improved tree growth algorithm (TGA) adopted for solving global optimization problems.Due to the fact that many problems from the domains of industry and service sectors innovation can be formulated as global NP hard problems, the improved TGA in the unmodified, or slightly adjusted implementations, can be easily applied to these problems as well, and this represents the clear contribution of this paper for this research domain.Also, we should emphasize that the TGA was not explored well, since only two papers that deal with this approach exist in the literature.

Relationship to Technological Innovation for Industrial and Service Systems
Currently, the 4th industrial revolution is taking place.The idea of this new revolution is represented by the terms Industry 4.0, Smart Manufacturing and Economy 4.0.During this revolution, many innovations of industrial and service systems will take place.Cloud computing, Internet of things, WSNs, and the integration of "exponential technologies" play an important role in these innovations.Also, during these innovations, increasing digitalization and interconnection of systems, products, value chains, and business models is needed.However, in order for innovations to be successfully implemented, many problems should be solved.As already, stated in the Section 1, most of such problems can be mathematically modelled and such they belong to the category of NP hard optimization.Since the swarm intelligence methods have proved to be efficient in solving NP hard problems, these methods can be applied for tackling NP hard problems when innovating industrial and service systems.
For example, cloud computing may be the production platform for technological innovations for industrial and service systems.According to the literature review, load balancing [19], as well as scheduling problems [20] in cloud computing environments were successfully tackled by using swarm intelligence metaheuristics.Swarm intelligence has also been successfully applied in numerous optimizations in telecommunications and routing systems [4].

Basic and Enhanced Tree Growth Algorithm
TGA is novel swarm intelligence metaheuristics developed in 2017 by Armin Cheraghalipour and Mostafa Hajiaghaei-Keshteli [17].TGA models the competition between the trees in the nature for acquiring light and food sources.
Inspired by the nature, one iteration of the TGA is divided into four phases.In the first phase, N1 better solutions (individuals) in the population perform the local search process.In the second phase, N2 solutions are moved to the distance between the close best solutions under different α angles.Third phase is conducted by discarding N3 worst solutions from the population, which are replaced by randomly generated solution from the feasible domain of the search space.Finally, in the fourth phase, N4 new solutions are generated, when each newly created solution is modified by the mask operator respect to the best solution in the set N1 that consists of best solutions in the population.
One execution (run) of the TGA can be summarized in the eight following steps: Step 1: Generate random initial population of N candidate solutions within the values of lower and upper parameters' bound and calculate fitness of each solution.In the case of minimization problems, the fitness is reverse proportional to the value of objective function.
Step 2: Sort population according to the value of fitness and find the current best solution T j GB, where j-th denotes the current iteration.
Step 3: For N1 better solutions in the population perform the local search for all solutions' parameters according to the Eq. ( 1) [17].
where    and   +1 denote the i-th solution in the population in the iterations j (old solution i) and j+1 (new solution i), respectively,  is the reproduction rate parameter, and r is uniformly distributed pseudo-random number between 0 and 1.When the new solution is generated, the selection between old and new solution is performed using greedy mechanism (the solution with the higher fitness value is retained).
Step 4: N2 solutions from the population are moved towards the two closest better solutions from the subpopulation N1 under different α angles.To find the closest best solutions, the following equation is applied for all solutions' parameters [17]: When the distance is calculated, then two solutions x1 and x2 with the minimal distance (di) from each solution   2  are selected for producing linear combination by using the equation [17]: where  represents the TGA control parameter and its value is between 0 and 1.Finally, all solutions   2  from the subpopulation N2 are moved between two adjacent solutions with α different angles by employing Eq. ( 4) [17].
Step 5: The N3 worst solutions are discarded from the population and they are replaced with the randomly generated solutions from the feasible region of the search space.
Step 6: New population N is created, where N= N1 + N2 + N3 Step 7: In this step, new set of N4 randomly distributed solutions are created and each solution is modified by using the mask operator respect to the best solution from the subpopulation N1.Solutions from the N4 subpopulation are then added to the population N.
Step 8: Population N+ N4 is sorted according to the fitness and the best N solutions are chosen as the initial population for the next iteration of the algorithm.In this step, tournament or roulette wheel selection process is employed.
In the presented steps, N, N1, N2, N4,  and  are the control parameters of the TGA.

Dynamic Tree Growth Algorithm
Two basic processes of any swarm intelligence algorithm, that guide the search, are the exploitation (intensification) and exploration (diversification).To obtain the satisfactory results, the balance between these two processes should be adjusted well.
If the balance between exploration and exploitation in not properly balanced in favor to exploitation, the algorithm may converge to suboptimal solutions in early iterations, and may be trapped in the local optimum, which is called the premature convergence.At the other hand, if this balance is in favor of exploration, it can happen that the algorithm may find the right part of the search space, but may not be able to fine tune around the current best solutions.As a consequence, the algorithm cannot find the optimum solution.
The exploration and exploitation in the original TGA are controlled by two control parameters  and , and the number of solutions N3, which are discarded from the population in each iteration.The  parameter directs the local search process, while the  parameter directs solutions towards the better solutions in the population.Both parameters conduct the process of exploitation.At the other hand, by adjusting N3, the process of exploration is conducted.All three parameters are static during the whole course of algorithm's execution.
By conducting empirical tests, we concluded that the TGA performance can be improved by dynamically adjusting the values of  and N3.In early iterations of algorithm's execution, the value of the  parameter should be lower, and the newly generated solution should be further from the current solution (Eq. ( 1)).Basic assumption is that the algorithm in early iterations has not found the right part of the search space.However, in later iterations, the value of the  parameter should be higher, with the assumption that the algorithm has found the proper part of the search space.The value of the parameter  should be static, since the nature of Eq. (3).
Also, the number of solutions that are discarded from the population (parameter N3) should be higher in early iterations, since in this phase more exploration power is needed.The change of N3 parameter is directly influenced by the values of N1 and N2.
By incorporating dynamic behavior of  and N3, we devised dynamic search TGA (dynsTGA) metaheuristics.Pseudo-code of dynsTGA is shown in Algorithm 1.

Algorithm 1 Pseudo-code of the dynsTGA metaheuristics
Initialization.Generate pseudo-random population, set the iteration counter t=1, maximum iteration number MaxIter, and initial values for  and N3 control parameters while t < MaxIter do Evaluate population and sort all solutions according to their fitness for all solutions in N1 Perform local search by using Eq. ( 1) Apply greedy selection between old and new solution end for for all solutions in N2 Algorithm was tested in 30 independent runs.We performed comparative analysis with outstanding stochastic algorithms from the literature that were tested on the same benchmark set, and compared obtained values for best, mean and standard deviation performance indicators.Comparative analysis is presented in Table 2, where the best results for each performance indicator and each benchmark test are bolded.
First thing that we want to emphasize is that from the Table 2, it is clear that the dynsTGA outperforms original TGA metaheuristics.For example, in F3 (BL Problem) benchmark, dynsTGA showed better performance for best, mean, as well as for standard deviation indicator.Similar, in the case of F8 (GRP) test, dynsTGA showed significantly better robustness than the basic TGA.The improvements over unmodified TGA are also significant in the F6 test, where dynsTGA obtained better values.
As can be seen from Table 2, comparative analysis with IBPA [17], LADA [17], TS [17] and WSA [17] was performed.In average, dynsTGA approach performs better than all other metaheuristics that are included in comparative analysis.The superiority of dynsTGA can be best pictured in the F3 (BL Problem) benchmark, where dynsTGA completely outperformed all other approaches for best, mean and standard deviation indicators.

Conclusion
In this paper, a modified and improved version of relatively new tree growth algorithm (TGA) metaheuristics was presented.Only few papers from the literature deal with the TGA approach.Original TGA was enhanced by introducing dynamical adjustment of exploitation and exploration search parameters.The performance of enhanced TGA metaheuristics was measured on standard set of global optimization benchmarks.In the experiments, similar parameter setup as in [17] was applied.To prove the robustness and solutions' quality of the dynamic TGA, comparative analysis with the basic TGA, as well as with other state-of-the-art algorithms for global optimization was conducted.From the presented side-by-side comparison, it is obvious that the dynamic TGA significantly improves original TGA and performs better than other algorithms that were included in analysis.
Since many problems from the domains of industrial and service systems can be modeled as global optimization tasks, dynamic TGA metaheuristics shows great potential in this area and can be adapted for tackling many real-world unconstrained and constrained optimization challenges.

Table 2 .
Simulation results for performance indicators and comparative analysis.