Genetic Search of Pickup and Delivery Problem Solutions for Self-driving Taxi Routing

. Self-driving cars belong to rapidly growing domain of cyber-physical systems with many open problems. In this paper, we study routing problem for taxies. In mathematical terms, it is well-known Pickup and Delivery problem (PDP). We use with the standard small-moves technique, which is to apply small changes to a solution for PDP in order to obtain a better one; and an approach that works with small-moves as mutations in genetic algorithms. We propose a strategy-based framework for managing set of small changes and suggest diﬀerent strategies. We tested algorithms for routing on real-world dataset on taxi orders to airports in United Kingdom. The results show that algorithms using mixed strategies outperform algorithms using a single small move.


Introduction
Usage of cyber-physical systems (CPS) such as medical devices, industrial robots or smart grid, showed extensive growth in the recent years [1]. It leads to high demand on algorithms for CPSs improving their performance and interaction with environment. Control systems of this type are thought to show high autonomy and to be able to solve typical tasks met in the domain of their application.
Self-driving cars (SDC), also known as autonomous cars, are one of the CPSs with rapidly growing importance: SDCs share market with ordinary (humandriving) vehicles, and potential effect of its introduction is often referred as a revolution [9,21]. Consequences include improvements in traffic, vehicle usage and even implementation of new social strategy, so-called mobility as a service [25,5]. List of organizations racing for SDC includes not only well-known vehicle manufacturing companies, but also Google, Uber [15] and Baidu [10].
One of the possible applications of SDC is a taxi service in airports. Airports are usually a very intensive hub for taxi routing, due to many people use taxis to get to an airport or get to a city after arrival. It explains why many airports all over the world have their own taxi services or at least taxi services accredited to be official for this airport. This policy shows several advantages. First of all, airport-associated taxis are less affected by the traffic situation in the city.
Second, tariffs for taxi are usually fixed or can be simpler adjusted to the current demand, since regular taxi services have to take into account all the demands for mobility across the city.
The last problems, namely routing, is actual for taxis. In its mathematical statement, this problem is known as Pickup and delivery problem (PDP) [16]. PDP is an optimization problem, relative to the most known travelling salesman problem (TSP). Both of them belong to the vehicle routing problem class, determining the optimal solution for problems of this class is known to be NPhard [22]. The TSP can be considered as a case of the PDP, since one is required not only to visit each point, but also pickup items in several points and deliver them to other points [27]. A lot of constraints can be added to the general case of PDP [28]. The taxi routing problem may be understood as last-in-first-out single vehicle pickup and delivery problem (SVPDPL): 1) we assume that we need to build a route for a single vehicle; 2) it cannot pick another item before it had delivered current one [8]. SVPDPL is known to be NP-hard problem, thus, a number of heuristics for solving it are suggested.
In this paper, we use variable neighborhood search (VNS) approach [4] for solution search that constructs a new solution by applying local changes (small moves) to an already found one. We use the framework suggested in [30] involving usage of strategies.
The remainder of the paper is organized as follows. In Section 2, we define SVPDPL, its solution via local changes and genetic algorithms. In Section 3, we describe a concepts of strategy and different strategies we use in this paper. In Section 4, we describe experiment setups and the results of algorithm comparison. Section 5 concludes.
2 Problem statement and small moves

Pickup and delivery problem statement
We will follow the notation from [7] and [30]. The Pickup and delivery problem is represented with a weighted graph G = (V, E, W ), where V is vertex set corresponding to the points, E is edge set corresponding to the routes connecting points, and W is a weight function that is defined on V and E. It corresponds to loads on vertices and cost of travelling for edges. In this paper, we use W representing the taxi routing problem: -W (v) equals 1 if v is a vertex with a person to be picked up (source), and −1 if v is a vertex, to which a person should be delivered (destination); -W ({u, v}) is a distance between the points corresponding to u and v.
Let P denote the source set and D denote the destination set, then V is splited into pairs (P, We enumerate all the vertices: a source vertex p i receives index i, and the corresponding destination vertex d i receives index i+n. An edge (i, j) can belong to E if 1) j = i+n; or 2) i > n and j ≤ n, i = n+j. We need to find a Hamiltonian route, which minimizes its cost:

Small moves
Since the search of an exact solution is NP-hard, an heuristic known as "small moves" was suggested [4]. The main idea is to apply small changes to a found solution in order to obtain a new, better one. Today it is state-of-the art approach. In this work we use the following five types of the most commonly used small moves.
Couple-exchange selects two orders (i, n + i) and (j, n + j), then it replaces i with j and n + i with n + j. Our illustrated example is presented in web 3 .
Point-exchange is analogous to the couple-exchange. The only difference is that not the pairs, but single points are exchanged.
Relocate-block (RB) inserts a sequence (i t , . . . , i t+n ) into the best possible place in the route. Our illustrated example is presented in web 4 .

Applying genetic algorithms
The small moves described in the previous subsection can be easily understood as mutation operations, which can be applied to a solution in order to obtain a new, better one. This idea was realized in [30] for applying genetic algorithms for the solution search.
The application of genetic algorithms requires the following problem formalization: spices are solutions of the SVPDPL, and fitness function is the quality of such a solution (the length of the route, which should be minimized). Mutation operation is the application of a small move. Each generation consists of K solutions, each solution gives a birth to n new solution by allying a mutation operation, then some of them are chosen via the selection operation to form the next generation. Totally, there are g generations.
Let N be equal to 50, which is the number of vertices in the graph. In [30], five different genetic algorithms were tested, we will use all of them.

Strategies
The question arises, which small moves can and should be used as mutation operations. The simplest way is to apply a small move of a certain type -this is a common practice. However, as it was shown in [30], applying different small moves is more beneficial. In that paper, one group of the genetic algorithms was allowed to pick mutation on each step randomly from the entire small move set (with the uniform distribution).
In this paper, we generalize such an approach with strategy concept. Assume we are given a set M = {m 1 , . . . , m M } of small moves. Then let strategy S = {q 1 , . . . , q M } be a vector of probabilities, with which each of small move should be applied on the next step.
The second group of strategies is formed according to the following empirics: 1) CE and PE perform similarly, this is why we use only one of them; and 2) according to the results in [30], as well as this study, L2O seems to be the strongest algorithm. This is why it makes sense to consider strategies, which use L20 with combination of other small moves. Thus, we add five new strategies for comparison: S mix−PE = (1/4, 1/4, 0, 1/4, 1/4). The last group of strategies we discuss in this paper are based on smallmoves properties. We run experiments on 150 different subsets and collected the following statistical evaluations: n j is the number of times, when ith small move is applied; f i is frequency of solution improvement by applying ith small move, and c i is frequency of solution improvement by applying ith small move ignoring cases, when it violates the problem constraints. Also let n i denote the computational complexity of the small move application. Thus: (1, 1, 1, 1, 1/N

Experiment setup and test data
VeeRoute company 5 kindly gave us dataset contacting information on 8,000 orders for taxies to transfer people from airports or to airports in the United Kingdom in March 2015. We used airport names and addresses to obtain geographical coordinates of the corresponding points using webservice mapquest 6 . Then we used the Mercator projection implementation library in library JMapProjLib 7 to transform them to 2D coordinates on a plane. Thus, we transformed the datasets into the dataset of two sets of points. Then we generated small datasets containing 50 random addresses as source and an airport as destination.
Each algorithm was run 10 times on 5 different subsets. Each experiment was held 10 times. We estimate optimization ratio (OR), which represents, how much we have decreased the length of the route in comparison with the original one: where d 0 is the length of the initial solution, and d 1 is the achieved length. We tested algorithms on g = 12 · N 2 generations.

Strategies comparison
We compared all the strategies, presented in the previous Section, for each of genetic scheme, presented in Section 2. Results are presented in Table 1.
As we can see, 1+1 genetic scheme outperforms all the other genetic schemes in general. Strategies of the second type outperform corresponding strategies of the first type, with the only exception for the mixed strategy. However, statistic strategies show the best performance. Usage of small-moves complexity leads to the two best results. This is can simply be explained by the fact, that implying a small move (RB) consumes time N greater, than any other small move, thus, applying such a normalization makes the mixed strategy more fair. However, it has no such a big improvement in other genetic schemes. When a more complex scheme is used, this difference becomes not so important. In these cases, statistic considerations have a higher impact. We can see, for the most of other schemes, S F , S C are the best, due to the breed is bigger, than one, and then unsuccessful application of RB will not be so crustal for reaching the optima.

Conclusion and future work
In this paper, we have suggested a novel solution for self-driving car routing, which is based on applying strategies: small-moves with assigned probabilities, which changes a found solution. We have suggested and analyzed several strategies with several genetic schemes for generating them. We have shown that small-moves ensembling is a very profitable step for results improvement. Also we have shown that the strategies that are based on the previous experience are very powerful. Another strong improvement was to connect probabilities of applying a small move with its computational complexity (time consumed).
We see several directions for future work. First, it may be useful to apply reinforcement learning in order to choose, which small move to apply next. Second, we plan to use meta-learning to predict the initial strategy and genetic algorithm scheme. Finally, it seems to be useful to choose the size of breed dynamically on each step (decreasing it), since strategies with big breed where good in the beginning of the breed.