Even Faster Elastic-Degenerate String Matching via Fast Matrix Multiplication

An elastic-degenerate (ED) string is a sequence of n sets of strings of total length N , which was recently proposed to model a set of similar sequences. The ED string matching (EDSM) problem is to ﬁnd all occurrences of a pattern of length m in an ED text. The EDSM problem has recently received some attention in the combinatorial pattern matching community, and an O ( nm 1 . 5 √ log m + N )-time algorithm is known [Aoyama et al., CPM 2018]. The standard assumption in the prior work on this question is that N is substantially larger than both n and m , and thus we would like to have a linear dependency on the former. Under this assumption, the natural open problem is whether we can decrease the 1.5 exponent in the time complexity, similarly as in the related (but, to the best of our knowledge, not equivalent) word break problem [Backurs and Indyk, FOCS 2016]. Our starting point is a conditional lower bound for the EDSM problem. We use the popular combinatorial Boolean matrix multiplication (BMM) conjecture stating that there is no truly subcubic combinatorial algorithm for BMM [Abboud and Williams, FOCS 2014]. By designing an appropriate reduction we show that a combinatorial algorithm solving the EDSM problem in O ( nm 1 . 5 − (cid:15) + N ) time, for any (cid:15) > 0, refutes this conjecture. Of course, the notion of combinatorial algorithms is not clearly deﬁned, so our reduction should be understood as an indication that decreasing the exponent requires fast matrix multiplication. Two standard tools used in algorithms on strings are string periodicity and fast Fourier transform. Our main technical contribution is that we successfully combine these tools with fast matrix multiplication to design a non-combinatorial O ( nm 1 . 381 + N )-time algorithm for EDSM. To the best of our knowledge, we are the ﬁrst to do so.


Introduction
Boolean matrix multiplication (BMM) is one of the most fundamental computational problems.Apart from its theoretical interest, it has a wide range of applications [37,52,29,27,46].BMM is also the core combinatorial part of integer matrix multiplication.In both problems, we are given two N × N matrices and we are to compute N 2 values.Integer matrix multiplication can be performed in truly subcubic time, i.e., in O(N 3− ) operations over the field, for some > 0. The fastest known algorithms for this problem run in O(N 2.373 ) time [30,54].These algorithms are known as algebraic: they rely on the underlying ring structure.
There also exists a different family of algorithms for the BMM problem known as combinatorial.Their focus is on unveiling the combinatorial structure in the Boolean matrices to reduce redundant computations.A series of results [7,9,15] culminating in an Ô(N 3 / log 4 N )-time algorithm [58] (the Ô(•) notation suppresses poly(log log) factors) has led to the popular combinatorial BMM conjecture stating that there is no combinatorial algorithm for BMM working in time O(N 3− ), for any > 0 [2].There has been ample work on applying this conjecture to obtain BMM hardness results: see, e.g., [44,2,49,33,43,42,17].
String matching is another fundamental problem.The problem is to find all fragments of a string text of length n that match a string pattern of length m.This problem has several linear-time solutions [22].In many real-world applications, it is often the case that letters at some positions are either unknown or uncertain.A way of representing these positions is with a subset of the alphabet Σ.Such a representation is called degenerate string.The first efficient algorithm for a degenerate text and a standard pattern was published by Fischer and Paterson in 1974 [28].It has undergone several improvements since then [36,39,20,19].The first efficient algorithm for a degenerate pattern and a standard text was published by Abrahamson in 1987 [3], followed by several practically efficient algorithms [57,47,34].
Degenerate letters are used in the IUPAC notation [38] to represent a position in a DNA sequence that can have multiple possible alternatives.These are used to encode the consensus of a population of sequences [21,4] in a multiple sequence alignment (MSA).In the presence of insertions or deletions in the MSA, we may need to consider alternative representations.Consider the following MSA of three closely-related sequences (on the left): These sequences can be compacted into a single sequence T of sets of strings (on the right) containing some deterministic and some non-deterministic segments.A non-deterministic segment is a finite set of deterministic strings and may contain the empty string ε corresponding to a deletion.The total number of segments is the length of T and the total number of letters is the size of T .We denote the length by n = | T | and the size by N = || T ||.This representation has been defined in [35] by Iliopoulos et al. as an elastic-degenerate (ED) string.Being a sequence of subsets of Σ * , it can be seen as a generalization of a degenerate string.The natural problem that arises is finding all matches of a deterministic pattern P in an ED text T .This is the elastic-degenerate string matching (EDSM) problem.Since its introduction in 2017 [35], it has attracted some attention in the combinatorial pattern matching community, and a series of results have been published.The simple algorithm by Iliopoulos et al. [35] for EDSM was first improved by Grossi et al. in the same year, who showed that, for a pattern of length m, the EDSM problem can be solved on-line in O(nm 2 + N ) time [32]; on-line means that the text is read segment-by-segment and an occurrence is detected as soon as possible.This result was improved by Aoyama et al. [6] who presented an O(nm 1.5 √ log m + N )-time algorithm.An important feature of these bounds is their linear dependency on N .A different branch of on-line algorithms waiving the linear-dependency restriction exists [32,48,18].Moreover, the EDSM problem has been considered under Hamming and edit distance [12].
A question with a somewhat similar flavor is the word break problem.We are given a dictionary D, m = ||D||, and a string S, n = |S|, and the question is whether we can split S into fragments that appear in D (the same element of D can be used multiple times).Backurs and Indyk [8] designed an Õ(nm 1/2−1/18 +m)-time algorithm for this problem (the Õ notation suppresses poly(log) factors).Bringmann et al. [14] improved this to Õ(nm 1/3 + m) and showed that this is optimal for combinatorial algorithms by a reduction from k-Clique.
Their algorithm uses fast Fourier transform (FFT), and so it is not clear whether it should be considered combinatorial.While this problem seems similar to EDSM, there does not seem to be a direct reduction and so their lower bound does not immediately apply.
Our Results.It is known that BMM and triangle detection in graphs either both have truly subcubic combinatorial algorithms or none of them do [56].Recall also that the currently fastest algorithm with linear dependency on N for the EDSM problem runs in O(nm 1.5 √ log m + N ) time [6].In this paper we prove the following two theorems.

Theorem 1.
If the EDSM problem can be solved in O(nm 1.5− + N ) time, for any > 0, with a combinatorial algorithm, then there exists a truly subcubic combinatorial algorithm for triangle detection.
Arguably, the notion of combinatorial algorithms is not clearly defined, and Theorem 1 should be understood as an indication that in order to achieve a better complexity one should use fast matrix multiplication.Indeed, there are examples where a lower bound conditioned on BMM was helpful in constructing efficient algorithms using fast matrix multiplication [1,16,13,45,24,55,59].We successfully design such a non-combinatorial algorithm by combining three ingredients: a string periodicity argument, FFT, and fast matrix multiplication.While periodicity is the usual tool in combinatorial pattern matching [40,23,41] and using FFT is also not unusual (for example, it often shows up in approximate string matching [3,5,19,31]), to the best of our knowledge, we are the first to combine these with fast matrix multiplication.Specifically, we show the following result for the EDSM problem.

Theorem 2.
The EDSM problem can be solved on-line in expected O(nm 1.381 + N ) time.
An important building block in our solution that might find applications in other problems is a method of selecting a small set of length-substrings of the pattern, called anchors, so that any relevant occurrence of a string from an ED text set contains at least one but not too many such anchors inside.This is obtained by rephrasing the question in a graph-theoretical language and then generalizing the well-known fact that an instance of the hitting set problem with m sets over [n], each of size at least k, has a solution of size O(n/k • log m).While the idea of carefully selecting some substrings of the same length is not new, for example Kociumaka et al. [41] used it to design a data structure for pattern matching queries on a string, our setting is different and hence so is the method of selecting these substrings.
I C A L P 2 0 1 9

21:4
Even Faster Elastic-Degenerate String Matching Roadmap.Section 2 provides the necessary definitions and notation as well the algorithmic toolbox used throughout the paper.In Section 3 we prove our hardness result for the EDSM problem (Theorem 1).In Section 4 we present our algorithm for the same problem (Theorem 2); this is the most technically involved part of the paper.

Preliminaries
Let T = T

Lemma 3 ([26]
).If p and q are both periods of the same string X, and additionally p + q ≤ |X| + 1, then gcd(p, q) is also a period of X.
A trie is a rooted tree in which every edge is labeled with a single letter, and every two edges outgoing from the same node have different labels.The label of a node u in such a tree T , denoted by L(u), is defined as the concatenation of the labels of all the edges on the path from the root of T to u.Thus, the label of the root of T is ε, and a trie is a representation of a set of strings consisting of the labels of all its leaves.By replacing each path p consisting of nodes with exactly one child by an edge labeled by the concatenation of the labels of the edges of p we obtain a compact trie.The nodes of the trie that are removed after this transformation are called implicit, while the remaining ones are referred to as explicit.The suffix tree of a string S is the compact trie representing all suffixes of S$, $ / ∈ Σ, where instead of explicitly storing the label S[i . .j] of an edge we represent it by a pair (i, j).
A heavy path decomposition of a tree T is obtained by selecting, for every non-leaf node u ∈ T , its child v such that the subtree rooted at v is the largest.This decomposes the nodes of T into node-disjoint paths, with each such path p (called a heavy path) starting at some node, called the head of p, and ending at a leaf.An important property of such a decomposition is that the number of distinct heavy paths above any leaf (that is, intersecting the path from a leaf to the root) is only logarithmic in the size of T [51].
Let Σ denote the set of all finite non-empty subsets of Σ * .Previous works (cf.[35,32,12,6,48]) define Σ as the set of all finite non-empty subsets of Σ * excluding {ε} but we waive here the latter restriction as it has no algorithmic implications.An elastic-degenerate string, or ED string, over alphabet Σ, is a string over Σ, i.e., an ED string is an element of Σ * .
Let T denote an ED string of length n, i.e. | T | = n.We assume that for any 1 ≤ i ≤ n, the set T [i] is implemented as an array and can be accessed by an index, i.e., For any ED string X and a pattern P , we say that P matches X if 1. | X| = 1 and P is a substring of some string in X [1], or, 2. | X| > 1 and P = P 1 . . .P | X| , where P 1 is a suffix of some string in X [1], P | X| is a prefix of some string in X[| X|], and We say that an occurrence of a string P ends at position j of an ED string T if there exists i ≤ j such that P matches T [i] . . .T [j].We will refer to string P as the pattern and to ED string T as the text.We define the main problem considered in this paper.
Aoyama et al. [6] obtained an on-line O(nm 1.5 √ log m + N )-time algorithm by designing an efficient solution for the following problem.

Active Prefixes (AP)
INPUT: A string P of length m, a bit vector U of size m, a set S of strings of total length N .OUTPUT: In more detail, given an ED text one should consider an instance of the AP problem per segment.Hence, an O(f (m) + N i ) solution for AP (with N i being the size of the i-th segment of the ED text) implies an O(n • f (m) + N ) solution for EDSM, as N = n i=1 N i .We provide an example of the AP problem.
For our hardness results we rely on BMM and the following closely related problem.

Boolean Matrix Multiplication (BMM)
An algorithm is called truly subcubic if it runs in O(N 3− ) time, for some > 0. TD and BMM either both have truly subcubic combinatorial algorithms, or none of them do [56].

EDSM Conditional Lower Bound
We show a conditional lower bound for the EDSM problem.Specifically, we show that TD can be reduced to the EDSM problem.We work with the decision version: the goal is to detect whether there exists at least one occurrence of P in T .
I C A L P 2 0 1 9

21:6
Even Faster Elastic-Degenerate String Matching Theorem 1.If the EDSM problem can be solved in O(nm 1.5− + N ) time, for any > 0, with a combinatorial algorithm, then there exists a truly subcubic combinatorial algorithm for triangle detection.
Proof.Consider an instance of TD, where we are given three N ×N Boolean matrices A, B, C, and the question is to check if there exist i, j, k such that Let s be a parameter to be determined later that corresponds to decomposing B into blocks of size (N /s) × (N /s).We reduce to an instance of EDSM over an alphabet Σ of size O(N ).
Pattern P .We construct P by concatenating, in some fixed order, the following strings: ED text T .The text T consists of three parts.Its middle part encodes all the entries equal to 1 in matrices A, B and C, and consists of three string sets X =X 1 • X 2 • X 3 , where: 1. X 1 contains all strings of the form v(i)xa j , for some i = 1, 2, . . ., N , x = 1, 2, . . ., s and It is easy to see that |P (i, x, y)| = O(N /s).This implies the following: By the above construction, we obtain the following fact.Fact 6. P (i, x, y) matches X if and only if the following holds for some j, k = 1, 2, . . ., N /s: Solving the TD problem thus reduces to taking the disjunction of all such conditions.Let us write down all strings P (i, x, y) in some arbitrary but fixed order to obtain P = P 1 P 2 . . .P z with z = N s2 , where every P t = P (i, x, y), for some i, x, y.We aim to construct a small number of sets of strings that, when considered as an ED text, match any prefix P 1 P 2 . . .P t of the pattern, 1 ≤ t ≤ z − 1; a similar construction can be carried on to obtain sets of strings that match any suffix P k . . .P z−1 P z , 2 ≤ k ≤ z.These sets will then be added to the left and to the right of X , respectively, to obtain the ED text T .ED Prefix.We construct log z sets of strings as follows.The first one contains the empty string ε and P 1 P 2 . . .P z/2 .The second one contains ε, P 1 P 2 . . .P z/4 and P z/2+1 . . .P z/2+z/4 .The third one contains ε, P 1 P 2 . . .P z/8 , P z/4+1 . . .P z/4+z/8 , P z/2+1 . . .P z/2+z/8 and P z/2+z/4+1 . . .P z/2+z/4+z/8 .Formally, for every i = 1, 2, . . ., log z, the i-th of such sets is: ED Suffix.We similarly construct log z sets to be appended to X : The total length of all the ED prefix and ED suffix strings is The whole ED text T is: Lemma 7. The pattern P occurs in the ED text T if and only if there exist i, j, k such that Proof.By Fact 6, if such i, j, k exist then P t matches X , for some t ∈ {1, . . ., z}.Then, by construction of the sets T p i and T s i , the prefix P 1 . . .P t−1 matches the ED prefix (this can be proved by induction), and similarly the suffix P t+1 . . .P z matches the ED suffix, so the whole P matches T , and so P occurs in T .Because of the letters $ appearing only in the center of P i s and strings from X 2 , every P i and a concatenation of having the same length, and the P i s being distinct, there is an occurrence of the pattern P in T if and only if X 1 X 2 X 3 = P t for some t and X 1 ∈ X 1 , X 2 ∈ X 2 , X 3 ∈ X 3 .But then, by Fact 6 there exists a triangle.

Note that for the EDSM problem we have
time, for some > 0, by choosing a sufficiently small α > 0 and setting s = N α we would obtain an O(N 3−δ )-time algorithm for TD, for some δ > 0. We partition the strings in S into three types, compute the corresponding bit vector V for each type separately and in different ways, and, finally, take the disjunction to obtain the answer for the restricted instance.
Partitioning S. Let = 8/9 • (10/9) k (to avoid clutter we assume that is an integer, but this can be avoided by appropriately adjusting the constants), so that the length of every string in S belongs to [9/8 • , 5/4 • ).The three types of strings are as follows: Type 1: Strings S ∈ S such that every length-substring of S is not strongly periodic.Type 2: Strings S ∈ S containing at least one length-substring that is not strongly periodic and at least one length-substring that is strongly periodic.Type 3: Strings S ∈ S such that every length-substring of S is strongly periodic (in Lemma 8 we show that in this case per(S) ≤ /4).These three types are evidently a partition of S and, before we proceed with the algorithm, we need to show that we can determine the type of a string S ∈ S in O(|S|) time.We start with showing that, in fact, strings of type 3 are exactly strings with period at most /4.

Type 1 Strings
In this section we show how to solve a restricted instance of the AP problem where every string S ∈ S is of type 1, that is, each of its length-substrings is not strongly periodic, and furthermore |S| ∈ [9/8 • , 5/4 • ) for some ≤ m.Observe that all (hence at most 1/4 • ) length-substrings of any S ∈ S must be distinct, as otherwise we would be able to find two occurrences of a length-substring at distance at most 1/4 • in S, making the period of the substring at most 1/4 • and contradicting the assumption that S is of type 1.
We start with constructing the suffix tree ST of P (our pattern in the EDSM problem) in O(m log m) time [53] (note that we are spending O(m log m) time and not just O(m) as to avoid any assumptions on the alphabet).For every explicit node u ∈ ST , we construct a perfect hash function mapping the first letter on every edge outgoing from u to the corresponding edge.This takes O(m log m) time [50] and allows us to navigate in ST in constant time per letter.Then, for every S ∈ S we check if it occurs in P using the suffix tree in O(|S|) time, and if not disregard it from further consideration.We want to further partition S into S 0 , S 1 , . . ., S log m that are processed separately.For every S k , we want to select a set of length-substrings of P , called the anchors, each represented by one of its occurrences in P , such that: 1.The total number of occurrences of all anchors in P is O(m/ • log 2 m). 2. For every S ∈ S k , at least one of its length-substrings is an anchor.3.For every S ∈ S k , at most O(log 2 m) of its length-substrings are anchors.We formalize this using the following auxiliary problem, which is a strengthening of the hitting set problem: for any collection of m sets over [n], each of size at least k, we can choose a subset of To reduce finding anchors to an instance of the NS problem, we first build a bipartite graph G in which the nodes on the left correspond to strings S ∈ S, the nodes on the right correspond to distinct length-substrings of P , and there is an edge connecting a node corresponding to a length-string H with a node corresponding to a string S when H occurs in S. Using suffix links, we can find the node of the suffix tree corresponding to every length-substring of S in O(|S|) total time, so the whole construction takes O(m log m + S∈S |S|) = O(m log m + N ) time.The size of G is O(m + N ), and the degree of every node on its left belongs to [1/8 • , 1/4 • ).We further partition G into a logarithmic number of graphs G 0 , G 1 , . . ., G log m where G k contains all nodes v on the right of G such that the number of occurrences in P of the corresponding length-string belongs to [2 k , 2 k+1 ).For every node u on the left of G we find k such that at least 1/8 • / log m of its neighbors exist in G k , add u as a node on the left of G k , and declare S k to consist of all strings S ∈ S corresponding to nodes on the left of G k .By construction, every S ∈ S corresponds to a node on the left of exactly one G k , so we indeed obtain a partition of S. For every S k we solve the corresponding instance of the NS problem to obtain its corresponding set of anchors.We can assume that all strings in S k are distinct, so there are at most m 2 nodes on the left of G k , the degree of each such node belongs to [1/8 • / log m, 1/4 • ] and, denoting by m k the total number of occurrences in P of strings corresponding to nodes on the right of G k , we have k m k ≤ m and there are at most m k /2 k nodes on the right of G k .At most O((m k /2 k )/( / log m) • log m) nodes on the right of G k are designated as anchors, making the total number of occurrences of all anchors O(m/ • log 2 m).Also, every S ∈ S k contains an occurrence of at least one anchor, and no more than O(log 2 m) such occurrences.
It is not immediately clear that an instance of the NS problem always has a solution.We show that indeed it does, and that it can be efficiently found with a Las Vegas algorithm.Lemma 10.A solution to an instance of the NS problem always exists and can be found in expected linear time.
Proof.We independently choose each node of V with probability p to obtain the set X of selected nodes.Then, we check if the size of X is small enough, every node in U has at least one selected neighbor, and O(α • log |U |) such selected neighbors.All these checks can be made in linear time in the size of the graph, so to show that a solution exists and can be found in expected linear time, it remains to show that we can adjust p to make the probability of failure equal to a constant less than 1.The expected size of X is obviously p|V |, so by Markov's inequality the probability that |X| > 4p|V | is at most 1/4.The probability that a node in U has no neighbors in X is at most (1 − p) d .Thus, by union bound the probability that there exists at least one such node is at most Its expected number of selected neighbors is pd .Thus, by Chernoff's inequality the probability that its number of selected neighbors exceeds (1 + δ)pd is at most e − δ 2 2+δ pd .By setting δ = 1, we obtain that the probability of the number of selected neighbors exceeding 2pαd is at most e − In the rest of this section we explain how to compute the bit vector V from the bit vector U after having obtained a set A of anchors for a set of strings S k of total length N k .For any S ∈ S k , since S contains an occurrence of at least one anchor H ∈ A, for concreteness S[j . .(j + |H| − 1)] = H, any occurrence of S in P can be generated by choosing some occurrence of H in P , say P [i . .In such case, we say that the occurrence of S in P is generated by H.By the properties of A, any occurrence of S ∈ S k is generated by at least one but no more than O(log 2 m) anchors.For every H ∈ A we create a separate data structure We separately describe what information is used to initialize each D(H) and how it is later processed to update V .We first construct the suffix tree ST of P and the suffix tree ST r of P r in O(m log m) time.We augment both trees with a structure for answering weighted ancestor (WA) and lowest common ancestor (LCA) queries that are defined as follows.For a rooted tree T on n nodes with an integer weight D(v) assigned to every node u, such that the weight of the root is zero and D(u) < D(v) if u is the parent of v, we say that a node v is a weighted ancestor of a node v at depth , denoted by WA T (u, ), if v is the highest ancestor of u with weight of at least .Such queries can be answered in O(log n) time after an O(n)-time preprocessing [25].For a rooted tree T , LCA T (u, v) is the lowest node that is an ancestor of both u and v.Such queries can be answered in O(1) time after an O(n)-time preprocessing [10].Recall that every anchor H is represented by one of its occurrences in P .Using WA queries, we can access in O(log m) time the nodes corresponding to H and H r , respectively, and extract a lexicographically sorted list of suffixes following an occurrence of H in P $ and a lexicographically sorted list of reversed prefixes preceding an occurrence of H in P r $ in time proportional to the number of such occurrences.Then, by iterating over the lexicographically sorted list of suffixes and using LCA queries on ST we can build T (H) in time proportional to the length of the list, and similarly for T r (H).To construct L(H) we start by computing, for every S ∈ S k and j = 1, . . ., |S|, the node of ST r corresponding to (S[1 . .j]) r and the node of ST corresponding to S([(j + 1) . .|S|] (the nodes might possibly be implicit).Using suffix links this takes only O(|S|) time.We also find, for every lengthsubstring S[j . .(j + − 1)] of S, an anchor H ∈ A such that S[j . .(j + − 1)] = H, if any exists.This can be done by finding the nodes (implicit or explicit) of ST that correspond to the anchors, and then scanning over all length-substrings while maintaining the node of ST corresponding to the current substring using suffix links in O(|S|) total time.After having determined that S[j . .(j + − 1)] = H we add (u, v) to L(H), where u and v are the previously found nodes of ST r and ST corresponding to (S[1 . .(j − 1)]) r and S[(j + ) . .|S|], respectively.By construction, we have the following property, also illustrated in Figure 1.Processing.The goal of processing D(H) is to efficiently process all occurrences generated by H.As a preliminary step, we decompose T r (H) and T (H) into heavy paths.Then, for every pair of leaves u ∈ T r (H) and v ∈ T (H) decorated by the same i, we consider all heavy paths above u and v. Let p = u 1 − u 2 − . . .be a heavy path above u in T r (H) and q = v 1 − v 2 − . . .be a heavy path above v in T (H), where u 1 is the head of p and v 1 is the 21:11

Initialization. D(H) consists of two compact tries T (H) and T r (H)
An occurrence of S starting at position i in P corresponds to a triple (i, L(ux), L(vy)) on some auxiliary list.
head of q, respectively.Further, choose the largest x such that u is in the subtree rooted at u x , and the largest y such that v is in the subtree rooted at v y (by the choice of p and q, u is in the subtree rooted at u 1 and v is in the subtree rooted at v 1 , so this is well-defined).We add (i, | L(u x )|, | L(v y )|) to an auxiliary list associated with the pair of heavy paths (p, q).In the rest of the processing we work with each such list separately.Notice that the overall size of all auxiliary lists, when summed up over all H ∈ A, is O(m/ • log 4 m), because there are at most log 2 m pairs of heavy paths above u and v decorated by the same i, and the total number of leaves in all trees T r (H) and T (H) is bounded by the total number of occurrences of all anchors in P , which is O(m/ • log 2 m).By Fact 11, there is an occurrence of a string S ∈ S k generated by H and starting at position i − j + 1 in P if and only if L(H) contains a pair (u, v) corresponding to S[j . .(j + |H| − 1)] = H such that, denoting by p the heavy path containing u in T r (H) and by q the heavy path containing v in T (H), the auxiliary list associated with (p, q) contains a triple (i, x, y) such that x ≥ | L(u)| and y ≥ | L(v)|.This is illustrated in Figure 2. From now on we focus on processing a single auxiliary list associated with (p, q) together with a list of pairs (u, v) such that u belongs to p and v belongs to q.
An auxiliary list can be interpreted geometrically: for every (i, x, y) we create a red point (x, y), and for every (u, v) we create a blue point (| L(u)|, | L(v)|).Then, each occurrence of S ∈ S k generated by H corresponds to a pair of points (p 1 , p 2 ) such that p 1 is red, p 2 is blue, and p 1 dominates p 2 .We further reduce this to a collection of simpler instances in which all red points already dominate all blue points.This can be done with a divide-and-conquer procedure which is essentially equivalent to constructing a 2D range tree [11].The total number of points in all obtained instances increases by a factor of O(log 2 m), making the total number of red points in all instances O(m/ • log 6 m), while the total number of blue points is O(N k / • log 4 m).There is an occurrence of a string S ∈ S k generated by H and starting at position i − j + 1 in P if and only if some simpler instance contains a red point created for some (i, x, y) and a blue point created for some (u, v) corresponding to S[j . .(j + |H| − 1)] = H.In the following we focus on processing a single simpler instance.
To process a simpler instance we need to check if U [i − j] = 1, for a red point created for some (i, x, y) and a blue point created for some (u, v) corresponding to S[j . .(j +|H|−1)] = H, and if so set V [i − j + |S|] = 1.This has a natural interpretation as an instance of BMM: we create an (5/4• )×(5/4• ) matrix M such that M [|S|−j, 5/4• +1−j] = 1 if and only if there is a blue point created for some (u, v) corresponding to S[j . .(j +|H|−1)] = H; then for every red point created for some (i, x, y) we construct a bit vector U i = U [(i − 5/4 • ) . .(i − 1)] (if i < 5/4• , we pad U i with 0s to make its length always equal to 5/4• ); calculate V i = M ×U i ; and finally set V [i + j] = 1 whenever V i [j] = 1 (and i + j ≤ m).Lemma 12. V i [k] = 1 if and only if there is a blue point created for some (u, v) corresponding to S[j . .(j + |H| − 1)] = H such that U [i − j] = 1 and k = |S| − j.

I C A L P 2 0 1 9
The total length of all vectors U i and V i is O(m log 6 m), so we can afford to extract the appropriate fragment of U and then update the appropriate fragment of V .The bottleneck is computing the matrix-vector product V i = M × U i .Naïvely, this might take O(N k / • log 4 m) time, because the total number of 1s in all matrices M is bounded by the total number of blue points.We overcome this by processing together all multiplications concerning the same matrix M .Let U i1 , U i2 , . . ., U is be all bit vectors that need to be multiplied with M , and z a parameter to be determined later.We distinguish between two cases: (i) If s < z we compute the products naïvely by iterating over all 1s in M , and the total computation time, when summed up over all such matrices M , is O(N k / • log 4 m • z); (ii) If s ≥ z we partition the bit vectors into s/z ≤ s/z + 1 groups of z (padding the last group with bit vectors containing all 0s).For every group, we create a single matrix whose columns contain all the bit vectors belonging to the group.Thus, we reduce computing all matrix-vector products M × U i to computing O(s/z) matrix-matrix products of the form M × M , where M is an (5/4 • ) × z matrix.M is not necessarily a square matrix, but we can still apply the fast matrix multiplication algorithm to compute M × M using the standard trick of decomposing the matrices into square blocks.Note that some M might be sparse, so we need to represent them as a list of 1s.Then, summing up over all matrices M and both cases, we spend O(N k / •log 4 m•z+ z ω−3 •m log 6 m) time.We would like to assume that ≥ log 4 m so that we can set z = / log 4 m.This is indeed possible, because for any t we can switch to a more naïve approach to process all strings of length at most t in O(mt 2 + N k ) time: for every possible length t ≤ t scan P with a window of length t , in every step check if the current window P [i . .(i + t − 1)] corresponds to a string S ∈ S, if so and U [i − 1] = 1 then set V [i + t − 1] = 1.After applying this with t = log 4 m in O(m log 8 m + N k ) time, we can set z = / log 4 m (so that indeed 5/4 • ≥ z as required in case s ≥ z) and the overall time complexity for all matrices M and both cases becomes O(N k + ω−2 •m log 6+4(3−ω) m).Summing up over all values of k and and taking the initialization into account we obtain O(m log 8 m+m ω−1 log 7+4(3−ω) m+N ) = O(m 1.373 +N ) total time.(We hide log O(1) m factors using the fact that ω < 2.373 [30,54]).

Wrapping Up
In the full version of the paper we design O(m 1.373 + N ) and O(m 1.381 + N )-time algorithms for an instance of the AP problem where all strings are of type 2 and 3, respectively.For type 2 strings, we follow the same ideas as for type 1 strings, except that instead of the NodeSelection problem we select the anchors by applying a periodicity-based argument.For type 3 strings, we need both fast matrix multiplication and FFT.Together with Theorem 14, this gives us Theorem 2.
For any c ∈ Σ, ||c|| denotes the total length of all strings in c, and for any ED string T , || T || denotes the total length of all strings in all T [i]s or the size of T , i.e., ||c|| = s∈c |s| and || T || = n i=1 || T [i]||.An ED string T can be thought of as a representation of the set of strings A( T ) = T [1] × . . .× T [n], where A × B = {xy | x ∈ A, y ∈ B} for any sets of strings A and B.

Example 4 .
Elastic-Degenerate String Matching (EDSM) INPUT: A string P of length m and an ED string T of length n and size N ≥ m.OUTPUT: All positions in T where at least one occurrence of P ends.Pattern P = GTAT ends at positions 2, 6, and 7 of the following text T .

4 An O(nm 1 .
381 + N )-time Algorithm for EDSM Our goal is to design a non-combinatorial O(nm 1.381 + N )-time algorithm for EDSM.It suffices to solve an instance of the AP problem in O(m 1.381 + N ) time.We further reduce the AP problem to a logarithmic number of restricted instances of the problem, in which the length of every string S ∈ S is in [(10/9) k , (10/9) k+1 ), for k = 0, . . ., log m/ log(10/9).If we solve every such instance in O(f (m) + N ) time, then we can solve the original instance in O(f (m) log m + N ) time by taking the disjunction of results.

Lemma 8 .Lemma 9 .
Let S be a string.If per(S[j . .j + − 1]) ≤ /4 for every j then per(S) ≤ /4.Given a string S we can determine its type in O(|S|) time.
[n] of size O(n/k • log m) that nontrivially intersects every set.Node Selection (NS) INPUT: A bipartite graph G = (U, V, E) with deg(u) ∈ [d, α • d] for every u ∈ U .OUTPUT: A set of O(|V |/d • log |U |) nodes from V such that every node in U has at least one selected neighbor but O(α • log |U |) such selected neighbors.

1 3
αpd .By union bound, the probability that this happens for at least one node is at most |U | • e − 1 3 αpd .We choose p = 3 ln(4|U |)/d.Then, the probability that the size of X exceeds 4p|V | = 12 ln(4|U |)/d • |V | = O( |V | d • log |U |) is at most 1/4, the probability that there exists a node in U with no selected neighbor is at most |U | • e −pd ≤ 1/4, and the probability that there exists a node in U with more than 2pαd = O(α • log |U |) selected neighbors is at most |U | • e − 1 3 αpd ≤ 1/4, thus the overall probability of failure is at most 3/4 as required.

Figure 1
Figure1 An occurrence of S starting at position i in P is generated by H: (u, v) corresponds to S[j . .(j + |H| − 1)] = H and i appears in the subtree rooted at u as well as the subtree rooted at v.

Fact 11 .
A string S ∈ S k starts at position i − j + 1 in P if and only if, for some anchor H ∈ A, L(H) contains a pair (u, v) corresponding to S[j . .(j + |H| − 1)] = H, such that the subtree of T r (H) rooted at u and that of T (H) rooted at v contain a leaf decorated with i.Note that the overall size of all lists L(H), when summed up over allH ∈ A, is O(N k / • log 2 m), because any S ∈ S k contains O(log2m) occurrences of anchors, and since each S is of length at least , there are only O(N k / ) strings in S k .

Lemma 13 .Theorem 14 .
If two N × N matrices can be multiplied in O(N ω ) time, then, for any N ≥ N , an N × N and an N × N matrix can be multiplied in O((N /N ) 2 N ω ) time.By applying Lemma 13, we can compute M × M in O( 2 z ω−2 ) time (as long as we later verify that 5/4 • ≥ z), so all products M × U i can be computed in O( 2 z ω−2 • (s/z + 1)) time.Note that this case can occur only O(m/( • z) • log 6 m) times, because all values of s sum up to O(m/ • log 6 m).This makes the total computation time, when summed up over all such matrices M , O( 2 z ω−2 • m/( • z) • log 6 m) = O( z ω−3 • m log 6 m).An instance of the AP problem where all strings are of type 1 can be solved in expected O(m 1.373 + N ) time.Proof.The total time complexity is first O(m + N ) to construct the graph G and then all graphs G k , then expected linear time to solve their corresponding instances of the NodeSelection problem, partition S into S 0 , S 1 , . . ., S log m and obtain their corresponding sets of anchors H.The time to initialize all structures D(H) is O(m log m + N k ).For every D(H), we obtain in O(m/ •log 6 m+N k / •log 4 m) time a number of simpler instances, and then construct the corresponding Boolean matrices M and bit vectors U i in additional O(m log 6 m) time.