Earth Mover’s Distance as a Comparison Metric for Analog Behavior

,


Introduction
Due to better technology scaling of digital blocks compared to analog blocks more and more parts of the analog implementation of modern IC designs are shifted to the digital domain.However, certain analog components are still indispensable, leading to mixed-signal designs.Hence, the demand for comprehensively verifying such designs is continuously growing and compared to the highly automated verication methodologies in the digital domain, pre-silicon verication in the analog domain usually implies a substantial amount of manual work and computational eort.
On analog module level, pre-silicon verication primarily targets the analysis of electrical characteristics as for instance gain, distortion or input resistance.
Although determining electrical characteristics is also targeted on chip top and system level, conrming the functional correctness and the application tness of the design by means of simulation is the main goal in this regard.The authors of [1], e. g., point out that it is still a common practice to manually inspect waveforms when performing the latter.This eectively obstructs the usage of automated checking and regression-based simulations inevitably leading to an increased error-proneness as well as vast expenditures of time.Therefore, they advise to leverage automated solutions to overcome these problems.We were also able to identify these diculties and we still see room for improvement in this eld.For that reason we set ourselves the goal to oer an approach which enables an automated way of comparing analog behavior and allows mixed-signal system level functionality to be veried in a regression-based way.
The contribution is structured as follows.First, we give an overview of related work that covers approaches for comparing analog behavior.Their results form the basis for the motivation of our work.Following this, we explain the theoretical principles of our developed approach.In Section 4 we select two competitive algorithms and dene a number of comparison criteria in order to show the strengths of our approach.Finally, a case study is outlined in which we point out the achieved benets.

Related Work
If analog behavior of a design shall be checked, one of the most common approaches is to make use of analog assertions.There are dierent works [2] [3] which successfully apply this technique for mixed-signal verication.However, assertions generally lack in abstraction since they have to be dened at signal level.Apart from that, they can usually be applied to check only one particular property of the behavior and are therefore highly application-specic.
Further works, which deal with this topic, are presented in [4][1] [5].They propose approaches which are based on application-specic checkers.These checkers continuously monitor and evaluate circuit characteristics like, for instance, openloop gain or output noise.However, their solutions require a recurring eort for the development of checkers since they have to be elaborated and implemented for each design characteristic.
Due to the aforementioned deciencies it is necessary to look for more generic approaches.For this reason we investigated several metrics that are capable of determining the similarity between discrete signals.One frequently used metric is, for instance, the Pearson correlation coecient (PC).The authors of [6] demonstrate how to employ this measure for the purpose of mixed-signal ver- ication.An overview of other potential metrics like quadratic mean error or Fréchet distance (FD) can be moreover found in [7].Their results marked the starting point of our work, in which we strove for nding a better suited similarity measure.This newly developed approach is presented and assessed in the subsequent sections.

Earth Mover's Distance
The algorithm we propose is based on the so-called earth mover's distance (EMD).This measure is widely used in the eld of content-based image retrieval and is capable to determine the degree of similarity between two distributions.
An essential part of this algorithm is the repetitive application of a ground distance.Therefore, we rst show how to construct an appropriate ground distance before dwelling on the overall concept of our EMD approach.

Ground Distance
In order to quantify the similarity of analog signals it is necessary to operate on discrete-time signals.A discrete-time signal x[i] ∈ R is a series of data with the discrete time i ∈ N 0 and i < n, where n ∈ N is the number of sample points, i. e.
the length of the signal.
In this publication, however, we rely on an alternative denition that regards the signal x[i] as a vector x ∈ R n : x = (x 0 , . . ., x i , . . ., x n−1 ) For our ground distance δ G (x i , y j ), we claim the following fundamental requirement: The ground distance should be bounded to the interval [0, 1], as it can be shown that a bounded ground distance is a prerequisite for the EMD to be bounded as well.
Several publications [8][9][10] utilize the Euclidean distance |x i −y j | as ground distance.This measure is obviously not compatible with the requirement of boundedness.However, there are methods to subsequently bound such distances, e. g. one could use a division by a parameter that incorporates information about maximum and minimum signal values.Alternatively, one could apply an appropriate function like hyperbolic tangent to it.All these methods have the crucial drawback that the resulting distance functions still treat pairs (x i , y j ) of big values dierently than pairs of small values, although the ratios of the pairs are the same.For example consider x 0 = 1000 This phenomenon eectively cancels out all sections of the involved signals where the signal's values have a relatively small magnitude.Thus, we claim another requirement: The magnitude of the ground distance's arguments should not aect the result, i. e. the ground distance has to be scale-invariant, such that the following property holds: In order to fulll this requirement, we drop the Euclidean distance and use a scaled version of the Tanimoto distance [11][12] instead: A 3D-Plot of the function is shown in Fig. 1.

Tanimoto distance
There are a number of important properties to be mentioned about the scaled Tanimoto distance: It is limited to the interval [0, 1] and becomes 0, whenever x i = y j .Also, it becomes 1 if the arguments are equivalent in magnitude, but dier in sign (for Furthermore, it is continuous for (x i , y j ) = (0, 0) and it is scale-invariant, i. e. the distance does not depend on the order of magnitude of the arguments.
Since the scaled Tanimoto distance does not take the position of the elements x i and y j into account, we need to extend it.This extension is done by calculating the dierence between the positions normalized to the biggest possible dierence n − 1.The extension and the scaled Tanimoto distance are then combined using the hypot function.Thus, the ground distance is dened as follows: The division by √ 2 is required to retain the limitedness to the interval [0, 1].Note that this expression can only be applied for n > 1.For the very unusual case that n = 1 we set δ G (x i , y j ) = δ T (x i , y j ).

Algorithmic Concept
The earth mover's distance is an approach for measuring the distance between two multi-dimensional distributions.Informally, if one must successively transport soil from one pile to another in order to equalize them, the earth mover's distance calculates the minimum cost for the total transport.The distributions are also called signatures and represent weighted feature vectors of both signals x and y.In the following the concept of this approach, which is based on [9], shall be explained.
Given are two signatures X and Y where x i , y j denote elements from the respective feature vectors x and y.Each signature consists of n clusters.
For each element x i , y j a particular weight w xi , w yj can be assigned.In our case a constant weight W is sucient since no element shall be prioritized: Moreover the possibility for transports between both signatures is dened as with f ij the ow between x i and y j .Now the idea is to nd a ow F that minimizes the costs C for the overall work: For our ground distance we chose the combined distance from Equation 3 since this distance can be used for scalars and is bounded above and below.
The optimization problem in Equation 5 must furthermore satisfy the following constraints where constraint 79 can be simplied according to our assumption in Equation 4: Constraint 6 allows only unidirectional ows from X to Y. Constraint 7 ensures that the weight in Y matched to x i does not exceed n −1 , while constraint 8 ensures that also the weight in X matched to y j does not exceed n −1 .Finally, constraint 9 forces the total amount of weight matched to be equal to the overall weight of each signature.
In summary this particular metric consists in solving the following transportation problem for a pair of distributions on condition that the above-stated constraints are satised: This expression represents a linear optimization problem.Hence, we can solve it by an appropriate algorithm like the simplex method.However, this method shows an exponential worst-case but polynomial average complexity [13], which in turn has a determining inuence on the overall performance of this measure.
The results for d EM eventually yield values within the interval [0, 1] where 1 symbolizes complete dissimilarity and 0 a full match.

Assessment
In order to assess the eciency of EMD, we compare its behavior with two other algorithms mentioned in Section 2; the Pearson correlation coecient and the Fréchet distance.
In this section we describe how we carried out the assessment and discuss its results.The assessment setup consists of a set of deviation algorithms that are used to generate distorted variants of a given source signal as well as implementations of the EMD, the PC and the FD that compare the source signal with its distorted variants.In this way, we can analyze the behavior of the dierent algorithms with respect to various types of signal deviations.

Deviations
A deviation D(x i , p) is dened as a function that maps the elements of a signal x to the elements of a signal y of the same length: where p ∈ R is the severity of the deviation.
In addition, we introduce the following symbols for the sake of readability: The amplitude x of a signal x and the swing ∆x of a signal x 1 1 Please note that we assume that x is not a constant signal, i. e. ∆x = 0 and hence x = 0. We make this limitation as constant signals would unnecessarily complicate Oset The rst deviation to be dened here is the oset.An oset is a constant, global shift of the source signal along the signal axis.Parameter p oset species the severity of the oset: In this denition, the actual oset is p oset • ∆x.This causes p oset to be relative to the swing of the signal.The advantage of this denition is that an oset with, say, p oset = 1 is always signicant.This allows for comparing dierent values of p oset independent of the actual signal.Edge Steepness This deviation describes a change in the edge steepness of the signal while retaining the signal's original characteristics: The applied changes to the signal can be regarded as a three-step process: The values of x are scaled to the interval [−1, 1].Then, the signal is compressed using tanh or artanh, respectively, and nally, it is rescaled to the original scale.some of the deviation denitions that we give in the following subsections without providing any further insight.For example, consider Equation 14, where ∆x = 0 would cancel out the eect of p oset .
The rst branch using tanh causes the signal's edges to become more convex, while the second branch using artanh causes the edges to become more concave.Neither of both branches is dened for p edge = 0, but it can be shown that lim p edge →0 D edge = x i for both branches.Hence, it makes sense to dene a third branch D edge = x i for p edge = 0, leaving the signal as it is.Examples are illustrated in Fig. 3. Delay The delay is a constant, global shift of the signal along the time axis.
When shifting a signal along the time axis, it is necessary to ll the emerging gap.Here we have chosen to ll the gap with the value x 0 leading to an acyclic delay.Thus, we dene the delay for 0 ≤ p delay ≤ 1 as follows: The severity p delay is relative to the signal's length n.
An example for delay deviation can be found in Fig. 4.
Scaling Scaling means that the signal is vertically stretched or compressed through multiplying its function by a constant factor p scale : Fig. 5 shows the eect of adding scale to a signal.Peak In contrast to any other deviations presented here, a peak is only a local phenomenon.For n being odd, we dene the peak as single spike in the middle of the signal: For n being even, we vary the two adjacent points in the middle of the signal: Examples for peak deviations are illustrated in Fig. 6.
Noise In principal, adding noise to a signal is like adding oset, except that the oset is now random over time rather than constant: where u i is a uniformly distributed random number with An example for noise deviation is provided in Fig. 7.

Comparable Algorithms
In this section, we present two additional algorithms that we use to evaluate our EMD approach.Pearson Correlation Coecient The Pearson correlation coecient is dened as where x and ȳ represent the arithmetic mean: x = 1 n n−1 i=0 x i and ȳ = 1 The result of s pea is a value in the interval [−1, 1], where 1 indicates a full match, 0 a total mismatch and −1 a full match except the sign.
It can be moreover shown that there is a direct relation between Pearson correlation coecient and normalized cross-correlation.The latter is given in [14] and dened as With the help of this equation it can be seen that Pearson correlation coefcient is equivalent to normalized cross-correlation at position j = 0: In other words, an evaluation of the Pearson correlation coecient additionally embraces an analysis of normalized cross-correlation, which in turn is a widely used measure of similarity in the eld of signal processing.Fréchet Distance For discrete time signals, the Fréchet distance can be dened as where α(k) and β(k Values for d F are in the same interval as for δ G .Since we use the ground distance δ G as given in Equation 3, the interval is [0, 1] where 0 indicates a full match and 1 a total mismatch.
The denition in Equation 24 is not very intuitive, which is why a more intelligible explanation featuring a dog and its owner has become popular [10]: The Fréchet distance is the shortest length of a leash required for a dog and its owner to walk on two routesrepresented by our signalsfrom one endpoint to the other.Both are allowed to change their speed, but they are not allowed to walk back.
Furthermore, the denition does not indicate how to implement the Fréchet distance.Luckily, [15] provides a pseudo implementation for discrete-time signals.

Results
The results of the assessment are shown in Fig. 8.In order to make the metrics comparable, we mapped the results of the FD and the EMD to the interval of the PC by subtracting their results from 1. Thus, 1 indicates a full match, 0 a total mismatch and −1 full match except the sign.Please note that the interval [−1, 0) is only applicable to the PC.The source signal x that was used to generate the results is a sine signal with an amplitude of 1 going over two full periods.Furthermore, n is set to 200.From the results it can be seen that the comparison algorithms judge the delay deviation very dierently.FD and PC react very pessimistic and, thus, suggest that source and deviated signal are hardly related to each other.In contrast, EMD detects the deviation but does not drop very low, indicating that there is still a relation between the two signals.The edge deviation is treated similarly by all algorithms, with FD being somewhat more pessimistic than the others.Noise is detected by all algorithms; again, FD being very pessimistic.
In addition, FD is not able to dierentiate between the noise severities.The oset deviation is detected well by EMD and FD.However, FD shows a peculiar behavior in the sense that it returns bigger values for severity 1 and 10.PC is not sensitive regarding oset.The peak is very well analyzed by PC, while not being detected at all by EMD.The FD detects the peak, however, it can't dierentiate between dierent severities.Scaling is being detected well by EMD and FD, whilst PC is not sensitive, except on the sign.horizontal axis shows the applied algorithms.The vertical axis shows the result of the comparison between the source signal and the deviated signal with the severity p of the deviation indicated by color.A value of 1 on the vertical axis stands for a full match, i. e. the metric states that the signals being compared are identical; 0 stand for a full mismatch, i. e. the metric states that the signals are completely dissimilar; −1 stands for a full match except the sign (note that only PC can produce such a result).For an interpretation of the results, see Table 1.
In order to summarize the assessment, we compressed the results from Fig. 8 into Table 1.That table indicates, whether a metric is able to detect (D) a distortion and whether it can still see a similarity (S) between the original signal and its distorted version.We count a distortion as detected if the average of the absolute value of resulting metric M is signicantly smaller than 1: For example, PC can't detect a scale distortion, as all four result in Fig. 8 are ±1 leading to an average of the absolute of 1 ≥ 0.99.
As a criterion for similarity being counted as detected, we dene the average of the absolutes to be greater than 0.66: It can be seen that the EMD can detect all kind of signal distortions, except peak.Referencing the explanation of EMD, where soil is transported from one pile to another, the peak is not detected because the amount of work that is needed to transport the peak is low.This is due to the peak being used in our assessment is very narrow, i. e. it does not contain much soil.However, the biggest advantage of EMD is that it is always able to correctly detect similarity between source and deviated signal.

Case Study
In order to test the reliability of the presented approach, we applied the EMD to a chip level verication of a real-world design.This case study is outlined in the following paragraphs.The design we applied the EMD to is micro-controller-based and features a LIN interface and power switches (see Fig. 9).Its application is focused on driving small electric motorse.g. for window lifts or cooling fansin cars.

Application
Being an automotive application, power consumption is of great concern for the design.In order to fulll the requirements regarding power consumption, the design features a sleep mode as well as wake-up mechanisms.The verication of sleep and wake-up on chip level is crucial, since it aects three design domains digital and analog hardware as well as software.The chip level verication was performed as a mixed-signal simulation in which the hand-shaking between the domains and the voltage curves of the power rails were checked for correctness.
The checking was done in such a way that the results of the initial simulation were inspected manually and stored in the le system.The results of all subsequent simulations were then automatically compared against the results of the initial run using EMD.
In Fig. 10, we show the voltage curve of a rail in the power management unit gained during dierent simulations.The rst curve is the one recorded during the initial simulation and serves as a reference for all subsequent simulations.The colored phases have been dened after the initial simulation, such that the automatic comparison of subsequent results with the ones gained during later simulations could be done phase-wise.Curves A to D represent simulation results after several modications to the design.

Results
Table 2, 3 and 4 show the results of the comparison between signal curves A to D and the reference using EMD, PC and FD, respectively.In order to detect a low similarity, we empirically assume a threshold value of 0.97 for each analysis."Empirically" means that the threshold value, which serves as a pass/fail criterion, is application dependent and is therefore to be dened by verication engineers on a per-circuit basis.Results which fall below this threshold indicate an insucient degree of similarity (i.e. fail) and are marked red in the tables.
First, it can be clearly seen that due to a bug the design does not wake up in curve A, resulting in an EMD value of approximately 0.5.This unintended functional behavior is also detected by the other two metrics.
A less obvious signal deviation can be found in curve B. The obtained results reveal a change of certain capacitors in the design leading to a slightly dierent discharging behavior in phase 3 (EMD ≈ 0.96 and FD ≈ 0.93).For this case PC already shows the rst deciencies since the metric is not able to detect the deviation (PC ≈ 0.99).The aforementioned change of particular external capacitances was actually not a bug, but a specication-compliant adjustment.
Therefore, the reference curve had to be replaced by curve B.
In the third case (signal C), wake-up is triggered at a later point in time from the test bench.In this application, this has no impact on the functional behavior of the design.Therefore, the second active phase, which is consequently being altered by a delay, shall not be agged as too dissimilar to the reference.However,   Switching on an external high load during wake-up leads to the signal shape illustrated by curve D. It is characterized by a peak in the fourth phase at 2000 µs, which can be traced back to a short-time voltage sag.Except for the peak, there is no signicant dierence between curve D and the reference signal.
However, PC and FD are again not able to compensate this deviation and would ag a mismatch (PC ≈ 0.86 and FD ≈ 0.87) whereas EMD correctly detects the functionally unchanged wake-up behavior (EMD ≈ 1.0).
While the critical sleep mode-bug in the above-stated mixed-signal design was detected by all three metrics, EMD reveals its strengths in particular when deviations like delay or peak are distorting the signals.In these cases PC and FD are not able to identify correct functional behavior, which in turn conrms our analyses from Section 4. A concluding summary of our results can be found in Table 5.We furthermore observed the aforementioned deciencies when analyzing several other metrics like Euclidean distance, cosine similarity and Hausdor distance, inspiring us with condence that our approach represents the most reliable one.

Conclusion and Outlook
In this contribution we introduced and described a novel technique for the au-  Furthermore, we assessed the approach by comparing it with related techniques and presented a case study, in which we applied the approach to a realword project conrming its usefulness and high reliability compared to other metrics.
Aside from the outlined benets in the eld of mixed-signal verication, the developed approach has also high impact on several other topics.It can, for instance, be leveraged to optimize objective functions, which in turn is helpful when performing model calibration.In this case an automated regression could be set up to successively t model parameters of interest.Another eld of application is the regression testing of SPICE-style simulators, which can be used to identify diverging simulation behavior caused by diering simulator versions.
Our future work will focus on the extension of the presented methodology regarding usability and exibility.The long-term goal is to provide a building box that covers the need of verication engineers to simulate and verify mixed-signal designs.This building box shall include methods and techniques for driving, monitoring and checking analog signals, as well as for coverage collection and reference modeling.

Fig. 2 5 Fig. 2 .
Fig. 2 depicts the signal change in case of an oset deviation.

Fig. 8 .
Fig. 8. Results of the assessment.Each diagram stands for a type of deviation.The

Fig. 9 .
Fig. 9. Block diagram of the SoC used for our analyses.

Fig. 10 .
Fig. 10.Voltage curves of a power rail in an automotive design during dierent design states gained by chip level simulation.The red phase shows the power-up of the design, green stands for the active state and yellow for sleep mode.
tomated comparison of analog behavior.The technique tackles the necessity of being able to automatically check the functional correctness of analog and mixedsignal ICs during design and verication.It moreover opens the possibility of reducing verication eorts by avoiding manual and error-prone investigations of waveforms since it can be easily leveraged for regression tests.Beyond that,

Table 1 .
Qualitative analysis of the assessment.The table shows whether an algorithm is able to detect (D) the deviation and whether it still correctly reports a similarity (S) between source and deviated signal.

Table 2 to
4show that the results for PC and FD fall below the predened threshold while EMD is able to compensate this deviation.This observation goes

Table 2 .
Comparison of the curves from Fig. 10 using EMD.

Table 3 .
Comparison of the curves from Fig. 10 using Pearson Correlation Coecient.

Table 4 .
Comparison of the curves from Fig. 10 using Fréchet Distance.

Table 5 .
Overview of results: EMD is the only metric that correctly detects, i. e. compensates all four deviation scenarios.