Metrics for Assessing Product Variety Utilization

. In mass customization, offering the right variety is critical and it is therefore proposed to develop an assessment system for the capabilities critical for mass customization success. This paper proposes different metrics and methods for assessing the utilization of a company’s product variety. Two different methods are selected for further testing which is done on historical data for three different product families from three different companies. It is concluded that different metrics and methods may be relevant for different products, reflecting variety and complexity. However, in general, monitoring utilization of variety has potential to improve business for mass companies.


Introduction
In any company, it is essential to offer products, which match the needs and desires of customers to achieve sales and profit. This is true for mass producers as well as mass customizers; however, in mass customization this issue is somewhat more complex than mass production due to a much higher variety and a more complex product structure. As pointed out by Salvador et al. [9], mass customizers need three fundamental capabilities to be successful: 1) Solution Space Development -Identifying the attributes along which customer needs diverge, 2) Robust Process Design -Reusing or recombining existing organizational and value chain resources to fulfill a stream of differentiated customer needs and 3) Choice Navigation -Supporting customers in identifying their own solutions while minimizing complexity and the burden of choice [4], [9]. In order for companies to be able to establish themselves as mass customizers or for existing mass customizers to improve performance, we proposed that an assessment system for Mass Customization performance is established [5], [6].
An essential element in mass customization is product variety. Variety is what differentiates mass customization from other business strategies, however the variety offered to the customer, the solution space, must be carefully designed, since a too high variety will imply higher costs and a too low variety will imply lost sales, since customers will not be able to buy the product matching their individual requirements [8]. Hence when developing an assessment system for MC, metrics addressing the size of the solution space and whether this size fits with the demand for variety is essential, since variety is a major driver for both cost and revenue. A number of metrics for this purpose have been proposed all related to solution space development [3]. These metrics however cover a broad area of capabilities related to solution space development, including the solution space at a certain point in time as well as dynamics of solution space development.
The metric "Used variety" (UV) introduced by Piller [7], referenced from [1] addresses how well the solution space is utilized by the customers, i.e. how much variety is offered vs. how much does actually make sense compared to the customers' requirements. The metric is calculated as indicated in the formula below: Using this metric may be difficult in practice, since the number of perceived variants is not readily available, as well as the number of all possible variants may be difficult to determine for various reasons, which will be addressed below.
The research question of this paper is: How can the utilization of a mass customizer's solution space be measured? The research question will be addressed with a basis on the "Used Variety" metric presented above, however due to certain practical issues in implementing this metric directly, further considerations need to go into defining the metric more specifically, which is done applying the method outlined below. The reasons for assessing the solution space include enabling mass customizers to optimize their solution space as well as their configuration systems, which both influence the performance in relation to solution space utilization.

Method
To answer the research question, a number of sub questions will need to be answered. In order to do this, the following methodical approach has been applied: 1. Clarify the purpose of the metric and based on this evaluate the feasibility of the originally proposed form of the metric. 2. Identify variables, which should be used to define and calculate the metric. 3. Evaluate the availability of data necessary for calculating a value for the metric. 4. Based on findings above, propose alternative methods to calculate the metric. 5. Verification of feasibility of calculating the metric by applying the proposed methods on different data sets containing actual historic product data. 6. Evaluate results from verification and identify the most suitable method in relation to the purpose identified in step 1.

Purpose and evaluation of existing metric
The basic purpose of the metric UV is to evaluate the utilization of a company's solution space. The reason to do so is that variety comes at a cost, and variety, which is not being sold, does not generate income and may imply economic loss. The reasons for variety not being utilized can be very different but relating to the fundamental capabilities of MC, we expect low utilization to be due to either lack in capability for solution space development or choice navigation. Low utilization due to problems within solution space development will imply that a company has developed variety, which is simply not demanded by customers, i.e. development of the wrong products. On the other hand, utilization issues due to poor choice navigation imply that the company may have exactly the right variety, but is not able to guide the customer towards buying the product that matches the specific requirements. Either way, the result is that companies have made investments in introducing variety, which is not paying off.
Hence the metric should be able to assess to what extent the variety offered to customers is actually being utilized, i.e. sold, in order to enable companies to react to low utilization. The metric should be applicable as a KPI and should thus be able to track utilization over time. In order to track performance over time, the KPI should be calculated on a regular basis and therefore it should be possible to calculate based on readily available data from the company's IT systems and not require expensive market surveys or qualitative assessments from the company.
The way of assessing the utilization of variety proposed by Piller [7] using the metric Used Variety implies identifying the number of perceived variants and dividing this by the number of all possible variants. In general, assessing variety based on the measure "number of variants" is intuitively an obvious choice since this is based on a simple count. However, in many cases this will be impractical. One of the reasons for this is that in mass customization, the number of theoretical variants easily becomes astronomical. For example, when configuring a Mini Cooper online the configuration choices presented to the customer will result in a number of possible variants well above a 20-digit figure. Another issue that may arise when determining the number of possible variants is how to count continuous configuration variables, e.g. the height and width of a window, which can be any value within a predefined interval or custom printed Tshirts, which can have any image printed on them. In these cases, it makes little sense to determine the number of possible variants, since this figure is literally infinite. This is also the case for products composed by predefined modules or components, which can be combined in various ways to configure a product, where the number of modules included in the configuration is not bound. In this case, the number of possible variants will also literally be infinite. Hence, when determining the value of the UV metric the value will in many cases approach a zero value, as the denominator will be virtually or literally infinite. Furthermore considering an example where a given solution space is expanded with one extra binary option, this will effectively double the number of possible variants (given this option is offered for all products). This again implies that the value of the UV metric will be halved. Considering again the Mini Cooper example, adding an option to include floor mats will double the number of possible variants, thereby halving the value of the UV metric, even though the actual change in product variety would clearly not imply that the utilization of variety is reduced by 50%.
The numerator of the UV metricnumber of perceived variantsmay be determined in a number of different ways. The term "perceived" implies that the number should rely on the perception of the customer, which naturally is difficult to determine quantitatively. However, a reasonable way of measuring number of perceived variants is to assume this is equal to the number of different products (configurations) that have been sold over a period. If assuming this, it will be feasible in many cases to calculate an actual value for this metric, however as indicated above the value may not be a very good representation of the actual utilization of the solution space.

Variables & calculation methods
As described by Brunoe et al. [2], the utilization of a product family's solution space can be addressed using set theory. The solution space can be described using three sets; Configurable variety, configured variety and ordered variety. The set configurable variety contains all possible product variants, whereas configured variety contains all variants that have been configured and ordered variety contains all variants that have been configured and subsequently ordered. To assess the utilization of a solution space, it must be addressed what the relationship is between the set "configurable variety" and the set "ordered variety, i.e. a small difference between the two sets implies a good utilization of the solution space. However to assess the difference between the two sets, it must be determined what are the elements of the sets.
One possibility is to define the elements in the sets as product variants, which corresponds to the approach suggested by Piller [7] for calculating the "used variety" metric. This is in some cases a simple way of measuring the utilization of the solution space, however as pointed out above, there are certain issues in other cases. Calculation of the solution space utilization (SSU1) can be done by applying the formula below: This form of the metric differs from the one defined by Piller [7], since the numerator simply counts the number of distinct configuration. The data needed to calculate the value of this metric is expected to be readily available, assuming the company has a product configurator, where the product family model defining the solution space provides the information needed for calculating the number of different product variants (simply by calculating the product of number of possible values for each configurable variable).
Another possibility for defining the elements of the sets is to let each outcome of each variable in a product configurator represent an element. E.g. if a configurator allows you to choose a product in three different sizes and three different colors, that would correspond to six elements. We define these elements as "configuration options". I.e. a configuration option corresponds to a specific value for a specific variable. Using the configuration options, the number of elements will not become astronomical when representing the solution space, but product configurators will typically contain configuration options in the order of hundreds or thousands. It is then possible, to calculate the solution space utilization based on the number of configuration options (SSU2): This metric will also be able to assess the utilization of the solution space, however without the drawback of using the SSU1 metric of e.g. reaching near zero values for larger solution spaces and being sensitive towards introduction of simple variety. On the other hand, if the sales of a product family reaches a certain level, it is expected that the value of the SSU2 metric will approach one, as it is likely every configuration option will be chosen by a customer at least once. This leads to the issue regarding this way of measuring solution space utilization, which is that a configuration, which is chosen only one time by a customer, will count the same as a configuration option, which has been chosen thousands of times by customers. Clearly, a configuration option chosen thousands of times indicates a better utilization than a configuration option chosen only once, which this metric does not take into account.
Another way of assessing the utilization, using the configuration options, but taking the issues described above into account, would be to calculate the frequency by which each configuration variable is chosen by a customer. By doing this, it can be determined for each configuration variable, how well it is utilized. Furthermore, to assess the solution space as a whole, a distribution of the frequencies can be determined and illustrated in e.g. a histogram, which will indicate whether all configuration options are chosen similarly often, or some configuration options are chosen almost in every configuration whereas others are chosen very rarely. To represent this distribution, mean values and standard deviation can also be calculated for the frequencies for which configuration options are chosen. Although this way of assessing utilization of the solution space is not a metric in it's simplest form, i.e. a metric which can be calculated as a single figure, we include it in this paper as it gives a deeper insight than the other simpler metrics. The data used for calculating a value for this metric is identical to the SSU1 metric In this paper, the two methods -SSU1 and determining the frequency and distribution of configuration options being chosen are tested on data from three different products from three different companies all producing mass customized products. The metric SSU2 has not been included in the test because of the arguments presented above and since initial test showed that, the metric for all product families would reach the value 1.

Data
The data used for testing the different metrics and calculation approaches have been obtained from three different companies. The product families represent three entirely different product types in terms of application and complexity. Product family A is a family of mass customized capital goods equipment of high complexity. The product family is close to being Engineer to Order products and have thus a very large product variety, however they are standardized and can thus be considered mass customized products. Product Family B is a family of products, which are used as sub components of larger systems in the process industry. The products have medium variety and are customized to meet individual demands, by combining different components. The complexity of these products is relatively low. Product Family C is a family of products for domestic utilities. The product family has medium variety but are slightly more complex than the products in family B but still much less complex compared to family A. Data obtained for the three product families consists of a product family model defining the variety, i.e. variables and there values, enabling determining the total number of variants and configuration options. Furthermore, historic data describing specific configurations which had been sold over a one to two year period was obtained for determining what variety had actually been configured.

Results
The results of testing the SSU1 metric are shown in table 1. At first glance, it can be seen that all three product families have SSU1 metric values very close to one, although product family A has a significantly lower value for the metric, for which the major reason is that the number of possible variants is significantly higher. Furthermore, it can be seen that the number of distinct configurations sold are not particularly high. This is due to the products being B2B products and thus not being sold in volumes comparable to consumer products. Furthermore, for families B and C it indicates that sales includes much repurchase of previously purchased configurations. I.e. one configuration is sold in larger volumes and in multiple orders.  Table 2 and Figure 1 show the results of analyzing the frequencies of which configuration options are chosen by customers. In figure 1, the frequencies are divided into 20 even intervals. The bars in the histogram indicate the number of configuration options, which have been chosen the number of times, which corresponds to the frequency interval. E.g. the first bar for product family A indicates that 358 configuration options have been chosen by customers between 0 and 22 times. The number below each bar indicates the upper bound of the interval.  Table 2 also reflects an analysis of the frequencies of configuration options being chosen, where the mean frequency and standard deviation are calculated. The mean frequency for e.g product family A indicates that configuration options are in average chosen 77 times by customers. The mean is also related to the number of configurations in row two in the table, where the value 0.231 indicates that the average configuration option is chosen in 23.1 % of the configurations. The table also shows the standard deviation for the frequencies, where it is notable that the standard deviation in all three product families is larger than the mean value indicating very spread frequencies, but also a long tail of configuration options where a few configuration options are chosen very frequently and a high number is chosen very rarely, which is also seen in figure 1.  22  43  65  86  108  129  151  172  194  216  237  259  280  302  323  345  366  388  409  431   Product   Family   B   0   10   20   30   40   50   60   70   80   9  18  27  36  45  54  63  72  81  90  100  109  118  127  136  145  154  163  172  181   Product   family   C   0   50   100   150   200   250   300   350   400   21  41  61  82  102  123  143  164  184  205  225  256  266  268  307  327  348  368  389  409 Product Family A 7

Conclusion
The basic idea of the Used Variety metric [7] is intuitively good, however insufficiently described to operationalize it. Using the SSU1 metric a very similar measure can be calculated, however testing on actual data showed that the method did not provide very useful information on utilization of product variety. The metric may however be useful in cases with low variety or in cases with a lower number of predefined product variants. Another method for assessing variety was tested, where the frequency of choosing a configuration option is counted, resulting in a distribution of frequencies, which indicates if there is a large spread in the utilization of configuration options and if there is a large part of the product variety which is rarely configured and thus potentially unprofitable.
The data used to test the methods is obtained from three different companies, however all operation in a B2B setting, limiting volume. The results may have been somewhat different if data were included for consumer products as well, however in terms of assessing the applicability of the method, we consider the testing done in this work as sufficient to draw the conclusions stated above.
Further consideration needs to go into the practical application of the methods, e.g. over how long time should data be sampled? Using data for 1 month will reveal a significantly different result than data for half a year. Furthermore, not all configuration variables may be relevant to include, since some may be redundant, if the configurator as an example contains variables for both user requirements and product characteristics.
In general, we expect the application of product variety assessment tools as a valuable tool for mass customizers in order to optimize their business in terms of determining the right level of product variety and diagnosing the configuration process.