A Proposal of Performance Indicators for Collaborative Business Ecosystems

. Business ecosystems enabled by the increasing use and improvement of communication networks, offer nowadays a powerful competitive advantage to business players and entrepreneurs. They form a collaborative new mean of economic and social value creation, addressing customers’ needs, overcoming constraints of individual firms, increasing capabilities for new business opportunities, and accelerating learning and innovation. This paper proposes a set of performance indicators to measure some of these collaboration benefits, therefore motivating the sustainability and resilience of the business ecosystem. The presented results are based on simulation models, which intend to characterize the roles and interactions of a real life collaborative business ecosystem.


Introduction
The increasing progress and use of information and communication technologies, namely computer networks, and collaboration platforms, have changed the ways of making business, which have moved from traditional industrial sectors to business ecosystems supported by collaborative platforms.The term Business Ecosystem was introduced by Moore [1], which used natural ecosystems as a metaphor to describe business environments.According to this author, a business ecosystem is "an economic community supported by a foundation of interacting organizations and individuals -the organisms of the business world.This economic community produces goods and services of value to customers, who themselves are members of the ecosystem".The players of the ecosystem "coevolve their capabilities and roles" [2] in a symbiotic business environment.
A recent business trends report [3] denotes the continued rise of business ecosystems as a highly relevant transition with substantial implications for the society, economy, and businesses.In fact, as argued by the author, a business ecosystem captures three main generic characteristics of any ecosystem: 1) Variety for a healthy ecosystem, due to the encouragement for participation of a diversity of organizations, that together can overcome the capabilities of any one of them; 2) Potential for ever more productive development of the ecosystem, due to the increasingly advanced ways of collaboration, boosted by high-tech tools of connectivity and interaction; and 3) Longevity and durability of the ecosystem, due to a collective awareness of its members to protect, foster, and sustain the ecosystem, motivated by common goals, benefits, and values.These characteristics of an ecosystem highlight and reinforce the purpose of this paper, which is driven by the following research question: "What is a reasonable set of key performance indicators to measure and assess collaboration benefits in a collaborative business ecosystem (CBE)?" [4] In order to address the above presented research question, the following hypothesis is considered: "Collaboration benefits can be evaluated and made explicit if a set of indicators is established through a holistic combination of concepts of value and benefit, derived from a number of research areas such as value systems, social networks analysis, supply chain performance, and complex adaptive systems".This hypothesis guides this research, which proposes a set of performance indicators and related metrics to assess benefits in a CBE, promoting its sustainability and resilience.
The remaining sections of this paper are organized as follow: section two presents a brief literature review, highlighting contributions from a number of related research fields, section three proposes a set of performance indicators to assess CBEs, and section four shows results of some metrics based on simulation data, to assess the coherence of the models.The last section discusses the contributions and future work.

Literature Review
The research area of collaborative networks [5], in particular their established reference models and taxonomies [6], provide a significant contribution to understand the structure and dynamics of a business ecosystem.A business ecosystem, as described in [6], is a sub-class of a Virtual Organizations Breeding Environment (VBE), i.e. a source network of organizations founded as a long-term strategic alliance.As such, a CBE is modeled as a community of organizations, whose connections are created during a given period (for instance one year), resulting in the formation of Virtual Organizations (VOs), according to collaborative business opportunities.
In order to propose performance indicators for a CBE, a literature research shows that there are already well-established indicators for individual organizations, of which the balanced scorecard (BSC) [7] is the most common example.However, to measure collaboration benefits in a network of organizations, or more specifically in a CBE, there is still a lack of solutions, despite some efforts made in a number of research areas.One example is a conceptual model for VBEs value systems using a BSC to monitor the VBE performance [8].Another example is the value systems proposed in [9], [10], which identify collaborative social and economic core values, suggesting mechanisms to assess the alignment of the value systems of the network members.These mechanisms allow identifying conflicts that may affect the performance of a CBE, although not suitable for performance measurement purposes.More concerned with collaboration benefits, an example presented in [11], identifies and characterizes benefits, such as innovation capacity, flexibility, agility, costs, and risks, among others, which constitute an important input for the establishment of the proposed metrics and associated performance indicators.
The research area of traditional supply chains, which have evolved to more collaborative platforms, known as supply chain collaboration (SCC), provides a wide number of performance models and metrics.For instance in [12], the authors made a comprehensive review of supply chain performance measurement methods comprising the period from 2000 to 2011, and found several techniques and models that were used, namely, AHP (Analytic Hierarchy Process), SCOR (Supply-chain Operations Reference Model), SEM (Structural Equation Model), BSC (Balanced Scorecard), Six Sigma, and combinations of two of them.In [13], more concerned with collaboration (SCC), the author conducted a survey in which he identified a number of mathematical and simulation models focused mainly on the performance evaluation.Some of these techniques and models can be adapted to CBEs; in particular, simulation models to characterize the business environment, and a subset of statistic models from SEM, widely used in the field of behavioral and social sciences.
Another related area is the social network analysis (SNA), which is currently a subject of much research work due to the growth of social networks.In SNA, the social structures are analyzed using network and graph theory, which provides measures of centrality (degree, betweenness, closeness, etc.), and cohesion (reciprocity, density, clustering, etc.).Inspired by SNA, several attempts have been made to apply these indicators to collaborative networks.For instance, a set of indicators to measure the value of social capital of a VBE [14], and a methodology for a network analysis of a business ecosystem [15], among others.However, the structural analysis of the network is limited when it comes to capture the value of the economic or social exchanges represented by the connections.To overcome these limitations, a value network analysis (VNA) model is proposed in [16], providing metrics to evaluate the complex dynamic exchanges of tangible and intangible values in the network.Even though these contributions, a solution to measure the performance of a CBE is not yet established.
The proposed performance indicators for CBEs described in the next section, are inspired mostly by these last presented approaches.

Metrics and Measures for Collaborative Business Ecosystems
The adopted research approach at this stage uses models with simulated data to assess the consistency and make a preliminary calibration of the measurement scale of the metrics and performance indicators proposed in this section.The approach is represented by the process depicted in Fig. 1.The analysis of the literature in a number of related research areas, such as enterprise performance indicators, value systems, collaboration benefits, social network analysis, and supply chain collaboration, inspired a set of metrics and measures with a potential applicability in CBEs.Then, several simulation models of CBEs are used to assess the metrics and calibrate the scales using ratios and standard normalization methods.
A CBE model is represented by a weighted direct graph, where nodes stand for organizations, and arcs correspond to the collaborative transactions between them.The weights of the arcs stand for the number of times the collaborative transactions were performed in a given period (by the creation of VOs in the CBE).In the used models, random series of data were also considered, to simulate existing (e.g.portfolio of competencies) or new value created by the organizations (e.g.products, services, or patents), resulting or not from the collaboration, therefore not directly characterized by the graph.Finally, the last step resulted in a first proposal of a set of performance indicators to assess CBEs.The performance indicators for CBEs introduced in this work and described in the following tables are: 1) Innovation Indicator (II), to measure the innovation potential of a CBE; 2) Contribution Indicator (CI), to evaluate the value creation of a CBE; 3) Prestige Indicator (PI), to assess the influence/prominence of each organization in the CBE; and 4) Resilience Indicator (RI), to assess the resilience of a CBE.
The choice of the above indicators is based on the performed literature analysis and mainly driven by societal, economic, and business concerns.The potential for innovation of a CBE reflects its health level, therefore contributing for social and economic value creation.On the other hand, in a business environment, the economic value is a strong argument for collaboration.As such, the contribution indicator is proposed to show the economic value of the collaboration, i.e. the value that an organization brought in the CBE, and the value that is gained by an organization from the CBE.The prestige indicator shows the distribution of collaboration among organizations in the CBE, revealing eventual prominence of one or more organizations, or even isolated organizations.A strong collaboration level enhances a common network identity, reinforcing collaboration trust, and increasing collaborative knowledge.At last, the resilience indicator in a business context assesses "the capacity for an enterprise to survive, adapt, and grow in face of turbulent change" [17].Table 1 presents a brief description of these indicators, explaining their meaning for individual organizations Oi in the CBE, and for the CBE as a whole.Considering these performance indicators, a set of appropriate metrics had to be found in order to identify key measurable characteristics of each indicator.The suggested metrics are shown in Table 2, including mainly counting measurements (e.g.number of new products, services, or patents created by each organization or by the CBE as a whole), as well as measurements related to the network structure, such as centrality, which is a measure of the activity or level of influence in terms of relationships among the organizations of the CBE.
The metrics shown in Table 2 to assess the resilience of a CBE were inspired by the four fundamental characteristics of resilient living ecosystems considered in [18]: Diversity, Efficiency, Adaptability, and Cohesion, and were tailored to business ecosystems.In fact, the resilience of the CBE can be calculated as a function of these factors, if each of them can be translated in terms of one or more quantified metrics.

PIi
At this stage, the performance indicators can then be formulated as a function of the presented metrics.The suggested formulas described in Table 3, are calculated in terms of ratios (case of II), or measures of degree centrality (case of CI), and betweenness centrality (case of PI).

Is measured by the ratio between the two variables #NewPd i and PortPd i
The ratio of the totals is weighed by the correlation r between the two variables: (#VO, #NewPd)

Resilience Indicator (RI)
Performance Indicators for CBEs Contribution Indicator (CI) The measures of centrality are adopted from [19], but they will have to be reformulated in order to incorporate the weights of the connections, which represent collaborative opportunities and contacts towards potential collaborative opportunities.Then, the resulting indicators are normalized in relation to the size of the CBE (number of organizations).Finally, RI is calculated as a function of the factors (Di, Ef, Ad, and Co), in which the evaluating metrics are weighted by a value according to their relevance in the context of the CBE.

Assessment and Calibration of the Performance Indicators
Among the performance indicators introduced in the previous section: Innovation Indicator (II), Contribution Indicator (CI), Prestige Indicator (PI), and Resilience Indicator (RI), II is chosen for the illustrative examples presented below.The II for a given organization Oi (IIi), measures the ratio between the new products, services, or patents created by that organization (#NewPdi) during a given period, and its portfolio (#PortPdi).The same indicator for the whole CBE, measures the ratio between the number of total products, services, or patents created in the CBE (∑i #NewPdi), and the total portfolio of the CBE #PortPdi).However, this ratio does not give any indication about collaboration, i.e., the indicator does not reflect a measure of the improvement of innovation due to collaboration.Thus, the ratio II for the whole CBE (IICBE) is weighted by a correlation coefficient (Spearman's or Pearson's) [20], denoted by r, between the number of VOs in which the organizations have participated (#VO), and the new products, services, or patents that they have created (#NewPd).
Pearson's correlation coefficient is a measure of the extent to which two quantitative variables (in this case #VO and #NewPd) are linearly related.It measures the strength of the association between the two variables, giving a value between +1 and −1 inclusive, where 1 is total positive correlation, 0 is no correlation, and −1 is total negative correlation.The Spearman correlation coefficient, is the usual Person r applied to data in the form of ranks, measuring how well the relationship between two variables can be described using a monotonic function.The process to rank data in order to calculate the Spearman correlation, consists of replacing each element of a set, by its ranking position according to its relative value (the lowest is rated one, and the higher the number of elements of the set).For instance, considering the set of data S1={9, 2, 8, 5}, the resulting rank set is RkS1={4, 1, 3, 2}.When the set contains repeating elements, for instance the set S2={9, 2, 5, 5}, the repeated elements are rated with the mean of their raking positions, e.g., the mean for the element 5 is (2 + 3)/2 = 2.5), resulting in the rank set RkS2={4, 1, 2.5, 2.5}.
To assess the metrics and coherence of the performance indicator II, as stated above, three simulated scenarios were created (CBE1, CBE2, and CBE3), and described in Table 4, Table 5, and Table 6.The three scenarios represent the same CBE composed of ten organizations, but showing different indexes of innovation, and collaboration performances.The goal of the first two scenarios is to show the influence of the correlation coefficient over the same value of the ratio II of the CBE as a whole, i.e., the smaller the correlation coefficient, the smaller the value of the IICBE.The third scenario shows how a high ratio II is pushed down caused by a very low correlation coefficient, meaning that the innovation capacity of the ecosystem was not due to collaboration.Table 4 represents the simulated scenario CBE1 exhibiting hypothetical series of data for the metrics #PortPd, #VO, and #NewPd, and the calculated ratio IIi, between 0 and 1, using linear normalization.The values of the IICBE (0.64 and 0.58), are calculated by multiplying the ratio II (0.77) by the Pearson (0.84) and Spearman (0.75) correlation coefficients, respectively.Table 5 depicts another scenario (CBE2) showing the same ratio II for the whole ecosystem (0.77).However, the collaboration coefficients are lower (0.51 and 0.50), resulting in a lower value (0.39) for the IICBE.The last scenario (CBE3), depicted in Table 6, exhibits a high ratio II (0.92), but a very low collaboration coefficients (0.13 and 0.10), resulting in equally low IICBE values (0.12 and 0.09), meaning that the high improvement of innovation is not due to the collaboration in the CBE.Considering the boundary conditions of the correlation coefficient (r = 1 and r = 0), the value r = 1 maximizes the result of the IICBE, meaning that the innovation capacity is all up to collaboration.The value r = 0, pushes the resulting IICBE down to 0, meaning that the innovation is not related to collaboration.A negative correlation coefficient is not considered, as it is not assumed that collaboration could influence negatively the innovation capability of the organizations.
The above analysis is based on simulated scenarios.For a further assessment of the proposed indicators, the ongoing work involves capturing real data from a concrete business ecosystem.This is being done for the ports area, a collaborative community offering integrated logistics and freight services.A port area is an interesting use case, as it offers a wide diversity of players and services, such as port authorities, customs, freight forwarders, multi-modal transport entities (rail, road, and maritime), suppliers, and customers, collaborating through increasingly integrated technological platforms, effectively contributing to the growth of the local economy and social value.

Conclusions and Further Work
The metrics and associated performance indicators presented along this paper, contribute as a first approach to accomplish the research question and related hypothesis stated in the first section, by proposing a set of indicators to assess collaboration benefits.A set of metrics were also suggested to quantify those benefits, and different scenarios of CBEs represented by models with hypothetical data were used in order to assess the validity of the measures associated to the indicators.The ongoing work, as mentioned above, encompasses the continuation of the assessment of the proposed indicators, but capturing real data from concrete businesses ecosystems.
The future work is aimed at finding an answer to address the second research question, which consists of: "How to promote collaboration sustainability and resilience within a business ecosystem?"[4] In order to respond to this question, the following hypothesis is considered: "Sustainability and resilience of collaboration in business ecosystems can be promoted if a system of incentives, combined with transparent assessment methods, is implemented at the ecosystem level".This hypothesis will guide the next steps of this research, which will be mostly supported by models using system dynamics (SD) combined with agent based (AB) simulation.The AnyLogic Multimethod Simulation Software [21], will provide the methods and simulation tools to create and explore these models.SD is a method to improve learning in complex systems, somewhat a method to develop computer models, to help learning about dynamic complexity, understand the sources of policy resistance, and design more effective policies [22].
The goal of applying SD at this stage of the research is first to capture and represent the organization of a CBE, considered as a complex system, which, along with stocks and flows structures, feedback processes, time delays, and other structures, determine its dynamic behavior.In a second phase, the performance indicators should be introduced to affect the dynamic behavior of the agents.Finally, a third phase will be dedicated to simulation to observe the agents' behavior, playing with the introduction of incentives/penalizations, resulting from the decisions of the managers.
The explained research approach, aims to specify, validate and make accepted by experts and the scientific community, a set of indicators for CBEs, which not only allow to evaluate the performance of the system, but also contribute to motivate its sustainability and resilience.

Fig. 1 .
Fig. 1.Process representing the used methodology to develop a set of PIs for CBEs.
rP and rS denote Pearson and Spearman correlation coefficient, respectively Rk(#VO, #NewPd) stands for rank data of #VO and #NewPd, used to find Spearman correlation coefficient

Table 1 .
A short description of the proposed performance indicators.

Table 2 .
A proposal of metrics to establish the proposed performance indicators for CBEs.

Table 3 .
Proposed calculation formulas for the performance indicators to assess CBEs.

Table 4 .
Simulated scenario of the CBE1 to assess the Innovation Indicator (II).

Table 5 .
Simulated scenario of the CBE2 to assess the Innovation Indicator (II). #N

Table 6 .
Simulated scenario of the CBE3 to assess the Innovation Indicator (II).