Revisiting Challenges in Using Discrete Event Simulation in Early Stages of Production System Design

. This paper presents challenges of using discrete event simulation when supporting decision in early stages of production system design, when significant changes are introduced. It was based on three real-time case studies performed at one manufacturing company during 2014-2016. Challenges in the cases were mapped to previous literature, pointing out discrepancies and highlighting three additional challenges, specifically related to issues in the early stages of the production system design process. The significant change introduced to the assembly system, and the early phases of evaluation put significant challenges to the use of discrete event simulation and the study points out further efforts needed to support manufacturing companies under change, with an established industrial structure and legacy systems to consider.


Introduction
As manufacturing companies currently meet major change drivers in terms of digitalization, technology development, sustainability and market behavior, many production systems need to deal with significant changes in terms of product portfolio, volumes, and its role in the value chain.In addressing these significant changes, modeling and simulation is a fundamental concept for analyzing behavior and performance in a future setting, and in the area of production system design, literature and practitioners have for long underlined the benefits of discrete event simulation (DES) [1], [2], [3].DES involves the modeling of a series of events when state changes occur at discrete points in time [3], and is used for decision support in production system design because of its ability to evaluate change dynamically [4], [5].DES is suggested to create knowledge, generate understanding, visualize results, and communicate production system design decisions across different functions within a manufacturing firm [6].The further development of streamlining simulation-based production systems engineering based on input data from digital twins is a current strong development [7].
However, research has long indicated the limited use of DES at manufacturing companies [8] where specific challenges have been pinpointed.As the DES use can be separated into the three model process phases of design, development and deployment [3], challenges have been discussed from different perspectives.This identification of challenges in DES use for decision support has preponderantly centered on difficulties in the development phase of DES models.This approach stems from the realization that a single DES model is often incapable of supporting all production system design decisions and that detailed questions about a production system's performance will arise during the design process [3].In line with this, Heilala et al. [10] analyze the support of production system design and operations decisions based on DES use.Also, the deployment phase are exposed to challenges due to the need to build trustworthy DES models for factory management to commit to production system design decisions.This situation underscores the issues that arise when determining which elements of a production system to represent when supporting decisions through DES during production system design and thereby achieve a successful implementation of DES models [11].In addition, contextual circumstances related to the introduction of significant changes, such as the lack of information necessary to commit to decisions during production system design, upset the validity of a model and question the credibility of its results [12].
Concluding, current understanding of challenges in DES use for manufacturing applications has so far focused on the use of DES during the operation of a production system or in the final stages of production system design.Thus, the aim of this paper is to revisit the challenges of DES use when supporting decision in early stages of production system design, when significant changes are introduced.We present empirical findings from three real-time case studies performed at one manufacturing company.Each case follows early design stages of a production system and where the DES was suggested to support design decisions.DES was used in two of the three cases, but in one of the cases DES was decided not to be used.The study explores the basis for the decisions of using, or not using, DES during the early design stages where, for the company, new assembly system concepts were to be evaluated.The study contributes to a more established scientific fundament in a practitioner's context, relying on a trial and error tradition.It presents live empirical evidence on challenges while using DES in early design stages of assembly systems for manufacturing companies with an established industrial structure and legacy systems to consider.

Material and method
Three real-time case studies were conducted at a Swedish manufacturing company from the heavy vehicle industry with sites around the globe.Live projects during 2014-2016 were followed and data were collected during 12 months for Case 1, 14 months for Case 2 and for 5 months for Case 3. Data collection for each case included interviews, participant observation, company documents, and discrete event simulation models.
Collected data included five interviews per each case.Interviewees were selected on the basis of their participation in each production system design project and included a manufacturing research manager, two logistics developers, two consultants, six manufacturing engineers, two project leaders, a manufacturing engineering manager, and one assembly system concept owner.Interviewees described the reasons behind the start of production system design, and activities in the design of the production system.Finally, interviewees reported on the conclusions and consequences of the designed production system.Interviews were recorded and transcribed and lasted between 38 to 120 minutes.Interviewees were individually contacted at a later date, and shown the transcribed texts for approval.Additionally, data were collected through participant observation including weekly project meetings, workshops, informal conversations, and floor shop visits.Also, company documents related to each production system design project were analyzed.Collected data included the examination of seven DES models developed in Cases 1 and 3 which focused on understanding the consequences of introducing production system changes during production system design.Data analysis followed three streams of concurrent activities: data condensation, data display, and conclusion drawing [13].Each case was first treated individually as a separate study.Then a cross case synthesis and analysis was employed to gain a comprehensive understanding across cases [14].The analysis was based on earlier identified challenges in using DES in manufacturing application from [9], [11] and [1].Conclusions were finalized based on the comparison and contrasting of theoretical findings with cross case synthesis.

Discrete Event Simulation in three cases
was interest in using DES in the production system design project team, but DES was ultimately not used during production system design.Case 3 investigated a production system design project that introduced a multi-product assembly system to replace the site's existing product-specific system in which different families of cabs could be assembled in the same line.The designed production system aimed to comply with the manufacturing company's strategic objectives as well as the site's need to increase production system efficiency and reduce assembly line space.The production system design project included the development of two DES models.Summarizing the findings, Table 1 classifies the identified challenges of DES support of production system design decisions for the three cases (C1, C2 and C3) based on the three DES model process phases [9].The list of challenges is put together from existing literature [9], [11], [1].
Table 1.Empirical identification of challenges of DES supporting production system design decisions in Cases 1, 2, and 3. List of challenges based on [9], [11] and [1].all cases identifying the specific questions that a DES model should address was a challenge.As expressed by the project manager in Case 1, "Are we doing the right thing?Should we do it this way?No one really had an answer for that."This related to specifying the level of abstraction of a model when incomplete and conflicting production system knowledge existed [1].The point in case is described by the manager in Case 3, "you can do it in a lot of ways and still get the same result.What should we choose?It's not only about the pieces, it's the system around you, and finding the right solution for you."This could be explained by the presence of high uncertainties in early design phases [16] and the lack of overall comprehension of the intricacies of a production system during its design [9].Specifying what to model and how to do so required the assistance of DES experts in all cases.This exposed the lack of resources with the right competence to build DES models [11], the absence of standards to aid the development of DES models [17], and the dependency on individuals instead of processes to achieve DES use during production system design.

Identified challenges in DES model development phase.
DES model development in its relation to input data and DES model verification was the most frequent of challenges of DES use when supporting production system design decisions, a situation consistent with current findings [18].The reasons behind this challenge were related to the absence of a real world from which to draw data and came as a consequence of introducing a change that was significantly different from the manufacturing company's current operations [19].This issue was increased by the high levels of uncertainty, which involve the probability that certain assumptions made during design are incorrect or that the presence of entirely unknown facts has a bearing on the future of a designed production system [20].This is instantiated by the project manager in Case 2, "There were many unknowns in this innovative project, which different from doing business as usual.We were doing this (transformation) for the first time with no recipe." Identified challenges in DES model deployment phase.Challenges associated with the deployment of a DES model in support of production system design decisions were also present.These were linked to the limited use of DES in the industry.Model developers had to continuously show in what ways DES contributed to the design of production systems [9], as exemplified by the consultant in Case 2, "There was a need to present simulation testing at very different levels (in the organization).Simulation is important, but it is really difficult to make a crisp conclusion for everyone to see.Especially when so little is known."Further, communication of DES model results required the interpretation of experts, a situation that limited effective decision making by project team members [10].Similarly, the low-reusability of DES models was linked to the lack of in-house resources with DES know-how that could modify or re-use developed models at later times.

Discrepancies between challenges in the case studies and earlier literature.
Empirically not all challenges in DES use outlined in [9], [11] and [1] were detected (see table 1), and different levels of concurrence of challenges in DES use existed across cases.The absence of previously identified challenges (Table 1) does not indicate a disagreement with extant theory.The dearth of evidence on this matter is explicated by contextual circumstances.The absence of challenges regarding representation of production system dynamics and complexity, as well as modelling the combinatorial explosion of options in a production process was a consequence of the choice of what production system to model and how to model.DES models incurred in assumptions and simplifications that limited the representativeness of complexity and options.Challenges affecting simulation model maintenance were not discovered as DES models were not re-used after presentation of model results had taken place.
Also, case data show the presence of challenges additional to those specified in literature when DES supported production system design decisions.First, DES model objectives did not establish a direct connection to the high level objectives pursued by the manufacturing company.This constitutes a severe challenge to the characterization of a production system during its design and the decisions involved [21].Second, DES use was severely limited by the absence of a rigorous production system design process with clearly defined milestones for DES use [10].The project manager in Case 1 exemplifies, "We could not specify an implementation (for DES) at the beginning.We had to investigate if it was a good idea first, look into this and start to generate ideas," and a production engineer from Case 2, "Of course we tried to look for benchmarking but there are not so many examples in this kind of industry for this type of a change.There is actually no case available where you can find something similar."

Conclusion
This study revisited challenges of using DES when supporting decision in early stages of production system design, when significant changes are introduced.It was based on three real-time case studies performed at one manufacturing company during 2014-2016.Challenges were mapped to previous literature, pointing out discrepancies and highlighting two additional challenges, specifically related to issues in the early stages of the production system design process.The significant change introduced to the assembly system, and the early phases of evaluation put significant challenges to the use of discrete event simulation.Future research efforts are needed support manufacturing companies under significant change, with an established industrial structure and legacy systems including: (1) model design in terms of alignment to high level business objectives and knowledge, (2) input data and model validation during model development, and (3) model knowledge, reuse, and interoperability during model deployment.