Process Planning Assessment Framework

. In order to rationalize process planning activities or to improve process planning outcomes, various process planning approaches with different degrees of computerized support and automation have been established. Ranging from general to very specialized approaches to specific problems, these approaches have specific advantages and drawbacks. When designing new approaches as well as selecting existing ones for application in the field, their properties have to be considered comprehensively. However, existing research only selectively touches on the objectives regarding the planning process or the quality of information generally requested from process plans. This paper presents the synthesis of a comprehensive process planning assessment framework, including planning process and outcome related criteria, based on an extensive literature review.


Challenges assessing process planning approaches
The necessity to rationalize process planning activities has led to the establishment of various planning approaches with different degrees of computerized support and automation [1].Next to an increase in planning efficiency, the proposed planning approaches aim at improving the quality of the process sheets, which in many cases do not meet the requirements of the stakeholders [2].
Ranging from general to very specialized approaches to specific problems, these approaches have specific advantages and drawbacks [3].As the importance of single properties of these approaches naturally depends on the specific application scenario, companies have to cover their basic process planning needs by producing reliable documents for production scheduling and control activities.Therefore, when designing new approaches as well as selecting them for application in the field, the properties and suitability of the approaches should be considered comprehensively.
While standard references in the production engineering field cover the conventional contents of the final process plans, their description of the planning process usually does not deliver a target system or criteria for optimization [1,4].Newly developed planning approaches mostly do state their specific motivation and the improvement potentials regarding the planning process or outcome they want to address.However, there are no comprehensive assessment frameworks for process planning available.
The paper is structured as follows.In Section 2, we give an overview over existing approaches for assessing process planning.We then outline the research methodology of the extensive literature review.In Section 3, we analyze the relevant evaluation criteria.In Section 4, we present the synthesis of the criteria in the assessment framework.In Section 5, we present a brief first case study for basic validation, and, in Section 6, we conclude the paper.

2
Theoretical framework

Deficiencies of existing approaches
Standard references in the field of production engineering and management provide a generally applicable and neutral definition of the planning process and the desired outcomes (e.g.[4]).Additionally, for evaluating the planning process success, some publications refer to the general objective of achieving an optimal relation between the use of resources and desired outcome (e.g.[1]).Beyond this hardly tangible objective regarding the characteristics of the resulting process plan, the authors describe general trends like Simultaneous Engineering (SE), the questioning of the separation of planning and control, as well as decentralization and automation of the planning process [1].However, stated benefits [4] of automating the planning process can be interpreted as assessment criteria for process planning: improved productivity, lower production cost, consistency, time savings and rapid integration of new production capabilities.
Research articles focusing on process planning in many cases focus on enhancing understanding and technology regarding specific planning problems, e.g. the automated selection of machining parameters [5], automated discovery of typical process routes [6] or selecting "optimal" process plans [7].The researchers usually either loosely state their objectives or feature explicit, even multi-criteria, objective functions [8] with regard to the specific planning problem at hand.Denkena et al. [9] provide research touching on a wider range of relevant criteria regarding technological and business considerations.However, to the knowledge of the authors, there are no research articles providing a comprehensive overview over all relevant criteria.
Although neither evaluation criteria for the planning process, nor the resulting process plans are well covered, methods for performance measurement of general business processes might partially be applicable to process planning.While widespread methods for BPM assessment (e.g.CMMI and BPMM) rely on rather abstract process maturity models, more tangible process KPIs are usually company and process specific.However, Fishermanns [10] provides several universal criteria that can partially be adapted to the planning process: repetitiveness, personnel qualification, personnel count, intensity of communication, intensity of documentation, process complexity, determinacy, task difficulty, consistency, external relations, relevance of quality, time and cost.

Research methodology
In order to achieve the stated objective of developing a comprehensive assessment framework, the evaluation criteria to be considered, are extended by a rigorous and systematic literature review.The review is considered an explorative background review, in order to inform the design process.This section shall provide information on the coverage of the review [11].
The review was largely organized sequentially [11].The search covered literature since 2000.The search string was ("process plan") AND (quality OR requirements OR assessment OR criteria) and its German equivalent.Fig. 1 provides an overview of the databases and search providers as well as the results.Many articles identified by the keyword searches ('hits') do not focus on the research question at hand, but merely offer insights into the topic incidentally as described in Section 1.In addition, the search string has widespread application beyond the field of production engineering.Therefore the articles have been evaluated (the most highly prioritized hits in the cases of google scholar and ScienceDirect) based on their abstracts, in order to assess their relevance for the review ('reviewed') [12].58 articles have been reviewed (cf.

Evaluation criteria for process planning
The relevant literature has been analyzed for references to criteria, requirements and objectives regarding the planning process and the resulting process plan.The criteria listed in Fig. 1 represent the references in the reviewed articles in the most objective way, but are not completely mutually exclusive.The criteria have been synthesized from the illustrated literature review, structured and aggregated in order to generate comprehensive listings to be used in the assessment framework described in Section 4.

4
Process planning assessment framework

Application
The process planning assessment framework is aimed at informing the decisionmaking process of researchers and practitioners.It is supposed to ensure that all relevant criteria are taken into consideration.Subject of the assessment are the planning process, albeit manual or automated, and the resulting process plan.The following sections 4.2 and 4.3 contain the comprehensive listings of the respective assessment criteria.To aid understanding and standardized assessment, descriptions on their relevance as well as examples are provided.As the criteria are multi-dimensional and not fully independent from each other, they partially complement or conflict with each other.Therefore, in application, the criteria have to be prioritized.The approach follows a traditional multicriteria utility analysis [13], as shown in Fig. 2.

Fig. 2. Assessment approach
The planning levels of process/operations planning (Fig. 3) serve as assessment dimensions for the process plan assessment.During assessment, the user applies the relevant criteria to the required information of the planning level of interest.The subprocesses shown in Fig. 4 are selectable assessment dimensions of the planning process.

Assessment criteria
The criteria for the assessment of the planning process (Fig. 4) are structured into cost related criteria, criteria regarding desirable process characteristics and criteria related to user interaction with the process and its tools.The criteria for the assessment of the resulting process plan (Fig. 5) are structured into objective data requirements as well as subjective optimization criteria.

Fig. 5. Process plan assessment
During assessment, the user applies the relevant criteria to the required information of the planning level of interest (Fig. 3).For example, when choosing or designing a process planning approach for individual production, the stakeholders might decide that the planning approach should primarily provide information on macro/process plan level because the operations planning is left to experienced machine operators.

First case study and basic validation
The assessment framework is illustrated with the example of an industrial partner in the metal working industry.All subprocesses are determined to be relevant dimensions, while only the macro planning level is of interest in this case (cf.Fig. 2, step 1).
Weighting the assessment criteria in a pair-by-pair comparison (cf.Fig. 2, step 2), effort and duration prove important for the planning process.Most important criteria for the resulting process plan are feasibility, integrity and correctness/accuracy. Fig. 6 shows two macro level process plans for a piston variant, generated manually and by a generative computer-aided process planning (CAPP) system.The relevant stakeholders have assessed (cf.Fig. 2, step 3) the rather quick manual planning process positive in terms of duration and effort.The automated and even faster generation by the CAPP system is rated very positive, however length and effort of the CAPP system setup and update process are rated very negatively.
Comparing the resulting process plans to several executed manufacturing orders for the parts number, the characteristics of the manually generated process plan are established as very positive.It is showing occasional deviations to other workstations with the same capabilities as well as statistical variations of processing and setup times.
The characteristics of the automatically generated process plan, without any further human intervention, are assessed as neutral.All necessary machining operations are included in the right order, however the operations sequence needs to be complemented.Two workstation IDs are missing for integrity, while the accuracy of the standard times is significantly off in several instances.According to the assessment regarding the criteria relevant to the company, it becomes obvious, that at the current stage manual process planning remains the norm, while further capability improvements of the CAPP system will be investigated.

Conclusion
The assessment framework presented in this paper offers comprehensive criteria and corresponding descriptions for both the planning process, as well as the resulting process plan.It can serve researchers developing new process planning approaches for reference and enables practioners reengineering their planning process to assess their respective options with the criteria relevant to them.The assessment framework was validated with an SME.Further research needs to be done on the measurability of the criteria.More validation needs to be done to further detail the approach and to ensure applicability in various fields/sectors.

Fig. 1 .
Fig. 1.Criteria referenced in the reviewed literature

Fig. 6 .
Fig. 6.Process plans for machining a piston variant

Table 1 .
Database search

Table 2
).Other Journals include the International Journal of Computer Integrated Manufacturing (3), Annals of the CIRP (2), Journal of Materials Processing Technology (2) and several more.

Table 2 .
Literature review