Towards 3D Visualization Metaphors for Better PLM Perception

. PLM tools mainly refer information systems based on a wide graph representing linked information. Such a PLM model is compatible with a meta-model often expressed in UML. UML graphics diagrams are paradigms to support sharing and understanding the concepts and relationships that must be supported by the PLM tool. Nevertheless instance diagrams contain so many objects that it is hard to present them in a holistic view that can be easily shared by collaborators. This paper presents a 3D approach to visualize an Object-Process Methodology (OPM) model, which provides a single holistic view of a complex system and supports sharing among collaborating parties. This approach has been applied to managing the complex information related to VISIONAIR – a European visualization infrastructure project, successfully enabling the management of 123 simultaneous projects.


Introduction
Product lifecycle management (PLM) integrates the theories and technologies to create, optimize and manage a holistic view of a product throughout its life cycle.Maintaining information coherence is a major PLM issue.Tools have been developed and commercialized to integrate information from the various domains involved in the various product life cycle stages.Such tools require detailed product models as well as the engineering processes to produce it.Models are often formalized through Unified Modeling Language (UML).The complexity of the modeled domain leads to huge class diagrams which represent the conceptual model of PLM.Some object diagrams are used to visualize the instantiation of the conceptual model, but they are seldom practiced because the complexity of relationships leads to very large, entangled graphs that cannot be understood.Hence, such diagrams are usually limited to small examples.Some user interface is needed to access information in models of real life products.Web-based user interfaces are currently practiced, leading to a large amount of tables linked together.A standard user accesses the system through a profile area, with a table of projects and products with which she or he are concerned.Selecting a project or product, he gets access to new tables providing specialized information.Each table is a partial view of the complete information system.Overall comprehension of the product life cycle requires navigating through a big set of tables and building a mental view of the product evolution or its related processes.
While PLM claims to provide a holistic management of the product life cycle, it fails to offer simple, user-friendly views.We present a new way to access needed PLM information.Using a 3D visualization framework, we take advantage of the third dimension to augment perception capacity.
In the next section we discuss various product and process models which were proposed to measure the PLM complexity.Section 3 focuses on Object Process Methodology (OPM) and the OPM-based presentation, which combines the processes and the object being processed, yielding a set of conceptual 2D object-process diagrams (OPDs).Section 4 presents the use case concerning the management of more than 120 projects simultaneously in the VISIONAIR FP7 European project.A main objective of the VISIONAIR project was to invite guest users to one of the 29 research institute laboratories participating in the VISIONAIR consortium.We created a conceptual view of VISIONAIR processes in an OPM model.Then, a 3D projection of the instantiation model was used to simultaneously visualize the status of various projects.This 3D approach is presented in Section 5.The paper concludes with the discussion about the added value of this visualization for process management.

PLM conceptual models
A wide literature covers PLM models.Design is mainly concerned with the product and the definition of its engineering processes via activity/process models or by product data management models which often integrate the product and its manufacturing processes.PLM also address models of customers (CRM), enterprise resources planning (ERP), and many other aspects.Some authors propose integration of these aspects.In this section we provide a rapid overview of product and activity models.

Product Models
The product breakdown structure is often used as a decomposition of the product for short-term projects.Indeed, this provides a local bill of materials, which was extended as the core model of product data management (PDM) systems.The bill of materials is a tree of articles, mainly corresponding to an assembly decomposition into sub-assemblies or components.It becomes quickly complex when every representation of article may be released and the corresponding versions are saved in the same data base.Moreover, various views, such as design view, manufacturing view, etc., are attached to the product life state.The bill of materials is a structure decomposition of the product, which provides reference to identify parts of complex products and to access a set of representations or views.Indeed the bill of materials is a structure decomposition of the product.
The structure decomposition does not provide a complete description of the product.Gero [1] highlighted the cognitive dimensions of product analysis as the function, behavior and structure descriptions (FBS).Many models were developed to integrate functional and behavioral representations of the products.The Core Product Model was an attempt to propose an extensible standard model.Its kernel definition was expressed by Fenves [2,3] through a UML class diagram (Figure 1).Bond-graphs were proposed to focus on function and behavior representations [4].The product part of IPPOP (Integration Process Product for Performance Optimization) model focused on establishing links between FBS dimensions [5].The FBS-PPRE model claimed to integrate manufacturing processes and resources to the FBS model [6].All these models fail in describing product behavior which remain a polysemous concept [7].No model succeeded in becoming the standard.A recent effort has been to extend UML to SysML by arguing that the product is a system and adding requirements and parametric diagrams [8], including to functional and behavior diagrams.
STEP is a special case which is often reduced to an exchange standard, but initially it was built to model the overall life cycle of a product [9].Indeed the norm the EXPRESS language acts as an editor of conceptual models [10], which is similar to the UML diagram class.As in UML, an EXPRESS diagram is a set of relationships between concept classifiers which have basic attributes.A diagram is a conceptual view of a partial domain as it will be modeled and exchanged through the standardized files, but it could be also directly used as a modeling data structure.The various chapters of the STEP standard define partial views of PLM aspects.STEP fails to provide a coherent holistic view for PLM, because the various parts were defined separately and intersection concept may differ with respect to the corresponding chapters.Moreover STEP releases are too often late and lag after commercial innovations, making STEP obsolete.
Indeed, product models are presented as 2D diagrams, and when they are instantiated to show all the actual instances of one or more object classes, perception of and interaction with the model becomes a real challenge.

Activity Models
Similar to product models, activity models have also been proposed in the PLM domain.A simple and comprehensive activity model widely used in industry for short term projects is the work breakdown structure (WBS) [11].WBS seems to be very efficient, because it is easily understood by any engineer.Yet, it is just a tree decomposition of a task.Such decomposition was widely adopted in most activity models starting by the series of IDEFx process models or within project management tools, such as Microsoft Project.Several UML diagrams convey activity: state machines, activity diagrams, sequence diagrams, and even use case diagrams.Business Process modeling Notation (BPMN) was proposed in 2006 by the Object Management Group (OMG).These activity models propose diagram representations that make them quite intuitive, as illustrated by the BPMN diagram in Figure 2.

Fig. 2. An example of a BPMN model [12]
Activity models usually present an a priori model of the expected process.The major problems with these representations concern three aspects: • These representations are not easy to update, share and create on the fly; they lack agility.• They are seldom reused and almost every project leads to a definition of a new model.Moreover, the model is supposed to fit all its instances.• Complexity is poorly managed, as decomposition is hardly undertaken.
Every non-trivial activity has to be decomposed into sub-activities, and this recursive process must be stopped whenever the right balance between modeling need and model comprehension efficiency is reached.

Complex information systems and user interfaces
A model is not an end in itself, rather, it is expected to support some human expert activity.Individuals should be able to extract, edit, and share information while maintaining a coherent model.This idealized view has not been reached, and therefore partial solutions are provided to support human experts as they advance along the PLM process: • Groupware is widely used to share models.To support coherence, the most popular solution has been a check-out/check-in protocol, which enables users to protect potentially modified data and inform collaborators that they are doing so.Synchronization solutions exist, but they are mainly practiced in software development projects.Application of such solutions to a wider range of domains have been investigated [13] but only partly realized.• Mapping to an expert view: expert dedicated models are produced either by manual reconstruction from scratch or by and by hand, by extraction and conversion of common models to a dedicated view.A mapping to a 3D model is a usual practice within product development, because it fits the physical view of a real product.This approach is used obviously in CAD systems, but also for manufacturing simulators and many other behavior analysis systems, such as finite elements method.Such mappings provide a metaphor to visualize and interact with a partial view of PLM and help resolve interoperability issues.Interoperability and Groupware are the major ways to create connections across a huge data set, and providing a more complete and holistic information PLM model.The research question in this regard is how to enable a holistic understanding of a complex information system?The assumption of this study is that a 3D model may be used to extend the capacity to present instances of object classes in an information system even if the information does not have an obvious 3D physical metaphor.
The next section presents a conceptual model of a real use case supported by a process-driven information system.A 3D representation of an instance of this process is then introduced and tested on various 3D viewers to assess how the 3D presentation helps manage this complex information.

An infrastructure project with a complex 4.1 VISIONAIR activities
VISIONAIR is a European infrastructure for research in visualization technologies, which operated between 2011 and 2015.The project acronym means "VISION Advanced Infrastructure for Research".The 25 partners involved in the project operate 29 high-level visualization and interaction facilities that share a technologyoriented vision.The 29 facilities were associated to four technological domains: 1. Augmented collaborative environments, including holography, were supported to enable local or remote collaboration of actors.2. Ultra-high definition and networking facilities provided ultra-high definition (UHD) displays (4K and 8K, even in 3D), including streaming the corresponding content on networks.

Virtual reality (VR) facilities included immersive environments such as
CAVEs, where the user is immersed in virtual scenes.4. Scientific visualization facilities were dedicated to the navigation and understanding of results of data output by high performance computing or from physical observation.
The infrastructure hosted external research projects.A call for projects was launched and any European research could apply for a Trans-National Access (TNA) project.A process, described in detail in the sequel, was designed to evaluate project proposals in order to accept or reject them.An information system was created to manage the 230 TNA applications received over the project's four years and ensure the realization of 122 of them, enabling projects that cover a wide spectrum of disciplines, including the following: This long list demonstrates the variety of projects managed within VISIONAIR and the difficulties associated with gaining control and managing the projects.Obviously, we implemented information systems to support both the evaluation of the proposals and the collaboration with the applicant.This use case is a service deployment rather than a product design activity and thus the corresponding "product model" is much simpler.In the following sections, we present the OPM conceptual framework to model this process.

OPM conceptual model
Object-Process Methodology (OPM) is an approach and a language for modeling complex systems using a compact set of concepts [14].It is a holistic graphical and textual paradigm for the representation and development of complex systems in a formal framework.OPM is a tool for system engineers and it enables representing systems simply in a single model, expressed in both graphics and equivalent natural language.
By using a single holistic hierarchical model for representing structure and behaviour, clutter and incompatibilities can be significantly reduced even in highly complex systems, thereby enhancing their comprehensibility.OPM has proven to be better in visual specification and comprehension quality when it was used for representing complex reactive systems compared to the standard in the field of systems engineering.
One major benefits of OPM is the combination of the product model through object definition and the activity process which consume an object in a given state and produce objects in a new state.The three concepts of the FBS paradigm are joined in a single 2D diagram: Structure is related to objects, Functions are related to activities and Behaviour is related to object states.This combination makes it ideal for collaborating and communicating knowledge and ideas, via visualization-based platform.
An OPM diagram is composed of 3 main symbols plus links: • Objects are represented by rectangles.The name of the rectangle identifies the concept, which is either an actor or an object.Within Figure 3, a "Research group", a "Visionair consortium", a "Project" are objects managed through the process.
• Object States are represented by rounded rectangles placed in the object rectangle.One object has a single active state and is identified with its current state.The "research group" in Figure 3, may have several states: "unaware", "aware", "waiting for feedback", "project rejected" and "project approved" • An activity is represented by an ellipsis.An activity is decomposed in several activities by drawing ellipsis inside the father ellipsis.There is no recursive decomposition on a single diagram.If a sub-activity must be decomposed this second level of decomposition will be drawn in a secondary diagram.• Several links are available: • consumption, and production link: • control link • inheritance link: the traditional specialization link of concept • link These concepts are presented in Figure 3. Through a first draft model of the transnational project activity process.

3 OPM description of VISIONAIR TNA process
Object-Process Methodology, OPM [14,15] is a systems modeling paradigm that represents the two things inherent in a systems: its objects and processes.OPM, which has become ISO 19450 Specification in Aug. 2014 [16], is an ideal visual-textual platform for not just engineering systems and managing human knowledge, but also for extending and evolving it by combining disparate facts within a formal, intuitive, consistent, reliable, and evolvable OPM model.
OPM is fundamentally simple; it builds on a minimal set of concepts: stateful objects-things that exist, and processes-things that happen and transform objects by creating or consuming them or by changing their states.Another fundamental advantage of OPM is that it represents the system simultaneously in graphics and natural language, and the two are completely interchangeable.OPM enables clear representation of many important features of a system at various levels of detail.OPM semantics was originally geared towards systems engineering, as it can model information, hardware, people, and regulation.
OPM was selected for modeling the TNA process because of the the clear conceptual overview of the process it porvides.Basically, a TNA process involves an external "Research Group" which is supposed to apply for a project.VISIONAIR consortium members are in charge to support this Research Group.First a Review Committee is organized for assessing the proposed "Project" and a "Host Facility" technically supports the project if and when it is accepted.
Objects are denoted in OPM by rectangles and the states of an object are denoted by rounded-corner rectangles inside the object.Examples of states of the object Host Facility are expecting an incoming project, preparing the execution of the project, hosting the Research Group, and finally completed experiment (see Figure 3).
Processes, denoted as ellipses, transform objects by ( 1) consuming them, (2) producing them, or (3) changing their state.The TNA process is comprised of the following main subprocesses: first promotion was made by the review committee and projects were solicited.An interested Research Group creates an account and can apply for a project.Then it waits for the results of the Reviewing process, which produces the informatical object Review Outcome.Depending on this outcome, the project may be rejected, accepted, or modifications may be requested.The acceptance activity notifies the Host Facility and the Research Group, which collaborate for remote preparation, then for co-located set of one or more experiments (see Figure 3).The 2D diagram in Figure 3 provides an overall view of the process, but the number of arrows gives an idea of the complexity of the managed process.Each object "knows" its current state and can clearly identify the process that should be processed next.As soon as a project was submitted, the Research Group knows that it must wait for review outcome.Every process may be split into sub-processes via OPM's in-zooming mechanism, but in this paper we do not make use of this refinement mechanism.
While each single actor (object) knows its current state and activity opportunities, the complete management of all the simultaneous projects, the corresponding research groups and host facilities remains a real challenge, as one needs to be on top of all the simultaneously occurring projects.to figure out which research groups and host facilities are concerned, or rapidly get the list of projects in a given state.We produced a database system that reflects the TNA project process.Only humans can achieve a holistic picture by integrating several partial results from querying the database.In the next section we present tour 3D metaphor for improved management of the TNA process.

3D visualization of the TNA process
A few works investigated the potential of visualization for information system.Bihanic and al. [17] proposed various way to visualize graph data.Hayka and al. [18] developed new metaphors for multitouch tables to visualize a PDM bill of materials.Sadeghi investigated the management of models for project reviews [19].The current work is a new step in these investigations.

3D metaphor
As argued, the OPM 2D diagram provides an efficient conceptual view of the process but it cannot visualize a set of object instances.The core idea of the 3D metaphor is to take advantage of the third dimension, which is available with current visualization technologies.The third dimension is used to list the instances concerned an object state.We then wish to assess the benefit of improved process control as a result of using the third dimension.An interaction behavior was encoded.A trackball system was designed to support free navigation in 3D over the scene.When selecting an object in a certain state, the list of object instances are drawn along an axis orthogonal to the diagram plane.Figure 4 shows a specific view where the "Host Facility" was selected at the state of "preparing a project".The projects which were accepted show up on the right column, identified by their index, e.g., Project #75 at the top, while the host facilities are listed by their name and (in parentheses) the number of projects that the Host Facility is preparing to host.
With this single view, a user gets a clear picture of the concerned facilities and projects in response to his or her graphics-driven query.By selecting an instance from one the two lists, additional information can be printed in an information window.For a project, one can get the project name, the Research Group, dates of submissions and of state change, etc. Depending on the working mode, selecting an instance may visualize the related items.For example, if a specific Project is selected, the relevant instances of the object classes Research group, Host Facility, Review Committee with their current state are presented.This visualization provides the system executives and operators with a clear view and rapid understanding of all the information related to a given project.

Projection on various device
Our 3D metaphor was tested on several devices.First, we used a stnadard 2D desktop.Figure 4 and Figure 5 were produced this way.To test for information overload, in Figure 5 several states of the Research Group object were selected, and indeed, as Figure 5 shows, there is too much information and the added value of the 3D instance presentation is lost.
We then used a 3D active stereoscopic full HD display with a Crystie back video projector on a wall 3 meters wide and 2 meters high.As Figure 6a shows, the OPD is oriented from top to bottom.
The metaphor was also tested on a powerwall.As Figure 6b shows, the powerwall is made of 9 Samsung © LCD 55" panels with very narrow, 5mm sides between any two adjacent displays.
Finally, the metaphor was tested on a head-mounted display (HMD) Oculus © DK2.This technology is both affordable and can be easily installed on standard desktop computers.However, interaction with this device is not convenience, and some dizziness may appear due to loss of contact with real space.In our case, such uncomfortable feelings due to large displacement are not an issue, since the global scene may be really to avoid large head movements.Since interaction with HMD is challenging, a mouse was used, which required some training.

Overall perception
The assessment focused on the overall subjective display perception rather than on interaction issues.While we did not have a large enough sample to allow for statistical analysis, the differences in perception were very obvious.Three factors appear to be significant: • Dimensionality: a traditional 2D desktop and the power-wall were used for projecting the 3D scene on a 2D display.The HMD and the stereoscopic back projection systems are referred to as 3D spaces.• Resolution: the tested 2D desktops were either at very high resolution 4K or Full HD.Both the 3×3 power-wall and the stereoscopic back projection system have Full HD resolution.The HMD is a Full HD display used in side-by-side mode, so each eye has half full resolution in the horizontal direction.
• Brightness: all the displays have high brightness, but the brightness of the power-wall is video projector.An important usage issue is text readability.Low resolution is a drawback, and this deficiency increases with stereoscopy, where even a small alinement defect hinders text readability, making it unusable for information systems.Thus, HMD is currently still an immature technology, while systems with higher resolution are preferred.The 3D superposition of text in on 2D displays makes them hardly readable.The 3D spaces are more adequate to perceive 3D superposition of text, but on the other hand, resolution in 2D displays is better, so the power-wall provides better rendering of text with 3D superposition.
The immersion in 3D space allows putting the diagram on a horizontal plane and perceiving the instances as being lists perpendicular to that plane.The object states far from the observer on the horizontal plane are the initial states and the closest states are easily perceived as the final states, making the project time line clearly aligned from far (origin, past) to close (destination, future).With full immersion and tracking, the observer can overcome the difficulties arising with text superposition in 2D desktops.However, the projection of text in 3D is at a lower resolution, making it less readable.When a lot of text is projected the user loses the ability to focus on the text, and some of it also occludes other text.Higher resolution is needed for this approach to be implemented in a valuable tool.
With respect to interaction comfort, using HMD, traditional desktop, and 2D power-wall, the user can sit down.We have not implemented 3D interaction at this stage and used a 2D mouse in this first test.3D stereoscopic back projection system appears to better fit a clear perception of 3D, as navigation becomes easier, but resolution must improve for better text readability.

Conclusion
Among existing conceptual modeling approaches and diagram kinds, OPM has proved to be very efficient, because it combines both processes and state-full objects -the only two building blocks in the universal OPM ontology.In our application we, have found it to be a clear and nice tool to collaborate about the conceptual definition and management of the complex trans-national access process.Since, like any other diagram kind, OPMs are not adapted to the description of instances, as soon as more than a single instance is running under the process, a new approach is in order.
The complexity of the VISIONAIR EU FP7 Project, in which more than 200 projects were simultaneously managed in 29 Host Facilities by one Review Committee, called for designing and assessing new management metaphors.
Using the 3D representation metaphor, we have demonstrated a clear advantage.Additional research and development must take place in order to formalize this approach and take advantage of the opportunities opened by this study.
The various rendering technologies and devices must also be assessed.Obviously, real immersion in the 3D model with ultra-high definition will provide higher added value than the devices we have tested.
The current state-of-the-art and expected trends promote usage of HMDs, which are becoming affordable and easier to deploy in an office environment.We plan to extend this work by implementing and surveying new devices, improving the 3D metaphor, and designing and assessing new interaction systems.
In any case this study opens widely new perspectives towards the usage of virtual environments in the PLM area.

Fig. 3 .
Fig. 3. First draft of a TNA Project Process

Fig. 5 .
Fig. 5. 2D Desktop visualization of several states of the same object A 3D viewer was used to visualize the OPM diagram.The original OPM diagram, called Object-Process Diagram (OPD), was simply drawn on a plane and thus remains unchanged with the same semantic.An interaction behavior was encoded.A trackball system was designed to support free navigation in 3D over the scene.When selecting an object in a certain state, the list of object instances are drawn along an axis orthogonal to the diagram plane.Figure4shows a specific view where the "Host Facility" was selected at the state of "preparing a project".The projects which were accepted show up on the right column, identified by their index, e.g., Project #75 at the top, while the host facilities are listed by their name and (in parentheses) the number of projects that the Host Facility is preparing to host.With this single view, a user gets a clear picture of the concerned facilities and projects in response to his or her graphics-driven query.By selecting an instance from one the two lists, additional information can be printed in an information window.For a project, one can get the project name, the Research Group, dates of submissions and of state change, etc. Depending on the working mode, selecting an instance may visualize the related items.For example, if a specific Project is selected, the relevant instances of the object classes Research group, Host Facility, Review Committee with their current state are presented.This visualization provides the system executives and operators with a clear view and rapid understanding of all the information related to a given project.

•
Improvement of engineering processes: interacting with fluid dynamics simulations, weather forecast, new style design, extended vision and perception of technical 2D drawings, manufacturing process analysis or simulation, augmented tools for manufacturing, manufacturing plant layout arrangement, and training on assembly and disassembly of products.• Remote collaboration: remote physical feedback by handling haptic devices from a long distance, co-working with shared 4K panels, co-organisation of nurse schedule, surgery tele-operation, and remote collaborative concerts involving musician and dancers located in various countries and participating in a same concert within a virtual environment.• New tools and methods for medical applications: ergonomic analysis to support sport analysis activities, training gun fighters, analysing rugby and basketball gestures, and enabling new protocols of remote interconnections between athletes and coaches.3D ultra-high definition quality with low bandwidth and low latency over networks to support this kind of remote collaboration was demonstrated.• Museum applications: storytelling and reconstruction of fragile historical sites for patrimony conservation and archeological studies.• Virtual visit: visits of faraway planets with data incoming from observatories, exploring seas and animals scanned from tomography, and other applications including studies about virtual environment perceptions and psychology, etc.

Table 1 .
Qualitative comparison of displays