A Multi-agent Approach to Implement a Reverse Production Virtual Market in Green Supply Chains

. The implementation of internal reverse production process programs often involves signiﬁcant allocations of capital and resources for the construction and implementation of all the steps in the process. But, what if we think of reverse production process as a service-based manufacturing network, in which all the activities are outsourced and the only thing that a manufacturing company needs in order to participate is an interface/service to “play” in that ecosystem. In this work we present an approach to implement reverse production process following a Service-Oriented Manufacturing paradigm by means of a virtual market supported by intelligent software agents.


Introduction
Over the last few years there has been a well-recognized need for achieving overall sustainability in manufacturing activities [6,8], due to several established and emerging causes: environmental concerns, diminishing non-renewable resources, stricter legislation and inflated energy costs, increasing consumer preference for environmentally friendly products, etc.In this way corporate environmental responsibility has gradually and more consistently extended beyond complying with increasingly stringent environmental regulation and also beyond the taking up of proactive initiatives.Many research works have demonstrated the positive effects of internal measures towards sustainability [9].Lately the goal has begun to shift to integrating all of the business value-adding operations, including purchasing and in-bound logistics, production and manufacturing, distribution and out-bound logistics, in such a way that activities associated with these functions have the least harmful environmental impact [13].Greening the supply chain is one such innovative idea that is fast gaining attention in the industry [3].
There are a number of definitions and philosophies on what a green supply chain may be.These vary from something as small as buying green products from a supplier to the much broader context of an industrial ecosystem [5].Sustainable Corresponding Author supply chain is an active research field in which new techniques are continuously proposed to reduce negative environmental impacts while pursuing production economy [13].Sustainable supply chain network consists of forward production processes and reverse (recycling) production processes (reverse logistics).In the specialized literature, most of the attention is targeted to the forward action of the supply chain [5], whereas few manufacturers have considered how this supply chain can or should work in reverse to reclaim products at the end of their lifecycle and return them through the supply chain for decomposition, disposal or re-use of key components.Systems in which the two approaches (forward and reverse) are combined can drastically reduce the negative impacts to the environment [17].These systems are called closed-loop production systems 1 .
The implementation of internal reverse production process programs often involves significant allocations of capital and resources for the construction and implementation of all the steps in the process.But, what if we think of reverse production process as a service-based manufacturing network [7], in which all the activities are outsourced and the only thing that a manufacturing company needs in order to participate is an interface/service to "play" in that ecosystem.
In this work we present an approach to implement reverse production process following a Service-Oriented Manufacturing paradigm by means of a virtual market supported by intelligent software agents [18].In a nutshell our approach consists of an ecosystem of services (i.e.web services) that implements a virtual market for buying and selling items subject to be recycled (i.e.old materials, wastes, used products, etc.).A company will participate in the market: (i) when the company gets economic profit from buying or selling those items, or; (ii) when the company is trying to reduce its "cuota" of negative environmental impact (i.e.reducing CO2 emissions, waste of materials, pollutants, etc.).

Green Supply Chains
Translating the definition of sustainable development to supply chains impose the need to migrate from a field that myopically addressed only operational and economic matters to one that comprehensively considers the broader environmental and social issues that face organizations of today [5].A consensus definition of green and sustainable supply chains does not exist (for a large review on definitions see [1]).It is identified as a sub-discipline of supply chain managing expanding the work in a variety of areas.An in-depth state-of-the-art review on green supply chains can be found in [5].In this work we consider a manufacturing supply chain as a system that consists of 5 layers, including raw material supply, manufacturing, wholesaling, retailing, and end-customers.
Whereas, a used-product and/or waste materials reverse logistics chain includes collecting points, recycling plants, disassembly plants, secondary material markets, and final disposal locations of wastes.All these activities are executed with the purpose of capturing value, or proper disposal.
The keys to the successful design and use of reverse logistics systems include costs, overall quality, customer service, environmental concerns and legislative concerns.Other factors to consider are: cost-benefit analysis, transportation, warehousing, supply management, remanufacturing and recycling, and packaging.All these factors require high economic costs for a company to implement a reverse production process.In this way financially attractive approaches are urgently required to facilitate the participation and commitment of manufacturing companies and customers in green supply chains.Bearing this issue in mind and trying to contribute to the state-of-the-art on research works related with reverse production systems, recognized as still low [5], we propose a virtual market of used-products and/or waste materials supported by artificial intelligent techniques and service oriented manufacturing approaches.

MAS and Intelligent Manufacturing Systems
New technologies are revolutionizing the way manufacturing and supply chain management are implemented.Initiatives such as Industrie 4.0, Smart Industry, Industrie du Futur, among others from Europe; Industrial Internet Consortium from USA; Industrial Value Chain Initiative from Japan; Made in China 2025 from China, are trying to integrate Internet and manufacturing systems in order to fully digitalize the factories of the future.This new production paradigm is based on concepts as autonomy and co-operation because both are necessary to create flexible behavior and thus to adapt to the changing production conditions.One such approach is the Intelligent Manufacturing Systems (IMS) paradigm.In IMS the manufacturing system is conceived as a distributed system in which its constituent components, such as machines, resources, products, and staff, have intelligent capabilities for acting in its environment pursuing global system goals; and have autonomous execution for decision making, social interaction with other intelligent entities, and collaboration for achieving the system goals.
Agent based Manufacturing Systems, are based on Multi-Agent System (MAS) technology [2].MAS is focused on the co-ordination of entities, called agents, with intelligent behaviors that interact in a group of (possibly pre-existing) agents.A closely related approach to MAS is Holonic Manufacturing System (HMS).HMS is based on the concept of "holonic systems" to describe autonomous manufacturing modules (holons) with distributed control.In a HMS, a holon can be used for transforming, transporting, storing and/or validating information and physical objects.A holon has an information processing part, in charge of the logical and intelligent computation; and often a physical processing part (optional), in charge of controlling a physical component: a machine, a tool, etc.In this way holons are a powerful approach to implement Cyber Physical Systems for the factories of the future in Industrie 4.0.The specialized literature in IMS, focused on MAS and HMS, offers successful approaches for achieving sustainability in manufacturing systems.The concept of Go-green Holon proposed in [15], is a pre-built development artifact that includes efficiency-oriented mechanisms (optimizing sustainability means), in addition to classical effectiveness-oriented mechanisms, to make a decision and/or execute an operation in IMS.Go-green ANEMONA [10] is a complete software engineering method for developing sustainable IMS.The method helps the designer to specify and implement sustainable optimization functions in IMS providing development guidelines and modeling templates.On the other hand, in [11] it is proposed the application of agent-based systems for supply-chain synchronized production planning including management of raw materials flow as well as flow of returned by customer obsolete products and defected semi-products and products that are refused by quality control within the factory.Whereas, in [12] a multi-agent system framework to achieve coherent and consistent workflows that can meet order requirements is proposed.

A Virtual Market approach for Reverse Production Systems
The overall idea behind the proposed approach is to facilitate the participation of stakeholders in the activities of green supply chains by means of Service Oriented Manufacturing Systems [7], providing an easy to use interface for the players of a wider reverse logistic ecosystem as a step towards virtual outsourcing of reverse manufacturing processes.The focus of the work presented in this paper is the market of products subject to be recycle.Figure 1 shows the structure of the virtual market.In the market we can find different roles: Buyers of items (old materials, wastes, used products, etc.) that want to further process the items or just want to re-sell them in other markets; Sellers that want to get economic profit (or even "green" profit) from the disposable items (i.e. if the seller is a manufacturing company, the items may usually come from their production activities, being wastes, old materials or used products.etc.); Market Staff which is a special role that is in charge of ensuring the supporting activities in the market (i.e.execution control activities, enforcement activities for the market rules, etc.).The virtual interaction among these roles are implemented by means of web services, that are published in the Cloud and managed by a Service Oriented Architecture (SOA) framework.The services provide the basic functions to play in the market ecosystem such as: Negotiation Protocol Service, Call for Bid Service, Bid Service, etc.
The three roles of the market are implemented by means of agents.In our approach we follow the guidelines from Go-green ANEMONA [10] in order to design the agents and the negotiation interactions in the market.The Buyer and Seller roles are specified as Go-green Holons [15].Whereas, the interaction functions of the market are encapsulated as web services following the process definition of Service Oriented Manufacturing System approach [7].
Picture a market, where customers (Buyers or Sellers) are involved in faceto-face negotiation, participate in auctions or convene to consolidate an offer to other large customer.A market with several negotiation dialogs, each with a specific negotiation protocol and each with its own access and termination conventions (and other rules).Moreover, a market that also allows information exchange among the participants, request to open or enter a negotiation dialog, invite players to negotiation dialogs, and participants to reconvene after leaving a negotiation dialog.We capture this global arrangement with the MAS approach described in next sub-sections.

The agents
The three main roles mentioned above are, in turn, specialized into five roles that may interact.The customer (or party agent) of the market that can participate as Buyer (b) and/or Seller (s).Finally, there are three types of Market Staff roles.The Mediator role (m) represents framework agents who run standard negotiation activities for example managing the users data, the structure, the specific parameters of the negotiation protocols, etc.The Negotiation Dialog Manager role (dm) represents framework agents who execute activities that are specific of a given negotiation protocol, for example accept valid negotiators, tune negotiation parameters of the dialog, mediate in the negotiation process, expel negotiators, etc.The Legal Authority role (la) represents framework agents who are in charge of activities for agreement enactments that are executed as a result of a successful negotiation process.
The ontology structure of the negotiation framework is defined by the following key entities in the market which constitute the core concepts that the framework recognizes: An item is defined as ι = τ ι , O, ρ ι , where τ ι is the item type, O is the ontology where the item ι is defined, and ρ ι is a set of property values that define the particular attributes that characterize the item itself.The list ρ ι is domaindependent, and is defined using the approach presented in [14].For example the following is a fragment description of a waste ItemX that also specifies its component materials.
Individual: ItemX Types: GenericProducts, Waste and (contains some((material-value some Glass) and (material-percentage value 35)) and (contains some((material-value some Plastic) and (material-percentage value 62))) and (has price((price-value 40) and (price-currency Euros))) ... This ontology description is used by the agents in the market to reason about the items and evaluate the profit they can get from the item negotiation.
A deal δ = {ι} is a sequence of items ι that can be negotiated.
An agreement is a full instantiated deal among two or more parties.It is defined by a tuple α = P, δ , d, st , where P is the set of parties that enact the deal; δ is an agreed deal from a previous negotiation process; d is the agreement date; st represents the stage where the agreement currently holds -it can take a value from a enumeration, e.g.pending, executed, cancelled, etc.

The Negotiation Structures
The virtual market provides a set of services that are organized in two different groups: informational, and negotiation (see Tables 1).The informational set of services lists ongoing deals and active agreements, and also provides invitations or call for negotiations.Whereas, the negotiation set allows to create a negotiation dialog, to participate into a negotiation dialog, and once an agreement on a sequence of items has been successfully reached, it allows to settled the agreement according to the given conventions.
In order to assure a regulated execution of the virtual market, special attention is made in order to enact an agreement.First of all, the Mediator checks whether or not the agreement satisfies some formal conditions.If the agreement complies with these, a transfer contract is agreed upon and signed by the customer agents involved, and then the agreement becomes active.Once an agreement is active it may be executed and, consequently, other customer agents may initiate a grievance procedure that may overturn or modify the agreement.Even if there are no grievances that modify a contract, customers may not fulfill the contract properly and there might be some contract reparation actions.If things proceed smoothly, the agreement subsists until maturity.

Conclusion and Future Works
In this paper a MAS supported Virtual Market approach for reverse production process was presented.The goal of the proposed approach is to facilitate the participation of stakeholders in the activities of green supply chains by means of Service Oriented Manufacturing Systems [7], providing an easy to use interface for the players of a wider reverse logistic ecosystem as a step towards virtual outsourcing of reverse manufacturing processes.The proposed framework define EnterNegotiationDialog inf ormRole(px, dm, dialog ID , pt), where px is the participant; dm is the negotiation dialog manager; dialog ID is the dialog ID; pt is the participation type, b or s.
Bid sendBid(px, dm, dialog ID , pt, πx, δ), where px is the participant; dm is the negotiation dialog manager; dialog ID is the negotiation dialog ID; pt is the participation type, b (the bid is a Put) or s (the bid is a Call); πx is the offered price; δ is the item set that is being negotiated.
three different players: Buyer, Seller and Market Staff.The virtual interaction among these roles are implemented by means of web services, that are published in the Cloud and managed by a SOA framework.The services provide the basic functions to play in the market ecosystem such as: Negotiation Protocol Service, Call for Bid Service, Bid Service, etc.The framework is currently under development in order to implement a first prototype to perform validation tests.Moreover, the authors are interested in enhancing the work in order to "close-theloop" allowing to retro-feed the internal sustainable manufacturing operations scheduling functions (such as [4]), that are running in the manufacturing companies, in order to fine-tune the values of the sustainable variables used by these internal algorithms for better optimizing the sustainable constraints.

Table 1 .
Negotiation Services ID , error), where m is the manager, px is the customer, dialog ID is the ID of the negotiation dialog, or a nill value when there is an error.ID , error), where α ID is the ID of the agreement.CallForBidWhen the negotiation dialog has a P ublic type of access, m broadcasts an invitation message to all the participants: inviteAll(m, dialog ID , protocol, δ, C), where dialog ID is the negotiation dialog that is receiving players; the negotiation protocol protocol used in that dialog; the set of issues, δ, that is being negotiated; and the set of constraints, C, to participate in are also made public.When the negotiation dialog has a P rivate type of access the invitation message is invite(m, py, dialog ID , protocol, δ, C), where each candidate py ∈ P dialog ID .CreateNegotationDialog request(px, m, open, protocol(params), δ, pt, at, C), where the semantic is as follows; Party agent px requests to the Mediator, m, to open a negotiation dialog with the negotiation protocol protocol.The negotiation protocol is instantiated with the set of values for the parameters params.The dialog is created to negotiate about a deal δ.The requesting party, px, will participate as pt that can take one of these values: p, that is an observer Party; a Buyer party b; or a Seller party s. at is the access type that can be: Public, any body can be invited; or Private, only Party agents that fulfill the set of constraints C can participate in the negotiation dialog.