Collaborative Model in Brain-Computer Integration

. Brain-computer integration is a new intelligent system based on brain-computer interface technology, which is integrated with biological intelligence and artificial intelligence. In order to make this integration effective and co-adap-tive biological brain and machine should work collaboratively. ABGP-CNN based environment awareness and motivation driven collaboration will be presented in the paper. Motivation is the cause of action and plays important roles in collaboration. The motivation leaning method and algorithm will be explored in terms of event curiosity, which is useful for sharing common interest situations.


Introduction
Brain-computer integration is a new intelligent system based on brain-computer interface technology, which is integrated with biological intelligence and artificial intelligence [1]. Brain-computer integration is an inevitable trend in the development of braincomputer interface technology. In the brain-computer integration system, the brain and the brain, the brain and the machine is not only the signal level of the brain machine interoperability, but also need to integrate the brain's cognitive ability with the computer's computing ability. But the cognitive unit of the brain has different relationship with the intelligent unit of the machine. Therefore, one of the key scientific issues of brain computer integration is how to establish the cognitive computing model of brain-computer integration. At present, brain-computer integration is an active research area in intelligence science. In 2009, DiGiovanna developed the mutually adaptive brain computer interface system based on reinforcement learning [2], which regulates brain activity by the rewards and punishment mechanism. The machine adopts the reinforcement learning algorithm to adapt motion control of mechanical arm, and has the optimized performance of the manipulator motion control. In 2010, Fukayarna et al. control a mechanical car by extraction and analysis of mouse motor nerve signals [3]. In 2011, Nicolelis team developed a new brain-machine-brain information channel bidirectional closed-loop system reported in Nature [4], turn monkey's touch information into the electric stimulus signal to feedback the brain while decoding to the nerve information of monkey's brain, to realize the brain computer cooperation. In 2013, Zhaohui Wu team of Zhejiang University developed a visual enhanced rat robot [5]. Compared with the general robot, the rat robot has the advantage in the aspects of flexibility, stability and environmental adaptability.
Brain-computer integration system has three remarkable characteristics: (a) More comprehensive perception of organisms, including behavior understanding and decoding of neural signals; (b) Organisms also as a system of sensing, computation body and executive body, and information bidirectional exchange channel with the rest of the system; (c) Comprehensive utilization of organism and machine in the multi-level and multi-granularity will achieve system intelligence greatly enhanced.
The core of brain-computer integration is the cognitive computation model of brain computer collaboration. Cognitive process of brain computer collaboration is composed by the environment awareness, motivation analysis, intention understanding and action planning and so on, in support of the perception memory, episodic memory, semantic memory and working memory to complete brain machine group awareness and coordinated action. Supported by the National Program on Key Basic Research Project we are engaging in the research on Computational Theory and Method of Perception and Cognition of Brain-Computer Integration. The main goal of the project is the exploration of cyborg intelligence through brain-computer integration, enhancing strengths and compensating for weaknesses by combining the biological cognition capability with the computer computational capability. In order to make this integration effective and co-adaptive, brain and computer should work collaboratively. We mainly focus on four aspects, environment awareness, cognitive modeling, joint intention and action planning, to carry out the research of cognitive computational model.
(1) The environmental awareness of brain computer collaboration. For brain computer bidirectional information perception characteristics, the integration of visual features of the Marr visual theory and Gestalt whole perception theory in the wide range, research on the environment group awareness model and method by combination of brain and computer. The discriminative, generative and other methods are applied to analyze the features of environment perception information, mine perception information patterns and knowledge, generate high-level semantics, and understand well the environment awareness, build brain computer collaborative group awareness model.
(2) Cognitive modeling of brain computer collaboration. This involves combining the mutual cognitive characteristic of the brain computer, utilize and study the achievement made in intelligent science about consciousness, memory, study the cognitive cycle in brain computer collaborative information processing. According to the characteristic of the collaborative work of the brain computer, utilize the research results of physiological mechanism of brain information processing, study information representation method and reasoning mechanism in episodic memory and semantic memory of human brain, carry on cognitive modeling to information processing procedure in the brain computer collaboration.
(3) The joint intention driven by motivation. The essential characteristic of the brain computer collaborative work is that the agents have common goals, commitments, intentions, etc. that are jointly restrained. In order to describe the characteristics that should have for the brain computer system, study the joint intention theory to describe union restrictions of autonomous agents and reason balance of agent mental state; To the needing of brain computer collaborative work, study the essence of the behavior motivation and generation mechanism, put forward the intention model driven by motivation; Study the joint intention method for multi-agents, offer theory support for constructing the brain computer collaborative work of multi-agents.
(4) Action planning for brain computer collaboration. Under the support of ontology knowledge system, study action planning method of brain computer collaboration; using reinforcement learning and Markov decision theory, study the part of perception of the planning method, present the action planning theory with learnability and optimization methods.
Collaborations occur over time as organizations interact formally and informally through repetitive sequences of negotiation, development of commitments, and execution of those commitments. Both cooperation and coordination may occur as part of the early process of collaboration, collaboration represents a longer-term integrated process. Gray describes collaboration as a process through which parties who see different aspects of a problem can constructively explore their differences and search for solutions that go beyond their own limited vision of what is possible. [6] In this paper, a collaborative model for cyborg intelligence will be proposed in terms of external environment awareness and internal mental state. The collaboration between brain and computer is driven by motivation which is generated dynamically.

Conceptual Framework of Brain-Computer Integration
An effective approach to implementing engineering systems and exploring research problems in cyborg intelligence is based on brain-computer integration methods [7]. Using these methods, computers can record neural activity at multiple levels or scales, and thus decode brain representation of various functionalities, and precisely control artificial or biological actuators. In recent decades, there have been continuous scientific breakthroughs regarding the directed information pathway from the brain to computers. Meanwhile, besides ordinary sensory feedback such as visual, auditory, tactile, and olfactory input, computers can now encode neural feedback as optical or electrical stimulus to modulate neural circuits directly. This forms the directed information pathway from the computer to the brain. These bidirectional information pathways make it possible to investigate the key problems in cyborg intelligence. How to interact between brain and computer at various is a critical problem in braincomputer integration. On the basis of the similarity between brain function partition and corresponding computing counterparts, a hierarchical and conceptual framework for brain-computer integration is proposed. The biological part and computing counterparts are interconnected through information exchange, and then cooperate to generate perception, awareness, memory, planning, and other cognitive functions.
For the brain part, abstracted the biological component of cyborg intelligence into three layers: perception and behavior, decision making, memory and consciousness ( Figure 1). We also divided the computer functional units into three corresponding layers: awareness and actuator, planning, motivation and belief layers. We also defined two basic interaction and cooperation operations: homogeneous interaction (homoraction) and heterogeneous interaction (heteraction). The former represents information exchange and function recalls occurring in a single biological or computing component, whereas the latter indicates the operations between the function units of both biological and computing parts. Homoraction is also modeled as the relationship between units within the same part. In the case of a single part in a brain-computer integration system, it will reduce to a biological body or computing device just with homoraction inside. Consequently, verifying the existence of heteraction is necessary for cyborg intelligent systems. As typical brain-computer integration systems of "animal as the actuators", rat cyborgs [8] [9], were developed to validate how the animals can be enhanced by the artificial intelligence. Ratbots are based on the biological platform of the rat with electrodes implanted in specific brain areas, such as the somatosensory cortex and reward area [10]. These electrodes are connected to a backpack fixed on the rat, which works as a stimulator to deliver electric stimuli to the rat brain. Figure 2 shows the physical implementation of the rat-robot navigation system [11]. In the automatic navigation of rats, five bipolar stimulating electrodes separately are implanted in medial forebrain bundle (MFB), somatosensory cortices (SI), and periaqueductal gray matter (PAG) of the rat brain. There is also a backpack fixed on the rat to receive the wireless commands. There are two components which are necessary to implement the automatic navigation. Firstly, the communication between a computer and a rat needs to be solved. The stimulation signals are delivered by a wireless backpack stimulator which is comprised of stimulating circuit, control processor and Bluetooth transceivers. The control processor receives the computer instructions through the Bluetooth transceivers. Then it sends commands to the stimulator to control the rat behaviors. By receiving commands from the machine, the rat can perform a lot of navigation tasks, e.g. walking around mazes, climbing bridges, and stopping at a special place. Secondly, a video camera device used to capture the rat movement is installed above the scenario. With the video captured by the birdeye camera, the machine can establish a map of the environment and analyze the real time kinetic state of the rat [12]. For vison-enhanced ratbots, a mini-camera is connected to the backpack to capture movement or the surrounding environment. A computer analyzes video stream input and generates stimulation parameters that are then wirelessly sent to the backpack stimulator to control the rat's navigation behavior by manipulating virtual sensation or reward. Paper [13] reported that vision-enhanced ratbots can precisely find "human-interesting" objects, i.e., human faces and arrow signs, identified by object detection algorithms.

ABGP-CNN Based Environment Awareness
Agent can be viewed as perceiving its environment information through sensors and acting environment through effectors. As an internal mental model of agent, BDI model has been well recognized in philosophical and artificial intelligence area. As a practical agent existing in real world should consider external perception and internal mental state of agents. In terms of these considerations we propose a cognitive model through 4-tuple <Awareness, Belief, Goal, Plan>, and the cognitive model can be called ABGP model as shown in Figure 3 [14]. Convolutional neural networks (CNN) is a multiple-stage of globally trainable artificial neural networks. CNN has a better performance in 2 dimensional pattern recognition problems than the multilayer perceptron, because the topology of the two-dimensional model is added into the CNN structure, and CNN employs three important structure features: local accepted field, shared weights, sub-sampling ensuring the invariance of the target translation, shrinkage and distortion for the input signal. CNN mainly consists of the feature extraction and the classifier. The feature extraction contains the multiple convolutional layers and sub-sampling layers. The classifier is consisted of one layer or two layers of full connected neural networks. For the convolutional layer with the local accepted field and the sub-sampling layer with sub-sampling structure, they all have the character of sharing the weights There are several methods developed for visual awareness [15]. Here we only describe CNN is used for visual awareness and construct agent model ABGP-CNN. For ABGP-CNN, the learning process of recognizing the natural scenes should mainly focus on how to train the CNN as its visual awareness module and how to build appropriate belief base, goals, and plans library. Training CNN includes what the multistages architecture is appropriate for the natural object recognition, what learning strategy is better.

Motivation Driven Collaboration
Motivation is an internal motive force and subjective reasons, which direct drive the individual activities to achieve a certain purpose, and the psychological state initiated and maintained by individual activities. Psychologists define motivation as the process that initiates, guides, and maintains goal-oriented behaviors. All kinds of behaviors and activities of the people can't be separated from the motivation. Consider the dual nature of motivation, that is implicit and explicit, the motivation process is complexity. In general, implicit motivational processes are primary and more essential than explicit motivational processes. Here we only focus on explicit motivation and hypothesize that the explicit motivational representations consist mainly of explicit goals of an agent. Explicit goals provide specific and tangible motivations for actions. Explicit goals also allow more behavioral flexibility and formation of expectancies. In cyborg intelligent system we have developed two approaches for brain computer integration, that is, needs based motivation and curiosity based motivation.

Needs based motivation
In 1943, humanistic psychologist Maslow put forward the demand theory of motivation. Maslow's assumption that people in need, the sequence of human motivation, from the most basic physiological and safety needs, through a series of love and respect, the complex needs of self-realization, and need level has great intuitive appeal [16]. Green and others advocated that the theory of motivation is divided into 3 categories of physiology, behavior and social [17]. Merrick proposed that the motivation theory is divided into 4 categories, namely, the biological theory, cognitive theory, social theory and the combined motivation theory [18]. The biological motivation theory tries to explain the motivation of the work process based on the biological level of natural system. The mechanisms of these theories are often explained by energy and motion, which make the organism toward a certain behavior. The existing research on artificial system has been used to create the simulation of software agent and natural system using the theory of biological motivation. Bach proposed the MicroPsi architecture of motivated cognition based on situated agents [19]. MicroPsi explores the combination of a neuro-symbolic cognitive architecture with a model of autonomous, polytelic motivation. The needs of MicoPsi cognitive system fall into three groups: physiological needs, social needs and cognitive [20]. Physiological needs regulate the basic survival of the organism and reflect demands of the metabolism and physiological well-being. Social needs direct the behavior towards other individuals and groups. They are satisfied and frustrated by social signals and corresponding mental representations. Cognitive needs give rise to open-ended problem solving, skill-acquisition, exploration, play and creativity. Urges reflect various physiological, social and cognitive needs. Cognitive processes are modulated in response to the strength and urgency of the needs.
According to brain computer integration requirements, a motivation could be represented as a 3-tuples {N,G,I}, where N means needs, G is goal, I means the motivation intensity [21].
A motivation is activated by motivational rules which structure has following format: R=(P, D, Strength(P|D)) where, P indicates the conditions of rule activation; D is a set of actions for the motivation; Strength(P|D) is a value within interval [0,1].

Curiosity based motivation
Curiosity based motivation is through motivation learning algorithm to build a new motivation. Agent creates internal representations of observed sensory inputs and links them to learned actions that are useful for its operation. If the result of the machine's action is not relevant to its current goal, no motivation learning takes place.
This screening of what to learn is very useful since it protects machine's memory from storing unimportant observations, even though they are not predictable by the machine and may be of sufficient interest for novelty based learning. Novelty based learning still can take place in such a system, when the system is not triggered by other motivations. Motivation learning requires a mechanism for creating abstract motivations and related goals. Once implemented, such a mechanism manages motivations, as well as selects and supervises execution of goals. Motivations emerge from interaction with the environment, and at any given stage of development, their operation is influenced by competing event and attention switching signals.
The learning process for motivations to obtain the sensory states by observing, then the sensed states are transformed mutually by the events. Where to find novelty to motivate an agent's interestingness will play an important role. Once the interestingness is stimulated, the agent's attention may be selected and focused on one aspect of the environment. Therefore, it will be necessary to define observations, events, novelty, interestingness and attention before descripting the motivation learning algorithm.

Definition 1: Observation Functions
Observation functions define the combinations of sensations from the sensed state that will motivate further reasoning. Observations containing fewer sensations affect an agent's attention focus by making it possible for the agent to restrict its attention to a subset of the state space. Where, a typical observation function can be given as: (1) The equation defines observation function S(t) in which each observation focuses on every element of the sensed state at time t.

Definition 2: Difference Function
A difference function  assigns a value to the difference between two sensations S L(t) and S L(t ′ ) in the sensed states S (t) and S (t') as follows: Events may be of varying length or even empty, depending on the number of sensations to change.

Definition 4: Novelty Detection Function
The novelty detection function, N, takes the conceptual state of the agent, c ∈ , and compares it with memories of previous experiences, m ∈ , constructed by long term memory to produce a novelty state, n ∈ : ∶ × → (5) Novelty can be detected by introspective search comparing the current conceptual state of an agent with memories of previous experiences [14].

Definition 5: Interestingness Function
The interestingness function determines a value for the interestingness of a situation, i ∈ , basing on the novelty detected, n ∈ : ∶ → (6) Definition 6: Attention Selection Selective attention enables you to focus on an item while mentally identifying and distinguishing the non-relevant information. In cyborg we adopt maximal interestingness strategy to select attentions to create a motivation.
The following describes the basic steps of novelty based motivation learning and goal creation algorithm in the cyborg system.

Motivation execution
In cyborg system, the realization of the motivation module is through agent model ABGP. The current belief of the belief memory storage contains the agent motivation base. A desire is a goal or a desired final state. Intention is the need for the smart body to choose the current implementation of the goal. In agent, the goal is a directed acyclic graph by the sub goal composition, and realizes in step by step. According to a directed acyclic graph a sub goal is represented by a path to complete, the total goal will finish when all sub goals are completed.

Collaboration
In brain-computer integration rat brain should work with machine collaboratively. Here rat brain and machine can be abstracted as agent, so the collaboration can be viewed as joint intention. Joint intention is about what the team members want to achieve. Each team member knows the intention specifically and achieves it by collaboration [22]. In the joint intention theory, a team is defined as "a set of agents having a shared objective and a shared mental state". The team as a whole holds joint intentions, and each team member must inform others whenever it detects the goal state change, such as goal is achieved or the goal is no longer relevant.
For the joint intention, rat agent and machine agent have three basic knowledge: first, each one should select its intention; second, each one knows its cooperator who also select the same intention; and last, each one knows they are a team. They can know each other through agent communication.

Simulation Experiments
ABGP-CNN as the detailed implementation for the conceptual framework of braincomputer integration, here we give a simulation application to significantly demonstrate feasibility of conceptual framework of brain-computer integration based ABGP-CNN Agent model. The following will mainly represent the actual design of the rat agent based on ABGP-CNN supported by the conceptual framework of brain-computer integration.
Under belief knowledge conditions, the goals (here mainly visual information) constantly trigger the awareness module to capture environment visual information, and the event module converts the visual information into the unified internal motivation signal events which are transferred to action plan module. Then the action plan module will select proper actions to response the environment.
In simulation application, we construct a maze and design a rat agent based on ABGP-CNN to move in the maze depending on the guidepost of maze path in Figure  4. The task of the rat agent is to start moving at the maze entrance (top-left of maze), and finally reach the maze exit (bottom right of maze) depending on all guideposts. In order to fulfill the maze activity, the rat agent is implemented all the three basic modules, <Awareness>, <Motivation>, <Action Plan>. In the rat maze activity experiment, the rat agent is designed to have 3 basic motivation behaviors moving on, turning left and turning right in the maze. In order to guide rat's behaviors we construct a true traffic guidepost dataset means 3 different signals, moving on, turning left and turning right. The different signal corresponds to different guidepost images like in Figure 5. When rat agent moves on the path, its goals constantly drive awareness module to capture environment visual information (here guideposts in the maze) and generate the motivation signal events to drive its behaviors plan selection. In the experiment, there are 3 motivation signals, moving on, turning left and turning right according to the guideposts in the maze path. Which means the agent can response 3 types of action plans to finish the maze activities.

Conclusions
This paper described the collaborative model of brain-computer integration, which is a new intelligent system based on brain-computer interface technology. In order to make this integration effective and co-adaptive biological brain and computer should work collaboratively. ABGP-CNN based environment awareness and motivation driven collaboration have been proposed in the paper. Motivation is the cause of action and plays important roles in collaboration. The motivation leaning method and algorithm has been explored in terms of event curiosity, which is useful for sharing common interest situations.
The future of brain-computer integration may lead towards many promising applications, such as neural intervention, medical treatment, and early diagnosis of some neurological and psychiatric disorders. The goal of artificial general intelligence (AGI) is the development and demonstration of systems that exhibit the broad range of general intelligence. The brain-computer integration is one approach to reach the AGI. A lot of basic issues of brain-inspired intelligence are explored in the book [1] in detail.