Exploring the Effects of Replicating Shape, Weight and Recoil Effects on VR Shooting Controllers

. Commercial Virtual Reality (VR) controllers with realistic force feed-back are becoming available, to increase the realism and immersion of first-person shooting (FPS) games in VR. These controllers attempt to mimic not only the shape and weight of real guns but also their recoil effects (linear force feed-back parallel to the barrel, when the gun is shot). As these controllers become more popular and affordable, this paper investigates the actual effects that these properties (shape, weight, and especially directional force feedback) have on performance for general VR users (e.g. users with no marksmanship experience), drawing conclusions for both consumers and device manufacturers. We created a prototype replicating the properties exploited by commercial VR controllers (i.e. shape, weight and adjustable force feedback) and used it to assess the effect of these parameters in user performance, across a series of user studies. We first analysed the benefits on user performance of adding weight and shape vs a conventional controller (e.g. Vive controller). We then explore the implications of adding linear force feedback (LFF), as well as replicating the shape and weight. Our studies show negligible effects on the immediate shooting performance with some improvements in subjective appreciation, which are already present with low levels of LFF. While higher levels of LFF do not increase subjective appreciations any further, they lead users to reach their maximum distance skillset more quickly. This indicates that while adding low levels of LFF can be enough to influence user’s immersion/engagement for gaming contexts, controllers with higher levels of LFF might be better suited for training environments and/or when dealing with particularly demanding aiming tasks.


Introduction
Current VR controllers are moving away from vibrotactile feedback, and controllers with more realistic force feedback are becoming available. Custom design controllers for FPS games like Strike VR [1], MAG P90 [2], Delta Six [3] or Haptec recoil systems [4] are available, which mimic the shape, weight and recoil of real guns. All aim to increase realism, immersion and sense of presence in VR environments. However, the effects of replicating these parameters (shape, weight and recoil feedback) on users' performance are unclear. In spite of their higher price tag, their benefits when compared to cheaper alternatives (e.g. passive controllers such as the PlayStation VR aim [5], Wii gun [6] or NES Zapper [7]) remain unclear.
On one hand, the use of more realistic controllers (e.g. those that replicate shape/weight) can improve learning time [8][9][10], presence [11] and involvement [12]. Including force feedback has been shown to improve hand-eye coordination [13], performance and potentially reduce learning time [14]. On the other hand, a strong recoil is also known to have negative effects (i.e. reducing aim accuracy [15,16], causing exhaustion and injuries [17]). Tactile augmentation (replicating shape) [18] has proved to enhance presence in VEs, but its effects on aiming performance are unclear. Besides unclear/conflicting factors, works exploring performance using LFF [14,19] are not consistent, not reporting or using different levels of LFF in each study. There is a lack of standardisation in characterising/reporting LFF levels in the literature, and commercially available gun controllers do not report their feedback levels either.
We first describe an experimental setup to deliver LFF (a pneumatic attachment for the HTC controller), and we characterise three levels of LFF (i.e. allowing for reproducibility of results) that are later used on our user studies. We describe a replicable testbed to measure LFF on FPS game controllers, based on standards for ballistic research. This provides an objective characterisation of the LFF levels used in our studies, allowing for reusability of our results and, more importantly, providing a replicable setup for future comparisons with this/other LFF controllers.
Our first study explores the shooting performance of a conventional VR controller (i.e. HTC Vive) with that of a passive prop controller. That is, a controller replicating the shape and weight of a commercial gun controller (i.e. as in a Nintendo Zapper [7] or Wii gun [6]), but not including actual force feedback, arguably the main factor driving up the costs of controllers such as StrikeVR. Our study revealed that, despite its weight (~1Kg), the performance was not decreased, and participants had a better subjective appreciation for a controller matching the gun's shape and weight.
Our second Study then explores the benefits of adding LFF to a gun-shaped controller with realistic weight, mimicking the cues provided by current VR FPS controllers (e.g. Strike VR [1], MAG P90 [2]) and testing three different levels of LFF. No further effects on performance could be observed due to the inclusion of LFF, but participants' subjective impressions improved, even for the lowest level of LFF.
The third study explores the effects of LFF on participants' learning curve, showing that higher levels of LFF improved skill acquisition, allowing participants to reach maximum aiming distance within fewer shots. We finish the paper by discussing the implications of our results for the future design and usage of LFF controllers for VR FPS.

Related Word
We focused the related work in three main areas: general approaches for haptic VR controllers; studies on the effects of force feedback in 3D pointing/shooting; and military literature describing recoil properties and measurement techniques.

Haptic VR controllers
Tethered force feedback devices [20][21][22] (i.e. not portable/wearable) offer high accuracy and precision. These devices have been extensively used in VR training for tasks that require LFF, replicating needle insertion [23], surgery training [24] or teleoperation [25]. Although very precise, these devices are normally expensive and better suited for research/industrial applications. Untethered interfaces trade accuracy or haptic fidelity for a portable setup. Exoglove designs like Dexmo [26] provide active forces on the movement of users' fingers. Asymmetric vibration, such as in Waves [27] or Traxion [28], has proved to be a feasible approach to deliver distinguishable/perceivable cues that help users navigate a space with push/pull effects. However, the magnitude of the force delivered is weak and therefore not suitable to replicate recoil effects. Electric Muscle Stimulation (EMS) has been used to deliver strong force feedback (e.g. punches [29]) by contracting the user's muscles. However, this technique cannot yet deliver precise and controlled LFF (i.e. vector direction defined by user's joints) and is not applicable to small muscle groups (i.e. individual fingers or wrist, which are greatly affected by recoil).
A series of task-specific controllers that enhance VR experiences have been published over the last couple of years. NormalTouch [30] recreates low definition shapes while HapticRevolver [31] provides a palette of textures (i.e. to match the surface properties of objects in the VE or rotating buttons). Claw [32] (among other features) provided vibrotactile feedback on the fingertip and force feedback on the trigger finger. Researchers highlighted how the users enjoyed the gun operation mode. Although these solutions deliver high-quality haptic feedback, none of them assessed the effects of the feedback/recoil on user performance.
Following the popularity of VR headsets (HTC & Oculus), several companies have commercialised controllers that replicate the shape, weight, and recoil effects of real guns. Strike VR [1] provides advanced controllers with LFF available for multiplayer and arena games. Other companies like MAG P90 [2] or Delta Six [3] offer comparable solutions. Similarly, Haptec [4] develops electromagnetic recoil simulators that cover from small guns to heavy weapons with a focus on training applications. Most of these controllers are aimed at improving users' immersion. However, no information is available on the specific levels of LFF used by these controllers or on their influence on the user's aiming performance.

2.2
Linear Force Feedback, shape and weight aid for aiming/shooting Pointing tasks within 2D Graphical User Interfaces (GUIs) have been studied extensively (e.g. Fitts' law [33]), even in combination with LFF. In a study comparing linear force, audio and visual feedback, the haptic condition yielded quicker motor response [34]. Later work used a multimodal mouse design with LFF (using a solenoid to stimulate the tip of the finger) and drag force control [35]. Their results showed that LFF reduced stop-time and the time to select a target after the cursor has stopped. Further research by Cockburn et al. corroborates that tactile feedback could reduce mean target acquisition time [36]. Although positive, these studies only explored 2D interfaces and, unlike recoil, the feedback was always delivered before the user action (feedforward). Understanding of pointing techniques in 3D and in combination with feedback techniques is not as mature. Modified models have been proposed for 3D pointing [21]- [23] that complement the original Fitts model, but they are not so broadly accepted. Moreover, other aspects, such as the role that supporting cues/modalities play is still unclear. For instance, Krol et al. [14] used a wireless controller (uWand) modified with a solenoid to provide LFF, reporting that 3D selection using LFF was faster than using visual or audio cues [14]. However, later work [19] using a similar system (Wii remote) found that haptic technology provided a more discrete improvement on performance than previously reported. Beyond using different hardware, studies fail to report the levels of LFF used, limiting their replicability and the scope where their results apply.
Studies using VR for military training are available [37][38][39] which, even if focussed on real guns and experienced shooters (instead of gamers), stand against the decision of using high levels of LFF for VR controllers. Recoil at the level of real military weapons is detrimental to aim [15] and can even lead to the development of injuries [40]. Research in self-transformation devices [38] also challenges the choice of commercial VR controllers to recreate the actual gun's shape and weight, suggesting that the controller's weight distribution is much more important to recreate a realistic device [41,42]. Precision in shooting is also affected by two main components; visual (i.e. aiming) and proprioception (i.e. gun-holding & posture). Several studies [43][44][45][46] have shown how knowledge of performance (KP -i.e. shooting accuracy) or knowledge of result (KR -stability, pose or balance) are both directly affected by the ergonomics of the gun and can improve user performance. This is particularly relevant for FPS VR games, where the camera view (i.e. head) is decoupled from the controller (i.e. hand) [47].

Recoil properties and measuring techniques
One of the reasons behind the mixed results about the influence of force feedback on 3D pointing could come from the level of force feedback/recoil used in each study. Commercially available gun controllers do not report their feedback levels, and there is also a lack of standard in reporting this in the literature, where the magnitude of the feedback is usually not characterized [7], [20].
To inform the levels of force feedback used in the user study (i.e. asses safety ranges and allow replicability), we looked at existing approaches to measure LFF. Two main approaches are more commonly used for ballistic measurements, reading impulse with load cells (N/s) or displacement using accelerometers (m/s 2 ). While acceleration is greatly affected by the user's grip on the controller, impulse provides a robust metric comparable across LLF devices, and the design os measuring setups can be adopted from equivalent rigs for real guns [48][49][50]. Other researchers have identified peak force as a more representative parameter to describe recoil felt by the user [50], and our setup and experiments will allow us to measure and report both impulses and peak forces.
Although rare, there have been cases where vibrations of game controllers have been related to injuries, such as the hand-arm vibration syndrome [51], making it advisable to characterise and limit the LFF to safe levels. Although these effects are dependent on the physiology of the person [17], some guidelines exist. Spine et al. recommends limiting recoil to a maximum of ~13.33 N/s to avoid injuries for real weapons [15], while the H&S Executive body in the UK sets limitations of vibration exposure to a maximum of 2.5 m/s 2 daily. We considered these restrictions within the design and implementation of our attachment, as described in Section 3.2.

Experimental Setup
Commercial recoil controllers use tracking systems based on existing VR solutions (e.g. HTC or Oculus systems) to maximise compatibility. We built a linear force feedback attachment (LFFa) for the handheld controllers of an HTC Vive as a replica of existing recoil controllers. The LFFa aimed to provide weight, shape and LFF comparable to existing VR gun controllers. The device was designed to reproduce a range of LFF level, tested during our studies. The following subsection will describe the design and operation of our LFFa when mounted on the controller and the LFF levels produced. We also describe a reproducible testbench implemented to characterise the impulses and peak forces of the LFFa, as these parameters are related to the perceived intensity of recoil. We then, describe the design of a hand dynamometer used to measure the handgrip strength and identify users' hand fatigue. This will allow us to put our results in perspective according to objective parameters. Furthermore, details of the design allow for replicability of the setup and testbed, providing a set of tools for future studies on the use of linear force feedback.

Haptic feedback attachment
We used the HTC Vive controller as the foundation for our recoil controller. The controller ergonomics have a similar design to that of a pistol grip (Fig. 2), with the top ring serving as an attachment feature. Using the controller's in-built tracking system also rendered equivalent accuracy to commercial recoil controllers. We aimed to reproduce the external shape of a futuristic gun as can be seen in some commercial controllers [1]. However, the extra footprint of this enclosure occluded IR receivers on the HTC VR controller and affected tracking performance. As a result, we settled for a minimum enclosure (Aluminium tube) and motivated the visual design in VR as a futuristic SCI-FI gun (see Fig. 1 & Fig. 3 left). The visual representation of the device (in VR) was designed to match its physical shape closely, so that it could still act as a passive haptic prop.
The attachment uses a pneumatic cylinder to displace a weight attached to the end of the piston. The weight attached is added to increase the kinetic energy of the moving piston and, in turn, the perceived recoil force. The moving piston and weight were housed in an aluminium tube to protect the user from finger entrapment. A metal cap at the end of the enclosure transferred the kinetic energy upon impact. The total weight of the final device was 950g (controller ~470g; recoil FA ~400g), closely matching the weight of an example MAG P90 VR Gun controller (~500g + controller).
The pneumatic piston used was an SMC Double Action Roundline Cylinder, CD85N20-125-B, connected to a 5/2 electrovalve (VUVG-BK10-B52) and powered by a 24V 0.2A power supply. Pneumatic components were modified by increasing the inside diameter to 3.8 mm as this reduced airflow constraints. An air compressor (Bambi models 150/500) supplied up to 8 bars of pressure to the system. We used 4mm outside diameter pipes to connect the compressor to the electrovalve and 2mm pipes to connect the electrovalve to the piston. A microcontroller (Arduino Nano) and TIP120 circuit were used to control the electrovalve. Communication with Unity was done via Serial COM at 2,000,000 baud speed.

Characterising LFF feedback: Impulse and peak force testbench
The design of our measuring testbench is based on ballistic research [49] and the principles described by Spine et al. [15]. This design was chosen as it is the most effective method to measure the impulse forces component on the horizontal axis. The logged data allowed us to compare the LFF with existing data [52] and to assess LFF effects against related guidelines. Furthermore, readings with this measuring are independent of the user's pose or grip. To build the testbench frame ( Fig. 3. left) we used V-slot linear aluminium rails (40x40mm) where the device rested. U-shape clamps with bearings fitted on the barrel minimised friction on the travel axis.
The end of travel cap of our LFFa rested against a Phidget load cell (0/200kg), which was used to convert the mechanical impact to an electric signal. To filter and amplify the signal, we used an INA125 IC circuit and logged the data using a Picoscope 2204A. After assembly, the load cell was calibrated using a series of known weights following a standardised process. The testbench was used to select the ideal moving mass of our LFFa. We measured the impulse response using various weight attached to the piston (60g, 120g, 180g, 240g and 300g). The heaviest weight tested (300g) increased the overall device weight above the 1 kg mark and produced no significant increment on the max impulse response (weight decreased acceleration on the moving mass). A weight of 240g proved to be the optimal weight -delivering the highest impulse recoil forces while retaining the overall weight of the LFFa bellow 500gr.
We conducted a short pilot-study to determine the minimum pressure of the pneumatic system that produced a realistic recoil. Below 2 bars of pressure, the piston acceleration was too slow to recreate an impact. With the 240g weight, the 2 bar setting delivered a peak of 337.5 N and lasted for approximately 1ms. The maximum pressure of the compressor used (8 bars) delivered a maximum peak force of ~500N, lasting also ~1ms. We then defined a middle LFF setting (3 bars), resulting in a peak force of 412N (midpoint from the 2 other settings). We used the data collected to calculate the impulse of each of the conditions: 2 bar delivers a 0.028 N/s, 3 bar= 0.035 N/s and 8 bar= 0.042 N/s, and we will refer to them as LOW, MED and HIGH levels in the rest of the paper.
The final impulse response from these three LFF levels is shown in Fig. 3. right. It must be noted that the short duration of the response is due to the inelastic impact measured (load cell, end cap and moving mass are steel). A real user holding the device will result in a much more elastic response, although the final impulse (i.e. summation of force over time) will still be the same.
To measure hand fatigue during our user studies and to avoid any potential ill effects, we measured each participant's hand grip force before and after each condition trial. Grip strength is directly related to the fatigue on hand and forearm muscles, and a decline in strength acts as an indicator of fatigue [53]. We build a precision digital hand dynamometer (Fig. 4), designed using CAD software and printed using a Mak-erBot 3D printer. We measured the grip force using a load cell Phidget 0/50Kg and similar amplifier (INA125 IC) and calibration procedure used for the recoil testbench. Reading values were logged using a Pico-scope 2204A and data processed using R. Grip force measurements followed the standardized protocol described in [54], measuring grip forces across 3 repetitions.  User studies

User study 1: Effects of shape and weight
This first study was aimed to assess the performance of a VR FPS controller reproducing the shape and weight of a gun (i.e. similar to passive props such as Wii gun) when compared to a conventional VR controller. Prior literature has shown that tactile augmentation (i.e. recreation of shape and weight) can increase immersion. However, most commercial recoil controllers are above 1kg, and such extra weight could hinder the experience and performance due to fatigue or momentum when moving the controller.

Experimental Setup
The experimental setup was designed using Unity 2017.3.1f1 and a HTC Vive headset. The VE consisted of an open field with no clear points of reference within the scene ( Fig.1 & Fig. 5 left), to avoid muscle memory of the position of the target and therefore carryover effects. The user was located on an elevated platform to allow shooting at targets above and below head level. A cross on the floor marked the centre of the platform as a reference for the user to remain at the same position. During the user study, a series of targets were successively presented in front of the user. Targets were arranged, on 4 planes at different distances (25, 35, 45 and 55 meters from the user's initial position) according to a 3x3 layout. The central target was rendered at the user's eye level (measured at the beginning of the experiment), and the remaining ones were presented at 2.5m around the central target (i.e. leaving a space of 0.5m between adjacent targets). During the trial condition, each target was displayed twice, adding to a total of 72 tasks per condition (4 distances x 9 targets x 2 repetitions). An initial countdown of 5s was presented at the beginning of the task (i.e. prepare the user for the task), and an in-game questionnaire (answers selected via touching, not shooting) was used at the end of each of the 72 trials. The shooting in the VE was implemented using a raycasting technique. Gravity, wind or any other environmental conditions were not considered, not affecting the linear trajectory of the shots.

Participants
A total of 14 participants (9 male and 5 female) of ages 18 to 46, average age 32.2 years (SD = 7.19) took part in the experiment. All participants were right-handed and had a normal or corrected-to-normal vision and were screened prior to the user study to ensure no hand injuries. Half of the participants were experienced in a similar task (FPS games). The study was approved by the local Ethics Committee. A priori statistical power analysis was performed for sample size estimation in G*Power. Running a power analysis on a repeated measures ANOVA with two feedbacks, a power of 0.95, an alpha level of 0.05, and a medium effect size (f=0.25, ηp2=0.06, critical F= 1.30) [55,56], suggested a sample size of 8 participants. Given the high response of participants, we incremented the number of participants to 14.

Method
In this initial user study, we compared the user's performance looking at two conditions: a controller with our LFFa as a passive prop (no LFF due to pneumatic activation) and a conventional HTC controller. We used a within-subjects design, counterbalancing the order of the two conditions. A schematic of the user study workflow is shown in Fig. 5 right. Each user study session started with participants filling in a background questionnaire (i.e. demographics, previous experience with related activities like paintball/clay shooting/FPS gaming), followed by a brief introduction to the VR system and the controller. Here, participants were explained their goal (hit as many targets in the centre as possible); they were shown a two-handed pose to hold the controller (i.e. weaver stance) and were shown how to use the sights to aim at targets. Users were instructed to shoot as soon as they were ready and informed that they had only one shot per target. The user study started with a training task with i = 36 targets at the closest distance (25m) followed by the two test conditions (n=2): passive LFFa and HTC controller. Participants had to take a 5 min break between tasks without the headset.
During each condition, participants shot targets as they appeared one at a time (with randomised position and distance). In each trial, a single target was rendered for 3 seconds, with a delay of 1.5 seconds between consecutive targets. If the target was hit the user received audio feedback and a red sphere was displayed showing the ray/target collision point.
Dependent variable indicators of performance in FPS games [57] were automatically logged. Time per shoot (TPS) measured the time elapsed since the target was rendered until the participant pressed the trigger; Hit registered a bool variable with true if the target was hit, while hit distance (HD) measured the distance between the hit point and the centre of the target in meters.
After each task, participants answered 4 questions using a 7-point Likert scale. These questions aimed to measure; enjoyment of the experience (Q1 -"How much did you enjoy the experience?"), perceived arm/hand fatigue (Q2 -"I felt that the condition that I used makes my arm/hand feel tired") and how much they believed the controller affected/aided their aiming (Q3. "I found the feedback useful to aid aiming within the gaming experience"). Finally, a forced choice question asked participants their preferred condition (Q4 -"What controller would you prefer using?").

Results
Significance was tested for p<0.05. We used Levene and Shapiro tests as well as QQ plots to test for ANOVA assumptions, and we will only refer to the required corrections used wherever they were necessary. Also, where mean or standard deviations need to be mentioned in the text, these will be noted as M and SD respectively. We analysed the questionnaire results using Wilcoxon signed-rank test as the distribution of the residual were not normal.
Performance results (TPS, in Fig. 6.B) were similar, with mean time values for HTC controller (M = 1.584, SD = 0.530) and passive controller (M = 1.577, SD = 0.517). Once again, no significant difference were found between conditions (F (1,13) =0.016, p >0.1). Differences could not be found in terms of Hit rate either ( Fig. 6.C). Analysis of performance according to user experience did not show any additional differences. Q1 (enjoyment - Fig. 6.D) and Q3 (effect on aim - Fig. 6.F) analysis showed no significant difference between the conditions. Q2 (perceive fatigue - Fig. 6.E) showed a significant difference between the HTC controller ( M = 2.36, SD = 1.60) and the passive prop controller ( M = 4.5, SD = 1.34), (Z = -1.882, p < 0.05). This suggests that participants using LFF had the perception of feeling more tired after the experiment. Finally, Q4 rated controller preference, with a total of 10 out of the 14 participants (71.4%) indicating a preference for the modified passive LFFa controller.
Participants answers to Q2 suggested a perceived increase in fatigue following the use of the LFFa. However, aim performance did not degrade over time and no significant difference was found on hand grip strength. A higher number of users showed a preference for the passive prop on Q4. This is in line with previous work that suggests that tactile augmentation increases enjoyment and immersion [18], but it could, however, be influenced by novelty effects. We found that providing tactile augmentation did not improve performance with our replica during the user study. As the passive LFFa did not influence performance we proceeded to assess if the addition of LFF could introduce any additional benefits.

User study 2: Effects of LFF on VR controllers
In this experiment, we investigate the effect on user performance when adding actual LFF on a controller that already replicates the shape and weight of the gun. This is included as a way to assess the extra benefits from a current commercial VR PFS controller (e.g. StrikeVR, MAG P90) when compared to cheaper alternatives using only a passive prop with the shape of the gun.

Method
Experimental setup and VE remained unchanged, but this user study tested four different conditions (n = 4): passive LFFa and LFFa with LOW, MED and HIGH levels of recoil (see section 3.2). As in Study 1, each condition included i = 72 trials, and the same variables (TPS, Hit) were recorded. As a difference, we measured error angle (EA) instead of distance to the centre as a more consistent measurement over different distances to target [58]. Reduce Latin square counterbalancing was used across conditions. Introduction and training remained unchanged from Study 1. However, the questions focused on the effects of the addition of LFF, versus the passive attachment: Q5 -"The haptic feedback matched the action in the demo"; Q6 -"I enjoyed the use of the controller in the demo"; Q7 -"The feedback made the controller feel more realistic", while Q8 remained as a forced choice, identifying their favourite condition. Hand grip force was measured before and after each condition (8 times in total), with no ill effects being detected for any participant or condition.

Participants.
Twelve participants were recruited (8 male and 4 female) of ages 18 to 46. The average age was 28.6 years (SD = 4.2). All participants were right-handed had normal or corrected-to-normal vision and were pre-screened prior to the experiment. Some participants had played video games before (n =6), and a smaller group had played FPS games (N = 3), some participants had previous experience with VR headsets (N = 5). We ran an estimation in G*Power, resulting in a required sample size of 8 participants.

Results
Data collected was analysed for significant effects using one-way repeated measures ANOVA and Friedman's test with Holm corrections [59] for the questionnaires. Results obtained for this study were in line with Study 1, showing no significant effects on users' aim performance, but with subjective differences in the questionnaire responses.
The differences on the EA were low (Fig. 7 showing differences between any of the 3 LFF conditions and the passive LFFa condition. This indicates that delays between the trigger being pressed and the recoil were low enough as not to affect participant's sense of agency [60,61], and that the force feedback had a positive impact for the representation of the shooting action. The fact that the specific LFF level (LOW, MED or HIGH) did not influence participants' impressions suggests that while some amount of LFF can improve this perception of action/consequence matching, we cannot justify the need of higher LFF levels for these tasks.
A similar result was obtained from the analysis of Q7enjoyment (see Statistical differences could be found in all cases (p<0.01), and paired analysis showed that both Q6 and Q7 were rated higher when using LFF. The results from our forced-choice question (Q8) also seemed to match the idea that a high amount of LFF is not required. From twelve participants 6 preferred the LOW condition, 3 participants MED and 2 preferred passive LFFa. All these results make it hard to justify the need for high levels of LFF for commercial VR FPS controllers (i.e. no gains in terms of performance, similar or lower levels of subjective appreciation).

User Study 3: Effect of LFF on the learning curve
Our previous studies showed no effects on users' performance, and only improvements on participants' subjective assessment due to feedback cues (shape, weight and LFF). This study analysed the effect of these feedback cues over time. That is, even if the feedback is delivered after the shot, forces could engage with the user's proprioceptive system, alleviating registration (i.e. a mismatch between the real position of the gun and where it is seen in VR) and perceptual errors (e.g. depth compression introduced by VR headsets). Better loops with the users' proprioceptive system could thus reinforce eye-hand coordination, which is key for shooting/pointing tasks To assess such carryover effects (i.e. learning curve) we modified the user study, gradually increasing task difficulty according to the participant's performance until a maximum shooting distance was achieved for each condition. The VE and hardware remained unchanged and only the experimental method was modified.

Methods
The experimental procedure remained unchanged from the previous study. Although the position of the target within its 3x3 layout remained randomized, the distance between the user and the target varied incrementally, following a staircase design (i.e. instead of random positions and distances, as in Study 2). Targets started at an initial distance of 25m from the participant, and moved in steps of ±4 meters, using a 'three up -one down' design. That is, participants had to hit 3 targets in a row for the task to increase in complexity (i.e. move target 4m away) and a single missed shot reduced the distance. The initial distance of the target was based on users' hit rate from Study 1 (96% hit rate at 25m), being suited for participants of any aiming skillset level. A minimum set distance of 8 metres from the user was used (but no upper limit). A counter of number of targets hit was shown on the top right of the virtual environment to motivate the user to perform well [62]. The step distance (±4 meters) was selected from a pilot study with 4 participants, settling in for a value that felt moderately incremental at each step but that still resulted in challenging distances within the duration of each test, even for skilled participants. As in Study 2, we tested four conditions (LFFa, LOW, MED, HIGH) with 72 trials per condition, producing a final set of 72 distances (d1 -d72). We performed an introduction at the start of the user study and measured the hand grip before and after each condition as previously. No questionnaires were used and handgrip tests corroborated no ill effects on participants' grip and no relevant differences across conditions.

Participants
We recruited 16 participants for a within-subject user study (11 male and 5 female), of ages 18 to 46. The average age was 32.3 years (SD = 5.5). All participants were righthanded had a normal or corrected-to-normal vision and were pre-screened prior to the user study to make sure they did not have any hand injuries. Some participants had played video games before (N = 11), fewer had previous experience with VR headsets (N = 6). With a sample size estimation in G*Power for a power of 0.95, an alpha level of 0.05, and a medium effect size (f=0.25, ηp2=0.06, critical F= 1.34) required a sample size of 8 participants.

Results
We analysed the data using repeated measures ANOVA on the different conditions. Given the different aiming skillset of participants, we first normalised DT values [63] for each participant. This allowed us to combine users' curves, computing the mean normalised distance per trial and obtaining a single characteristic curve of performance per condition. Using the data taken from all the participants we calculated the mean average curve per condition and performed a linear model fitting per condition [64]. We compared the result models using Friedman's test with Holm-Bonferroni corrections. A log model provided the best fitting results, using the form; f(x)=a+logb (x). Parameter a determines the starting offset value on the slope, while b determines the slope of the curve (lower values of b indicate a steeper slope on the log plot, associated with fewer shots required to reach maximum distance skillset and, hence, higher learning speed).
All the curves showed a good fitting coefficient R 2 >0.93 (see Fig. 8). The coefficients of the models per curve are shown in Table 1. HIGH shows the lowest value on b, indicating that participants reach their maximum skillset at a faster rate than using the other conditions. To compare the effect of the LFF on individual conditions (normalised d1 -d72), instead of on the mean fitting curve, we used Friedman's test. Significant differences were found between passive LFFa & LOW/MED (p<0.05) and passive LFFa & HIGH (p< 0.001).
The analysis suggests that LFF does help to reach users' maximum skillset faster than with the other conditions on our experimental setup. All the participants achieved similar results on the max distance value at the end of each condition, hence the improvement from condition HIGH is not the result of the degradation of a participant's max distance. Following the previous hand measure results, we did not find any  LFFa indicator of fatigue. We suspected a decline in aim accuracy after a given amount of time due to fatigue (each condition trial lasted ~12 min where the participant held the controller with extended arms). However, although subjective fatigue was present, at no point on the condition trial did participants' maximum distance decline, suggesting our controller did not affect performance negatively.

Discussion
This paper explored the effects that a controller replicating the cues of current commercial VR FPS controller (i.e. shape, weight and recoil) has on aiming performance. The results also indicate that, when performance is considered (hit rates, accuracy, shooting time), the addition of these feedback cues did not alter immediate aim. As such, consumers of existing recoil VR FPS controllers should not expect an immediate improvement on performance when using this type of controllers.
These feedback cues did provide improvements in users' subjective appreciation. While the inclusion of LFF (i.e. recoil) did improve appreciation over a passive prop (shape and weigh alone) or a conventional controller, higher LFF levels did not necessarily translate on better subjective assessments of immersion, engagement and realism. From our discussions with participants, they highlighted how a low LFF was enough to recreate the action, and higher levels did not increase the reality or involvement, which matched our observations from the questionnaire responses and forced choice questions on their preferred feedback configuration.
Hence, device manufacturers could consider using lower LFF levels, as our results suggest that this would not lead to any significant loss in subjective appreciation. At the same time, lower levels of LFF could help reduce the costs of the hardware (e.g. impulse levels do not increase linearly with pressure, as illustrated by the three levels LFF we used), which would allow them to reduce their price tag and reach a broader audience. Similarly, lower LFF levels cause less interference with tracking system components (i.e. a strong recoil can disrupt readings from the accelerometers used by tracking systems, such as HTC Vive), reducing technical challenges related to isolating tracking from recoil feedback in their devices, and also contributing to reduced costs.
Also, the fact that impulse responses and peak forces did not affect (increase) aiming performance could be informative for eGame competitions, who could consider allowing participants to use them during their competitions.
Our studies also revealed that the inclusion of LFF resulted in an increase in the learning curve slope, which did increase for higher levels of LFF. Thus, the inclusion of such higher levels of LFF would still remain relevant for devices intended for professional or training environments, which could find the related increases in cost justified in exchange for improved skillset acquisition.
Readings from handgrip strength tests showed no significant differences across any of the conditions, suggesting the LFF levels used were not enough to induce significant fatigue, even for the relatively long duration of our studies (~1h). However, it is worth noting that, even if physiological effects (i.e. handgrip) remained safe and accuracy was not decreased over time (e.g. towards the end of the trials in Study 3), participants did report increased levels of perceived fatigue, which should be carefully considered particularly when applied to entertainment applications.
It must be noted that our results and recommendations should be considered within the context of devices and parameters tested (weight, impulse response and peak forces), as characterised by our proposed testbed measuring setup. Other factors, such as weight distribution, trigger quality, materials and grip shape will also influence the appreciation and success of such controllers. Similarly, more advanced techniques for feedback control (e.g. based on voice coils), can result into more sophisticated haptic patterns (i.e. compared to our testing device) which could offer other improvements and/or support the action of the game beyond simple recoil effects.
Even in this case, the parameters explored here (shape, weight and LFF) remain key for this kind of systems, and the recoil measuring testbed and experimental setups presented in the paper will provide a valuable means to create replicable and reusable LFF levels and experiences, as well as providing a way to interpret and compare results and findings from future studies in terms of quantifiable metrics, such as impulse and peak force.

Conclusions
This paper has presented an exploration of the effects of shape, weight and LFF on 3D shooting/pointing task motivated by the recent commercialisation of gun-shaped VR controllers. Our results reveal that these types of controllers provide improvements for user immersion and learning time. We also found that tactile augmentation of controllers using LFF did not significantly affect users' immediate performance. Our findings suggest that FPS controllers do not require high levels of recoil to increase objective appreciation when used by people with little or no experience of real guns. Following previous research on LFF, we encountered that different approaches to the implementation of LFF and modelling do not follow a particularly methodical system. The lack of reporting on force magnitude or on the controller properties could be the potential reasons for the previous contradictory results. Through our study we have illustrated in detail a simple approach to characterise these effects, allowing for reusability of results and replication of our findings. Any future work on linear force feedback could benefit from using similar techniques to model and report their applications and results in a standardised and comparable manner.