Evaluation of Leadership in Nontechnical Skills

. Among the knowledge, skills, and abilities related to safety that are required in healthcare practitioners, the present study focuses on skills other than specialized techniques (nontechnical skills), which have drawn much interest, especially in recent years. From our previous research, we know that nontechnical skills may greatly increase the safety of operating rooms, which is also increasingly being recognized by medical workers. Therefore, in this research, we investigated the usefulness of evaluation of nontechnical skills in Japan. We collected data for evaluation at a hospital from between February to and August 2014 using the nontechnical skills(NOTSS) assessment system. The total number of data sets was 270. 18 doctors participated, including (cid:12)ve evaluators. We found it diﬃcult for us to evaluate leadership in NOTSS. To perform NOTSS in Japan, diﬀerent evaluation indicators may be necessary that change to leadership. We will propose that indicator in the future.


Introduction
Among the knowledge, skills, and abilities related to safety that are required in health-care practitioners, the present study focuses on skills other than specialized techniques (nontechnical skills), which have drawn much interest, especially in recent years.Research concerning human error in aviation started in the 1960s because flight crew were unable to say anything to the aircraft pilots, who wielded absolute power, which led to aircraft accidents [1] [2].Previous studies [3][4] [5] have shown that nontechnical skills are essential to increase safety standards further, which is increasingly being recognized by health-care practitioners.The construction of an effective and efficient evaluation system for nontechnical skills is extremely important, and the development of educational programs to link the results of such an evaluation system with the improvement of nontechnical skills is essential [6].Nontechnical skills can be examined in all facets of medical care, however, because surgical treatments are invasive and put patients at high risk, increasing safety is especially in line with policy priorities.Therefore, the present study focuses on the field of surgery [7].The principal aim of the present study is to evaluate of leadership in nontechnical skill for surgeons (NOTSS).It is often said that Japanese do not demonstrate leadership, so we confirm it in actual surgical operation.In this study, it is an analysis of leadership using medical data, but we think that this study has needs in a wide field including such as management engineering.

The Importance of Nontechnical Skills
It has become clear that when performing medical activities, most problems that occur during operative treatment are related to nontechnical skills (Table 1).It has been reported that 43% of surgical errors are due to a lack of communication [8] and 97% of bile duct injuries are due to a failure to recognize a situation [9].Furthermore, the World Health Organization (WHO) has published a standardized guide for hand hygiene and a safety checklist for operative patients [11], and a global movement to improve patient safety has begun.The WHO safety checklist for operative treatment has been tested in various countries worldwide and the incidence of complications and mortality associated with operative treatment decreased 36% and 47%, respectively, after the introduction of the checklist.
There has been a trend to focus on skills related to medical treatment to achieve good results, but whether or not the medical practitioners possess superior skills, the team performance is low when individual situation awareness and decision-making skills are insufficient, when communication with other team members cannot be performed well, or when leadership cannot be exercised.Therefore, we introduce an educational program for the specialization of nontechnical skills for surgeons during operative treatment to ensure the maximization of the operative team performance [12].

Organization of nontechnical skills
To utilize the NOTSS system and assess behaviors effectively, Yule, S., Rowley, D., Flin, R., et al., [13] noted that the following three points are necessary: (i) fundamental knowledge concerning NOTSS and human performance and error management, (ii) fundamental principles in the use of psychometric tools to understand the NOTSS system and to evaluate behaviors in the clinical setting, and (iii) training in the use of a corrective program to construct objective evaluation criteria.To achieve these points, Yule, S, et.al., [13] categorized nontechnical skills into four groups, and each skill was further subcategorized into three elements (Table 2).Using evaluation criteria as such as that shown in Table 3, each category was scored and the evaluation criteria were standardized.Evaluations were performed on site using the NOTSS assessment system.We collected data at a hospital from between February to and August 2014.The medical facility had 527 beds and 150 doctors.Nineteen surgeons, including six residents, work in the 60-bed surgical ward.About 1200 cases are operated annually.We assume a target performance of one operation per day.We included surgeons who have performed the same procedure more than 10 times.Therefore, high-risk operations were not included and infrequent procedures in which surgeons do not have much experience were not included.We also assumed that emergency surgeries would be excluded from this study because the NOTSS assessment system should be used in operations when the surgeon has plenty of time to prepare for the operation and can usually operate without being stressed during the procedure.
Evaluation was performed by having a single evaluator evaluate a single surgeon.The total number of data sets was 270.Eighteen doctors participated, including five evaluators.We removed an executive surgeon from the operator evaluations for the statistical analysis.The evaluation data was input by each evaluator using a personal computer or tablet.In the data tabulation, each element was scored out of a total of 48 points, with three elements in each category.Each element had a maximum score of four points.Therefore, each category had a total of 12 points, with 48 points in total.Data for elements that did not have an evaluation value were treated as missing values as appropriate.The NOTSS evaluation data is shown in Table 4. From these results, the evaluation of situation awareness has high scores and small variations, but the leadership evaluation shows that the scores vary with low scores.For the correlation coefficient, the value related to leadership is low.Next, when performing the principal component analysis, the results are as shown in the Fig 1 .From Table 5, principal component 2 has a positive leadership value, and the influence of leadership is significant.From Fig 1, we get that the variation in the PC 1 direction is small, but the variation in the PC 2 direction is large.From this result, we think that the quality of evaluation differs from that of other categories in the evaluation of leadership.

Analysis by two-way factorial ANOVA without replication.
We are interested in whether NOTSS was different for every surgeon and every category.Using a statistical method, we analyze the obtained data to derive our results.We categorize two types, such as surgeon and evaluator, and analyze by two-way factorial analyses of variance (ANOVA) without replication.We focus on the mean for each surgeon.(Table 6).From Table 6, we find P = 4.84E − 23 for surgeons.Therefore, significant difference is found between surgeons for a mean NOTSS with 95% significance.Similarly, we find P = 1.87E − 4 for categories.Thus, there are significant differences between categories.

ANOVA of categories
We know there are significant differences in means and variance for surgeons (Table 6).Here, we explore which pairs of categories have significant differences.
As shown in   of the leadership.First, we use the Bartlett test to confirm the homogeneity of variances (P = 0.116) for these categories.We set the significance level at 5%.The null hypothesis is that the mean of the population of each category is equal.An alternative hypothesis is that there is a difference in the means in at least one set in each category.We use the Bonferroni adjustment method to obtain P-values (Table 7).Thus, we find that leadership differs from the awareness, decision, and communication categories with 5% significance.

Analysis using surgeon's experience and Surgical difficulty
Next we analyzed using surgeon's years of experience and surgical difficulty level [14].Surgical difficulty is classified in three level.The correlation coefficient be- tween experience years and surgical difficulty level is 0.70, which shows that a veteran doctor is conducting a surgery with a high degree of difficulty.From the Fig 4, the higher the surgical difficulty level, the higher the leadership evaluation tends to be.However, there are similar trends in other categories, not limited to leadership.Scatter plot of surgical difficulty level and leadership evaluation is shown in Fig 5 .The correlation coefficient between experience years and leadership evaluation is 0.64, which is the largest among the categories.The variance of the leadership evaluation of doctors with experience years less than 10 years is 5.09, which is the largest among the categories.On the other hand, the distribution of the leadership of doctors with more than 10 years of experience has fallen sharply to 1.34.From this data, we consider that the years of experience has an influence on leadership.

Conclusion
In this study, it is difficult for us to evaluate leadership in NOTSS.There are differences in values for the respective roles and we need to standardize this aspect of evaluation.Especially for Japanese, leadership is difficult to evaluate.
To perform NOTSS in Japan, different evaluation indicators may be necessary that change to leadership.We will propose that indicator in the future.These study results show that the experience of the surgeon is influential for NOTSS.Therefore, hospitals should necessarily cooperate with an education system to provide NOTSS to their staff [15].We hope that NOTSS will be applied widely in hospitals.
Fig 2 indicates four categories of means for each surgeon.We used two-way factorial ANOVA without replication for Fig 2 to produce an ANOVA table

Fig 2 ,
the horizontal axis indicates the Surgeon ID.Fig 3 indicates the boxplot for four categories.We can find the whole to have a low item

Fig. 2 .
Fig. 2. Four categories of means for each surgeon.

Fig. 3 .
Fig. 3. Boxplot for the surgeon means in each category.

Fig. 4 .
Fig. 4. Histogram of surgical difficulty level and leadership evaluation.

Fig. 5 .
Fig. 5. Scatter plot of surgical difficulty level and leadership evaluation.