Classification of Chinese and Western Painting Images Based on Brushstrokes Feature

. Painting is an important witness of the development of human civilization. In the communication and collision of Chinese and Western culture and art, because of the differences in political, geographical, historical and cultural backgrounds of the two countries, we find that there are great differences in the process of creating Chinese and Western art. As an expressive form of painting language, it truly and accurately reflects the painter's personality and unique psychological activities. The innovation of this article is intended for distinguishing Chinese from Western paintings by leveraging the brushstroke characteristics of paintings carefully. In particular, we run edge detection method and Sobel operator to extract the characteristics of brushstroke ; meanwhile, this research uses a 3 * 3 filter of image filtering to obtain image edge line. Considering the continuity of the painting brushstroke, we use morphological operation to remove noise and track to correction, connect and filter the edge of the line that are detected, aiming to extract the brushstroke features of painting. On this basis, combined with the deep learning model, we propose a new Chinese and Western painting classification framework, which helps to describe the style of painting works and improve the accuracy of Chinese and Western painting classification. Regarding Chinese and Western painting database constructed in the article, SVM shows its unique advantages compared with four commonly used classifier methods. In addition, this paper compares the classification based on brushstroke features to that without, the results show that the accuracy of classification based on brushstroke is nearly 10% better.


Introduction
Painting is one of the most important cultural heritages in the world.At present, using advanced machine learning algorithm based on big data to study painting image has become a popular research topic in the field of image processing and computer vision.
Nowadays globalization has become a developmental trend.As the frequent communication between Chinese and western painting art, we can better find the differences between these two kinds of art.Different people have different aesthetic, and therefore they choose to watch different paintings.In order to enjoy different paintings easily, using a computer to analyze the classification of Chinese and western painting has become an urgent need.
In the field of computer vision, natural images mainly describe the real scene objectively , so it has small difference in content.On the contrary, drawing images is the manual work featured in painter style and artistic genre, its content deviates from the reality.And it reflects the painter's emotion.Besides, for viewers, appreciating paintings tends to self-knowledge, and they can not accurately distinguish artist style and artistic genre.Therefore, this article uses feature extraction and computer image analysis to learn a lot of knowledge from painting images, and then extracts the features of painting images, and conducts sentiment analysis and style classification.
Traditional painting image researches mainly focus on analyzing the image, and pay more attention to the content of the painting itself, such as shape and key points.However, the artistic style of the image is a high-level semantic concept.In order to better analyze the painting works, the researchers look into painting techniques in the painting process and classify paintings by color, texture, overall structure, etc.However, new perspective generates new problems.Researchers manually extract complex features from their personal experience, which cause the loss of details and hence poor model generalization ability, challenged for the classification of painting images.Based on their characteristics of Chinese and Western paintings , the shapes, key points, colors and textures used in these techniques, no matter which technique is used, are to vividly depict the human and object images in the image, enrich the picture effect and enhance the taste of art.So different brushstrokes are left in the paintings, and it is a feature that can reflect different styles.
Brushstroke is the trace left by the painter in the process of painting, and it is also the characteristic of the painter in painting.Different brushstrokes present different feelings.The brushstroke is considered to be an important reflection of painting techniques, and a natural exile of the artist's personality, taste, and artistic endowment such that it can be used to distinguish painting style.
In this paper, we propose a new method of extracting brushstoke.The edge detection method is used to extract the stroke features.Considering that one stroke may not be sharp or broken, morphological operations are used to remove the noise and seal the edge.After the detected edge is connected and filtered, the CNN model is used to extract the deeper features which are input into the SVM classifier afterwards, in this process the classification of Chinese and western paintings is achieved.Compared with the non-brushstrokes classification method, the experimental results show that the accuracy of the classification based on the brushstroke features increases by nearly 10%.

Related works
International scholars have studied the brushstroke feature of painting.In China, Li et al. [1] proposed a new automatic brushstrokes extraction system to analyze the characteristics of Vincent van Gogh's unique brushstrokes style with scientific arguments.
Based on the comprehensive methods of edge detection [2], clustering and image segmentation, achieving automatic brushstroke extraction system, in this system, the feature of brushstroke are divided into the interactive features which depend on the distribution of neighboring brushstroke and the independent features of the geometric shape of brushstroke.And a large number of features of brushstrokes automatically are compared by means of statistical analysis.Guo et al. [3] summarized and expounded the image features used in the image complexity evaluation method from the aspects of information theory, image compression theory, image feature analysis and eye movement data, and she concluded the problems of classification and regression in image complexity modeling.Fan et al. [4] studied Wu Guanzhong's works, and analyzed the effect of different brushstroke thickness on the visual complexity.In the end, he proposed a method to estimate the thickness of the brushstroke by calculating the color change of the neighboring pixels.Sun et al. [5] also analyzed the feature of length, curvature and density of traditional Chinese painting brushstrokes.First, they extracted the painting brushstrokes with the method described in literature [1], and then proposed a method of histogram synthesis of the three features of the brushstrokes, including boundary length, flatness and average density.In abroad, Ting [6] discussed a specific Van Gogh's painting that requires some manual operations to complete the process of extracting brushstrokes.In order to find the brushstroke feature, David [7] and others need to manually input the details of the painting works, while for other completely different style works, other methods are needed.
In foreign countries, in terms of classifying painting images by extracting features, Condoroviei et al. [8] proposed a system for automatically identifying different styles of digital painting.It uses literature [9] combine the theory and framework to extract brightness and shape features, and use literature [10] Gabor Energy method to extract texture and edge information, 7 different classifiers are used to classify more than 3400 paintings of 6 different styles.For domestic work related to the classification of painting images, Yang [11] analyzed the artistic styles of western paintings and extracted the features of the two different artistic styles based on the ratio of color pairs, white space features and light consistency, which are classified by Support Vector Machine.Bai et al. [12] analyzed the different forms of expression and the causes of the formation of Chinese and Western painting, summed up two research methods of painting image aesthetics: experimental aesthetics and computational aesthetics, and he also summarized the development of painting image computational aesthetics from the perspective of painting image classification.Zou et al. [13] classified Chinese DunHuang frescoes by describing the appearance and shape features of the paintings, refined SIFT features into a four-layer deep learning network, and coded them using unsupervised deep learning methods.Through the comprehensive analysis using the word bag method, they found that the average recognition rate is 84.24% when the classification support vector machine is trained and tested, which has a significant advantage over the classification result of literature [14], which is 76.67%.Jia et al. [15] summarized the development of Chinese and Western painting feature extraction techniques and classification methods based on the details of the stroke feature, color features, shape features and texture features, and blank features of painting images, summed out the common machine learning methods, such as support vector machines, decision tree, artificial neural network and deep learning, outlined the advantages of various methods, drawing emotion analyze the features of image extraction and classification.

Characteristics of Chinese and Western paintings
The characteristics of Chinese and Western paintings are different, especially the stroke characteristics, as shown in Figure 1.

Brushstrokes feature extraction methods
In order to extract brushstroke features, we first use a method based on edge detection.After the edge lines around the stroke are identified, morphological operations are used to remove noise and closed edge lines.Figure 2 shows a flowchart for extracting stroke features:

Image preprocessing
Edge detection is a basic method in image processing and computer vision.It is not affected by changes in overall light intensity.Most of them are based on edges, but the edge lines detected by different operators have different effects.Through comparison experiments, as indicated in the box shown in figure 3, it was found that the edge lines detected by Prewitt operator would be lost or not clear enough.Therefore, as shown in figure 3, we selected the result of edge detection of a painting in the database.In this stage, the input image is first converted into a grayscale image.Then a 3*3 filter is used to filter the image, and the filter is convolved with the image to obtain the gradient image.
The results show that the 3 * 3 filter not only has less calculation time, but also has a slightly higher accuracy.So 3 * 3 is the best filter.The results show that the 3 * 3 filter not only has less calculation time, but also has higher accuracy.So 3 * 3 is the best filter.In the edge detection process, the results show that the calculation time of 3 * 3 filter is about 45.02 seconds, which is 20.12 seconds less than 5 * 5 filters and 33.08 seconds less than 7 * 7 filters.In the final classification accuracy, the classification accuracy of the filter with size of 3 * 3 is 3% higher than that of 5 * 5 and 2% higher than that of 7 * 7.
Because edge detection emphasizes image contrast, in other words, the difference in brightness, it can enhance the boundary features in the image.The target boundary is actually the step change of the brightness level, while the edge is the position of the

Morphological operations
As shown in figure 4, the edge line detected from the image may not be a stroke.Because the edges are not completely sharp and complete, when the fracture occurs, morphological manipulation is performed to complete the closure of the edges.Morphological operation in digital images refers to the use of digital morphology as a tool to extract image components that are useful for expressing and describing regional shapes, such as boundary, skeleton and convex hull, as well as morphological filtering, refinement and pruning after processing.Figure 5 is the image after edge detection, and is the result after morphological operation.

5
The proposed method After extracting the brushstroke features through edge detection algorithm and morhological operation, this paper combines the brushstroke features of painting images, studied the selection of classifiers in the classification model of Chinese and western paintings, and finally selected the appropriate classifier to classify Chinese and western paintings.The experimental steps are shown in figure 6.

CNN model
In this study, after performing morphological operations on the image after edge detection, the data is fed into a CNN to extract the deep learning feature.The architecture of our five-layer ConvNet model is shown in figure 8.

Fig. 6. Experimental flow chart
The data with size of 64 ×64 is inputted.In layer C1, the first convolutional layer filters the input data with six kernels of size 5 × 5, producing six maps with size of 60 × 60.In following layer S1, with a subsampling ratio of 2, each map reduces the size of feature to 30 × 30 by performing max pooling.In layer C2, the second convolutional layer filters the data with 12 kernels of size 5 × 5, producing 12 maps with size of 26 × 26.Then the feature size of 12 map is reduced to 13 × 13 in layer S2.In the last layer, a full connection layer is designed to get a 2028-dimensional vector from layer S2.Finally, a 1014-dimensional feature is obtained in the output layer.In this study, the CNN classifier is not used, but the last layer of CNN is inputted into the final SVM classifier.The support vector machine makes the output as a new While the objective function is:

Chinese and Western painting datasets
where C is the regularization parameter, i  is a slack variable.The above objective function can be reformulated to solve the optimization problem using quadratic programming: where is the kernel function.In the following work, we choose Support Vector Machine Classification.

Support Vector Machine Classification
The SVM is used to perform the classification in our work since it is confirmed to be effective.The basic idea of the SVM is to convert a nonlinear separable problem into a linear separable problem by searching an optimal hyper-plane.The optimal solution is to maximize the distance of each class from the hyper-plane.The SVM was proposed for a two-class classification problem originally.It shows unique advantages in solving the problem of non-linear and highdimensional pattern recognition in small sample data sets.As an effective method, SVM overcomes the traditional dimensional disaster, and is successfully applied to text classification, speech recognition, image classification, and so on.
For the classification of binary images, the purpose of SVM is to learn the decision function: The goal of the SVM is to search the optimal hyper-plane by maximizing the width of the margin between the two classes.The hyper-plane is: is the label of the image.For the test image, assume that its feature description is x, if 1 ) ( = x f ，it is classified as a positive sample, otherwise, it is a negative sample.During the experiment, a linear support vector machine was selected for classification, and the parameter settings used universal values.In this study, two types of classification problems need to be solved.SVM is used to determine whether a test image belongs to Chinese painting or Western painting.

6
Experiment and result analysis In order to verify the superiority of the model, we also selected some similar works from the two different types of painting in China and the West to classify in our model.In the experiment, the data set is divided into two parts: three quarters are used for training and one quarter are used for testing.Figure 7(a) shows samples of Chinese paintings.Figure 7(b) shows samples of Western paintings.

Classification results
In order to verify the importance of stroke features for the classification of Chinese and Western paintings, we conducted two classification experiments: classification based on stroke features and classification without stroke features.In the classification based on stroke features, the pre-processed image is inputted into the CNN model to automatically learn the good features of the image edges.In the experiment without brushstrokes, there is no image preprocessing, edge detection and morphology operations.In the absence of stroke features, we directly input 64×64 data images into the network model shown in Figure 7.The experimental results are shown in Table 1.
The classification accuracy of the classification model based on stroke features proposed in this paper reaches 89.04%, which is 10% higher than the classification model without brushstrokes.In addition, this paper also compares several commonly used classifiers (ID3, Decision tree, KNN, Naive Bayes).Proposed by Ross Quinlan [17], both ID3 and C4.5 algorithms, a classification prediction algorithm, are algorithms in decision tree.The core of ID3 is to construct decision tree recursively by applying information gain criterion to select features at each node of decision tree.By calculating the information gain of each attribute, it is considered that the one with high information gain is a good attribute, and the attribute with the highest information gain is selected as the partition standard for each partition.The process of C4.5 algorithm is the same as ID3 algorithm, but the method of feature selection is changed from information gain to information gain ratio.KNN algorithm is one of the simplest methods in data mining classification technology.Referring to reference [18], we set the parameter K value to 5. Naive Bayesian classifier originated from classical mathematical theory, so we can directly find out the joint distribution P (x, y) P (x, y) of feature output y and feature x, and then use the following formula to obtain.
By comparison, the experimental results of the support vector machine classifier reach 89.04%.Compared with other classifiers, SVM can obtain better classification accuracy.The results are shown in Table 2.
It is verified that different network parameters have impacts on runtime and accuracy of the classification.As shown in Table 3, the training speed with epochs data with a size of 50 is nearly twice faster than the training with epochs data with a size of 100, but the accuracy decreases slightly.

Discussion
In order to verify the classification performance of brushstroke features, we also did a comparative test without extracting stroke features, and found that extracting brushstroke features important factors for the success of the classification system.In summary, brushstrokes are considered to be one of the important techniques for identifying image styles, and can accurately classify images.
In the process of researching and analyzing the painting characteristics of brushstrokes, classifiers are often used to make predictions in different application fields.Each classifier has its own characteristics, and different classifiers are suitable for different situations.In order to evaluate the performance of each classifier, the algorithm extracts features then feeds them into the classifier.It can be seen from Table 2 that support vector machines have superior classification performance, so this study uses support vector machines as the final classifier.
By comparing the difference in training time and accuracy of different data sizes, we found that data size is determined by the original subset of pixels.By inputting d-

Conclusion
In this paper, the edge detection method is used to extract the stroke features.Considering that one stroke in the stroke may not be sharp or broken, morphological operations are used to remove the noise and seal the edge.After the detected edge is connected and filtered, the CNN model is used to extract the deep learning features and input them into the SVM classifier, which can better achieve the classification of Chinese and western paintings.Compared with the non-brushstrokes classification method, the experimental results show that the accuracy of the classification based on the brushstrokes characteristics increases by nearly 10%.We demonstrate that the brushstroke is an significant element in classification.

Fig. 1 .
Fig. 1.Paintings with different characteristics of brush strokes In Chinese painting, flowers and birds, figures and mountains and rivers are the main features.The painting style mainly includes two categories: ink painting and murals.The representative painters are Qi Baishi, Xu Beihong, Zheng Banqiao and Wu Guanzhong.The painting usually takes the line as the basic modeling means, uses the color as the auxiliary characteristic, and does not pay attention to the light and shade.Regardless of landscape, flowers, birds, figures, painters always use different lines to draw the outline, and supplemented the colors.They mainly focus on the implicit, concise and euphemistic styles and pay attention to the performance and freehand brushstrokes.Western paintings are mainly based on relatively direct and intuitive painting techniques, most of which are represented by portraits and religious paintings, focusing on reproduction and realism.Painting styles include Renaissance style, Romantic style, change.The horizontal and vertical grayscale values of each pixel in the image are combined by the following formula, and the grayscale size of this point is calculated:

Fig. 3 .Fig. 4 .
Fig. 3. Original image and Effect diagram of Sober operator and Prewitt operator

Fig. 5 .
Fig. 5.The left picture shows the effect picture of edge detection, and the right picture shows the effect picture after morphological operation finally classify.Giving the training data (  ,   ) of "m" instances, it can be defined as:Training data: {(  ,   |  ∈   ,   ∈ {−1,1}，}  = 1,2,3 ⋅⋅⋅  .Where x i is from the N-dimensional feature space x and y i indicates the class, to which the corresponding xi belongs.

Table 1 .
Comparison of brushstroke characteristics and no brushstroke

Table 2 .
Compare the classification results of different classifiers

Table 3 .
Comparison of training time and accuracy under different amounts of data sizes into CNN, a competitive accuracy can be obtained, and the training speed is much higher than that of the original pixel system.Therefore, choosing an appropriate data size can reduce training time and obtain better classification performance.