Quantitative Analysis of Comprehensive Similarity in Restoration of Ancient Building Walls Using Hue–Saturation–Value Color Space and Circular Local Binary Pattern

Abstract

Evaluating the effects of wall restoration on ancient buildings has been a difficult task, and it is important that the overall appearance of the restored walls of ancient buildings is similar, harmonious, and uniform. This paper used a hue–saturation–value (HSV) color space and Circular Local Binary Pattern (CLBP) to analyze the comprehensive similarity between a restored wall and the original walls in Qi Li Ancient Town. The results show that the values of the comprehensive similarity calculation of ancient buildings based on the color and texture were consistent with the actual situation. The method is suitable for evaluating the degree of matching between wall repair materials and the appearance of the original wall materials of ancient buildings, and it can also be used to assess the comprehensive similarity between the repair materials and the original building walls before carrying out the wall repair in order to select more suitable materials for wall repair and achieve the best repair effect. And it is flexible and objective compared to human judgement. Through the accurate restoration of ancient buildings, not only can we protect cultural heritage and continue the historical lineage, we can also enhance the aesthetic value of buildings and meet people’s needs for historical and cultural tracing.

1. Introduction

As a part of cultural heritage, ancient buildings carry historical memories and people’s emotions [1,2], and have high artistic and scientific research value [3,4]. However, being subject to the erosive effects of human activities and the natural environment, the appearance and texture of these ancient buildings have gradually deteriorated and faded, and some of them have also suffered structural damage for various historical reasons [5,6,7]. Therefore, there is an increasingly urgent need to preserve and restore ancient buildings. The study of ancient building restoration as a means of preserving the art of ancient architecture is often a scientific study that integrates material science, artistic aesthetics, structural mechanics, and architecture [8]. In the study by Kharfi et al., ancient Roman ceramic bricks were successfully dated by the thermoluminescence method. In addition, the origin of the materials used to make the bricks was determined by X-ray fluorescence analysis (XRF) [9]. Currently, research on the restoration of ancient architectural materials with the help of various chemical analysis instruments, such as scanning electron microscopy (SEM), X-ray diffraction (XRD), and X-ray photoelectron spectroscopy (XPS), is quite extensive [10,11,12,13]. When restoring ancient buildings, it is key to use materials with similar chemical properties [14], but the overall matching of the restored building structure is more important; thus, 3D technologies and virtual reality have become part of the restoration of ancient buildings [15,16,17]. The use of digital technology to construct a virtual interactive display system for the restoration of ancient buildings with three-dimensional effects achieves the illustration of real spaces and is of great significance for the conservation of ancient architectural sites [18,19]. Virtual reality technology can recreate the overall effect of the restored ancient buildings. However, it is an important task to match the similarity between restoration materials and the original building in the actual ancient building restoration project [20,21], which is related to the overall effect of the actual completed ancient building. Traditional restoration methods rely on the experience and intuition of professional technicians in matching the appearance of ancient buildings, which has many limitations. First, the overall restoration outcome of a building is frequently influenced by the individual’s skill level and experience, leading to results that may be subjective and lacking scientific rigor and reproducibility. Second, traditional restoration methods usually require a lot of time and have a high cost, especially in the process of reworking and error correction. Moreover, the effect of an ancient building wall restoration is influenced by the personal aesthetics and artistic style of technicians, which makes it difficult to ensure the accuracy and consistency of the restoration. With the development of science and technology, we can use computer image processing techniques to measure the degree of matching more accurately between restored parts of ancient buildings and the original parts. HSV is a popular tool that is used for color mixing, because it agrees well with human color perception, so an HSV color space is widely used in computer vision and image processing for color-related tasks and calculation of image similarity [22,23,24]. By comparing the color features of images, the similarity or difference between images can be quantitatively evaluated for functions such as color-based image retrieval and image matching. An HSV color space only considers color information and ignores the texture data. In turn, texture features are particularly important in matching the appearance of old and new masonry wall materials in ancient buildings, and texture is also a key element of human visual perception [25]. Therefore, the results of using a single HSV color feature to determine the match between ancient buildings and restored ones are less accurate. There are many methods for extracting image texture features, such as the Gray-Level Coevolution Matrix (GLCM) [26,27], Autocorrelation-based Approaches [28], the Histogram of Gradient Magnitudes [29], the Local Binary Pattern (LBP) [30], etc. The LBP is widely used because of its advantages, such as a relatively simple computation process, fast extraction of image features, good robustness to illumination changes, and ability to extract texture features that are independent of illumination changes [31,32].

In this paper, we propose to use an HSV color space and CLBP-based extraction method to determine the matching degree between the masonry materials used in an ancient building wall restoration and the original walls by extracting the color and texture features of the appearance pictures of the old and new wall masonry materials in the process of an ancient building restoration, which can make ancient building wall restoration more efficient and accurate and improve the restoration effect and matching efficiency of the old and new buildings.

2. Methods

2.1. HSV Color Space

The HSV color space was created by Alvy Ray Smith in 1978 [33] and is also known as the Hexcone Model. It is a color representation based on human visual perception [34]. Color is composed of hue, saturation, and value in the HSV model, and the model is shown in Figure 1 [35]. Hue is measured using an angular scale from 0° to 360° (counterclockwise rotation), with 0° (360°) representing red, 90° yellow, 120° green, 180° cyan, and 240° blue. The horizontal direction indicates the saturation, which indicates the degree of color that is close to the spectrum of color; the higher the saturation is, the darker the color is, and the lower the saturation is, the lighter the color is; when it is closer to white, the range is 0–1. A saturation of 0 means pure white. The vertical direction indicates the value, which determines the brightness of the color space; the higher the brightness is, the brighter the color is, and the range is 0–1. A value of 0 indicates pure black (the darkest color currently).

The HSV color space model shows that it is more consistent with the way that humans perceive color. Generally, images are represented in the red–green–blue (RGB) color space, where the color representation is based on the combination of the three components of red, green, and blue, which is less intuitive and not fully consistent with human visual perception. In the RGB color space, the same color may show different RGB values under different lighting conditions, which leads to an instability of the color characteristics [22]. In contrast, the hue and saturation components in the HSV color space are more stable with respect to light variations, which can better capture the essential characteristics of colors and improve the robustness of color features. This fits well with our goal that the color of the old and new materials’ appearances after the restoration of ancient building walls should be harmonious and uniform, because the observation and image acquisition for the appearance of ancient building walls may be conducted at different points in time, so the wall images may be taken with different lighting conditions, and HSV overcomes this problem well. Therefore, in this study, HSV color space is used to extract color features from the original wall’s appearance in images and the restored wall’s appearance in images.

Generally, images are stored in the RGB color model, so to extract color features in the HSV color space, the images need to be converted from the RGB color space to the HSV space [33], and the conversion equation is as follows:

$$M=\max (R,G,B),$$

(1)

$$N=\min (R,G,B),$$

(2)

$$H=\left\{\begin{array}{l}{ 0}^{°} (M=N)\\ {60}^{°}\times \left(\frac{g-b}{M-N}+360\right) (M=r and g<b )\\ {60}^{°}\times \left(\frac{b-r}{M-N}+120\right) (M=g)\\ {60}^{°}\times \left(\frac{b-r}{M-N}+240\right) \left(M=b\right)\end{array}\right. ,$$

(3)

$$S=M-N,$$

(4)

$$V=\max (R,G,B),$$

(5)

2.2. Local Binary Pattern

The Local Binary Pattern is an operator that is used to describe the local features of an image, and LBP features have significant advantages such as grayscale invariance and rotation invariance. It was proposed by T. Ojala, M. Pietikäinen, and D. Harwood in 1994 [36], and LBP features have been widely used in many fields of computer vision because of their computational simplicity and better results [37]. In the restoration of ancient building walls, the more similar the surface texture of the restored building walls will look to the surface texture characteristics of the original building wall material—in addition to the requirement of matching the color of the restored restoration material with the original building wall material—the more harmonious the restored building wall’s appearance will look, so the similarity of the texture is also very important in matching the restoration material used for the restoration with the original building wall material.

The basic idea of the LBP operator is to maximize the relationship between each pixel point and the surrounding pixel points, either as greater than or less than, so that the local features can be more obviously represented. The main idea is as follows: let (x,y) be any pixel point in the pixel matrix that we want to process, as in the table below, and choose a 3*3 matrix for the central pixel point, as shown in Figure 2. As shown in the table above, the pixel points around (x,y) are compared with (x,y), assuming that the pixel value of the point (x,y) is γ. If the surrounding pixel value is less than γ, then the position information in the matrix is marked as 0; otherwise, it is marked as 1. This gives us a 0,1 worth matrix, which is a binary matrix. As we can see, the binary matrix after comparison and assignment can be composed of an 8-bit binary number in clockwise order. We call this binary value the LBP eigenvalue, and then, using 8 bits as the most basic surrounding pixel value standard, the generated binary array is 11010100, and the obtained LBP eigenvalue is 212.

2.3. Circular Local Binary Pattern

The above LBP operator method is simple, but the drawback is also obvious, which is that the coverage area is fixed and only surrounded by 8 bits, and the requirement to find multiple scales of multiple directions of the texture is very clumsy. In order to adapt to different scales of texture features, and to achieve the requirements of gray scale and rotation invariance, a circular LBP operator was proposed [38], which is a kind of LBP operator based on the radius and angle design of the LBP operator, which can obtain texture features at multiple scales and angles. The improved LBP operator allows for multiple arbitrary pixel points in a circular neighborhood of radius R. Thus, an LBP operator such as a circular region of radius R containing P sampling points is obtained, as shown in Figure 3.

2.4. Cosine Similarity

The cosine distance is also called cosine similarity. Cosine similarity is a widely used metric that is both simple and effective [39]. It is a measure of the difference between two individuals by the cosine of the angle between two vectors in the vector space. The closer the cosine value is to 1, the closer the angle is to 0 degrees, that is, the more similar the two vectors are, which is called “cosine similarity” [40]. In this paper, we extract the color features and texture features of the images separately, calculate the color feature vector and CLBP texture feature vector, and then calculate the color similarity and texture similarity of the images by using the cosine similarity.

3. Ancient Building Wall Restoration Method Based on Integrated Similarity of Images’ Color and Texture

3.1. Research Steps

In old buildings, color and texture are important factors to determine whether the overall appearance of the building is harmonious and matching, so this paper intends to use the HSV color space and CLBP to calculate the similarity of the outer surface of the old and new wall materials in the process of an ancient building wall restoration to determine whether the used old restoration materials match the appearance of the original building wall. This can guide the process of wall restorations in ancient buildings in terms of how to choose the most suitable material from the many old materials for construction. The main steps are shown in Figure 4.

3.1.1. Image Acquisition

In this paper, based on the Qi Li Ancient Town Cultural Tourism Project, the restored walls of Qi Li Ancient Town were photographed and selected as the research object of this paper. As shown in Figure 5, the effect of part of the restored wall is shown, and the overall effect shows the difference between the original building wall and the new masonry wall. In this paper, we used the photos of the original wall and the new masonry wall in Qi Li Ancient Town to conduct experiments to evaluate the effect of wall restoration in Qi Li Ancient Town and to verify the accuracy of the method in this paper based on the final picture similarity calculation. A total of four groups of images of the repaired walls were selected for the experiment (each group of pictures was taken under the same lighting conditions), as shown in Figure 6.

3.1.2. Image Conversion

Normally, images are stored in the RGB color space in the computer, so to calculate the color and texture similarity of an image, the image needs to be converted. Image color feature extraction using the HSV color space requires converting RGB images to HSV images, as shown in Figure 7, as well as converting RGB images to the HSV space and outputting the images of the H channel, S channel, and V channel, respectively. In addition, the process of extracting texture features from images using CLBP entails the initial step of converting the images into a grayscale format, as shown in Figure 8.

3.1.3. Color and Texture Feature Extraction

In the HSV color space, the color distribution image represents the distribution of the image on the hue (H), saturation (S), and value (V) channels, respectively. The distribution image can be obtained by HSV color feature extraction, as shown in Figure 9.

As shown in Figure 8, the horizontal coordinate Bins indicates the range of values of the hue, saturation, and value, and the vertical axis indicates the number of pixels in each interval. Then, the texture features of the image are extracted using the CLBP, and the image texture feature histogram, as well as a texture feature based on the CLBP, can be obtained, as shown in Figure 10 and Figure 11.

3.1.4. Comprehensive Similarity Calculation

The color similarity and texture similarity of the images are calculated using cosine similarity, and the combined similarity is calculated by combining the color and texture feature weights of the images. The formula for calculating the comprehensive similarity is shown in Equation (6), and the values of the weights w are shown in

Table 1. Weight selection criteria.

Criteria (Color Compared to the Texture)	${\mathit{w}}_{\mathit{c}\mathit{o}\mathit{l}\mathit{o}\mathit{r}}$	${\mathit{w}}_{\mathit{t}\mathit{e}\mathit{x}\mathit{t}\mathit{u}\mathit{r}\mathit{e}}$
Very important	0.9	0.1
Relatively important	0.8	0.2
Important	0.7	0.3
Little importance	0.6	0.4
As important as each other	0.5	0.5

. In the study presented in this paper, we discussed the restoration of the walls of relatively large ancient buildings, and we must generally consider the overall effect of the restored buildings, so the importance of color matching is more important than that of texture matching; therefore, the color weight is taken as 0.7, and the texture weight is taken as 0.3 in the study of this paper. Finally, according to

Table 2. Determination of matching degree of ancient building wall restorations.

Total Similarity	Result
0.9–1	Good restoration effect
0.8–0.9	Average repair effect
Less than 0.8	Poor restoration effect

, the degree of matching between the restoration materials used in the restoration of ancient building walls and the original wall materials of ancient buildings is judged.

$$TS=w_{color}\ast S_{color}+w_{texture}\ast S_{texture},$$

(6)

4. Result and Discussion

4.1. Image Color Feature Extraction

The color features were extracted from the original wall part and the new wall part of the four collected groups of images, and the comprehensive similarity was calculated. We take the third group’s images as an example to show the analysis process. Figure 12 shows the three channels of the HSV of the third group of images. In Figure 12, the three channels of the HSV color space for the third group’s images are depicted. The HSV color distribution of the original and new segments of the third group’s images is shown in Figure 13.

As shown in Figure 12 and Figure 13, there is a certain difference in the values of the H, S, and V channels of the third group of images, and the same visual effect as that perceived by human eyes shows that there is a large color difference between this group of restored walls and the original walls.

4.2. Image Texture Feature Extraction

Two parts of the image (the original wall surface and the new wall surface) will be randomly intercepted for texture feature extraction. As shown in Figure 14 (taking the third group’s images as an example), the use of CLBP can extract the more prominent texture features of the ancient building walls.

The texture feature histogram is obtained based on the texture features extracted from the circular LBP, as shown in Figure 15.

The texture feature extraction results for the third group show that the restoration of this wall has a good match in texture.

4.3. Comprehensive Similarity Results

In this paper, four groups of restored walls in Qi Li Ancient Town were selected to calculate the comprehensive similarity of the restored walls with the original building walls, and the calculated results are shown in

Table 3. Comprehensive similarity calculation results.

Number	Color Similarity	Texture Similarity	Comprehensive Similarity
1	0.4415492	0.9663997	0.5990044
2	0.5559807	0.9951636	0.6877356
3	0.5009628	0.9961462	0.6495178
4	0.9031662	0.9974285	0.9314449

.

The overall effect of the wall restoration of the first group is the worst, and the most harmonious effect is achieved in the fourth group of images. According to Table 2, the first group to the third group’s wall restoration effects are poor, and the fourth group’s repair effect is good. As can be seen from Table 3, the comprehensive similarity of the calculated pictures of each group basically matches, indicating that the method of comparing the similarity of wall appearance images of ancient buildings based on the HSV color space, as well as CLBP, that is proposed in this paper is highly accurate and can be used as a method to judge the matching degree between the artificially made old wall restoration materials and the original wall materials in the process of wall restorations of ancient buildings. The research in this paper is not only beneficial to the protection of cultural heritage and the continuation of historical continuity, it can also enhance the aesthetic and ornamental value of ancient buildings and meet people’s demand for traceability and aesthetics of historical and cultural monuments.

4.4. Discussion

From the above results, it can be seen that it is feasible to evaluate the restoration effect of ancient architectural walls through texture and color similarity. Using the HSV-based method to calculate the color similarity from the comparison of the three color channels, is to a certain extent more objective and accurate than the human eye in its ability to distinguish the color similarity, but also to avoid the human eye’s fatigue and miscalculation, and its evaluation results are also consistent with the actual situation. This paper proposes that the extraction of color and texture feature weights is also more flexible and can be assigned according to the actual needs. In addition, this method can not only be used for the evaluation of the restoration effect of the restoration of ancient wall surfaces, but also to evaluate the design of the wall before the beginning of the restoration program based on the color and texture of the ancient bricks to evaluate the restoration of the wall, such as whether it is reasonable, so that a more suitable material can be selected for the restoration of wall surfaces in order to achieve the optimal restoration effect.

5. Conclusions

In the restoration of old buildings, not only the physical and mechanical properties of the building materials should be considered to meet the requirements of the building, but also, the principle of “repairing the old as the old” should be followed in order to keep the overall style of ancient buildings from being destroyed. In this paper, we use HSV and CLBP to analyze the appearance of old and new walls and study the evaluation method of wall restoration in order to achieve accurate matching of old and new wall materials, and we reach the following main conclusions:

(1) The comprehensive similarity calculation method for restored materials of ancient building walls based on the HSV color space and CLBP was utilized for four groups of restored wall pictures, and its analysis results were fully consistent with the actual situation through comparative analysis, indicating that the method is suitable for evaluating the matching degree between wall restoration materials and the appearance of the original wall materials of ancient buildings.

(2) In order to adapt to the evaluation of the restoration effects of different scales and types of ancient buildings, this paper proposes a weight assignment method for the color and texture characteristics of ancient building walls, which can be assigned different values according to the actual needs of the site, improving the flexibility and applicability of the method.

(3) In addition, the method can also be used to evaluate the comprehensive similarity between the restoration materials and the original building wall before the commencement of the wall restoration and to select more suitable materials for the wall restoration to achieve the best restoration effect.

In summary, in the restoration of old buildings, their unique textures and colors are essential to preserve and restore the authenticity of ancient buildings, and the human eye is subject to some subjective errors in judging color textures. In this paper, the evaluation method for the effects of ancient building wall restorations based on a comprehensive image similarity analysis has important theoretical significance and application value in cultural heritage conservation and architectural aesthetics. In addition, the method is more flexible, as long as the comparison image is taken in the same light and the angle of the captured part is the same. The method can improve the effectiveness and efficiency of ancient building restoration, protect and perpetuate the historical value of ancient buildings, and meet people’s demand for history and culture.