The Role of Artificial Intelligence and Machine Learning in preserving Cultural Heritage and Art Works via Virtual Restoration

: Artifacts including paintings, drawings, prints, mosaics, sculptures, historical structures and monuments, as well as archaeological sites present a key part of our cultural heritage. It consists of Intangible culture (such as folklore, traditions, language, and knowledge), tangible culture (such as buildings, monuments, landscapes, archival materials, books, works of art, and artifacts), and natural heritage (such as biodiversity and culturally significant landscapes) .Now we will concentrate on tangible culture and its problems and how to handle them. One of its biggest problems is that over the years the nature of the materials used in the creation of the artwork make them prone to cracks, fractures, stains, and colors fading and blurring. The causes of their damage could be of natural or human-related reasons. The natural causes range from war, fires, earthquakes, natural disasters and the human-related causes range from accidental events like to pollution which results of climate changes, which like acid rain. It is a must to consider the environment in which you store your artwork. Our regular environment's light, heat, moisture, and pollution levels can lead to harmful chemical and physical reactions in artwork. There are several reasons why it is necessary to preserve ancient works of art. The fact that it enables us to comprehend the historical and cultural context of the era in which it was made is one of the key factors. Then preserving our artworks is a must and this can be done by manual techniques or using machine learning algorithms.


INTRODUCTION
Preserving our artworks is an essential because they can help us gain understanding of the attitudes, values, and viewpoints of those who made it as well as the society in which they lived.Furthermore, by conserving older works of art, future generations will be able to learn from the skills and methods of earlier artists and craftspeople, as well as about the cultural legacy of their own nations.Furthermore, art is a reflection of the human experience and preserving it helps us to understand our shared history and the human condition.Finally, preserving older art can also be seen as a way of honoring the artists and the cultures that produced it, and as a way of ensuring that their contributions to the world are not forgotten.Manual techniques for preserving artworks involve traditional methods and practices used to protect and maintain the integrity of art pieces like: varnishing, cleaning, Restoration and repair.These techniques typically require physical interventions and specialized expertise.One of the key disadvantages of manual preservation techniques is the inherent subjectivity involved and that it can be time-consuming and costly.This limitation can pose challenges, particularly for smaller art institutions or individual collectors with limited resources, so using a computer aided model to protect our heritage is important.AI is particularly adept in digitizing and preserving vast collections of works of art and historical artifacts.AI can analyze and classify texts, photos, and other data using machine learning techniques, which facilitates effective information organization and retrieval.The long-term preservation and accessibility of cultural treasures are ensured by this digitization process, which aids in the creation of extensive databases that are simple to access and share.AI aids in the preservation and restoration of artwork.AI algorithms can help conservators restore missing or deteriorating materials by finding patterns, colors, and textures in high-resolution scans or images of damaged objects.AI also contributes to the restoration and conservation of artworks.By analyzing high-resolution scans or photographs of damaged pieces, AI algorithms can identify patterns, colors, and textures to assist conservators in recreating missing or deteriorated elements.This technology aids in the reconstruction of damaged artworks, helping to preserve their original aesthetic and historical value.Deep learning techniques can be used to build models that are used in the reconstruction of damaged paintings.It presents multiple approaches that help in the restoration of damaged artworks by training the model using the original pictures of the artwork before the damage, it will create a repaired image for the artwork that will help in the restoration of artworks.This will be time saving, less expensive compared to human dependent repairs and sure it will be with higher quality of results because its output depends on the original pictures and videos of the painting or mural.However, it's critical to recognize the ethical concerns and limitations of AI when it comes to protecting cultural assets.Because AI algorithms depend on the data they are trained on, biased or inaccurate data may have an impact on the results and interpretations.Additionally, as it may include subjective choices that need for human skill and sensitivity to maintain the originality and integrity of the artwork, the application of AI in restoration and conservation should be treated cautiously.In conclusion, AI has revolutionized cultural heritage preservation by facilitating effective digitalization, assisting with restoration efforts, and improving accessibility.We may use technology to conserve and promote our cultural history for future generations, encouraging a greater knowledge and appreciation of our unique cultural past, by fusing the strengths of AI with human skills.
Traditionally, cultural heritage buildings and artworks were preserved and restored through manual human interventions and to ensure their availability to future generations.Recently, the world is witnessing huge technological leaps, which can dramatically aid preserving cultural heritage.Digital and computer transformations are capable of lowering costs for services, while improving the final output and evaluation results.Artificial intelligence, particularly machine and deep learning, provide powerful algorithms which are altering the prospect of many sectors, including physics and the humanities.
[9].Machine learning (ML) equip modern systems with a wide variety of various algorithms and procedures to enable learning from large volumes of intricate data.Current ML techniques showed competence in extracting evidence with distinction for effective decision-making.The available algorithms are proficient in extracting high-level features and computer vision related tasks.Hence, it is crucial to employ the versatility and potential these techniques may have in the cultural heritage (CH) sector.In particular, ML techniques are well suited to the evaluation of enormous amount of extremely complicated data often available in the form of images or point cloud data.Automated diagnostics and virtual restoration of CH are significant for maintaining historical monuments and buildings.Machine learning helps remove a notable degree of error [10], which is often contributed to arguably subjective human input.Therefore, the use of artificial intelligence (AI) is expected to show potential in virtual restoration of artworks, leading to saving time, decreasing human effort, reducing the possibility of errors, and increasing the efficiency of results.By utilizing the strength of deep network designs, deep learning [9], a subset of machine learning, has revolutionized the restoration of artworks.Initially, the restoration of 2D paintings and photographs was the main use for these models.Deep learning algorithms are able to understand complex patterns and characteristics by analyzing enormous volumes of data, enabling the restoration of missing or damaged elements in artworks.These models have shown to be quite helpful in jobs like eliminating noise, fixing fractures, and restoring faded colors, thereby giving degraded artworks new life.Researchers expanded the use of deep learning to restore three-dimensional artwork as the field has developed.To manage the repair of statues, sculptures, and architectural structures, advanced models were created.These sophisticated models can handle 3D files, which provide comprehensive data on the geometry, texture, and structure of the relevant artwork.Deep learning algorithms may detect regions that need to be rebuilt, complete any gaps, and even mimic the artwork's original look by examining these files.The utilization of deep learning in 3D restoration has opened up new possibilities for preserving and bringing back to life culturally significant sculptures, historical landmarks, and architectural marvels.These models enable experts to visualize the original state of a damaged artifact and guide the restoration process with greater accuracy and precision.By harnessing the power of deep network architectures, the restoration of three-dimensional artworks has become more efficient, effective, and faithful to the original craftsmanship.Deep learning techniques' continually evolving gives potential for improving the restoration of artworks in a variety of mediums and dimensions.We could expect ever more advanced algorithms that can solve difficult restoration issues as researchers improve and enhance these models, assuring the preservation of our cultural legacy for current and future generations to experience and appreciate.
One of our primary objectives is to push the boundaries of deep learning by developing highly efficient and sophisticated algorithms capable of processing diverse types of data encountered in CH projects.These algorithms are specifically designed to handle both human-obtained data, such as historical documents, handwritten manuscripts, or archaeological findings, as well as photos of artworks, artifacts, and architectural structures.Due to the complexity of CH data, algorithms must be able to overcome a variety of obstacles, including as noise, deterioration, changes in lighting conditions, and occlusions.In order to solve these problems and provide outstanding levels of accuracy and resolution in the analysis and restoration of cultural artefacts, we are working to create deep learning models that are capable of processing both humanobtained data and photos, which are characteristic of most CH projects, to the greatest levels of accuracy and resolution.
AI and its applications have an effective role in saving our heritage, it can be used as: 1-Sensing technology for the analysis and surveillance of cultural assets in the arts and architecture.For example, sensors may be used to track environmental variables like temperature, humidity, light levels, and air quality in the subject of art conservation.2-Best practices and cutting-edge approaches for safeguarding the natural world to protect and preserve our environment, biodiversity, and natural resources.It requires a comprehensive and multidisciplinary approach that considers ecological, social, and economic aspects.3-Innovation and research in the subject of architectural heritage conservation and recovery, embracing both 3D and 2D by including a range of researches and improvements aiming at preserving and restoring architectural heritage.4-Technologies for earth observation and geoscience to estimate the risk to cultural heritage by the use of remote sensing, geospatial data, and geoscience techniques to monitor and assess the potential threats and vulnerabilities faced by cultural heritage sites and artifacts.Also, deep learning techniques can be used to build models that are used in the reconstruction of damaged paintings.It presents multiple approaches that help in the restoration of damaged artworks by training the model using the original pictures of the artwork before the damage, it will create a repaired image for the artwork that will help in the restoration of artworks.This will be time saving, less expensive compared to human dependent repairs and sure it will be with higher quality of results because its output depends on the original pictures and videos of the painting or mural.
AI also can be used in the prevention of the forgery of artworks by applying artwork authentication by proving that it has indeed been created by the artist which will make the process of making unreal copies become harder.
Our challenge in this domain is to build a model that works for different scene representations such as faces, objects, landscapes …etc.and which solve different artworks problems like stains, fading colours, cracks, missing parts…etc.Restoring a painting requires a thorough examination of the original surface.Old, faded paintings may need to be cleaned before the start of restoration process.(2D restoration) [11].Beyond the restoration of two-dimensional paintings and murals, the field of art restoration encounters challenges.It also includes the preservation of three-dimensional artworks, such as sculptures, temples, and the exteriors of historic structures.Technology is essential in finding creative solutions for the issues presented by restoring these complicated three-dimensional items.In order to do this, sensors are used to carefully examine the object's volume, shape, and texture.These sensors may consist of photogrammetry methods, 3D scanners, or laser scanning apparatus.An extremely precise and comprehensive 3D digital representation of the artwork may be produced by taking exact measurements and data points from the surface of the piece.The generated 3D file is used throughout the restoration as a resource and a roadmap.The resulting 3D file serves as a reference and a guide throughout the restoration It allows restorers to digitally regenerate missing or damaged parts of the artwork, ensuring that the restored elements align seamlessly with the original design.The 3D file provides invaluable information about the object's original form, proportions, and intricate details that may have been eroded or lost over time.By utilizing this digital reference, restorers can make informed decisions about the materials, techniques, and interventions needed for the restoration.

VIRTUAL RESTORATION AND INPAINTING SOLUTIONS
In this section, a comparison is conducted between multiple solutions that used machine learning techniques to restore damaged artworks.Each study is described separately by mentioning the used dataset, their proposed model and finally the results that they achieved.
Varun Gupta et al.
[1] used Art Images: Drawing/Painting/Sculptures/Engravings [2], it is a dataset with about 9000 images containing 5 types of arts: Drawings and watercolors, Works of painting, Sculpture, Graphic Art and Iconography (Old Russian art).The presented model comprised two-stages for virtual restoration of digitized artwork: First, Mask R-CNN is used for creation of a mask automatically for the damaged irregular regions of the ruined artwork image, which is given as input.Then, images are fed to the modified U-Net architecture with their created masks.The mask is automatically updated by the modified architecture, which substitutes partial convolutions for regular convolutions.Three domain experts evaluated the output qualitatively, while quantitative evaluation of the results was performed by making use of the structural similarity index (SSIM) and mean square error (MSE).PSNR measures the difference between corresponding pixels, whereas SSIM evaluate similarity in terms of various aspects such as brightness, contrast and structure.In addition to these, the L1 loss computing the overall error in pixel values.When comparing the results of multiple models, it is essential to take into account a variety of performance metrics while comparing the outcomes of different models.A variety of measurements is frequently required since no single measure can fully capture every aspect of model performance.
Additionally, it is essential to make sure that the comparison is fair, carried out using the same dataset, and validated using identical assessment techniques.In summary, performance measures provide objective metrics to assess and compare the results of multiple models.Accuracy, precision, recall, F1score, MSE, ROC curves, and AUC are commonly used measures.Cross-validation and statistical tests can be employed to validate and compare performance.Utilizing multiple measures and ensuring fair evaluation practices contribute to a more robust and reliable comparison of different models.Now we will make a comparison between the results of the mentioned architecture models that we have mentioned in the previous section, but first let's have a quick summarization of the used performance measures: 1-Mean square error (MSE) which calculated the degree of inaccuracy in statistical models.
Between the observed and projected values, it evaluates the average squared difference.

MSE = (1)
Where MSE = Mean squared error n = number of data points Yi = observed values Xi = predicted values 2-Structural similarity index (SSIM), which is a technique for estimating how well-liked digital photos and movies will be.3-L1 loss, which calculates the total amount of pixel value error.4-Peak Signal to Noise Ratio (PSNR), which is a term for the proportion of a signal's greatest achievable strength to the power of distorted noise that impairs the accuracy of its representation.

PSNR = 10
(2) Where R = is the largest variation in the picture data type used as input.MSE = Mean squared error 5-Fréchet Inception Distance (FID) to calculate the Wasserstein-2 distance between false and real pictures).6-Inception score (IScore) which is an algorithm used to assess the quality of images created by a generative image model such as a generative adversarial network (GAN).7-The learned perceptual image patch similarity (LPIPS) which is used to determine the perceptual similarity between two pictures.The next table shows a comparison of the results of 3 mentioned models that we have talked about before, they almost used the same performance measures.

OUR PROPPOSED MODEL
We can make some modifications on Varun Gupta et al. 's proposed model to improve the accuracy of the reconstructed image, we will start with the same step by generating masks for irregular regions in the paintings using R_CNN, and then we will pass the images with the created masks to Recurrent Feature Reasoning.Recurrent feature reasoning (RFR) is a progressive inpainting approach, proposed by Li et al. in 2020, to overcome the problems of prior traditional and deep learning approaches.The word "recurrent" here refers to refining the quality of inpainting over many steps and using the previous step's output for each subsequent step.
Since RFR-Net aims to enhance the quality of low-resolution images by generating high-resolution versions with improved details and sharpness.The model achieves this by leveraging a recursive refinement approach that progressively refines image features at multiple scales.

CONCLUSION
In conclusion, preserving our cultural heritage and artworks is of high importance as it allows us to connect with our past, understand different cultures, and appreciate the artistic achievements of humanity.However, the task of preservation is often challenging due to the deterioration and damage that these artifacts may suffer over time.In recent years, artificial intelligence (AI) and deep learning have emerged as powerful tools in the preservation of cultural heritage.Particularly deep learning has transformed the field of cultural heritage preservation.Deep neural networks are able to learn complex patterns and characteristics in artworks because they can be trained on large datasets.Then, using this knowledge, the artefacts' overall quality may be improved by replacing missing pieces and repairing damaged ones.Deep learning algorithms are excellent at tasks like inpainting, style transfer, and picture super-resolution, enabling the accurate restoration of original features and aesthetics.Deep learning and AI also make cultural assets more accessible and widely shared.These technologies enable broader audiences to investigate and interact with cultural artefacts without respect to physical or geographic restrictions by digitizing artworks and producing virtual reproductions.Platforms and tools driven by AI offer engaging interactions, instructional materials, and preservation efforts that promote a broader understanding of our common past.
The preservation of cultural heritage has advanced significantly due to AI and deep learning, but it's necessary to recognize that human skills and cross-disciplinary cooperation are still crucial.Domain expertise, conservation science, and art historical analysis are essential for directing the creation and use of AI systems.The blending of technology breakthroughs and human insights guarantees a comprehensive approach to maintaining our cultural legacy.To summarize, maintaining our cultural legacy and artistic creations is crucial to preserving cultural variety and comprehending our common past.In this endeavor, AI and deep learning are crucial since they provide strong tools for accessibility, restoration, and documentation.We can celebrate the beauty and importance of our creative past while preserving our heritage for future generations by utilizing the possibilities of these technologies.

FUTURE WORK
Image inpainting is an active area of research, and there are many ways to make the suggested approach better like: 1-Using bigger datasets is one technique to improve the performance.2increasing the size of the dataset used for training can provide the network with more diverse examples, enabling it to learn a richer set of features and improve its ability to inpaint missing regions accurately.3-Deeper network modelling can also catch patterns and structures that are more sophisticated in the pictures, perhaps producing superior inpainting outcomes.4-Training on more powerful GPUs can hasten learning to facilitate the usage of deeper models.The suggested approach can also gain from the addition of new mathematical models or finer tuning layers.By modifying network parameters precisely for the job at hand, fine-tuning layers improve the model's performance on inpainting.The recommended models can be supplemented by mathematical models, improving the quality and accuracy of their output.In order to direct the inpainting process and guarantee the preservation of significant aspects and qualities of cultural assets and artworks, these models might integrate domainspecific information, restrictions, or priors.Future work in this field may concentrate on creating new inpainting algorithms that make advantage of well-known drawing techniques, network architectures, and loss functions.
The precision and realism of the produced pictures can be increased by investigating other drawing techniques, such as stroke-based or texture synthesis-based approaches.Additionally, by successfully capturing and maintaining the unique qualities of cultural heritage and artistic creations, network topologies and loss functions may be improved to produce superior inpainting outcomes.

Figure 1 .
Figure 1.The 2 stages of Varun Gupta et al.'s model: Visual representation of an automated mask creation stage and visual representation of the image-inpainting stage Jieting Xue et al. [3] tested his proposed model on six of the most famous China's ancient paintings (Along the River During the Qingming Festival, One Hundred Horses …etc).A variant of deep Generative Adversial Networks (GANs) called Wasserstein GAN (WGAN) was used to finish Chinese paintings that had irregular gaps.The used variant aims to promote parameters search and learning stability.The introduced generator provide the synthesized lines together with

Figure 4 .
Figure 4.The results comparison of MFR-GAN with Ground truth and RFR Xianlin Peng et al. [6] used two datasets: Places2 and a home collected Murals1 dataset.Their proposed model is named as Content-constrained convolutional network (C3N).The network is based on an encoder-decoder structure.The network utilizes six types of layers.Two convolution layers variants namely partial convolution (PConv) and dual-domain partial convolution (DPConv).For activation function spacevarying activation unit (SUnit) and leaky space varying activation unit (LSUnit) are used.Two auxiliary layers of nearest neighbor interpolation (NN), batch normalization (BN), are also implemented.They evaluated their results qualitatively by using a variety of assessment measures, including the mean loss, the standard PSNR and SSIM, the inception score (IScore) , the learned perceptual image patch similarity (LPIPS) and the Frechet inception distance (FID).

Figure 5 .
Figure 5. Network architecture of C3NThe following figure will show the comparison of the results of their proposed method (C3N) with the results obtained by GL, PCN, LBAM and RFR.

Figure 6 .
Figure 6.C3N model repaired images results comparison with the output obtained by GL, PCN, LBAM, and RFR output may result from fine-tuning, which enables the model to modify its learnt features to the unique properties of the target dataset.4-Data Augmentation: Using strategies for data augmentation during training, such as random rotations, flips, or crops, can assist broaden the range of the training data.The model's capacity to manage changes in picture content may be enhanced by this augmentation, which can also enhance the output images' quality.5-Regularization Techniques: To avoid over fitting and increase the model's capacity for generalization, regularization techniques like L1 or L2 regularization, dropout, or batch normalization can be used.These methods help the model learn more reliable and accurate characteristics, which improves the output.6-Ensemble Methods: An ensemble of many RFR-Net models may be utilized to create the final output image rather than depending just on one model.Different initializations or architectural designs are possible for each model in the ensemble, resulting in a variety of predictions.Averaging or voting these predictions together can increase the output pictures' overall quality and resilience.

Table 1 .
Comparison table between the results of 3 mentioned architecture models because the higher PSNR, the better quality of the reconstructed image.By comparing the SSIM, we will find that XianlinPeng et al. [6]achieved the highest value following them GuangyaoLi et al. [4]then Jieting Xue et al.[3].So in term of PSNR are XianlinPeng et al. [6]the best because 1 indicates perfect similarity.And by Comparing L1, XianlinPeng et al. [6]is better too because they has less error in pixels and by comparing FID, XianlinPeng et al. [6]is better too because they have less distance between fake and real image.So the 4 performance measures, XianlinPeng et al. [6]got the best results.
By comparing the results in the previous table, we will find that XianlinPeng et al. [6]achieved the highest PSNR and following them GuangyaoLi et al. [4]then Jieting Xue et al.[3].So in term of PSNR are XianlinPeng et al. [6]the best It consists of two main components: the Recursive Feature Extraction (RFE) network and the Recursive Feature Fusion (RFF) network.The RFE network extracts hierarchical features from the input low-resolution image, capturing information at different scales.These features are then fed into the RFF network, which iteratively refines the features and generates high-resolution image predictions.[12]