Workflows and the Role of Images for Virtual 3d Reconstruction of No Longer Extant Historic Objects

3D reconstruction technologies have gained importance as tools for the research and visualization of no longer extant historic objects during the last decade. Within such reconstruction processes, visual media assumes several important roles: as the most important sources especially for a reconstruction of no longer extant objects, as a tool for communication and cooperation within the production process, as well as for a communication and visualization of results. While there are many discourses about theoretical issues of depiction as sources and as visualization outcomes of such projects, there is no systematic research about the importance of depiction during a 3D reconstruction process and based on empirical findings. Moreover, from a methodological perspective, it would be necessary to understand which role visual media plays during the production process and how it is affected by disciplinary boundaries and challenges specific to historic topics. Research includes an analysis of published work and case studies investigating reconstruction projects. This study uses methods taken from social sciences to gain a grounded view of how production processes would take place in practice and which functions and roles images would play within them. For the investigation of these topics, a content analysis of 452 conference proceedings and journal articles related to 3D reconstruction modeling in the field of humanities has been completed. Most of the projects described in those publications dealt with data acquisition and model building for existing objects. Only a small number of projects focused on structures that no longer or never existed physically. Especially that type of project seems to be interesting for a study of the importance of pictures as sources and as tools for interdisciplinary cooperation during the production process. In the course of the examination the authors of this paper applied a qualitative content analysis for a sample of 26 previously published project reports to depict strategies and types and three case studies of 3D reconstruction projects to evaluate evolutionary processes during such projects. The research showed that reconstructions of no longer existing historic structures are most commonly used for presentation or research purposes of large buildings or city models. Additionally, they are often realized by interdisciplinary workgroups using images as the most important source for reconstruction as far as important media for communication and quality control during the reconstruction process.


INTRODUCTION
3D reconstruction technologies offer a chance not only to digitize historic objects which are still extant, but also to reconstruct and visualize objects virtually which are no longer extant physically and only known from descriptions or historic depictions.Beyond a reconstruction of tangible or former tangible objects technologies could make "intangible" historic and cultural objects imaginable (Arnold and Geser 2008).A main element of all 3D reconstruction projects is the creation of a virtual 3D model, which is a virtual, subjective, simplified image of a historic original (Stachowiak 1973).To make these models visually perceptible it is necessary to transfer them into rendered images or physical objects again, i.e. as visualizations, animations, interactive content or manufactured prototypes.
Images and pictorial representations have always been an important medium for teaching and illustrating historical facts and items.While historical picture sources usually provide elusive and fragmentary impressions, digital three-dimensional reconstructions of historical objects and their depictions offer the chance to convey holistic and more accessible impressions.Apart from the uses for educational purposes (El Darwich 2005) or as a means to visualize historic items for the public (Greengrass and Hughes 2008), 3D technologies have gained importance as research tools (Favro 2006) and for cultural heritage management purposes.While workflows for virtual reconstructions of extant objects are mostly technologically or logistically challenging, a virtual reconstruction of non-or no longer existent items adds tasks such as interpretation and inclusion of historic sources.The various research and modeling competences required for the creation of 3D reconstructions usually leads to interdisciplinary teams (Münster 2011).Visual media and visual communication play several important roles for such reconstructions -as a source, as tools during the reconstruction workflow and, unsurprisingly, to make such virtual models visible for an audience.

Images and 3D reconstruction
Generally, research about the usage of images is nothing new and takes place in relation to their usage in various contexts, such as engineering, design, architecture or for scientific and research context (Gooding 2004).Regarding a quality of images as visual signs there are many possible dimensions, such as similarities to a depicted object, visual styles, or creation processes (Bresciani 2013).The use of images in a research related context would not only include functions such as memorization, documentation or communication within projects or of results.Such images would also be important for a solution finding process and related purposes such as information sorting and solution negotiation (Sachse 2002).Particularly the humanities, and especially archaeology, history of arts, and history of architecture, deal with historic images as sources of reconstruction.Types of sources and their relevance for 3D reconstruction are a prominent topic in scientific literature (Hermon 2008;Remondino, El-Hakim et al. 2009).But these are not new phenomena: Especially for the reconstruction of architecture, as the most prominent type of objects reconstructed by such projects (Münster and Köhler 2012), a communication via images has had a long tradition since early modern times (Carpo 2001).
Results of 3D reconstruction are mostly static images, animations, or even interactive visualizations like computer games.An approach to classify is delivered in the Engagement taxonomy by Grissom et al., which differentiates six degrees of interactivity for such visual output (Grissom, McNally et al. 2003).Closely related are questions for information transported by such visualization.Such aspects are theorized by several approaches as, for example, communication theories (Watzlawick 2000) or visual learning theories (Gagné, Briggs et al. 1988;Pahl and Ahlborn 1998;Schwan and Buder 2006).
Especially for 3D modeling and visualization there have been many technological developments during the last years.Generally these inventions are speedily adopted for reconstruction purposes, too.In publications there is a huge bandwidth of technologies described which are used or sometimes "abused" for such purposes, i.e.CAVE-like environments for visualization.In addition, modern technological trends like smartphones or web 2.0 are quickly adopted, as well as current socio-technological trends such as crowdsourcing (Gerth 2005;Havemann 2012;Labrador 2009) or mobile computing.
Another trend is to move on from a presentation of static objects to complex and lively impressions of history, involving enhancement of visualization with dynamic elements like crowded places (de Heras Ciechomski, Schertenleib et al. 2005;Maim, Haegler et al. 2007;Feneley, Chandler et al. 2008).Other trends are concerned with a presentation of content.This includes an increasing use of interactive Web presentation environments like Google Earth (Erving, Rönnholm et al. 2009) or Augmented Reality technologies (Zöllner, Becker et al. 2010) as well as a materialization of virtual 3D models via rapid prototyping methods or multi sensual presentation possibilities (Ch'ng 2009).

Research objectives
While most investigations focus on certain aspects or theoretical issues of images as sources or as results, there has been no systematic research specifically on roles and functions of images within the 3D reconstruction process and based on empirical findings until now.My main objective was to sketch a current state based on an empirical analysis of recent publications.What are research questions?On the one hand, my research interest was for phenomena visible within such projects.This includes the question for workflows, constellation and media produced.On the other hand, I investigated the role of depictions during the creation process, either as sources or as tools for collaboration or as results.

METHODOLOGY
To cope with these challenges the author performed three stages of analysis using widely established methods from social sciences: the first stage was a qualitative content analysis for publications to examine current usage scenarios, * its outcome and purposes (Münster and Köhler 2012).The investigation included a sample of 452 journal articles and conference proceedings dealing with 3D reconstruction of historic items.To enable a selection of relevant publications experts were asked to examine the most valuable conferences and publications.As a scope for conference proceedings, whole volumes were included, and an identification of relevant journal articles took place via keyword search.All publications * Methods: (Mayring 2008).included had to be written in English and available online.In that analysis a sample of 452 journal articles and conference proceedings were included.In nearly all cases a creation of visual outcome took place.More than 80% of the projects described in those publications dealt with data acquisition and model building for existing structures.Only 20% of projects focused on no longer extant architecture in terms of no longer physically existing buildings.That kind of project in particular seems to be interesting for a study of the importance of historic depiction during the work process.To examine this, the authors applied for a second stage a qualitative content analysis with a sample of another 26 international publications dealing with a reconstruction of no longer extant objects (Münster 2011 The leading paradigm for an evaluation of that stage was a mixed methods approach including heuristic frameworks and Grounded Theory (Charmaz and Bryan 2010;Strauss and Corbin 1996).As data in all four cases, a total of 9 interviews with key role team members took place; in addition, 6 direct and participating observations of team meetings were carried out.Also a significant number of documents, including logfiles, communication data, protocols, sources and model renderings were included in investigation.† Sample: (Münster 2011 This contribution has been peer-reviewed.The double-blind peer-review was conducted on the basis of the full paper.198

Images as sources
Even if sites and remains would give an impression about spatial relations for an interpretative reconstruction of no more extant or never realized historic objects, historic images are widely the only sources to reconstruct no more extant objects.
As finding in investigated projects the reconstruction of the depicted objects from such sources is challenging because not all of the required information is provided or directly measureable from these images, which often do not show plans but perspective projections instead.As a practical consequence, a reconstruction workflow dealing with such panorama leads to strategies like a counting of items or the comparison of sizes in the historic images.Besides that, also some smoothness or inexactness inherit in sources could be demonstrated during the modeling processes.In general, the depiction of single buildings is mostly consistent in historic perspective drawings, but the arrangement and connections between these buildings would be often not coherent; all the necessary information for a proper reconstruction is usually not provided by historic sources.To bridge such uncertain or contradictive information, interpretation would include logical implications such as analogies to similar objects, requirements of systems such as the Vitruvian architectural system, or would simply rely on innermodel logic as with common boundaries of modeled parts.
Moreover, different sources depict different planning levels or states.In these cases it is necessary to set and document which source should be the leading source for reconstruction.As a recommendation it would be helpful to arrange all material available at the beginning of a project in graphical production plans, including information about spatial arrangement and time * Based on: (Hermon, S., 2008;Francescoa, and d'Andrea 2008).
stamps.Nevertheless, additional information and multiple sources would have to be added at later stages of a work process.Especially if these are contradicting to former work could it lead to enormous challenges.Recommended strategies would be to set priorities as to which of this new information would be most important to be included and to progress this list as long as resources are available.Not implemented information should be documented at a final project description.

The modeling process
While a reconstruction of still extant objects is challenging for mostly technical aspects such as complete and accurate data acquisition and handling or an efficient and comprehensive algorithmic model creation, technical workflows for a VR-or CAD-modeling of no longer extant or never realized objects are widely established and similar to other 3D modeling tasks in engineering and design.Several factors influencing such a process and lead to certain strategies for a qualification of team Such projects are typically influenced by the concurrent objectives of scientific accuracy, coherence, visually impressive levels of detail, and budget limits, from which scientific accuracy and coherency of the created model were designated as the top priorities.Regardless of whether an object is extant or not physically existing, there are two major modes of reconstruction available during the production process.A digitization mode assumes well-researched attributes which are available as data or images for modeling and could be used for model creation without further research.Such stages usually progress quite quickly, while a second mode requires an interpretation or interpolation and progresses much more slowly.This could occur if no directly accessible information is available, sources do not agree with one another, or different scientific hypotheses are available and former research is required to gain information needed for reconstruction.While digitization stages of easy to transfer parts mostly occur at the beginning of reconstruction projects, especially beginning project teams would set a benchmark for qualities to achieve from such fast progression.In later stages interpretation work gains more importance and causes a slower progress of projects.As another phenomenon there is the common tendency, especially for beginning project teams, to reconstruct items as detailed as possible.While another major objective would be to reach a coherent level of detail that would lead to a significant amount of interpretation and research to reconstruct less documented items with the same quality as well documented items.

Images as tools
Images and visual media are widely used during the production process for communication and for quality negotiations.A major challenge of interdisciplinary projects results from differing disciplinary terminologies.Images, drawings, or sketches in investigated projects are widely used for interdisciplinary communication to bridge these gaps and to build a "common ground" (Wilkes-Gibbs and Clark 1992) for understanding.
Often such strategies were not established from the beginning of such projects, but became more important during the production process and enhanced the efficiency and understanding of information transfer, especially in combination with textual or oral explanations.While widely established symbols (Bertin and Scharfe 1982) like arrows are used in all projects, some projects created their own graphical codices or metaphors for communication, such as color schemes or abbreviations.Some of the projects investigated successfully adopted highly standardized conventions from architectural or engineering drawings for interdisciplinary exchange.
Another challenge in observed projects had been that team members of such work groups are often located at different locations.As a result, several strategies and media were used for distant communication and cooperation.For such purposes, in a mixture of synchronous and asynchronous strategies, widely established technologies like telephone, email and teleconferencing were employed.A selection of synchronous or asynchronous tools was highly dependent on the urgency of information request, their complexity and skills, and rank of the communication partner.In all of the projects observed, distant communication was aided by periodic meetings.These meetings were dedicated to the explanation of progresses as far as for an observation, negotiation, and discussion of quality.Participants were either project employees or external experts.As a most important workflow in that revision and correction process often a direct comparison between historic image sources and 3D renderings -"photographed" in a similar way -took place, providing the chance to notice differences and lackings of the current work.

Images as results
Nevertheless, in most researched project reports and case studies created virtual models will be presented to an audience as images, too.Formats used are static images, animations, or even interactive visualizations like computer games.Apart from this in some cases a data output is used for further investigation like finite element analysis or for rapid prototyping.Prominent topics in practice are user-friendly and impressing presentation as well as scientific accuracy of content shown.Another aspect is to refer to sources included in the model creation process.Most common approaches are to add overlays to display information on demand directly in the model or to provide a supplementary and mostly textual statement.

IMPLICATIONS
At a glance, images are widely used for the reconstruction of extant objects.Especially reconstructions of no more extant or never realised objects are mostly practiced by interdisciplinary teams including modelers and researchers.For these projects images are the most important sources.That results in comparison to textual sources not only from their richness and clearness of included information about visible aspects like objects, situations, geometries or sometimes surfaces, but also from their easy visual transferability and comparability with visual 3D models.One major problem related to this is the inconsistency of such sources, which leads to a high need for interpolation and research to gain all information needed for a reconstruction.Regarding a workflow it is recommended to structure these source materials visually, i.e. via "production plans", to define leading sources for reconstruction, and to find strategies concerning how to cope with information from sources or new research insights which would occur at later stages of a reconstruction process.
Especially for beginning project teams an aimed level of detail for a reconstruction would be defined within the very first and mostly fast progressing digitization stages and would often underestimate the time consuming reconstruction work during later stages.Even if most of the projects observed find a way to increase their budgets to solve this dilemma, it would be suggested to strive for a lower level of detail than it seems reachable at the beginning of a project and to implement strategies to control and adopt or redefine operative objectives and next tasks periodically, i.e. via priority lists.In the same matter, identify implications from project and innovation management to structure project work and implement strategies and employ persons to periodically control project workflows and adapt to current needs.
Many challenges for 3D reconstruction projects are connected to a lack of interdisciplinary understanding.Intensive support by images during a reconstruction process could foster especially interdisciplinary communication and could be used as a "creoles" (Collins 2007) for exchange and sharing of mental models.Here it is necessary to synchronize visual codes like symbols, colors, or tags.Such decisions and tasks should be started at an early project stage and should be controlled and adapted over the whole project process.Ideally, such visual coding schemes would be a team shared mental model of all project mates and would be documented and oriented with ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume II-5/W1, 2013XXIV International CIPA Symposium, 2 -6 September 2013, Strasbourg, France This contribution has been peer-reviewed.The double-blind peer-review was conducted on the basis of the full paper.200 either extant coding schemes, i.e. from engineering, or would use "natural" codings like physical analogies or concrete depictions (Tversky 2002) to make these issues recognizable at later times or even accessible for later works.But in all cases images would only support communication and, especially for complex tasks and interdisciplinary exchange, personal communication would be more useful than distance communication.
Moreover, for both professional and non-professional recipients it would be important to know or be informed about what the benefits and limitation are of reconstruction and depiction.Beyond a digitization of extant objects 3D reconstruction could make former tangible no longer extant objects or intangible historic issues "tangible" and imaginable from descriptive sources.New technical possibilities such as rapid prototyping and "lively" presentations facilitate a presentation and education of historic objects.
Especially since results are connected to a high level of suggestive power, a creation process of such reconstruction should be transparent and open.Reconstruction and images not only "depict" history, but are also the result of including many subjective and possibly biased sources and including a subjective decision process.All interpretations of history reconstructions just provide a more or less probable hypothesis.
In comparison to texts it would have to been taken into account that images and reconstructions are nomothetic and could hardly express uncertainty or probabilities.Beyond that it would be most important to uncover a genesis and a "chain of evidence", not only as a link to the sources on which created images and 3D models are based, but also to make decisions and production processes as a link between sources and results transparent (Bentkowska-Kafel, Denard, and Baker 2012).Professional vision as ability to focus on certain aspects (Goodwin 1994) and visual reasoning (Tversky 2005) both play an important role for production processes, and a control of a model created by experts is mostly taking place using visual "pattern recognition" (Simon 1990) strategies, i.e. by comparing rendered 3D model views with either pictorial sources or mental models.
While a documentation of projects is often incomplete, it would be most useful to keep artifacts like sketches, protocols and model states to support a remembrance of creation processes as well as to offer opportunity to negotiate decisions.

GENERAL CONCLUSIONS
What are new implications for the field of Cultural Heritage?3D reconstruction and the genesis of virtual images bridge the gap between intangible and tangible sources and support a contextualization of sources.Even if an intensive use of images especially as sources for 3D reconstruction of no longer extant objects and for a depiction of results were widely known, the range of functions and a crucial importance of images within such reconstruction processes seems surprising.Especially for communication and for negotiations, visual media are widely used within workflows.Other insights for performing 3D reconstruction historic objects could be that many suggestions from project and innovation management would fit for that kind of task and could foster working processes.While there are many efforts to make sources used for reconstruction visible in results, too, there are only a few possibilities and practically used approaches to making the decision and creation processes transparent.In addition, it would be important to find strategies to include a creation process in scientific discourse.That seems especially important, since implicit professional vision processes and visual reasoning are quite important within such processes.An archiving and preservation of documents and artifacts created during a production process would not only support a later remembrance and reconstruction of workflows for project members, but potentially also for other researchers to review such processes.
What are limitations?This study performs a qualitative, explorative research using methods from social sciences.That indicates a possibly subjective decision taking.Moreover, the sample is, in relation to similar investigations, relatively small and based mostly on ex post material and should be further increased.Regarding the extracted suggestions and recommended strategies, a major and unanswered question would be how projects evolve over time and how important learning or development by experience is.To investigate, future work should perform long term studies observing all project stages.

Figure 2 .
Figure 2. Paradigmatic model of production process members, communication, cooperation, and quality management [Figure 2].Specific challenges for such interpretative reconstructions are more their need for coping with historic sources or interdisciplinary workflows.Most of these projects were realized by interdisciplinary teams.

Table 1
). † . Sample (Publications) While such reports provide only a retrospective view, a third stage included four case studies exploring 3D reconstruction projects of historic buildings over time to examine aspects of visual communication and their evolution during a project creation process.