A COLUMN-BEARING LION BY LUPO DI FRANCESCO
3D Graphics for Cultural Heritage, University of Pisa
1) The photographs
The first step of the acquisition technique Dense Stereo Matching consists in taking a series of pictures of the object to be reconstructed. To take pictures of the sculpture, I used a Canon EOS 500D digital SLR camera with an 18–55 mm lens, without the zoom or flash. I took a number of pictures moving around the sides of the lion in a circular path; I made sure I had a good overlap of successive pictures and tried to take pictures covering the entire surface of the sculpture.
2) Reconstruction of the point cloud
A first attempt with Dense Stereo Matching starting with photographs was done using the Arc3D Web Service, which permits remote 3D reconstruction sent to the user by email. However, the results were not satisfactory.
Therefore, I tried again using Visual SFM software. With Visual SFM the Dense Stereo Matching processes can be followed step by step in four stages:
-
photo upload
-
automatic calculation of image matches
-
sparse reconstruction of the scene
-
dense reconstruction of the point cloud
Since the results were not satisfactory, I took more pictures of the sculpture to get better images (more in focus, with a more suitable white balance and better overlap of the sequences of pictures). Therefore, I made several attempts with Visual SFM using composite sets from various photographs. However, reconstruction with Visual SFM still did not yield satisfactory results. In fact, the front of the lion, which has extensive relief work, was reconstructed well, but the hindquarters and the sides of the belly and legs, which are very smooth, had big gaps. Even when I supplemented the set of pictures with details of the most problematic areas and attempted a dual reconstruction of the sculpture by dividing it into two parts, the same problems continued to arise and the gaps remained very evident.
Therefore, a set of 49 photos was uploaded using commercial software, AgiSoft Photoscan, which made it possible to obtain a dense point cloud suitable to continue the work.
3) Processing of the model with MeshLab
The next steps of the work to develop the model and process the data were conducted from this point on using MeshLab software, to which I uploaded the .ply file with the dense point cloud.
3.1) Cleaning
In the dense point cloud I reconstructed everything that could be observed in the photos, and thus an entire section of the room in which the lion was located. Therefore, first of all I had to remove vertices not related to the sculpture using various selection and deletion tools and filters available in the program (Select vertices, Delete vertices, Select->Invert selection). After completing an approximate deletion of many of the excess points around the lion, I continued with detailed cleaning aimed at sparing only the vertices related to the sculpture in the most precise way possible. For example, I deleted the vertices around the lion’s mouth. I then tried to remove the dark points distributed along certain areas of the surface as much as possible. This was delicate work that required a great attention to avoid creating holes in the point cloud. Furthermore, the work had to be saved frequently, always keeping the “Normal” box ticked so that the point cloud would not lose the normals per vertex.
3.2) Simplification
Since the dense point cloud was composed of a high number of points, in order then to launch the reconstruction filter of the trangulated model on a “lighter” model and keep the system from crashing, I simplified it, reducing the vertices from around 1.500.000 to an half using the Sampling->Mesh Element Subsampling filter.
3.3) Reconstruction of the triangulated model
Before launching the filter that transforms the point cloud into a triangulated model, I activated the Decorator Show Normals to verify that, during the various saves, the point cloud had maintained the normal per vertex.
Then I launched the Remeshing, Simplification and Reconstruction-> Reconstruction: Poisson reconstruction filter. I applied the values of 12 and 10 to the “Octree Depth” and “Solver Divide” parameters, and instead left the default value of 1 unchanged for the “Samples per Node” and “Surface offsetting” parameters.
Starting from the vertices and their normals, the Poisson reconstruction then transformed the dense point cloud into a 3D model composed of triangles.
As it can be seen, the triangulated model reconstructed with Poisson presents two fundamental characteristics:
-
it is completely closed;
-
compared to the point cloud, it lost the colour information and is a uniform white.
3.4) Cleaning
Therefore, the Poisson filter reconstructed a completely closed model and added more geometry with respect to that of the real object. In particular, Poisson reconstructed the lower face of the base of the sculpture, which was obviously not photographed and thus was not present in the point cloud. Furthermore, Poisson closed the lion’s jaws and reconstructed the zones that had more gaps in the point cloud, such as the haunches and the lower surfaces of the plinth above the animal. These parts “invented” by Poisson are immediately recognizable on the model not only with the “Flat” viewing mode, because they are particularly “convex” compared to the rest of the surface, but also with the “Wireframe” mode, through which we can see that the triangles composing them are larger than the others.
Therefore, the model had to be cleaned to remove the superfluous geometry reconstructed by Poisson. First of all, using the Select faces in a rectangular region and Delete selected faces and vertices tool, I eliminated the lower surface of the base on which the lion stands, which had been completely invented by the filter. To obtain a lower border that would be as even as possible, I performed remeshing before cutting. To do this, I used the Remeshing, Simplification and Reconstruction->Surbdivision surfaces: Midpoint filter to subdivide the larger triangles and decrease their size. This made it possible to select and cut the surface, yielding more linear borders.
Initially, in this phase of the project I tried to cut the base of the sculpture precisely. However, once I arrived at the initial results of the work, I noticed that I had accidentally cut too much geometry in that zone. In fact, on the completely white model I did not manage to be accurate and had sacrificed part of the reconstruction of the base. Therefore, I decided to go back to the model reconstructed with Poisson and, in the cleaning phase, to maintain some of the “invented” surface at the base and then eliminate it at a later stage, when it would be possible to use colour re-projected onto the model as a guideline to execute more precise cuts.
The same thing happened with the lion’s mouth. Poisson had filled in the jaws, which are instead open on the sculpture. However, manual selection and subsequent deletion of these parts “invented” on the completely white model were rather arbitrary and thus inaccurate. Therefore, here as well I decided to do the cleaning at a later stage so I could take advantage of the guidelines offered by the colour re-projected onto the model.
Instead, I decided to maintain the two lower surfaces of the plinth. In fact, although these geometric parts were largely “invented” by Poisson, they refer to a visible zone of the sculpture. Therefore, deleting them would have yielded a final model with large and very evident gaps.
I completed this cleaning phase of the model using the following filters present in the Cleaning and Repairing submenu:
-
Remove duplicated faces
-
Remove duplicated vertices
-
Remove zero area faces
-
Select non manifold edges->Remove
-
Select non manifold Vertices->Remove
-
Remove Unreferenced Vertex.
"Flat" viewing mode
"Wireframe" viewing mode
3.5) Smoothing
The model reconstructed with Poisson had noise that was very accentuated above all on the lion’s hindquarters, where the surface of the real object is instead very smooth. Furthermore, in the areas where the vertices in the initial point cloud were not as dense (the lion’s haunches and the lower surfaces of the plinth) I noted artefacts that were not realistic and were unattractive.
These results led me to attempt to improve the model.
a) I tried to return to the point cloud and relaunch Poisson, raising the value of the “Samples per Node” parameter. By doing this, the reconstructed model unquestionably showed less noise and was smoother, but this also meant a very evident loss of details, above all to the detriment of the reliefs of the lion’s mane.
b) I tried launching the Smoothing, Fairing and Deformation->Laplacian Smooth filter on the model reconstructed with Poisson. Here as well, through the repeated use of this filter on the entire geometry, the model lost a great deal of noise but also numerous details.
In both cases, I felt that such a large decrease in the details of the reliefs of the lion’s mane was not ideal, as the most distinctive part of the sculpture would have been lost. At the same time, however, I felt that the smoothness of the animal’s hindquarters had to be restored to illustrate the unique contrast between these two parts of the sculpture.
Therefore, I decided to take a different approach.
First of all, I tried cleaning the point cloud better. Then I intervened on the Poisson model (values 12-10-1-1-) through targeted smoothing. I combined use of the Smoothing, Fairing and Deformation->Laplacian Smooth filter applied only to selected zones with local manual smoothing executed through the specific function in the Paint tool. This made it possible to smooth only the rear part of the model, while maintaining the front part. Therefore, I kept working on the noisier parts, such as the lion’s haunches and the lower surfaces of the plinth, adjusting the dimensions and hardness of the manual smoothing instrument.
However, these particularly problematic zones showed the formation of artefacts that even smoothing was unable to improve. Therefore, I decided to select and delete the artefacts in order to close the holes that had been formed, using the Remeshing, Simplification and Reconstruction->Close Holes filter. Then I performed manual local smoothing on the parts that had just been reconstructed in order to make them the same as the adjacent surface. Furthermore, I used the filter Remeshing, Simplification and Reconstruction->Subdivision Surfaces: Midpoint, in order to decrease the dimensions of the larger triangles reconstructed by the Close Holes filter.
I also tried to apply this procedure to remove the excess geometry that Poisson had reconstructed between the lion’s right haunch and its belly. In this case, however, the results were not satisfactory. In effect, once this part of the geometry was deleted, the Remeshing, Simplification and Reconstruction->Close Holes filter and smoothing did not improve things. At the same time, however, not closing such a large hole would have greatly compromised the model. Consequently, I decided to maintain this part of the geometry, even though it is not present on the real sculpture.
3.6) Colour
As we have seen, the model reconstructed with Poisson had no colour information, as Poisson does not preserve the colour attributes of the vertices.
To obtain a coloured model, I took two different approaches.
a) Color-per-Vertex
This method envisages transferring the colour from the point cloud, in which every vertex has a colour attribute, to the triangulated model.
I used the Sampling->Vertex Attribute Transfer filter, taking the point cloud as "Source Mesh" and the previously cleaned and improved triangulated model as "Target Mesh". With this approach, the colour is contained in the geometry. In fact, each vertex of the model is assigned a colour value and the space between points is filled in through interpolation.
If we look at the result obtained on the model with this filter, we can see that in the areas “invented” by Poisson and those where the vertex density in the point cloud is lower, the colour is spotty and does not reflect the actual situation. In fact, to obtain good results many points need to be close together. Therefore, in order to improve colour I tried to increase the vertices in the triangulated model using the Remeshing, Simplification and Reconstruction->Subdivision Surfaces: Midpoint filter. Nevertheless, the colour was still imperfect in the more problematic areas, such as the lion’s haunches and chin and the lower surfaces of the plinth.
With respect to Color-per-Vertex, on this model the colour detail obtained with Texture Mapping is greater, above all on the lion’s mane, where the result is very good and accurate. Here as well, however, in the problematic zones of the lion’s haunches and the lower surface of the plinth, not covered well by the photographs, the colour is imperfect, and has pale rings and dark spots. Furthermore, the colour obtained with the Texture Mapping was darker than the real sculpture’s one. Therefore, I increased the Gamma value using the Color Creation and Processing->Vertex Color Levels Adjustement filter.
3.7) Cleaning
Both the model obtained with the Color-per-Vertex and the one obtained with the Texture Mapping, required a following cleaning intervention. Therefore, I cut the both models’ base lower borders more precisely. Furthermore, I “punched” again the lion’s mouth, that Poisson reconstruction beforehand closed.
b) Texture Mapping
Another colour-transfer method envisages projecting colour information onto the triangulated model from the set of photographs used for Dense Stereo Matching.
To attain this type of colour projection, in MeshLab I opened a file in the .mlp format, with the images aligned to the point cloud. I then uploaded the triangulated model and launched the Texture->Parameterization+Texturing from registered rasters filter. As a result, the texture from the related aligned images present in the Raster Layers was projected onto the various parts of the model. As opposed to Color-per-Vertex, in this case the colour information is not contained in the geometry but comes from an external file with a 2D image, and it is not recorded on the vertices but on the faces of the triangles.
3.8) Scaling
After obtaining the two final models coloured and the one with Ambient Occlusion, they had to be scaled to restore the measurements of the real object. This is because the models reconstructed with MeshLab present an arbitrary scale and their unit of measure is not the same as that of the real world. I used the following procedure to scale my models.
-
I took two real measurements of parts of the sculpture that are also clearly visible on the 3D model, i.e., the length of one of the sides of the base on which the lion is set (96,9 cm) and the length of one of the sides of the plinth (27.8 cm).
-
With the Measuring tool, on the model I took the arbitrary measurements that MeshLab assigned to the same portions of the sculpture on which I’d taken real measurements (7,383 and 2,118).
-
I calculated a scaling factor through the Real measurement/Model measurement formula. With both reference measurements, by approximation the scaling factor was 13.
-
I applied the Normals, Curvatures and Orientation->Transform: Scale filter, inserting the scaling factor to the X-axis and ticking the “Uniform scaling” option.
The MeshLab arbitrary measurements
The scaled model with real measurements
3.9) Positioning on the axis
Lastly, I positioned my final models correctly on the axis.
First of all, to verify the position of my model with respect to the axis I activated the Decorator Show Axis. I then used the Manipulator tool, which through translate and rotation commands made it possible to shift the model manually and align it correctly on the X, Y and Z coordinates. To make the new position of the model permanent I used the Mesh Layer->Freeze current matrix filter.
Now the lion is in the correct position in MeshLab from the moment the file is opened.
