Good Things Come to Those Who Wait: a more detailed look into merging 3-D models

The title of this blog sums up this past week pretty well. I work for OCPS as a middle school teacher and only have time at night or on the weekends to work on this project and no longer have the luxury to work whole days on classes as I have done in the past. This is prevalent because the merging processes took around 30-45 minutes each time I ran the process. I created many models this way before I realized an error in what I was doing. I could not separate the points based off of confidence as I learned last week because I rendered all the models with he advanced setting “calculate point confidence” turned off by default. Having this enabled is necessary to making the model clean as I will demonstrate in this weeks blog. This caused me to rerun these merging processes again, however enabling the “calculate point confidence,” caused processing time to increase even more from 30-45 minutes to between an hour to an hour and 15 minutes. Once more, if I realize there is an area of the model that is shaky and could use more photos then this waiting process has to start over again. This is why I describe this week as a lot of waiting as a single mistake can cost me a whole evening of just waiting.

                After hours of researching on the internet why I was having such difficulty merging models I discovered that Metashape has an automatic alignment method without the need of finding key points of similarity around the model. It took some time to get the hang of it but this made the process much easier by having the program itself detect the similarities.

                Once the models were merged I realized I forgot to cut away the table portions where the artifact was sitting before I merged the models but I was still able to cut it away after I merged them and it was not too much a problem considering the alternative was another hour of waiting time. I was now able to see the model in a heat map style of view that is organizing all of the points based of off how confident the program is on their placement. Red and orange dots are all those points that the program is unsure of and are therefore clouding the model giving it a dirty or fuzzy appearance. All those models that shift from green to blue are high confidence points where the program is confident in its placement. I was able to filter out the view to separate high and low confidence points and once there I could delete all low confidence points leaving behind a clean model.

This is the confidence map. The key in the lower left helps understand that the colder the color the higher the confidence value is.

The red and orange points are filtered out from the high confidence points and then deleted. These points are what clouds the model and gives it a fuzzy appearance due to their inaccurate location.

Only the high confidence points left behind

The dense point cloud after the confidence cleaning process. Notice how this is cleaner than any other dense point cloud I showcased thus far.

                From here I realized I would need to go back and take even more pictures on the bottom side of the model due to the lip of the conch being undefined and curved as opposed to a strong harsh edge. This artifact was included in the collection to be one of the more difficult model to create and so far it has and will continue to cost me more processing time as I try to get the inner fold of the shell to render but now with this streamlined process of cleaning the model I’m confident that this once will be completed as well.

The above photo shows the end product that is fully textured. This is the cleanest model I have made yet and looks perfect.

This is the problem mentioned in the above paragraph. The inner lip of the conch did not render any points due to a shortage of light and photos.

The Merging process

 

This past week I worked on figuring out how to merge two models together. This process is little longer than I anticipated but does not change my view on it being worth it. I did not encounter this problem earlier because the test models I was capturing were sitting on a base where I did not need to capture and in retrospect it would have probably been best to learn the merging process earlier.

I managed to successfully merge one of the artifacts together while the rest are in different stages of this process. The process involves placing a set of points on each model with the key being to place point 1 in the same location on both models. So point one will be in the same location on the artifact just placed on both models. This process is continued to at least three times, however the more points placed the higher the accuracy is during the merging processes. The purpose for placing these points is to tell the program which points match up across both models so they can align correctly.  The difficulty is in the form of finding a place on the model that is identifiable which is not always easy depending on the model, finding the right ridge on an arrowhead takes time as well as finding multiple points that are identifiable.

                Thankfully I learned that, once merged, Metashape can categorize all of the millions of points in the point cloud and separate these based off of a certainty rating that looks like a heat map. I can then easily remove all points that Metashape created yet was uncertain of its exact location while preserving the majority of points that are slotted into their correct position. This can save me a lot of time cleaning the models and removing the uncertain points by hand. This is an advanced feature I haven’t seen in free modelling programs and the longer I use this program the more I grow to understand more of its complexity.

                I still need to take this process slowly as I double check and review the steps that I am taking to get more used to it but I will go ahead and print a copy of a broken arrowhead that was created with the merged models and one with the blank side filled in by default to best see and compare the differences between them. Perhaps the detail will transfer well into the physical form or perhaps it will be too hard to tell.

A Tale of Two Models: Problems with Merging two 3D models together

This past week I made two models of every artifact that captures both sides of the models and built each model to be textured which is the last step in the process. I am still figuring out how to merge the models into one complete whole that leaves no gaps. There are a few different methods of alignment that I tried and so far I have had no luck with either. The goal moving forward is to find the best alignment method which should not take long to complete once I figure it out, but this will allow me to have complete models for each of the six artifacts that can then be printed.

                Also this past week I decided that the newer models I would create would have a lot more pictures in the profiles then I normally do. The other profiles had on average 50-80 pictures to create the model and I wanted to try capturing more pictures to see what type of difference that would make, so I captured 125 photos for one of the models which drastically increased the detail of the model I was working with but it took much longer to process. On average, to build the dense point cloud for the other models only took 2-5 minutes at the most, however, this conch shell that had 125 photos in it took almost an hour just to build the dense point cloud. Other steps in the process also took longer than normally. This created a scenario where I had to go work on something else while this processed, but I believe that it is worth the wait overall since the detail increased significantly. This is noticed just by looking at the sparse point cloud. Normally the outline of the object was suggested before, but the conch shell is already very prominent. Dense point clouds for all other models ranged between 1 to 1.6 million points while the Conch model had 3.4 million points which explains the processing time.

                This past week I printed a cylinder seal for Dr. Tiffany Spadoni and I also printed one for myself to use in a lesson on Mesopotamia for my 6^th grade classes. The ability to print artifacts like this off a cheap home printer has allowed me and will allow Dr. Spadoni to use these replicas as a teaching tool. I was able to show the students what a cylinder seal not only looks like but also how it works by demonstrating it on playdough. This is part of the benefit of the internship that I am working on, this allows a hands on approach to the past that not only interests students but creates a lasting impression of engagement.

Progress on plastic comparisons and the modelling process

This past week I have been crafting the 3D models based off of each artifact as well as cleaning and comparing the resin and plastic test prints. Over the next week I’ll be finishing some of the models and begin the printing process.

                The digital side of things each model has brought its own difficulties but none have so far proved too difficult. Many of the models need the assistance of playdough to remain upright so I can capture both sides, however this means I am not capturing the piece placed into the playdough. I have two choices; my first option is to leave the blank hole there and solidify the model in another program which would just fill the hole in and make it smooth, or my second option is to flip the model and capture the bottom side and then merge both models together to fill the hole. The second option is preferable but learning how to do this took up a larger portion of my time but I feel it will be worth it due to its accuracy. Additionally the glare on the sharpening tool was not as much of a problem as I anticipated as I could just turn down the lighting in my apartment and use more diffused lighting which allowed me to capture it without having random holes in it due to reflections.

                The plastic to resin comparison is following what I anticipated but new points have been brought to light as I broke the plastic models from their supports. Breaking the supports off of the model caused support marring on the models as anticipated that needs to be cleaned up more accurately either through sanding or with a fine edge tool, however some of the models were difficult to remove from their supports and I actually tore the arrow head into two pieces as I cracked the model cleanly along the layer line as I was trying to move the supports. The supports were fused to the model in some places and those touch points were stronger than the actual model itself. Additional support issues come from the wax seal and lock as there are many small little strands of plastic obscuring the detail that are a slow process to remove as well as the quality of the lock. The lock was printed on resin and FDM using the same digital model yet the plastic version has tiny holes and layer lines that seemed frayed and pulled out of position creating more holes.

Notice that some layers seem pulled out of position and some gaps become visible

This photo is similar to the one displayed last week. Notice that amount of thin strands that need to be cut and pulled off of the model's face. Additionally, the depth of each letter presents a visible difference

Overall, I would rather deal with the resin post processing any day because it is so much easier to not damage the model, preserve its detail, and faster to process. The plastic models, while they printed in a quarter of the time it took to print the resin models, the post processing is rather labor intensive and create a higher risk for damaging the replica. I should note, however, that I have a lot more experience with printing resin models than I do with FDM, especially because I do not place the supports on FDM nor do I control any of the other printing options and there may be ways to adapt the printing process to make the post processing more viable, less damaging, and less labor intensive.

Plastic Comparisons and Artifacts for Project

This past week I met up with Emma Dietrich from FPAN and talked about fixing the low-resolution problem I was having last week. Emma also brought the plastic replicas from the original test set so I could compare them to resin, as well as the actual artifacts that I will capture in 3D and print.

The plastic models were left on their support structure to help me better understand the post processing that goes into cleaning the models after they print while also having another aspect that I could compare between the two printing methods. Once I properly clean the support structures off I will be able to compare the two sets more fully, however initially the plastic prints look much better than I anticipated and do not bare strong layer lines which is traditionally associated with FDM prints. The plastic prints are also lighter and printed in a fourth of the time it takes to print their resin counter parts. While I will wait until I can compare them more fully next week, there was already an initial problem with the tip of the arrowhead pictured below. The plastic tip seems to be thinned, hallowed, and appears as if it isn’t complete. This could be a problem during the printing process, a limitation of FDM, or perhaps damage the model sustained after printing. Ill be sure to reach out to see if holding sharp points on FDM is normally an issue.

                I received six artifacts in total to preserve with two of them being purposefully selected to be more complex and difficult to capture and print. The first four models I will focus on are the easiest to capture and are three arrow heads and a fishing weight for a net. All are straightforward matte objects which will help me start this project off by working out any remaining kinks as I start from the beginning of the process and progress all the way to printing the model off and cleaning off support structures. The two other artifacts are a sharpening tool that has a slight gloss finish and a conch that was used as a hammer. The sharpening tool provides an additional challenge due to its gloss finish which reflects light back at the camera and creates blanks spots In the model as Metashape cannot place dots in these areas because it cannot detect what is supposed to be in that spot. The conch shell provides a challenge because of its overhang but also its size. A model like this will require support structures throughout the overhang and will overall require more care throughout the printing and post process.  

    I went ahead and started capturing one of the arrow heads and by the time of writing this blog I realized a mistake I made that I will go ahead and display here. So far the model looked great but I failed to take enough pictures on a particular side of the model and it left a dead space of data. This tells me that I need to go back and capture more photos from this angle to fix this problem. However, getting the model to this point did not take long at all due to the time I spent previously working in the program.

A Successful Test Run

 

One of difficulties of this internship is finding a time when two full-time workers can meet in a COVID-19 world. The story the past few weeks has been one of scheduling conflicts, whenever my supervisor was free to meet one night, I had an obligation for work and vice versa. Emma and myself have slotted another Face-to-Face meeting for Tuesday, October 13 where I can hopefully start the work on the deliverable part of the internship.

I have gotten a good grip on all the software and hardware used in this process from taking the photos to create the image profiles, to using Metashape to render the 3D model, then to Meshmixer to make the model watertight and solid, and finally to load it into a slicer and actually print the model and handle and cure it properly. I am confident that this process will not take long once I get the artifacts to print as the longest amount of time was figuring out how to use all these programs and hardware. Emma has also reassured me that the FDM test models were printed. A potential future problem could be the turn around time on the FDM prints for the five artifacts. I would have to complete each 3D render and send if off to FPAN for printing, then meet in person once again to exchange the artifacts and FDM replicas in time to start writing the conference paper so my goal is to have a quicker turnaround time after Tuesday’s meeting.

The final step in the 3D modelling process that I spent the last week working on was taking the fully rendered model in Metashape and exporting it into Meshmixer which is a free, basic, modelling platform. I previously mentioned Blender which is a free, yet highly advanced program but ditched the effort as it was overkill and difficult to understand with the time frame and scope of the project. From here I took the model and separated many of the stray islands of data found inside the hollow model and deleted them so only one single continuous model remained. Then, with Meshmixer, I was able to make the model solid and close any gaps or holes on the surface of the model. From here, I exported the model into the 3D printing slicers and the model was good to print.

I did not print this test model because I have printed hundreds of models at this stage to know how it will turn out physically so there is no need to waste the resin considering there was a problem with the resolution. the final product lacked a lot of detail on the surface which was disappointing and this problem could have a lot of causes which makes it difficult to pin down. Could this be a limitation of low-cost photogrammetry and an issue with the resolution of the images? Could this be a limitation of the model itself that I choose, being that it is at a 32mm scale with a lot of detail? Could this be a limitation of Meshmixer or the exporting process? Or could this problem stem from a small step that I missed, such as exporting the texture of the model separately and working with that? These questions will have to be answered when I work with the actual artifacts because this model I choose is a very small and complex figure with lots of thin pieces, fine details, and sharp edges.

This is the final, yet low-resolution model. Notice the lack of detail all around the model.

This is the actual model itself that I chose as a test model because of its small size yet high detail.

This was the model finished in Metashape which bears more resemblance to the original than to the printable version which is what gives me so many questions.

Overall, I am happy with this internship so far. The first half of it I spent learning the software and hardware associated with it and these skills can all directly translate as usable skills for what I want my thesis to be on which saves me the time in the future with figuring it all out. Many people outside the history department hear the project that I am doing and glaze over it as they do with any conversation relating to graduate research, however, when I show the three test prints I have done previously they light up and become interested and want to handle and touch the replicas. This internship, and the concept of replication as a whole makes me excited because it is taking the past and making it a tangible experience for some where the deliverable is a product that can be held in hand which has even brought smiles to a few faces.

 This past week I've spent moving and unpacking my apartment and tried my hand at photogrammetry once again to see if I could solve my previous problem. I tired a few different methods and built the models through the rendering processes and had different measures of success.

The first method I tried was the same one I used two weeks ago but without the turn table and I just turned the model by hand. This method failed once again as the cameras would not align properly which only ends up rendering half of the model. The reason for this failure is that there is a visible texture in the background of all of the photos because of photobox. 

The second method I used was the same as above put I moved the camera much closer to the model so nothing but the model and the immediate area around it was in frame. This way, not only would more detail be captured, but there would be no extra data or texture that would confuse the program. This method was a success and the cameras were actually aligned around the model meaning it was rendering three dimensionally.

Each blue square is where the program detects where my camera was when I took it in relation to the model.

From here I cleaned up any stray points in the sparse point cloud to make the dense point cloud cleaner with only the data I need, then I built the mesh which takes all the dense points and essentially connects them all which starts to turn the model into something that could be solid. I then did the last step and built the texture which brings all the definition onto the model. Since this model is just the test model as I wait for the artifacts, I did not spend too much time cleaning away all the white noise around the model. Cleaning this would make the model more defined and sharp but it takes some time to do as you have to go in by hand and delete away each segment without deleted the rest of the model it is attached to.

This is the rendered texture, the final step in Metashape.

This is a close up of the wireframe

The third method I did was to actually move around the model, yet with worse lighting. I decided it was worth doing still despite the success of the previous method so I can compare and decide which produces a better result. This method was a success and all of the cameras rendered in the correct area, however, it is clear that these cameras do not make a perfect circle or were taken from exactly the same height as I took each photo by hand as opposed to having a tripod. Despite this I was able to take a lot of photos from around the model and the results turned out really well. There was more work cleaning out the stray data such as part of my couch, TV, and desk. I completed the above mentioned steps to render the model and it came out with what appears to be a higher resolution because of the additionally photos I took. Overall, I will try physically moving around the model more often since it produced great results despite shaky hands, uneven photo heights etc.

These cameras are spaced unevenly and are at uneven heights because I took this by hand as I moved around the model.

This is the rendered texture for the method where I physically moved around the model.

The next step is to important these models in blender and fill in gaps and holes and make them printable on a 3D printer where I can even test print it out to see how accurate it was to the original. Hopefully within the next week or two I can start the work on the actual artifacts.