Getting)started) - Jarod Schultz · 2013-05-23 · 2 2"|Page" " Getting)started)...
Transcript of Getting)started) - Jarod Schultz · 2013-05-23 · 2 2"|Page" " Getting)started)...
2 2 | P a g e
Getting started
One of the most fundamental factors for getting a good 3D model reconstruction from photos is to take GOOD photos and ENOUGH photos that have GOOD overlap. Please refer to the end of the document for tips on good shooting strategies.
Getting started -‐ A360 Sign in
To get access to the Photos-‐to3D Autodesk® ReCap™ Photo online service, you will need an account on Autodesk’s online cloud platform A360 https://360.autodesk.com
To run A360, you must use Google Chrome as an internet browser. Safari and Internet Explorer are not supported. Once you have A360 running in your browser, do the following:
• Login on: https://360.autodesk.com/landing with your A360 account • If you do not have one, hit the Create button.
3 3 | P a g e
Basic Workflow: From photos to a 3D model
Upload photos and submit to invoke the photo-‐to-‐3D service
1. In A360, select Documents (top left of your screen), and create a new folder for your project. 2. Upload the photos in that folder. Note that the service currently can only process JPEG images.
It is currently not possible to create a folder on the fly, while uploading the photos. You will need to make one prior to the upload process. It is a best practice to always create a new folder for each project. This will make your photos manageable. Otherwise everything will be landing in the main directory of the folder and soon you will have a mess of photos and .rcp files from various projects.
Select all uploaded photos and right click over them to get the Create 3D model option. To select all photos either check the first one and then the last one holding the Shift key, or
check the little folder icon on top left of the Documents screen. Your selection must only contain photos, otherwise the ‘Create 3D model’ option will not show!
3. After all your photos have been selected, right click on them and click the ‘Create 3D model’ button.
4 4 | P a g e
4. A Photo-‐to-‐3D pop-‐up window will open. On the left side of the window, you will find various settings specific to the 3D reconstruction.
Here you should do the following:
• Name your project – At the top left in the side bar there is a place to name your project.
• Select a mesh density /quality There are 3 choices for mesh densities/qualities for the generated 3D models:
§ Draft – you can use this mode to get a quick result and see if your stitching (3D model creation) will be successful. In draft mode, you can view only the reconstruction, but you cannot export the mesh. (The export options are greyed out in this mode) § Standard or Maximum – These mesh densities give you a high quality mesh model that can be exported in various formats.
• Select the file formats in which you wish to save the resulting 3D model. You can save as .OBJ, .FBX, .IPM and .RCS (Autodesk point cloud format), format. A .RCP file will be generated automatically with every 3D model reconstruction. This is the native Reality Capture Project file and it is that file that you will be opening to view in your browser.
When done with your selections, submit your project by clicking on the rocket icon (top right or bottom right part of the screen). Once you have clicked on ‘Submit Project,’ wait until this message appears:
This means that your data is sent to the cloud for calculation, and that you will receive an email that will inform you when the calculation is finished. Note that the email will be sent to the email address with which you logged in / registered on A360. You will notice that upon the submission, a new .RCP icon is added to your folder as a placeholder, indicating work in progress.
5 5 | P a g e
From this moment on, the cloud service is calculating the reconstruction of the 3D model. When the service is done calculating, you will receive an email informing you of the success of the reconstruction and a link to the 3D model. This will take some time (especially if you selected Standard or Maximum mode) so, there is no need to wait here. You can leave your A360 space and continue working on something else, until you receive an email from the service. The email will be sent by ‘[email protected]’. Note: if you selected ‘Draft’ mode, the calculation process will be fairly quick. However, if you selected ‘Standard’ or “Maximum’ mesh resolution, the process can take from 30 minutes up to a couple of hours, depending on the number of photos you submitted and their resolution. For high quality mesh generation, you will probably be dealing with 70+ photos, so expect that this process takes at least an hour. Also note that the service is currently optimized for projects with less than 250 photos.
Preview your 3D model results in A360 web viewer and download it on your desktop
In the email there will be a hyperlink that when clicked, opens the .RCP model directly in the A360 web viewer. Alternatively, you can open the model from within A360 folder. When the .RCP file is ready, you will see a thumbnail image of your model there.
5. If you are working from within your A360 folder, click on the .RCP icon and this will initiate the Web 3D Viewer where your scene will be displayed.
6 6 | P a g e
A few things to note here: You might get confused when you first look at the generated scene, as it probably will not look like the object you tried to reconstruct (a warrior statue in the example case). This is because the software calculates the entire scene around the object as well, not just the object itself. You will need to zoom inside the scene and find the target object. You will need to clean up the rest of the scene that you don’t desire in any mesh software (3dsMax, Maya, Mudbox etc.)
Secondly, as our service is young and we want to track its accuracy, You will be prompted to let us know if you are satisfied with the reconstruction. To navigate the 3D scene, you can use either the navigation tools displayed in the side bar on the right, or the mouse: left button for orbiting, middle button for zoom and middle mouse + hold for pan.
7 7 | P a g e
You can view the generated 3D model as a textured model (Default setting) or as mesh only. To switch between the two, click on the monitor icon in the menu on the right side and select the globe icon for mesh only display. To come back to texture display, click again on the globe icon.
Figure 1 -‐ Switch to mesh mode
You can also display the camera positions where the photos were taken, by clicking the camera icon in this sub-‐menu.
Resubmitting your scene:
You can, at any time, resubmit your scene to be calculated again. There are various cases in which you will need to resubmit your scene:
a) If you first started with ‘Draft’ mode, and after checking the result in the Web 3D Viewer, you want to create a higher mesh quality (Standard or Maximum Resolution).
b) After you have manually stitched photos that have not been included automatically. These would be then considered in a new reconstruction.
To resubmit a scene, go to the ‘Action’ menu while in the Web 3D Viewer. You can also access this by right-‐clicking the .RCP icon.
8 8 | P a g e
When you click ‘Resubmit 3D model’, the ReCap photo settings UI opens again and here you can change the settings to Standard or Maximum quality, and pick which file formats you wish to export to.
Then, you click the Submit button again to restart a new calculation.
After resubmitting your scene in Standard / Maximum resolution, you will again receive an email. The model you will see on the screen will be visibly better quality, although you should note that you are still only viewing a proxy of the actual mesh, not the full resolution of the mesh, even if the mesh density you set was Standard or Maximum
The following message pops up when opening the Web 3D viewer:
Its important to understand that this is just a viewing limitation that does not give justice to the quality of the 3D meshes that are created. The generated .OBJ or .FBX meshes are high quality meshes. To view those high resolution meshes, download the .OBJ or .FBX and use applications such as 3ds Max, Mudbox, Maya or similar to view or further edit the meshes.
9 9 | P a g e
MANUAL STITCHING
The above-‐described workflow is straightforward process and is all you need to do if you had good number and quality of photos. However, until you learn how to take photos specific for photogrammetry and gather experience based on what works best, you will probably need to do some manual work before arriving to a good quality mesh output. When there are issues with your photos, after the scene is reconstructed for the first time, you might see that you have a few ‘unstitched’ photos, displayed in the side bar (usually you need to scroll down). Unstitched photos are photos that the service couldn’t take into account for the model generation, because they have little similarity with the rest of the photos (for example, the contrast between two photos is too high, or the overlap is too small etc.). Adding those photos might increase the quality of the generated 3D model, but it requires a manual stitching process.
Manual stich:
Manual stitching means that you will manually define matching of points between an unstitched photo and stitched photo so that the system takes the unstitched photo in account.
To start a manual stitch, from the left side bar, scroll down to see unstitched images, pick one and select STITCH on the top right of the screen
In the new manual stitch window, the photo that you selected to be stitched will be displayed on the left side on the screen. On the right side, you will be offered a choice of multiple, already stitched photos to pick from. You should pick an image that has many similar points with the unstitched one.
10 10 | P a g e
Expand the little icon on the left corner of the right screen to see the photos to choose from.
You should now start identifying matching points between the two photos that are common in order to help the system understand where the unstitched photo fits. Defining 4 matching points in total is usually good practice.
When you click on a point you wish to set as common between the two photos, the cursor turns into a magnifying glass that helps more precisely define the point. You can change the magnification factor with your mouse wheel. Start finding a point on the left click. It will be marked as Point 1. Then try to find the same point on the photo on the right side. That will also be marked as Point 1. Continue until you do this with 4 matching points. You should see something like this when you are done.
11 11 | P a g e
On the bottom left of the screen you will also have a display of all matching points and the corresponding selections. To view all points you have identified, hover over the first icon. Then you can click on any of the other points to see the corresponding matching points.
When done, click on the Actions menu, select Resubmit model and in the settings window click Submit project. The service will restart the calculation and you will receive an email when its done.
ADVANCED FEATURES AND WORKFLOWS
ReCap Photo has advanced features and those are separated from the straightforward photo-‐to-‐3D process. You can see the additional advanced tools by clicking on ‘advanced tool’ button on the bottom left of the sidebar in the initial settings screen.
This will invoke new tools that will appear in the blue ribbon on the top of the screen: Registration and Texture. These tools should be used prior to a 3D reconstruction, at the very beginning of the process.
Registration tools
In the registration panel you can do two things: work with survey points, and use reference distances to scale your model.
Survey Points
Survey points are collected on-‐site using ‘Total Stations’ or laser scanners. They are ‘hard’ 3D points used in the stitching process to make it more robust, and to scale or locate your 3D model. To add a survey point, select a point on the photo for which you know the coordinates, and once clicked, fill in the X,Y and Z coordinates on the bottom left of the screen:
12 12 | P a g e
Reference distances
Photogrammetry does not have a sense of scale. When we create a 3D model from photos the actual size of the reconstructed 3D object is unknown. If you know the distance between points on your original object you can set a distance to your model and thus scale the model to its actual size. You can do this with the help of the reference distance tool which is located on the bottom-‐right corner of the Registration screen.
Note that you should do this PRIOR to submitting your photos for 3D reconstruction.
To start the process for setting a reference distance, select the advanced tools in the side bar and then, in the blue ribbon, click on Registration.
A UI with two windows, similar to the manual stitch window, will open. Here you pick one image in the left pane that shows the part of the object for which you know the reference distance, and then select another image in the right pane where you can also see the same reference for which you have a distance value.
13 13 | P a g e
The process will require you to identify a pair points in both images and then set a reference distance between them.
Start by clicking in the left pane to create Point 1. In the same pane create point 2.
Now move to the right side, and in the right pane match point 1 and point 2. It is very important that you activate point 1 in the left pane before selecting the corresponding point in the right pane. To do that just move over point 1 in the left pane and click on it. It will become active.
14 14 | P a g e
Do the same for point 2. Click first in the left pane on Point2 to activate it and then click in he right pane on the same point.
You should end up with point 1 and 2 in both the left and the right pane.
Only now should you click on the reference distance tool, located in the lower right corner of the screen. When opened, click on the tape measure icon and then click once on Point 1 in one of the panes, and then on Point2. The distance field in the measure UI will now be active, and you should type the known distance value in there. To finish, select the TAB key (that is Enter for browsers).
What you should end up with is a blue line that connects point 1 and point 2 in both of the panes.
15 15 | P a g e
Note that in the current release we do not let you define the units for the distance.
Texture
By default, ReCap photo uses all the photos both for the geometry creation, as well as for the texture creation. If you are an advanced user, understand photogrammetry well and have decided to do some work in photoshop to make the texture look prettier for visualization purposes, you will be able to decide to keep the original images for the geometry creation only (these are displayed as Calibration images) and use the new, visually improved texture images for calculating the texture (these are displayed as Texture images).
In the Calibration images row you see all original images used for mesh creation.
In the Texture images row you see those used for texture creation.
They are initially the same, but the texture images can be either turned off by simple click over them (this is when you don’t want them to participate in the texture creation because their colors are off) OR you can replace them all with new ones, after treating them in a photo editing software, by clicking the ‘use different texture images’ option on the left of the pane. Its very important to leave the original name of the photo in the edited photo and just add an index to it (a,x,etc.)
When replacing a texture image with a new image, the left pane will show the photo that was used for mesh creation, while the right pane the one that will be used for texture creation
16 16 | P a g e
Known issues, limitations and bugs
Limitations
§ BROWSER: You will need to use Google Chrome as the browser. We are using a Web GL based 3D model viewer to preview 3D models in A360 and thus Safari and Internet Explorer are not supported.
§ Try not to exceed 200 photos per reconstruction. § There might be a 5 min time lag between the moment you receive the e-‐mail notifying you that
the 3D model is ready, and the moment your project files are actually available in your A360 folder.
§ GoPro cameras are not yet supported by ReCap Photo. They use a fish-‐eye lens, which requires a different mathematical model than the one currently implemented in ReCap Photo.
Bugs
§ If you are using Mac, you might run into an issue that the .OBJ and .FBX after being uncompressed are not readable in Maya or 3dsMax. We found out that some decompression software turns them into ‘No access’ files. Go to the properties of the unzipped files (Get info on rmc) and change the permissions from ‘No access’ to ‘Read and write’.
§ When creating a new folder on A360, sometimes the folder does not show up immediately. Try to refresh the page few times, until it appears.
3D Viewer
As previously mentioned, note that the viewer we use on A360 to preview the generated 3D scene has limitations, and thus what you will be viewing on screen is only a proxy and will not do justice to the actual quality of the mesh and texture, when processed in Standard or Maximum Resolution. You can see the high quality results once you import the exported meshes in Maya, 3DsMax etc.
Enjoy using ReCap Photo and please send any questions, suggestions and wishes on the ReCap Photo forum http://feedback.autodesk.com/cloudservices/products/cloudservices_recap
Your ReCap Team
ADDENDUM
How to take good photos for photogrammetry
17 17 | P a g e
What makes photos good for photogrammetry?
The photogrammetry process looks at series of photos and reconstructs the camera location, orientation, lens type and lens distortion. It does that by comparing and matching pixels/points across photos in the Entire Scene and match/ triangulate on visual features. This means that you should follow the following guidelines:
ENVIRONMENT (Lighting conditions)
You should avoid any environment in which the light creates strong contrast shadows over the object you shoot. Shadows are enemy to the reconstruction process.
• If you shoot in interior o The object must be placed in a room with diffuse light that does not produce contrast light
or shadow. o Do not use the flash.
• If you shoot outside o Avoid shooting at midday under strong sun. Best times are early in the morning or later in
the afternoon
EQUIPEMENT
• You can get good results with any camera. However, plastic lenses only go so far in terms of sharpness. If you are serious about this process a good lens makes a difference.
• Try to shoot the photos with a fixed lens. Fixed lens have no focal change which is good for the photogrammetry process. A 50ml lens is good.
• Whenever possible, shoot with a tripod and if possible a clicker. You need this because in order to make the entire photo sharp, you will have to set a wide depth of field. This means that a very small amount of light will be coming in the camera so you will need to set your shutter speed long. This makes even a tiniest of shake of the camera result blurry photos. The tripod and a clicker will assure sharp photos.
SHOOTING STRATEGIES – SCENE SET UP
• Make sure that he object you shoot is always in the middle of the photo and fills most of the frame • Use rich background. If you shoot a single object, avoid placing them on a monochromatic surface.
If the surface is monochromatic, add many other smaller objects around the target object. The algorithms need as many different references in the space to do better calculation.
• Avoid green photographers screens (ChromaKey). Photogrammetry doesn’t do well with them. • Don’t move objects around during the shoot and don’t have people walk around. Movements in
the photos create uncertainty in the reconstruction. • Shoot all around the object to cover each aspect of it with multiple overlapping photos. Shooting
each 5-‐15 degrees on the same height of the object is a good practice. If you want to have a good level of detail, you should shoot at 2-‐3 different heights of the object, ending up with 70-‐100 photos for a small to medium object and 180-‐200 photos for a building.
18 18 | P a g e
• Take enough photos. It is not enough to have many photos, but they must also have a good overlap
between them. If the overlap is too small or non-‐existent, the software might not take an entire cluster of photos in account for the reconstruction, and the reconstruction will be inferior to its potential.
• When shooting Interiors, follow the directions on the diagram below. Instead of standing in the
middle of the room and rotating around your own axis to shoot the photos, move along the walls to shoot in the other direction
•
When shooting Exteriors, again, avoid standing in one spot and shooting ‘panorama like’ spherical photos but move along the façade with the camera.
• Don’t have people walking cross the scene while the photos are taken. Anything that makes a
movement in the scene makes it hard for the algorithm to compare pixels between the photos
19 19 | P a g e
PHOTO QUALITY
• The photos must be in focus SHARP and CRISP throughout the entire scene ( not just the object of desire) as we compare pixels from the entire photo. If you happen to have few blurry photos among the entire set, remove them before making the reconstruction.
• Avoid blurriness in any part of the photos (ah, that beloved Depth of field effect). That will not work very well for photogrammetry. Do not assume that all is well if the object you try to shoot is sharp. We calculate the entire scene, not just the target object and any blurriness will confuse the algorithm.
• Do not change the zoom if you don’t have a fixed lens. Rather move towards and away from the object.
• Do not change exposure during the same acquisition. • Set ISO to 100, it works on pretty much any reconstruction. Or simply keep the ISO low. High iso
brings noise which should be avoided. • The ReCap Photo service currently accepts JPEG format only. However, if you shoot in RAW, you
will have the ability to do some changes in the RAW images that WILL affect the quality of the 3D model. You can always re-‐export JPEG’s from the Raw images using photo editing sowftare such as Adobe Photoshop or Bridge.
KNOWN TECHNOLOGICAL LIMITATIONS OF PHOTOGRAMMETRY
The photogrammetry process does not work on shiny, transparent or highly glossy objects. This is due to the fact that these reflect the surrounding which make the same point on one photo to be in a different color on another photo, making it impossible for the algorithms to calculate the scene.
20 20 | P a g e
For any comments or suggested changes or additions to this document, feel free to drop a note to [email protected]