© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.

CMP321

Deploying photogrammetry &

3D models in the cloud with
RealityCapture
Kurt Scheuringer Jerry Tejada
(he/him) (he/him)
Principal Prototyping Architect Partner Solutions Architect
Amazon Web Services Amazon Web Services

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Challenges with photogrammetry

Manual cleanup Consuming local resources Owning the pipeline

Large photogrammetry projects Traditionally, users run Enterprise customers want to
typically require significant cleanup photogrammetry workloads locally own the pipeline process

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
RealityCapture (RC)

RealityCapture is a fully featured

photogrammetry software for
automatically creating virtual
reality scenes, textured 3D
meshes, orthographic projections,
and much more from images,
videos, or laser scans

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Photogrammetry use cases

Source: Epic Games Learning

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Photogrammetry cloud process

Source: Epic Games Learning

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Input image dataset examples
Notch Peak sample dataset
Epic Games
173 pictures at 100+ Mpix

Romania dataset from

Epic Games case study

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Sample images, videos, and datasets provided by Epic Games Capturing Reality Case Studies
RC user interface – Notch Peak

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Sample images, videos, and datasets provided by Epic Games Capturing Reality Case Studies
Process flow in the cloud
Post-processing

Object Base model Manual

masking for processing enhancement
removal

Prepared
high-resolution
Image dataset Amazon Rekognition Align, base model Control points, mesh models and
image and texture cutting, etc. textures

Base models
and textures
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Pre-/post-processing examples
Add control points – Use cache project files to reprocess faster and add GCPs
Additional datasets – Adding datasets such as structured-ordered LiDAR data
RealityCapture Model export – Export to other supported 3D formats (e.g., OBJ, FBX, PLY, GLB)
Model preparation – Fine-grain model cleanup (e.g., lasso selection, hole removal)
Mesh segmentation – Divide and cut high-density meshes into smaller parts

2D image preparation – Advanced image masking, blur detection, and other cleanup
3D scene analysis – AI/ML to label full 3D scenes
Other tools
Model preparation – Advanced model cleanup and decimations
Format conversions – Convert to other 3D formats (e.g., GLTF, USD)

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Model outputs from a basic pipeline

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Sample images, videos, and datasets provided by Epic Games Capturing Reality Case Studies
Common pipeline CLI command sequence
RC Command Function CLI - RealityCapture.exe*

Run headless instance -headless -set "appQuitOnError=true"

-set "appProcessAction=ExecuteProgram" -set "appProcessActionTime=0"
–stdConsole
-writeProgress "ProcessingProgress.txt"
Activate instance with license (Param: License token) -activate ${licenseToken}
New scene/add images -newScene -addFolder “./s3-input-images"

Align images/set auto-reconstruction region -align -selectMaximalComponent -setReconstructionRegionAuto

Construct high-res model -calculateNormalModel -renameSelectedModel HighPoly

-calculateTexture
Construct low-res model, unwrap, and reproject high-res -simplify ${triangleCountTarget} -renameSelectedModel LowPoly
textures (Param: Triangle count target) -unwrap -reprojectTexture HighPoly LowPoly
“./TextureReprojectionSettings.xml”
Cleanup marginal (model-edge) triangles -selectModel LowPoly -selectMarginalTriangles
-removeSelectedTriangles
Export low-res model and save project files -exportModel LowPoly “./LowPolyModel.fbx”
-save “./Project.rcproj”
Exit application instance at end -quit

*RealityCapture supports CLI command chaining

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon EC2 instance benchmarking
Number of images: 473 x 42 Mpix (small dataset)
EC2 instance type equivalents Normal resolution reconstruction High-resolution reconstruction
(process time hh:mm:ss) (process time hh:mm:ss)
12 cores 4.00 GHz, RTX A5000, 0:29:52 2:22:13
128 GB – ~g4dn/g5.4xlarge
64 cores 2.70 GHz, RTX 3090, 0:25:59 1:40:35
256 GB – ~g4dn/g5.16xlarge

Number of images: 230 x 102 Mpix, 70 x 440 Mpix (large dataset)

EC2 instance type equivalents Normal resolution reconstruction High resolution reconstruction
(process time hh:mm:ss) (process time hh:mm:ss)
12 cores 4.00 GHz, RTX A5000, 3:00:12 13:53:13
128 GB – ~g4dn/g5.4xlarge
64 cores 2.70 GHz, RTX 3090, 1:32:18 6:08:39
256 GB – ~g4dn/g5.16xlarge
Benchmarks provided by Epic Games Learning
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Architecture for a basic RC pipeline

Parameter Store,
a capability of
AWS Systems Manager
License Key

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Masking images for model exclusion
Initial image Detect vehicle labels RealityCapture
example image bounding boxes depth mask image layer
(DJI1234.jpg) Amazon Rekognition (DJI1234.jpg.mask.jpg)

Sample images, videos, and datasets provided by Epic Games Capturing Reality Case Studies
© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Create and view jobs with a web interface

Example pipeline job stages

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Web platform and AI/ML integrations

Web user interface Image masking pre-processing

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Post-processing step example in RC

Mesh segmentation
Model divider

CLI scripts

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Sample scripts and tutorials by Epic Games Learning
Scalability extensions and other pipeline support

Amazon ECS Pipelines

VAMS pipeline integration

Pipeline container scalability (Visual Asset Management System)

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
RealityCapture cloud architecture review

Parameter
ParameterStore,
Store
a capability
License Keyof
AWS Systems Manager
License Key

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
RealityCapture resource
Need help? Scan for Epic Games
CapturingReality documentation

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Continue your Compute learning
Learn at your Increase your Earn your
own pace knowledge Compute badge

Expand your knowledge Use our Ramp-Up Guides Demonstrate your

with Learning Plans via to build your Compute knowledge by achieving
AWS Skill Builder knowledge digital badges

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Thank you! Please complete the session
survey in the mobile app

Kurt Scheuringer Jerry Tejada

linkedin.com/in/kurt-scheuringer/ linkedin.com/in/jerrytejada/

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.