Exploration seismology and the return of the...
Transcript of Exploration seismology and the return of the...
![Page 1: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/1.jpg)
Exploration seismology and the return of the
supercomputer
Exploring scalability for speed in development and delivery
Sverre Brandsberg-Dahl
Chief Geophysicist, Imaging and Engineering
![Page 2: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/2.jpg)
Marine seismic acquisition
![Page 3: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/3.jpg)
Seismic imaging, this is where HPC enters the scene…
![Page 4: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/4.jpg)
Seismic imaging: Create an image of the subsurface from surface
measurements
data(s,r, t) ®vel
image(x, y, z)
![Page 5: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/5.jpg)
How exploration seismology has been carried out for decades…
tim
e
Distance (receiver channel)
10 sec
Reflections (echos) from the subsurface
Firing a source and recording the
reflection from the sub-surface:
• This forms a seismic shot record
• Treating each shot as an individual
wave equation realization lead to the
“embarrassingly parallel regime” of
seismic imaging
![Page 6: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/6.jpg)
So how do we image, we solve the wave equation
It is used (in approximate ways) to model seismic waves as they
propagate in the earth
¶2
¶xi2U(xi, t) =
1
v2
¶2
¶t2U(xi, t)
![Page 7: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/7.jpg)
Some issues we face when imaging seismic data
• The seismic waves propagate through an ‘unknown’ medium, we can only measure on the (sea) surface
• We need to recover the medium properties from the data itself
• The subsurface can have large velocity and density contrasts, making the above a very complex task…
• Exponential growth in compute cost as function of frequency
– wave equation has 3 spatial dimension and 1 temporal, so solution schemes scale as f4
![Page 8: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/8.jpg)
Seismic image (migration) compared to reflectivity
A typical RTM image
The true reflectivity
Image formed by applying adjoint of modeling
operator.
No correction for data aperture, (incomplete)
data geometry or illumination.
![Page 9: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/9.jpg)
Why PGS did computers a bit differently
![Page 10: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/10.jpg)
In a good position to be different
• Relatively small market share in data processing, but growing
• Focused company: marine streamer seismic
• In-house software platform used only by PGS
• “Sticking to the plan”, business and technology strategy aligned and in place since 2009
This allowed us to do things
differently, and not copy what
was current standards or
adopted trends in the industry
![Page 11: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/11.jpg)
PGS Imaging
• To deliver the highest quality images of the subsurface in the shortest possible timeVision
• To become the market leader in high-end imaging
• To achieve a market share similar to that of PGS marine acquisition
Mission
• Improve quality and consistency of services
• Grow behind our fleet
• Increase productivity (more automated processes)
• Develop new interaction with customers
Strategy
![Page 12: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/12.jpg)
Guiding principles for PGS Imaging
• Acquire “complete data”: spatial sampling and bandwidth
• 3D operators throughout the processing flow, from wavefield separation through de-multiple
• Use the complete data in VMB and imaging: reflections, refractions and multiply-scattered waves
• Move towards full wavefield, and full-survey processing and imaging (ensemble-based processing)
• Leverage angle-domain image space for analysis, residual corrections and conditioning
• Inversion algorithms in processing and imaging
![Page 13: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/13.jpg)
-
1,000,000
2,000,000
3,000,000
4,000,000
5,000,000
6,000,000
7,000,000
8,000,000
9,000,000
10,000,000
11,000,000
12,000,000
13,000,000
2010 2011 2012 2013 2014 2015 YE
Compute Consumption
Late to the game, catching up to the data explosion…
![Page 14: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/14.jpg)
But also facing up to new realities…
![Page 15: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/15.jpg)
-
1,000,000
2,000,000
3,000,000
4,000,000
5,000,000
6,000,000
7,000,000
8,000,000
9,000,000
10,000,000
11,000,000
12,000,000
13,000,000
2010 2011 2012 2013 2014 2015 YE
Compute Consumption
Relying on Linux clusters, but not at scale
![Page 16: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/16.jpg)
-
1,000,000
2,000,000
3,000,000
4,000,000
5,000,000
6,000,000
7,000,000
8,000,000
9,000,000
10,000,000
11,000,000
12,000,000
13,000,000
2010 2011 2012 2013 2014 2015 YE
Compute Consumption
Missed the wave on GPU clusters…
![Page 17: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/17.jpg)
-
1,000,000
2,000,000
3,000,000
4,000,000
5,000,000
6,000,000
7,000,000
8,000,000
9,000,000
10,000,000
11,000,000
12,000,000
13,000,000
2010 2011 2012 2013 2014 2015 YE
Compute Consumption
Rather than trying to implement what others were already doing…
Needed to look for a radically different solution that is:
1. Cost effective (price performance) for current applications
2. Fitting the needs of our technology pipeline
3. Enabling implementation (at scale) of new algorithms
4. Helping reduces development time
![Page 18: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/18.jpg)
-
1,000,000
2,000,000
3,000,000
4,000,000
5,000,000
6,000,000
7,000,000
8,000,000
9,000,000
10,000,000
11,000,000
12,000,000
13,000,000
2010 2011 2012 2013 2014 2015 YE
Compute Consumption
Small XC30 system for R&D
![Page 19: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/19.jpg)
-
1,000,000
2,000,000
3,000,000
4,000,000
5,000,000
6,000,000
7,000,000
8,000,000
9,000,000
10,000,000
11,000,000
12,000,000
13,000,000
2010 2011 2012 2013 2014 2015 YE
Compute Consumption
Moving to the next level; spurred by growth in business- and data-volumes,
and readiness in the organization
![Page 20: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/20.jpg)
Grow behind our fleet, and keep up with a growing
data flow
![Page 21: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/21.jpg)
The big data challenge… in the seismic business
1995
6 streamers
2005
16 streamers
2015
24 streamers
![Page 22: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/22.jpg)
Titan Class vessel
![Page 23: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/23.jpg)
Seismic acquisition data volumes
• A typical streamer is 8000 meters long and contains 1280 receivers
• Data is recorded in time chunks or as continuous series, 2ms sample interval,
generating 500 samples per second per receiver
• A streamer (single sensor) generates 640,000 samples per second
• One streamer spread (10 streamers) generates 6,400,000 samples per second
• Big spread, 20 streamers, dual sensor will generates 25,600,000 samples per second
• Typical acquisition can generate multi-TBs of data per day
![Page 24: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/24.jpg)
Outline
• Why PGS did computers a bit differently
• Grow behind our fleet, and keep up with a growing data flow
• Triton, imaging a 660TB dataset using distributed memory and fast interconnect
• New geophysical science on a Cray XC40 supercomputer
– WEI: Wave Equation Inversion
– Deconvolution imaging condition
• Future challenges; software, people and machines
![Page 25: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/25.jpg)
Triton, imaging a 660TB dataset using distributed
memory and fast interconnect
![Page 26: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/26.jpg)
Triton acquisition
281 sail lines over 3
azimuths
2.6 million CMP
km acquired
enough to go to the
moon…
…and back
almost 3.5 times
![Page 27: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/27.jpg)
Triton pushes us to the next level
Reverse Time Migration (RTM) 16km aperture, 16km depth
6 Azimuths, 46 Angles
RTM Problem size approaches 200 GB per shot
Hardware requirement -- 6000 nodes with 256GB memory.
… while waiting for the next bigger survey that would push
us over 256GB of memory.
![Page 28: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/28.jpg)
So it was decided to do something radically different
• Move RTM to a massively-parallel computer
with distributed memory
• In the process the “embarrassingly parallel
paradigm” was busted
• RTM with angle/azimuth gather output was
implemented, and Triton production imaging
was run on a Cray XC40 system
![Page 29: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/29.jpg)
PGS Abel supercomputer
This is a key technology differentiator for PGS
• Will help with delivery of current projects
• Will enable development of new imaging technology
• 24 cabinet Cray XC40
• All CPU
• Almost 600TB of memory
• One-step all to all communication
![Page 30: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/30.jpg)
Implementing RTM on the Cray
0
1000
2000
3000
Commodity /scratch 3d ffts Transposes
• Our Cray has a fantastic network, but nodes have no scratch disk.
• Legacy RTM implementation uses lots of /scratch to store wavefield snapshots
• RTM with pseudo-spectral extrapolator (FFT-based)
Each node writing snapshots to a shared
lustre /scratch disk created a large problem.
![Page 31: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/31.jpg)
Implementing RTM on the Cray
0
1000
2000
3000
Commodity /scratch 3d ffts Transposes
Parallelizing one shot over multiple nodes and
storing snapshots in memory is better, but
distributed memory 3d FFTs slow things down.
![Page 32: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/32.jpg)
Implementing RTM on the Cray
0
1000
2000
3000
Commodity /scratch 3d ffts Transposes
Reorganize math to reuse transposes and work
two dimensions at a time. Total aggregate
runtime is faster than single node
• Each shot migrated using tens of compute nodes: no more limits on frequency and aperture
• Transcending the standard RTM implementations: densely sampled gathers, scalable
• Improved runtime over commodity: one node – one shot
![Page 33: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/33.jpg)
Summed over the full output space
of 11,847km2 (red polygon)…
…6 azimuths x 11,847km2 = 71,082km2
46 angles x 71,082km2 = 3,269,772km2
with a trace spacing of 30m x 30m…
RTM imaging – fun facts
…or for each surface location there is 276
angle traces i.e. images
Or stated differently: a 3D image with a
pixel size of 30x30x10 meters, that
covers an area about 5 times the size of
Texas, to a depth of 16km.
![Page 34: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/34.jpg)
So why? The sub-salt imaging challenge
![Page 35: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/35.jpg)
Leveraging a supercomputer in 3D image optimization
• When imaging beneath complex geology, it is common
that the signal of interest is confined to just a few partial
images (angle-range and azimuth).
• Selective stacking optimizes the signal by combining the
gathers that best illuminate the target while discarding
noisy parts that would deteriorate the final image.
• The aim is to improve the Signal/Noise in the final image
Azimuth Sectored
Angle gathersThe 5D output data volume is about 33TB, we now want to analyze
and edit this to improve the image further i.e. post processing, and
generate a final 3D image
![Page 36: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/36.jpg)
276 angle traces in a 5D hypercube giving a different view of the geology
00
Azimuth Sectored
Angle gathersButterfly Angle gathers
Angle
+ + 00
Angle
+ + 00
Angle
+ +
Az1
![Page 37: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/37.jpg)
Azimuth sectored stack sections
AZ 5 AZ 2
Az1
![Page 38: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/38.jpg)
Angle domain is great for combining data (5D binning)
• Utilizing full azimuth
RMO picks from near
and far offsets.
• Enhanced illumination
from offsets up to
16km.
0o60o 30o
AZ0
AZ60 AZ120Near offsets only, 0 –
8km
Near + far offsets, 0 –
16km
7500
14000
7500
14000
![Page 39: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/39.jpg)
A different perspective on the RTM computational domain
24km
Images are formed for
individual supershots, each
shot is processed inside a
computation box with a
surface area of 576km2.
The survey contains
>100,000 supershots
On the other hand, only 18
such computational boxes
are needed to cover the
complete survey area
![Page 40: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/40.jpg)
…and the size of a single image pixel
30 m
So more than 100,000
images were in reality
formed, each with a
computational domain of
24x24x16km, and with a
cell size of 30x30x10m.
Future: treat the full survey
area (100km x 100km) as
as one domain, rather than
100,000 separate
(overlapping) experiments
![Page 41: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/41.jpg)
Outline
• Why PGS did computers a bit differently
• Grow behind our fleet, and keep up with a growing data flow
• Triton, imaging a 660TB dataset using distributed memory and fast interconnect
• New geophysical science on a Cray XC40 supercomputer
– WEI: Wave Equation Inversion
• Future challenges; software, people and machines
![Page 42: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/42.jpg)
New geophysical science on a Cray XC40
supercomputer
![Page 43: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/43.jpg)
RTM imaging conditions: leveraging shared memory and fast interconnect
to implement new geophysics
• Move away from the classic cross-correlation imaging condition
• Better image conditions = more data available during imaging step
• Using the full wavefield in imaging, not just a synthesized source (snap shots), including very long,
continuous records with energy from multiple sources
• This field is in constant development, it is key to move fast from concept to implementation
![Page 44: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/44.jpg)
Classic acquisition scheme – individual shots
Shot N-1
Shot N
Shot N+1
This data structure lends itself to
(embarrassingly) parallel processing on
clusters, one shot per machine,
then accumulate individual images to
form final image. Data reduction.
![Page 45: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/45.jpg)
Emerging – continuous shooting and recording, irregular sampling
Shot N-1
Shot N
Shot N+1
Shot N-2This data structure does not lend itself to
individual shot processing.
New approach, massive data volumes,
very large linear systems
Map data into image space, no/little data
reduction, treat survey as one ensamble
![Page 46: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/46.jpg)
recorded seismic data
modeling operator
reflectivity
Migration = Imaging with an adjoint operator
dobs =Lm
dobs
L
m
![Page 47: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/47.jpg)
recorded seismic data
migration operator
migrated image
Migration (RTM) = Imaging with an adjoint operator
mmig = ¢Ldobs
dobs
¢L
mmig
![Page 48: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/48.jpg)
Least Squares Imaging - Wave Equation Inversion (WEI)
d =Lm
m̂= ¢LL( )-1
¢Ldobs
Massive matrix, dimension is Data x Data
![Page 49: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/49.jpg)
WEI: fast solvers for sparse linear systems
Ax = b
Am̂= ¢LL( )
-1¢Ldobs
![Page 50: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/50.jpg)
WEI: fast solvers for sparse linear systems
AThe size of A is 3TB for the example
that follows.
For the full Triton survey it is >400TB .
![Page 51: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/51.jpg)
WEI: fast solvers for sparse linear systems
Ax = b
A
Need to solve an sparse linear system
with an scalable (parallel) solver
A fast, low-latency interconnect is needed
to obtain good performance
![Page 52: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/52.jpg)
WEM
52
![Page 53: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/53.jpg)
WEI
53
![Page 54: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/54.jpg)
WEI
54
WEM
WEI
![Page 55: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/55.jpg)
Depth Slices WEM (35Hz)
1200m 1500m
![Page 56: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/56.jpg)
Depth Slices WEI (35Hz)
1200m 1500m
![Page 57: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/57.jpg)
Illumination below salt and Wave Equation Inversion
WEI applied in a small target area of 375 km2z INLINE XLINE
3 km 25 km 15 km
![Page 58: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/58.jpg)
Amplitudes extracted along target horizon
Migration result WEI result
![Page 59: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/59.jpg)
Some thoughts on what’s to come
• Relentless focus on turnaround and cost
– price-/performance is key, will pursue anything that provides improvements
– Balanced against speed of development: how long to bring new physics to market
• Bigger problem size
– More and more data, multi-component, continuous recording, density of sensors
– Non-uniform data: data with less inherit parallelism
– Inversion… very large linear systems
• Rapidly move from concept to large-scale deployment: “From Matlab to 200,000 cores in a week...”
– Small nimble teams, stable hardware
– Programming model is a challenge
![Page 60: Exploration seismology and the return of the …hpcuserforum.com/presentations/austin2016/PGS.pdf · How exploration seismology has been carried out for decades… me Distance (receiver](https://reader033.fdocuments.in/reader033/viewer/2022051801/5add0f317f8b9a9d4d8ca3ae/html5/thumbnails/60.jpg)
Conclusions
• How our size and strategy for growth pushed us to look beyond typical solutions used by peers
• The challenge of adapting new HPC hardware while running a small margin service business
– Organizational capabilities: commodity vs. integrated solution
– RTM ported to Cray XC40
• New algorithms brought to market thanks to
– Large memory pool
– Fast interconnect
– Acceptable programming model and “uniform” hardware
• Ensemble based processing, moving from single trace to full survey, then be able to dynamically focus on the parts that are of interest (anomalies)
– Challenges the software environment and programming model
– Will pose a challenge also for hardware