A Deep-Learning-Based Geological Parameterization Method

for History Matching

Yimin Liu Wenyue Sun Louis J. Durlofsky

SESAAI Meeting

March 30, 2018

Outlineq Background

q Deep-learning-based neural style transfer

q Convolutional neural network PCA (CNN-PCA) for geological Parameterization

q Parameterization results

q History matching results

q Conclusions and future work

2

History Matching

3

History Matching Problem

𝐚𝐫𝐠𝐦𝐢𝐧𝒎

(

𝟏𝟐 𝒅 𝒎 − 𝒅𝐨𝐛𝐬 𝑻𝐶234(𝒅(𝒎) − 𝒅𝐨𝐛𝐬)

+𝟏𝟐 𝒎− 𝒎8 𝑻𝐶934(𝒎 −𝒎8 )

:

4

q Decision variable: 𝒎 - model parameters

q Challenges:

§ 𝒎 can be high dimensional

§ 𝒎 should preserve geology

q Solution: map 𝒎 onto lower dimensional space

Log permeability

Reparameterization for History Matching

5

q Map 𝒎 to a new variable 𝝃

q Favorable properties:

§ dim 𝝃 ≪ dim(𝒎)

§ 𝝃 is uncorrelated

§ 𝒎@ preserves geological realism

𝒎 ≈ 𝒎@ = 𝒇(𝝃)

𝐚𝐫𝐠𝐦𝐢𝐧𝝃

(

𝟏𝟐 𝒅 𝝃 − 𝒅𝐨𝐛𝐬 𝑻𝐶234(𝒅(𝝃) − 𝒅𝐨𝐛𝐬)

+𝟏𝟐 𝝃 − 𝝃D

𝑻𝐶𝝃34(𝝃 − 𝝃D)

:

q Optimization variable: 𝝃

Principal Component Analysis (PCA)

q PCA: Oliver (1996), Sarma et al. (2006)

Ø Generate 𝑁F realizations using geostatistical algorithm

Ø Perform SVD and reduce dimension

Ø Generate new realization:

6

𝒎𝐩𝐜𝐚 = 𝑼𝒍𝜦𝒍𝟏/𝟐𝝃𝒍 +𝒎8

l << N𝑪 (N𝑪:#ofgridblocks)

𝒀 =𝟏

𝑁F − 𝟏[𝒎𝟏 − 𝒎8 ,𝒎𝟐 − 𝒎8 ,… ,𝒎 _̂ − 𝒎8 ]

𝒀 = 𝑼𝜦𝟏/𝟐𝑽𝑻 ≈ 𝑼𝒍𝜦𝒍𝟏/𝟐𝑽𝒍𝑻

𝝃𝒍~𝑵(𝟎, 𝑰)

PCA Representation for New Realizationsq Works well when 𝒎 follows Gaussian distribution

7

PCA 𝒍 = 70SGEMS N𝑪 = 3600

𝒎𝐩𝐜𝐚 = 𝑼𝒍𝜦𝒍𝟏/𝟐𝝃𝒍 +𝒎8 𝝃𝒍~𝑵(𝟎, 𝑰)

Non

-Gau

ssia

nG

auss

ian

PCA 𝒍 = 70SGEMS N𝑪 = 3600

Optimization-based PCA (O-PCA)q Optimization-based PCA: Vo and Durlofsky (2014, 2015)

Ø Formulate PCA as an optimization problem with regularizationØ Objective: minimize difference to 𝒎𝐩𝐜𝐚 and original histogram Ø Essentially post-process 𝒎𝐩𝐜𝐚 with point-wise mapping

8

O-PCA 𝒍 = 70SGEMS N𝑪 = 3600

𝒎fghi= 𝐚𝐫𝐠𝐦𝐢𝐧𝒙

𝒎ghi(𝝃𝒍) − 𝒙 𝟐𝟐+ γ𝒙𝑻(𝟏 − 𝒙) 𝑥m ∈ [𝑥o, 𝑥p]

(figures from Vo and Durlofsky, 2014)

Limitations of O-PCAq Underlying PCA honors only two-point correlations

q O-PCA point-wise mapping honors single point statistics

q Difficult to preserve multiple point statistics

9

SGeMS Random Real. 1 O-PCA Random Real. 1 O-PCA Random Real. 2

Unconditional Realizations

Neural Style Transfer

10

Content Image

Style Image

Output Image

q Recent work in deep learning and computer visionq Gatys et al. (2016), Johnson et al. (2016)q Enables transfer of photo into artistic style

(images from Johnson: github.com/jcjohnson/fast-neural-style)

Neural Style Transfer

11

q Recent work in deep learning and computer visionq Gatys et al. (2016), Johnson et al. (2016)q Enables transfer of photo into artistic style

Content ImagePCA Model

Style Image - Training Image

Output ImagePost-processed model

Neural Style Transfer Algorithm

12

q O: output image, C: content image, S: style image

q 𝐿hfrstrs 𝑂, 𝐶 : difference between output and content image

q 𝐿vswxt 𝑂, 𝑆 : difference between output and style image

q Output image preservesØ Content (objects) in the content image

Ø Style (color, texture) in the style image

Ø Object and texture are characterized by multipoint statistics

𝑂 = argmin|

{𝐿hfrstrs 𝑂, 𝐶 + 𝛾𝐿vswxt 𝑂, 𝑆 }Content Loss Style Loss

Analogy to O-PCAq Neural style transfer algorithm

q O-PCA

𝑂 = argmin|

𝐿hfrstrs 𝑂,𝐶 + 𝛾𝐿vswxt 𝑂, 𝑆Content Loss Style Loss

13

𝒎fghi= argmin𝒙

𝒎ghi − 𝒙 𝟐𝟐 + 𝛾𝒙𝑻(𝟏 − 𝒙)

Style LossContent Loss

q O-PCA losses based on point-wise difference

q Neural style transfer based on high-level features extracted from deep convolutional neural network (CNN)

Artificial Neural Networkq Nonlinear function 𝑦 = 𝑔(𝑥) with multiple layers of neurons

q Linear function: 𝑊�ℎo34 + 𝑏q Nonlinear activation function 𝑓 � , e.g., ReLU, sigmoid

14

ℎo = 𝑓(𝑊�ℎo34 + 𝑏)

q Convolutional neural networks (CNN):Ø Convolutional layersØ Suitable for image input

Neural Style Transfer Algorithm

15

𝐿h������ 𝑂, 𝐶 =�1

𝑁o𝐷o𝐹o 𝑂 − 𝐹o 𝐶 �

�

o

𝐿vswxt 𝑂, 𝑆 = �1𝑁x� 𝐺o 𝑂 − 𝐺o 𝑆 �

�

o

q Limitations of neural style transfer algorithm: Ø Need to solve optimization onlineØ Derivatives of the output image w.r.t input images not clear

Fast Neural Style Transfer Algorithm

q Train a transform net, Johnson et al. (2016)Ø Hour glass shape deep CNN with same input and output size

q Same loss function, optimize parameters in the transform net

16

Fast Neural Style Transfer Algorithmq Construct PCA with 1000 SGeMS realizationsq Train the model transform net on 3000 random PCA modelsq Training takes 3 minutes on 1 GPU (NVIDIA Telsa K80)q Final step: threshold at 0.5

17

Outlineq Background

q Deep-learning-based neural style transfer

q Convolutional neural network PCA (CNN-PCA) for geological parameterization

q Parameterization results

q History matching results

q Conclusions and future work

18

Unconditional Binary System

19

q Binary facies model:

q Training image size: 250 x 250

q Model size: 60 x 60, 𝑁� = 3600

q No hard data

q Goal: low-dimensional representation

q Reduced dimension: 𝑙 = 70

Training image

One model realization

Unconditional Binary System

20

PCA Real. 1

PCA Real. 2

O-PCA Real. 1

O-PCA Real. 2 CNN-PCA Real. 2

CNN-PCA Real. 1

Conditional Binary System

21

q Binary facies model

q Training image size: 250 x 250

q Model size: 60 x 60, 𝑁� = 3600

q Hard data at 16 well locations

q Reduced dimension: 𝑙 = 70

q 200 new random realizations

q Additional hard-data loss

Training image

One model realization

argmin|

{𝐿hfrstrs 𝑂, 𝐶 + 𝛾4𝐿vswxt 𝑂,𝑆

+𝛾�𝐿�isi(𝑂)}

Conditional Binary System

PCA Real. 1

PCA Real. 2

O-PCA Real. 1

O-PCA Real. 2

CNN-PCA Real. 1

CNN-PCA Real. 2

22

q All 200 CNN-PCA models match all hard data

Outlineq Background

q Deep-learning-based neural style transfer

q Convolutional neural network PCA (CNN-PCA) for geological parameterization

q Parameterization results

q History matching results

q Conclusions and future work

23

History Match Conditional Modelq Oil-water, 60 x 60 grid

q 2 injectors, 2 producers, BHP controlled

q ksand = 2000 md, kmud = 0.02 md

q Data: production and injection rates for 1000 days every 100days

q Number of data: 𝑁� = 80

q Goal: 30 RML posterior models

q Optimizer: PSO-MADS

24

Injector Producer

Permeability Estimation

25

True model

One O-PCA prior model O-PCA posterior model

One CNN-PCA prior model CNN-PCA posterior model

PROD-1 Water Rate

26

O-PCA Prior models O-PCA Posterior Models

CNN-PCA Prior models CNN-PCA Posterior Models

Infill Well Prediction

infill well locations

q Two infill wells P3, P4q Drilled at 1000 daysq Prediction to 2000 days

27

Permeability Estimation

28

True model

O-PCA #1 O-PCA #2

CNN-PCA #1 CNN-PCA #2

O-PCA #3

CNN-PCA #3

Infill Well PROD-4 Prediction

29

O-PCA Water Rate CNN-PCA Water Rate

24 curves 13 curves

Field Prediction

30

O-PCA Oil Rate CNN-PCA Oil Rate

15 curves 6 curves

Conclusions

q Developed CNN-PCA by combining deep-learning-based neural style transfer algorithm with PCA

q CNN-PCA better preserves channel geometry compared to O-PCA

q CNN-PCA history matching solutions provide more accurate prediction for infill wells

31

Future Work

32

q Extend CNN-PCA to bimodal, three-facies and three dimensional reservoir models

q Apply CNN-PCA with gradient-based history matching

q Implement CNN-PCA treatment with ensemble-based history matching methods

Acknowledgmentsq Stanford CS 231N course

Ø Instructors: Justin Johnson, Serena Yeung, Fei-fei LiØ Course project teammate: Yuanlin Wen (Google)

q Open source code:Ø O-PCA: Hai Xuan Vo (Chevron)Ø Neural style transfer (PyTorch): Hang ZhangØ Fast neural style transfer (PyTorch): Abhishek Kadian

q PSO-MADS: Obi Isebor

q Stanford CEES

33

34

Thank You!Q & A

Backup Slides

35

Convolutional Layersq Each neuron connects to a local region in previous layerq Convolution: No filters 𝑤o sliding through previous layersq Convolutional layer 𝑁o channels:

Ø Each channel is called a feature map of 𝐷o neuronsØ All channels form a feature matrix 𝐹o ∈ 𝑅^�× �

36

Convolutional Neural Networkq Neural network consists of mainly convolutional layersq Nonlinear layers:

Ø Activation layer, e.g., ReLU 𝑓 𝑥 = max(0,𝑥)Ø Pooling layer: down sampling to reduce dimension

q Image classification:Ø Input: image of size 𝐻×𝑊×3Ø Output: score for predefined image classes

37

VGG-16 Deep CNN (Simonyan and Zisserman, 2015)

Feature Matrix and Gram Matrix

38

q Content representation: feature matrix 𝐹o ∈ 𝑅^�× �

q Style representation: Gram matrix 𝐺o = 𝐹o𝐹o�/(𝑁o𝐷o)�

* Images modified from Gatys et al. (2016)

CNN-PCA for Reparameterizationq Construct PCA with 𝑁F SGeMS realizations

q Post-process 𝒎𝐩𝐜𝐚 with fast neural style transfer algorithm

39

𝒎𝐩𝐜𝐚 = 𝑼𝒍𝜦𝒍𝟏/𝟐𝝃𝒍 +𝒎8 𝝃𝒍~𝑵(𝟎, 𝑰)