Second EAGE Workshop on High Performance Computing for Upstream

Reservoir simulation of large scale projects is becoming increasingly complex, requiring more than simple black oil models and vertical well models to capture the behaviour of unconventional, fractured and highly heterogeneous production zones. Nvidia provides an array of accelerated linear algebra libraries to deal with the equations that must be solved in these situations. Accelerating sparse linear algebra on the latest GPU architectures has real potential for performance gains of hundreds of percent over carefully tuned multi-core CPU-only implementations, but at what cost in complexity? This talk will address the programming approaches needed to utilize GPUs at scale for today’s most challenging problems, and give a glimpse of the path forward to pre-exascale applications.

Today’s high-end multicore systems are characterized by a deep memory hierarchy, i.e., several levels of local and shared caches, with limited size and bandwidth per core. The ever-increasing gap between the processor and memory speed will further exacerbate the problem and has lead the scientific community to revisit numerical software implementations to better suit the underlying memory subsystem for performance (data reuse) as well as energy efficiency (data locality). The authors propose a novel multithreaded wavefront diamond blocking (MWD) implementation in the context of stencil computations, which represents the core operation for seismic imaging in oil industry. The stencil diamond formulation introduces temporal blocking for high data reuse in the upper cache levels. The wavefront optimization technique ensures data locality by allowing multiple threads to share common adjacent point stencil. Therefore, MWD is able to take up the aforementioned challenges by alleviating the cache size limitation and releasing pressure from the memory bandwidth. Performance comparisons are shown against the optimized 25-point stencil standard seismic imaging scheme using spatial and temporal blocking and demonstrate the effectiveness of MWD.

In this paper, we will discuss how to achieve the optimal price/performance and the optimal price/energy values in a complex compute environment like reservoir simulation. We will also discuss our experimentation with different CPU models and the impact of our selection on the users’ experience and data center electric power consumption using thevaried CPU models of the same generation with different number of cores and different clock speeds. For example, Intel® Ivy Bridge CPU have different SKUs with different power consumption, number of cores, threads, clock speeds, etc. Buying the highest clock speed with the highest number of cores may not be the optimal choice for a large system in a computer center. Some applications will not scale well as the number of cores increases keeping the same number of memory channels. Some applications also will not benefit from high clock speed.

Wave field simulation are needed in many algorithms in seismic data processing. Reverse time migration and full waveform inversion is a couple of this kind of algorithms. Simulation of wave propagation in industrial scale requires huge computational cost. Therefore, an extreme computational cost is the main challenge of reverse time migration and full waveform inversion. This extreme computational cost leads to these algorithms becomes less applicable in industrial scale.

This paper presents the reduced order modeling technique as a very efficient method for reduction of computation in the simulation of wave propagation. We consider the efficiency of this algorithm in the computational cost of the reverse time migration. A small part of 2D SEG/EAGE synthetic model is used for this purpose, and the performance of reduced order modeling technique is compared with application of the conventional finite element algorithm.

The obtained results demonstrate the capability, and accuracy of the reduced order modeling technique for simulation of wave propagation for some cases like reverse time migration that needs many simulations. The efficiency of this method is increased while the number of simulations or time of the simulations is increased.

Full Waveform Inversion is a high-resolution imaging method that has raised considerable interest in the oil industry since a decade. It has been mainly used as a P-wave velocity modeling building tool, while extension to multi-parameter elastic anisotropic reconstruction is now an active field of research. In this context, designing computationally-efficient and versatile FWI softwares, in which different numerical schemes for seismic modeling, wave physics and optimization algorithms can be interfaced easily, is of crucial interest. In this study we describe an object oriented framework based on the definition of abstract interfaces of components involved in frequency or time domain FWI workflow. We point out the numerical modeling of the full seismic wavefield as the central component and that the proposed design is suitable to interface different modeling approaches involving different discretisations and physics with the optimization kernel. Then we demonstrate the capability of the framework to preserve parallel scalability and efficiency of kernels even in an object oriented programming context. Lastly we present a concrete realisation of this abstract framework via an application of an acoustic 3D time-domain FWI on the Valhall field using a staggered grid finite difference scheme.

Sparse matrix-vector multiplication (spMV) is a fundamental building block of iterative solvers in many scientific applications. spMV is known to perform poorly in modern processors due to excessive pressure over the memory system, overhead of irregular memory accesses and load imbalance due to non-uniform matrix structures. Achieving higher performance requires taking advantage of the features of the matrix and choosing the right sparse storage format to better exploit the target architecture. In this paper we describe an efficient spMV for geophysical electromagnetic simulations on Intel Xeon Phi coprocessors. The unique features of the matrix resulting from electromagnetic problems make it hard to handle with classical sparse storage formats. We propose a matrix decomposition and a tuned storage format that obtains a 4.13x performance improvement over the optimized CSR spMV kernel on Xeon Phi coprocessors.

The current trend in seismic imaging aims at using an improved physical model, considering that the Earth is not rigid but an elastic body. This new model takes simulations closer to the real physics of the problem, at the cost of raising the needed computational resources. On the hardware front, recently developed high-performing devices, called accelerators or co-processors, have shown that can outperform their general purpose counterparts by orders of magnitude in terms of performance per watt. These new alternatives may then provide the necessary resources for making possible to represent complex wave physics in a reasonable time. There might be, however, a penalty associated to the usage of such devices, as some portion of the simulation code might need some re-writing or new optimization strategies explored and applied. In this work we will show some optimization strategies evaluated and applied to an elastic propagator based on a Fully Staggered Grid, running on the Intel® Xeon Phi(TM) coprocessor. It is important to remark, that the propagator is able to reproduce elastic wave propagation, even for an arbitrary anisotropy.

Leveraging petascale systems effectively and efficiently is not a trivial challenge. A good harmony between system architecture, algorithms, and application architecture is the key to success. In addition, when complex algorithms build up a workflow it becomes ever more challenging to get optimal execution. Successful business application requires not only the innovation in algorithm space but also efficient execution leveraging HPC environment. Positioning these complex technologies in a robust, scalable commercial usable form makes the business successful.

Here is a case study where different technologies are integrated in a software framework leveraging HPC environment. Also, leveraging a HPC system introduces additional complexities of achieving high level of parallelism. In the HPC group, as we started commercializing these technologies, we added them to a software framework to reduce end users complexities, improve technology uptake and effective HPC resource utilization.

Full waveform inversion (FWI) is an increasingly popular algorithm for automatically improving earth models in seismic exploration. The method is incredibly computationally expensive because the earth model is built up with many iterations of forward modelling and reverse time migration (RTM). Current research is helping to reduce the number of iterations and improve the robustness of convergence, but the prohibitive cost of FWI makes running real world datasets impractical for many researchers. Furthermore, information regarding the geological region can be used to accelerate convergence. Therefore, an FWI implementation must be both high performance, and flexible. This commercial case study outlines our approach to minimizing the cost of a flexible implementation.

The key feature of our approach is that we divide the algorithm into low cost and high cost steps. Fortunately, the low cost steps in the FWI algorithm are also the ones subject to the most research. The extremely high cost full wave modelling steps are comparatively consistent between variations of the FWI algorithm. Therefore, the full wave modelling is hardware optimized and all other steps can be quickly rewritten in Python.