EAGE Workshop on High Performance Computing for Upstream

In this paper, we present a scheme of Prestack Kirchhoff Time Migration(PKTM) with multi-GPU. Firstly, we intorduced three main optimization points of GPU code of PKTM, then we test the code with a real field data. After analysis the efficiency curve, we proposed a multi-GPU flowchart of PKTM. it first splits the seismic data to different GPU nodes according to the offset range and collecting and sorting the result to CRP gather , the proposed method can reach the maximum efficency of GPU PKTM code.

Computational and workflow limitations restricted the scope of FWI applications, with the method only using very low frequencies and transmitted arrivals. We have now overcome these limitations and can invert the full bandwidth of the data and include the reflected wavefield. The new, full-bandwidth, high-resolution FWI products have broad utility, beyond merely being used as migration velocity models. In particular: (1) subsurface scales not visible in traditional seismic can now be seen in FWI models; (2)the models are used for improved reservoir characterization and 4D seismic interpretation; (3) the method offers an effective approach for handling novel,non-conventional acquisition geometries and reduce cycle time. At the presentation we will show examples demonstrating the methods and benefits of Full-Bandwidth FWI.

We analyze the parallel performance of the Algebraic Multiscale Solver (AMS) for the heterogeneous pressure system that arises from incompressible flow in porous media. We propose special modifications to the algorithm to improve its computational efficiency on massively parallel architectures. AMS is a two-level linear solver algorithm, based on the idea of non-overlapping domain-decomposition with a localization assumption, where the local solutions in each domain are used to construct the coarse operator. The overall scalability of AMS is strongly tied to the choice of parameters and algorithms involved. These choices additionally impact the convergence properties of the solver. We focus on the basis-function kernel, which dominates the setup phase, and on the local smoother, which dominates the solution phase. We carefully consider the balance of computational scalability and convergence rate to ensure high overall performance while maintaining robustness. We present test results for highly heterogeneous problems derived from the SPE10 benchmark, and ranging in size from millions to tens of millions of cells. The parallel AMS code is run two different architectures: a multi-core architecture and a massively parallel Knights Corner architecture. We also compare the performance and robustness of AMS with the widely-used SAMG solver, running on the multi-core architecture.

The paper introduces Key Performance Indices (KPIs) that cover four major service areas of importance to HPC: System Utilization, Network Health and Performance, System Availability, and Jobs Scheduling Efficiency. Development and implantation of the proposed indices are discussed in details. It has been found that the proposed methodologies play an important role in clarifying the overall representation of our systems, allowing us to proactively and optimally administer and manage our HPC resources for improved simulation processes.

We have identified challenges upcoming hardware development is going to impose on RTM implementations. The increasing heterogeneity and complexity of target machines needs to be transparently mapped into the software layer. An efficient fault tolerance mechanism needs to be provided and I/O latencies need to be efficiently hidden. We have introduced a framework for RTM which is able to solve these problems. The framework is data dependency driven on two granularity levels. On the coarse level, concurrent computation of shots is powered by GPI-Space, a parallel development and execution framework. GPI-Space boosts our RTM framework by introducing a fault tolerant execution layer, an efficient topology mapping and an on the fly resource management. On the fine level, the computation of one shot is handled by domain decomposition in a task based model. The tight coupling between neighbouring domains is efficiently relaxed by the one sided asynchronous communication API GPI-2.0. Weak synchronization primitives allow for a fine granular and application specific breakup of data synchronization points with optimal overlap of communication by computation. Our framework has an inherent separation of parallelization and computation. Domain experts concentrate on the implementation of domain knowledge. Computer scientist can simultaneously do the parallelization and optimization.

Algorithmic adaptations are required if anticipated exascale hardware is to be used near its potential for upstream applications, since the existing code base has been engineered to minimize floating point operations. Programmers must now minimize synchronizations, memory usage, and memory transfers, while extra flops on locally cached data are almost “free”. High concurrency requires greater freedom to redistribute data while power-efficient design of the individual cores will likely require greater fault tolerance from algorithms to relieve the hardware. Stencil operation-intensive hyperbolic solvers present different opportunities for improving data locality in different regimes of dimension, number of components, discretization order, stencil structure, coefficient characteristics, and hardware characteristics. Today’s elliptic solvers exploit frequent global synchronizations, ultimately reflecting the global Green’s function for the Laplacian, yet execute few flops to cover these latencies. After decades of algorithm refinement during a period of programming model stability, new programming models and algorithms must be developed simultaneously. In this presentation, we briefly recap the architectural constraints and roadmap, highlight ongoing work at KAUST, and outline future directions.

There a two long standing trends in reservoir simulation, firstly to increase resolution of models and secondly to run multiple realization workflows to better characterize the inherent uncertainty associated with the sub-surface. These need to be balanced when faced with defined computing resources. Moore’s law has continued to revolutionize the computing environment. Recently it has become a requirement that parallelism be exposed at all levels to exploit computer power. Therefore the GPU has become an important tool for HPC applications. This paper demonstrates that models (~1M cells, fully implicit, black oil) can be run effectively on a single GPU taking advantage of its parallel nature and more importantly the high bandwidth to memory relative to a CPU. The second parallelism utilizes a cluster of GPUs to handle multiple realization (MR) workflows, where the MR capability is implemented within the simulator itself and is optimized to take full advantage of the hardware. The “sweet spot” of being able to efficiently run response surface assisted history matching runs and Monte Carlo type analyses as a single batch process holds out the prospect of changing how simulation is used on a day-to-day basis, making it practical to move away from a single deterministic model.

Complex computer systems exhibit many different characteristics that the best parameter choice becomes impossible to define. The range of parameters impacting the performance is too large to be solved by simple trial and error when considering manual tuning techniques, the domain decomposition influence, the compiler capabilities and hardware impact. Then auto-tuning appears now as an elegant solution to optimize source codes before compilation, using different compiler flags or at run time by tuning the input parameters. Starting from the basic implementation of a 3D finite differences kernel, we describe first the methodology to get an estimate of the best performance an algorithm can deliver. To get close to this theoretical achievable performance we present several tuning steps from the basic up to a full intrinsic implementation in order to improve parallelism, vectorization and data locality. Then to get to the best set of parameters, we introduce an auto-tuning methodology based on a genetic algorithm search. We are able to optimize for cache blocking sizes, domain decomposition shapes, prefetching flags or even power consumption, among others. From the un-optimized to the most optimized version, we achieved more than 6x performance improvement on the E5-2697v2 and almost 30x improvement on Xeon Phi.

Reverse Time Migration (RTM) technique is based on many successive solutions of the full wave equation. The use of the Discontinuous Galerkin Method (DGM) as space discretization technique, associated with a leap-frog time scheme, leads to a numerical problem with inherent parallelism, an interesting property to take advantage of, on the current and future, heterogeneous many-cores architectures. Nowadays, the use of supercomputers is mainstream. However, efficiently exploiting the hardware heterogeneity remains a challenging task, with a lack of long term vision regarding the performance portability and the implementation perennially. This highlights the limits of the current programming paradigms. In this context, the use of programming paradigms based on task-graph approaches allows automatic parallelism scheduling over dynamic runtime systems. In this paper, we will present our ongoing effort to introduce a task-based programming model into a industrial elastodynamics simulation code.