Scalability and Load Balancing Problems in Parallel Reservoir Simulation

Scalability and Load Balancing Problems in Parallel Reservoir Simulation <br><br>New parallel reservoir simulator software designed for Linux clusters enable to overcome hardware limitation and to simulate models with large amount of data. Reservoir Engineering industry is very interested in using ever growing dataset with more and more complex physics and detailed models. The key issue still remains running simulations in an acceptable CPU time. As, the trend in hardware technologies is not to improve drastically the performance of individual CPUs but to facilitate the aggregation of computation facilities (with high bandwidth network, multi-core architectures, ...), the challenge is to improve the efficiency of reservoir simulation software on a large number of processors.<br><br>New numerical difficulties and performance problems appear when the number of cells and the number of processors are growing. As a matter of fact, the architecture of Linux clusters is very sensible to memory distribution and load balancing:<br><br>- the cost of parallel solver algorithm is usually sensible to the size of the reservoir model (lack of scalability) and the consequences on CPU performance can no more be neglected;<br>- the domain decomposition algorithms used to distribute data between processors have a great influence on the computing load balancing between processors;<br>- using adaptive numerical schemes with dynamic space criteria (AIM schemes, flash algorithms based on the thermodynamical state of each cell) is a source of unbalance that cannot statically be resolved;<br>- simulation result storages on irregular data structures, such as unstructured grids, multilateral smart wells and perforated cells, lead to store an important amount of information during the simulation. With the variety of IO subsystems found on Linux clusters the simulator must be able to adapt its IO strategy to the underlaying IO library/file system and hardware.<br><br>In this paper, we present different approaches to overcome these kinds of problems. We discuss technical choices such like :<br>- advanced scalable linear solver algorithm ;<br>- load balancing issue with different domain decomposition strategies ;<br>- mesh partitioner strategy and parallel solver performance management;<br>- flexible IO strategy from simple file system to more complex parallel file system or database.<br><br>We have developed and benchmarked these different solutions on published reference large scale problems and actual case studies with several tens millions of cells. We analyze the results and discuss the efficiency of each solution to overcome the scalability difficulties and performance limitations due to load unbalance.