1887
  1. Home
  2. A-Z Publications
  3. First Break
  4. Volume 43, Issue 2
  5. Article
Volume 43, Issue 2
  • ISSN: 0263-5046
  • E-ISSN: 1365-2397

Abstract

Abstract

Traditional workflows using machine learning interpretation of seismic data rely on iterative training and inference on single datasets, producing models that fail to generalise beyond their training domain. Self-supervised training and scaling of 3D vision transformer (ViT) architectures enables seismic interpretation with improved generalisation across diverse datasets. We address the complexities of large-scale training on a global dataset of 63 seismic volumes using the masked autoencoder (MAE) architecture with the ViT-H model consisting of 660 million parameters. We leverage a cloud-native, digitalised seismic data infrastructure to address the data engineering challenges, avoiding duplication. For a downstream task, a salt segmentation model trained using interpretation labels from the Gulf of Mexico and Brazil demonstrated zero-shot generalisation on a West African survey. These findings underscore the potential of pre-trained foundation models to overcome the limitations of iterative approaches and extend seismic interpretation across diverse basins, marking a significant advancement in scalable machine learning for subsurface challenges.

EAGE Publications BV
Loading

Article metrics loading...

/content/journals/10.3997/1365-2397.fb2025016
2025-02-01
2025-02-19

  • From This Site

    /content/journals/10.3997/1365-2397.fb2025016
    dcterms_title,dcterms_subject,pub_keyword
    -contentType:Journal -contentType:Contributor -contentType:Concept -contentType:Institution
    10
    5
Loading full text...

Full text loading...

References

  1. He, K., Chen, X., Xie, S., Li, Y., Dollár, P. and Girshick, R. [2021]. Masked Autoencoders Are Scalable Vision Learners. ArXiv. https://arxiv.org/abs/2111.06377.
     [Google Scholar]
  2. He, K., Chen, X., Xie, S., Li, Y., Dollár, P., and Girshick, R. [2021]. Masked Autoencoders Are Scalable Vision Learners. ArXiv. https://arxiv.org/abs/2111.06377.
     [Google Scholar]
  3. Zhai, X., Kolesnikov, A., Houlsby, N. and Beyer, L. [2021]. Scaling Vision Transformers. ArXiv. https://arxiv.org/abs/2106.04560.
     [Google Scholar]
  4. Dehghani, M., Djolonga, J., Mustafa, B., Padlewski, P., Heek, J., Gilmer, J., Steiner, A., Caron, M., Geirhos, R., Alabdulmohsin, I., Jenatton, R., Beyer, L., Tschannen, M., Arnab, A., Wang, X., Riquelme, C., Minderer, M., Puigcerver, J., Evci, U. and Houlsby, N. [2023]. Scaling Vision Transformers to 22 Billion Parameters. ArXiv. https://arxiv.org/abs/2302.05442.
     [Google Scholar]
  5. Sheng, H., Wu, X., Si, X., Li, J., Zhang, S. and Duan, X. [2023]. Seismic Foundation Model (SFM): A new generation deep learning model in geophysics. ArXiv. https://arxiv.org/abs/2309.02791.
     [Google Scholar]
  6. Ordonez, A., Wade, D., Ravaut, C. and Waldeland, A.U. [2024]. Towards a Foundation Model for Seismic Interpretation. 85th EAGE Annual Conference & Exhibition, 2024, 1–5. DOI: https://doi.org/10.3997/2214–4609.2024101119.
     [Google Scholar]
  7. Roberts, M., Warren, C., Lasscock, B. and Valenciano, A. [2024]. A Comparative Study of the Application of 2D and 3D CNNs for Salt Segmentation. 85th EAGE Annual Conference & Exhibition, 1–5.
     [Google Scholar]
  8. Warren, C., Kainkaryam, S., Lasscock, B., Sansal, A., Govindarajan, S. and Valenciano, A. [2023]. Toward generalized models for machine-learning-assisted salt interpretation in the Gulf of Mexico. The Leading Edge, 42(6), 390–398.
     [Google Scholar]
  9. Lasscock, B.G., Sansal, A. and Valenciano, A. [2024, September 17–20]. Encoding the Subsurface in 3D with Seismic [Paper presentation]. IMAGE 2024, Houston, TX, United States.
     [Google Scholar]
  10. Sansal, A., Kainkaryam, S., Lasscock, B. and Valenciano, A.2023. MDIO: Open-source format for multidimensional energy data. The Leading Edge, 42(7), 465–470. https://doi.org/10.1190/tle42070465.1.
     [Google Scholar]
  11. Dask Development Team [2016]. Dask: Library for Dynamic Task Scheduling. Available at: http://dask.pydata.org [Accessed: 9 December 2024].
     [Google Scholar]
  12. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J. and Chintala, S. [2019]. PyTorch: An Imperative Style, High-Performance Deep Learning Library. Advances in Neural Information Processing Systems, 32, 8024–8035.
     [Google Scholar]
/content/journals/10.3997/1365-2397.fb2025016
Loading
Scaling Seismic Foundation Models
First Break 43, 69 (2025); https://doi.org/10.3997/1365-2397.fb2025016
/content/journals/10.3997/1365-2397.fb2025016
/content/journals/10.3997/1365-2397.fb2025016
Loading

Data & Media loading...

  • Article Type: Research Article

Most Cited This Month Most Cited RSS feed

This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error