1887
Volume 31, Issue 4
  • ISSN: 1354-0793
  • E-ISSN:

Abstract

Biased forecasts lead to biased decisions. An essential pre-drill input in petroleum exploration is the geological probability of success (PoS). If the PoS is biased and lacks accuracy and reliability, the result is value erosion. We used the Brier score, skill score, bias and attribute diagram to assess the quality of PoS forecasts for prospects on the Norwegian Continental Shelf (NCS) from 1990 to 2022. Pre-drill and post-drill information about the prospects have been reported to the Norwegian Offshore Directory. As the reported PoS was obtained by multiplying the probability of source, reservoir and trap, verification measures were also applied to these three factors. Overall, NCS forecasts tend to exhibit pessimism and overconfidence, with some improvement in bias over time. The trap forecasts consistently exhibit a negative skill score over time, indicating a performance no better than the standard reference class. Furthermore, PoS forecasts are not reliable. Biased forecasts indicate that forecasters need to revise their judgements, while negative skill scores imply that a forecast may not add value. Poor reliability indicates that prospects estimated to have a high PoS may not be successful, and vice versa. These shortcomings can lead explorers to prioritize non-viable prospects while missing the more promising ones.

© 2025 The Author(s). Published by The Geological Society of London for GSL and EAGE. All rights, including for text and data mining (TDM), artificial intelligence (AI) training, and similar technologies, are reserved. For permissions: https://www.lyellcollection.org/publishing-hub/permissions-policy. Publishing disclaimer: https://www.lyellcollection.org/publishing-hub/publishing-ethics

Loading

Article metrics loading...

/content/journals/10.1144/petgeo2025-014
2025-10-21
2026-01-18

  • From This Site

    /content/journals/10.1144/petgeo2025-014
    dcterms_title,dcterms_subject,pub_keyword
    -contentType:Journal -contentType:Contributor -contentType:Concept -contentType:Institution
    10
    5
Loading full text...

Full text loading...

References

  1. Bao, C.2016. Verification for Chance of Geologic Success. Norwegian University of Science and Technology (NTNU), Trondheim, Norway.
     [Google Scholar]
  2. Bickel, J.E. and Kim, S.D.2008. Verification of the weather channel probability of precipitation forecasts. Monthly Weather Review, 136, 4867–4881, doi: 10.1175/2008MWR2547.110.1175/2008MWR2547.1
     https://doi.org/10.1175/2008MWR2547.1 [Google Scholar]
  3. Bratvold, R.B., Mohus, E., Petutschnig, D. and Bickel, E.2020. Production forecasting: Optimistic and overconfident – over and over again. SPE Reservoir Evaluation & Engineering, 23, 799–810, doi: 10.2118/195914-PA10.2118/195914‑PA
     https://doi.org/10.2118/195914-PA [Google Scholar]
  4. Brekke, H.2000. The tectonic evolution of the Norwegian Sea Continental Margin with emphasis on the Vøring and Møre Basins. Geological Society, London, Special Publications, 167, 327–378, doi: 10.1144/GSL.SP.2000.167.01.1310.1144/GSL.SP.2000.167.01.13
     https://doi.org/10.1144/GSL.SP.2000.167.01.13 [Google Scholar]
  5. Brennand, T.P., van Hoorn, B., James, K.H. and Glennie, K.W.2009. Historical review of North Sea exploration. In: Glennie, K.W. (ed.) Petroleum Geology of the North Sea: Basic Concepts and Recent Advances. 4th edn. Blackwell Science, Oxford, UK, 1–41, doi: 10.1002/9781444313413.ch110.1002/9781444313413.ch1
     https://doi.org/10.1002/9781444313413.ch1 [Google Scholar]
  6. Brier, G.W.1948. Review of the Verification of Weather Forecasts. Koninklijk Nederlandsch Meteorogisch Instituut, de Bilt (Nederland) No. 125, Mededeelingen en Verhandelingen, Serie B, Deel 1, No. 2, by W. Bleeker. Bulletin of the American Meteorological Society, 29, 475–475, doi: 10.1175/1520-0477-29.9.1.47410.1175/1520‑0477‑29.9.1.474
     https://doi.org/10.1175/1520-0477-29.9.1.474 [Google Scholar]
  7. Brooks, H.E. and Doswell, C.A., III1996. A comparison of measures-oriented and distributions-oriented approaches to forecast verification. Weather and Forecasting, 11, 288–303, doi: 10.1175/1520-0434(1996)011<0288:ACOMOA>2.0.CO;210.1175/1520‑0434(1996)011<0288:ACOMOA>2.0.CO;2
     https://doi.org/10.1175/1520-0434(1996)011<0288:ACOMOA>2.0.CO;2 [Google Scholar]
  8. Buizza, R., Houtekamer, P., Pellerin, G., Toth, Z., Zhu, Y. and Wei, M.2005. A comparison of the ECMWF, MSC, and NCEP global ensemble prediction systems. Monthly Weather Review, 133, 1076–1097, doi: 10.1175/MWR2905.110.1175/MWR2905.1
     https://doi.org/10.1175/MWR2905.1 [Google Scholar]
  9. CAWCR2009. Forecast verification – issues, methods and FAQ. Centre for Australian Weather and Climate Research (CAWCR), https://www.cawcr.gov.au/projects/verification/verif_web_page.html
  10. Cloke, H.L. and Pappenberger, F.2009. Ensemble flood forecasting: a review. Journal of Hydrology, 375, 613–626, doi: 10.1016/j.jhydrol.2009.06.00510.1016/j.jhydrol.2009.06.005
     https://doi.org/10.1016/j.jhydrol.2009.06.005 [Google Scholar]
  11. Flyvbjerg, B.2006. From Nobel Prize to project management: getting risks right. Project Management Journal, 37, 5–15, doi: 10.1177/87569728060370030210.1177/875697280603700302
     https://doi.org/10.1177/875697280603700302 [Google Scholar]
  12. Gluyas, J.G. and Swarbrick, R.E.2021. Petroleum Geoscience. Wiley-Blackwell, Hoboken, NJ.
     [Google Scholar]
  13. Gneiting, T. and Raftery, A.E.2007. Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association, 102, 359–378, doi: 10.1198/01621450600000143710.1198/016214506000001437
     https://doi.org/10.1198/016214506000001437 [Google Scholar]
  14. Gotautas, V.A.1963. Quantitative analysis of prospect to determine whether it is drillable. AAPG Bulletin, 47, 1794–1812, doi: 10.1306/BC743B65-16BE-11D7-8645000102C1865D10.1306/BC743B65‑16BE‑11D7‑8645000102C1865D
     https://doi.org/10.1306/BC743B65-16BE-11D7-8645000102C1865D [Google Scholar]
  15. Henriksen, E., Bjørnseth, H. et al.2011. Uplift and erosion of the greater Barents Sea: impact on prospectivity and petroleum systems. Geological Society, London, Memoirs, 35, 271–281, doi: 10.1144/M35.1710.1144/M35.17
     https://doi.org/10.1144/M35.17 [Google Scholar]
  16. Hermanrud, C., Abrahamsen, K., Vollset, J., Nordahl, S. and Jourdan, C.1996. Evaluation of undrilled prospects – sensitivity to economic and geological factors. Norwegian Petroleum Society Special Publications, 6, 325–337, doi: 10.1016/S0928-8937(07)80027-010.1016/S0928‑8937(07)80027‑0
     https://doi.org/10.1016/S0928-8937(07)80027-0 [Google Scholar]
  17. Hsu, W.-r. and Murphy, A.H.1986. The attributes diagram: A geometrical framework for assessing the quality of probability forecasts. International Journal of Forecasting, 2, 285–293, doi: 10.1016/0169-2070(86)90048-810.1016/0169‑2070(86)90048‑8
     https://doi.org/10.1016/0169-2070(86)90048-8 [Google Scholar]
  18. Jolliffe, I.T. and Stephenson, D.B. (eds) 2012. Forecast Verification: A Practitioner's Guide in Atmospheric Science. 2nd edn. John Wiley & Sons, Chichester, UK, doi: 10.1002/978111996000310.1002/9781119960003
     https://doi.org/10.1002/9781119960003 [Google Scholar]
  19. Kahneman, D.2011. Thinking, Fast and Slow. Farrar, Straus and Giroux, New York.
     [Google Scholar]
  20. Lasabuda, A.P., Johansen, N.S. et al.2021. Cenozoic uplift and erosion of the Norwegian Barents Shelf – a review. Earth-Science Reviews, 217, doi: 10.1016/j.earscirev.2021.10360910.1016/j.earscirev.2021.103609
     https://doi.org/10.1016/j.earscirev.2021.103609 [Google Scholar]
  21. Megill, R.E.1984. An Introduction to Risk Analysis. PennWell, Tulsa, OK.
     [Google Scholar]
  22. Milkov, A.V.2015. Risk tables for less biased and more consistent estimation of probability of geological success (PoS) for segments with conventional oil and gas prospective resources. Earth-Science Reviews, 150, 453–476, doi: 10.1016/j.earscirev.2015.08.00610.1016/j.earscirev.2015.08.006
     https://doi.org/10.1016/j.earscirev.2015.08.006 [Google Scholar]
  23. Milkov, A.V.2017. Integrate instead of ignoring: base rate neglect as a common fallacy of petroleum explorers. AAPG Bulletin, 101, 1905–1916, doi: 10.1306/032717162281700310.1306/0327171622817003
     https://doi.org/10.1306/0327171622817003 [Google Scholar]
  24. Milkov, A.V.2022. Pre-drill assessments and drilling outcomes in Mexico in 2018–2022 and historical experience from Norway and the Netherlands: lessons learned and recommendations for future petroleum exploration. Natural Resources Research, 31, 2483–2504, doi: 10.1007/s11053-022-10074-310.1007/s11053‑022‑10074‑3
     https://doi.org/10.1007/s11053-022-10074-3 [Google Scholar]
  25. Milkov, A.V. and Navidi, W.C.2020. Randomness, serendipity, and luck in petroleum exploration. AAPG Bulletin, 104, 145–176, doi: 10.1306/0506191812810.1306/05061918128
     https://doi.org/10.1306/05061918128 [Google Scholar]
  26. Murphy, A.H.1997. Forecast verification. In: Katz, R.W. and Murphy, A.H. (eds) Economic Value of Weather and Climate Forecasts. Cambridge University Press, Cambridge, UK, 19–74, doi: 10.1017/CBO9780511608278.00310.1017/CBO9780511608278.003
     https://doi.org/10.1017/CBO9780511608278.003 [Google Scholar]
  27. Murphy, A.H. and Winkler, R.L.1987. A general framework for forecast verification. Monthly Weather Review, 115, 1330–1338, doi: 10.1175/1520-0493(1987)115%3C1330:AGFFFV%3E2.0.CO;210.1175/1520‑0493(1987)115%3C1330:AGFFFV%3E2.0.CO;2
     https://doi.org/10.1175/1520-0493(1987)115%3C1330:AGFFFV%3E2.0.CO;2 [Google Scholar]
  28. Nandurdikar, N. and Wallace, L.2011. Failure to produce: An investigation of deficiencies in production attainment. Paper SPE-145437-MS presented at theSPE Annual Technical Conference and Exhibition, October 30–November 2, 2011, Denver, Colorado, USA, doi: 10.2118/145437-MS10.2118/145437‑MS
     https://doi.org/10.2118/145437-MS [Google Scholar]
  29. Nesvold, E. and Bratvold, R.B.2022. Debiasing probabilistic oil production forecasts. Energy, 258, doi: 10.1016/j.energy.2022.12474410.1016/j.energy.2022.124744
     https://doi.org/10.1016/j.energy.2022.124744 [Google Scholar]
  30. NOD2024. Open data. Norwegian Offshore Directorate (NOD), Stavanger, Norway, https://www.sodir.no/en/facts/data-and-analysis/open-data/
  31. NOD2025. FactPages: Exploration wells in the North Sea. Norwegian Offshore Directorate (NOD), Stavanger, Norway, https://factpages.npd.no
  32. NPD2016. Resource Report 2016. Norwegian Petroleum Directorate (NOD), Stavanger, Norway, https://www.sodir.no/49096f/globalassets/1-sodir/publikasjoner/ressursrapporter-arkiv/resource-report-2016.pdf
     [Google Scholar]
  33. NPD2020. Resource Report 2020 – Exploration. Norwegian Petroleum Directorate (NOD), Stavanger, Norway, https://www.sodir.no/en/whats-new/publications/reports/resource-report/resource-report-2020/
     [Google Scholar]
  34. NPD2022. Resource Report 2022. Norwegian Petroleum Directorate (NOD), Stavanger, Norway, https://www.sodir.no/en/whats-new/publications/reports/resource-report/resource-report-2022/
     [Google Scholar]
  35. NPD2023. Prognoses and Results from Wildcat Wells More Than 20 Years Old. Norwegian Petroleum Directorate (NOD), Stavanger, Norway, https://www.sodir.no/en/facts/wells/wildcat-wells-more-than-20-years-old/
     [Google Scholar]
  36. Ofstad, K., Øvretveit, A., Kullerud, L. and Heggland, K.2000. Probability of discovery and the reasons for dry wells: results from the project: evaluation of Norwegian Wildcat Wells (Article 3). Norwegian Petroleum Society Special Publications, 9, 47–55, doi: 10.1016/S0928-8937(00)80008-910.1016/S0928‑8937(00)80008‑9
     https://doi.org/10.1016/S0928-8937(00)80008-9 [Google Scholar]
  37. Otis, R.M. and Schneidermann, N.1997. A process for evaluating exploration prospects. AAPG Bulletin, 81, 1087–1109, doi: 10.1306/522B49F1-1727-11D7-8645000102C1865D10.1306/522B49F1‑1727‑11D7‑8645000102C1865D
     https://doi.org/10.1306/522B49F1-1727-11D7-8645000102C1865D [Google Scholar]
  38. Petropoulos, F., Apiletti, D. et al.2022. Forecasting: theory and practice. International Journal of Forecasting, 38, 705–871, doi: 10.1016/j.ijforecast.2021.11.00110.1016/j.ijforecast.2021.11.001
     https://doi.org/10.1016/j.ijforecast.2021.11.001 [Google Scholar]
  39. Richards, F.L., Richardson, N.J., Rippington, S.J., Wilson, R.W. and Bond, C.E. (eds) 2015. Industrial Structural Geology: Principles, Techniques and Integration. Geological Society, London, Special Publications, 421, doi: 10.1144/SP421.010.1144/SP421.0
     https://doi.org/10.1144/SP421.0 [Google Scholar]
  40. Rose, P.R.2001. Risk Analysis and Management of Petroleum Exploration Ventures. AAPG Methods in Exploration, 12, doi: 10.1306/Mth1279210.1306/Mth12792
     https://doi.org/10.1306/Mth12792 [Google Scholar]
  41. Stensrud, D.J., Brooks, H.E., Du, J., Tracton, M.S. and Rogers, E.1999. Using ensembles for short-range forecasting. Monthly Weather Review, 127, 433–446, doi: 10.1175/1520-0493(1999)127%3C0433:UEFSRF%3E2.0.CO;210.1175/1520‑0493(1999)127%3C0433:UEFSRF%3E2.0.CO;2
     https://doi.org/10.1175/1520-0493(1999)127%3C0433:UEFSRF%3E2.0.CO;2 [Google Scholar]
  42. Stephenson, D., Coelho, C., Doblas-Reyes, F. and Balmaseda, M.2005. Forecast assimilation: a unified framework for the combination of multi-model weather and climate predictions. Tellus A: Dynamic Meteorology and Oceanography, 57, 253–264, doi: 10.1111/j.1600-0870.2005.00110.x10.1111/j.1600‑0870.2005.00110.x
     https://doi.org/10.1111/j.1600-0870.2005.00110.x [Google Scholar]
  43. Wilks, D.2001. A skill score based on economic value for probability forecasts. Meteorological Applications, 8, 209–219, doi: 10.1017/S135048270100209210.1017/S1350482701002092
     https://doi.org/10.1017/S1350482701002092 [Google Scholar]
  44. Wilks, D.S.2011. Statistical Methods in the Atmospheric Sciences. 3rd edn. International Geophysics Series, 100. Academic Press, Cambridge, MA.
     [Google Scholar]
  45. Yielding, G.2015. Trapping of buoyant fluids in fault-bound structures. Geological Society, London, Special Publications, 421, 29–39, doi: 10.1144/SP421.310.1144/SP421.3
     https://doi.org/10.1144/SP421.3 [Google Scholar]
  46. Ziegler, P.A.1990. Geological Atlas of Western and Central Europe. Shell Internationale Petroleum Maatschappij, The Hague, The Netherlands.
     [Google Scholar]
/content/journals/10.1144/petgeo2025-014
Loading
Probability forecast verification in petroleum exploration: insights from the Norwegian Continental Shelf
Petroleum Geoscience 31, petgeo2025-014 (2025); https://doi.org/10.1144/petgeo2025-014
/content/journals/10.1144/petgeo2025-014
/content/journals/10.1144/petgeo2025-014
Loading

Data & Media loading...

  • Article Type: Research Article

Most Cited This Month Most Cited RSS feed

This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error