Machine Learning Scalability Requires High Performance Computing Strategies

References

1. Araya-Polo, M., Jennings, J., Adler, A. and Dahlke, T.
   [2018] Deep-learning tomography. The Leading Edge, 37(1), 58–66.
   [Google Scholar]
2. Ben-Nun, T. and Hoefler, T.
   [2019] Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis. ACM Comput. Surv., 52(4).
   [Google Scholar]
3. Bergen, K.J., Johnson, P.A., de Hoop, M.V. and Beroza, G.C.
   [2019] Machine learning for data-driven discovery in solid Earth geoscience. Science, 363(6433).
   [Google Scholar]
4. Bouthillier, X., Laurent, C. and Vincent, P.
   [2019] Unreproducible Research is Reproducible. In: Chaud-huri, K. and Salakhutdinov, R. (Eds.) Proceedings of the 36th International Conference on Machine Learning, Proceedings of Machine Learning Research, 97. PMLR, Long Beach, California, USA, 725–734.
   [Google Scholar]
5. Chetlur, S., Woolley, C., Vandermersch, P., Cohen, J., Tran, J., Catanzaro, B. and Shelhamer, E.
   [2014] cuDNN: Efficient Primitives for Deep Learning.
   [Google Scholar]
6. Dozat, T.
   [2016] Incorporating Nesterov momentum into Adam. In: In Proceedings of ICLR Workshop.
   [Google Scholar]
7. McCandlish, S., Kaplan, J., Amodei, D. and Team, O.D.
   [2018] An Empirical Model of Large-Batch Training. CoRR, abs/1812.06162.
   [Google Scholar]
8. Ronneberger, O., Fischer, P. and Brox, T.
   [2015] U-Net: Convolutional Networks for Biomedical Image Segmentation. CoRR, abs/1505.04597.
   [Google Scholar]
9. Sergeev, A. and Balso, M.D.
   [2018] Horovod: fast and easy distributed deep learning in TensorFlow. CoRR, abs/1802.05799.
   [Google Scholar]
10. Shallue, C.J., Lee, J., Antognini, J.M., Sohl-Dickstein, J., Frostig, R. and Dahl, G.E.
    [2018] Measuring the Effects of Data Parallelism on Neural Network Training. CoRR, abs/1811.03600.
    [Google Scholar]