TY - GEN
T1 - Scaling Up a Multispectral Resnet-50 to 128 GPUs
AU - Sedona, Rocco
AU - Cavallaro, Gabriele
AU - Jitsev, Jenia
AU - Strube, Alexandre
AU - Riedel, Morris
AU - Book, Matthias
N1 - Publisher Copyright: © 2020 IEEE.
PY - 2020/9/26
Y1 - 2020/9/26
N2 - Similarly to other scientific domains, Deep Learning (DL) holds great promises to fulfil the challenging needs of Remote Sensing (RS) applications. However, the increase in volume, variety and complexity of acquisitions that are carried out on a daily basis by Earth Observation (EO) missions generates new processing and storage challenges within operational processing pipelines. The aim of this work is to show that High-Performance Computing (HPC) systems can speed up the training time of Convolutional Neural Networks (CNNs). Particular attention is put on the monitoring of the classification accuracy that usually degrades when using large batch sizes. The experimental results of this work show that the training of the model scales up to a batch size of 8,000, obtaining classification performances in terms of accuracy in line with those using smaller batch sizes.
AB - Similarly to other scientific domains, Deep Learning (DL) holds great promises to fulfil the challenging needs of Remote Sensing (RS) applications. However, the increase in volume, variety and complexity of acquisitions that are carried out on a daily basis by Earth Observation (EO) missions generates new processing and storage challenges within operational processing pipelines. The aim of this work is to show that High-Performance Computing (HPC) systems can speed up the training time of Convolutional Neural Networks (CNNs). Particular attention is put on the monitoring of the classification accuracy that usually degrades when using large batch sizes. The experimental results of this work show that the training of the model scales up to a batch size of 8,000, obtaining classification performances in terms of accuracy in line with those using smaller batch sizes.
KW - Distributed deep learning
KW - classification
KW - convolutional neural network
KW - high performance computing
KW - residual neural network
KW - sentinel-2
UR - https://www.scopus.com/pages/publications/85101965031
U2 - 10.1109/IGARSS39084.2020.9324237
DO - 10.1109/IGARSS39084.2020.9324237
M3 - Conference contribution
T3 - International Geoscience and Remote Sensing Symposium (IGARSS)
SP - 1058
EP - 1061
BT - 2020 IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2020 - Proceedings
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2020 IEEE International Geoscience and Remote Sensing Symposium, IGARSS 2020
Y2 - 26 September 2020 through 2 October 2020
ER -