Pavements are considered a main transport infrastructure. Hence, in this study, we leverage the advancement of a lightweight customised LinkNet with several innovations of adding a bottleneck, Mobile Inverted Bottleneck convolution (MBConv), an enhanced ASPP (Atrous Spatial Pyramid Pooling), and the pre-trained ResNet encoder for accurately segmenting damaged pavements. This innovation optimises the signal propagation thanks to the bottleneck, recognises small or complex objects of the enhanced ASPP, reduces the network complexity by MBConv and exploits the trained parameters of the ResNet encoder from ImageNet to improve the model's accuracy and efficiency. We trained the customised network, the original LinkNet, U-Net, FPN (feature pyramid network), PSPNet (pyramid scene parsing network) and PAN (path aggregation network), for comparisons with the flood-induced damaged pavement dataset from the Korea Intelligence Information Society Agency. Although the complexity of the assembled architecture has decreased (10M fewer parameters), the model is slightly slower than the original LinkNet by 13 ms (with ResNet50) and faster by 19 ms (with ResNet152), and both achieve around a 5% accuracy improvement. Damaged roads and pavements are well segmented and mapped, and the customised network is very fast and recommended for rapid segmentations and real-time operations on edge-portable devices.