Congratulations to Dr. Mohamad Alipour for successfully defending his Ph.D. dissertation!! It has been amazing having Mohamad as a member of the MOBLab. Looking forward to what comes next for such an outstanding scholar. Mohamad's dissertation builds from his early studies related to large scale infrastructure systems and emphasizes as series of vision-based strategies for assessing the performance and condition state of deteriorating infrastructure.
Dissertation Title: Deep Learning for Robust and Efficient Automated Defect Recognition in Critical Infrastructure
Abstract:
Smart and automated management and maintenance of critical infrastructure is not only a concern for the departments of transportation and city officials, but also a necessity for the realization of next generation smart cities. The shortage of resources in the face of the size of the national infrastructure, and the subjectivity, limitations, and cost of traditional visual inspections demand innovative solutions that can reduce manual inspection and maintenance operations while enhancing infrastructure quality assurance. Examples of such solutions include robotic inspection and crowd-sourced monitoring. The intersection between such strategies is their use of images to record and document the state of health of a structure, and to allow for the detection of issues of interest using automated methods.
This dissertation explores some of the challenges facing the use of deep learning for robust image-based condition assessment of infrastructure. First, a framework for scalable multi-class urban defect recognition was proposed that leverages two cost-effective alternative data sources, namely web images and Google Street View scenes. Self-training with data distillation using a set of geometric transformations on the unlabeled images was employed to increase the learning potential of the system. Results showed that the proposed framework helped increase the accuracy of the model by nearly 20% to achieve a final accuracy of 80% over a selected set of urban defect categories. A sensitivity analysis and an error analysis also helped determine the influential factors affecting the performance of the model.
Next, cracks as a highly prevalent mode of structural deterioration were selected for an in-depth analysis on detection and measurement algorithms. Three strategies were proposed to create robust models that can detect cracks in more than one material. It was shown that using the proposed strategies, an existing pre-trained model can be adapted to work for other materials and a single model can be jointly-trained on different materials. Through the proposed solutions and comparisons with deep learning and edge detection baselines, the potential to increase the robustness and flexibility of deep learning crack detection models for practical real-world applications was demonstrated.
To improve the state of the art in defect detection in terms of the level of detail, a patch-level model was repurposed into a fully convolutional neural network, which was trained end-to-end on a dataset of high-resolution images manually annotated at the pixel-level. Aside from the fully convolutional architecture employed, three key techniques were leveraged to produce competitive performance. These include cost-sensitive learning to combat class imbalance, data augmentation with rescaling to diversify training data and cover multiple scales of cracks, and the inclusion of crack-like distractors to make the model robust to extraneous objects. The model was shown to be successful in correctly detecting over 92% of crack and 99.9% of non-crack pixels.
Finally, the success of the proposed pixel-level crack detection model was then leveraged to quantify cracks, which is highly important in assessing the type and importance of the damage and the required maintenance measures. A definition that aligns well with the intuitive interpretation of a crack width was proposed and smoothing of width measurements was explored to increase robustness to jagged edge noise. Comparisons with both human crack width measurement and a number of state-of-the-art baselines demonstrated the consistency, usefulness and advantages of the method, which can be used to facilitate and accelerate manual structural inspections.
Through the presented results, this dissertation highlights the power of the emerging computer vision models based on deep learning in the field of automated structural inspections and introduces new opportunities for deployment using automated robotic and crowd-sourced inspection systems.