IEEE INTERNATIONAL CONFERENCE ON APPLIED ROBOTICS FOR THE POWER INDUSTRY (CARPI 2010), MONTREAL, CANADA, OCTOBER, 2010

Image Processing to Automate Condition Assessment of Overhead Line Components

Wai Ho Li, Arman Tajbakhsh, Carl Rathbone, and Yogendra Vashishtha

AbstractCondition monitoring of overhead electricity transmission line assets is essential to network operation. Traditionally, the condition of overhead lines are assessed visually. Visual inspection is difcult to apply to phase conductors due to their height above ground. As such, aerial imaging surveys seem to be an ideal solution to this problem. However, the large number of high resolution images generated by aerial surveys are costly to inspect in terms of time and labour. This paper presents an image processing system that automates conductor localization and spacer detection in order to reduce the work required in visual inspection. The implemented system was tested on over four thousand video images from actual aerial surveys of quadconductor transmission line assets. Experimental results show highly accurate conductor localization and a robust hit rate for spacer detection. These results suggest that image processing can be used to help automate labour intensive tasks in the condition assessment of overhead line components. Index TermsImage processing, computer vision, condition monitoring, condition assessment, conductor localization, spacer detection, automation, aerial survey, aerial imaging, line detection

I. I NTRODUCTION P AusNet monitors the condition of its overhead electricity transmission line assets to ensure the safe, cost effective and reliable operation of its network. Prioritisation of maintenance works requires decision making systems which rely on knowledge of the asset base condition that is current, complete and of sufcient quality. Bare overhead ACSR conductors and conductor ttings such as spacers, dampers and joints are generally expected to remain in service for a period of 50-75 years but this can vary substantially. Lines traversing benign environments are expected to last longer, whereas lines in more aggressive environments or defective lines can suffer from early or accelerated deterioration. The condition of overhead line assets has traditionally been assessed primarily using visual techniques. Difculties arise when trying to inspect phase conductor visually due to its height above ground, electrical hazards and inaccessible underlying terrain. SP AusNet proposes to develop an image capture system which uses helicopter-mounted high resolution
Dr. Wai Ho Li works at the Department of Electrical and Computer Systems Engineering, Monash University, Clayton, Melbourne, Australia; Email: wai.ho.li@eng.monash.edu.au A. Tajbakhsh works in the Asset Innovation and Research group at SP AusNet, Australia; Email: arman.tajbakhsh@sp-ausnet.com.au C. Rathbone works in the Lines Asset Management group in SP AusNet, Australia; Email: carl.rathbone@sp-ausnet.com.au Dr. Y. Vashishtha is Asset Innovation and Research manager at SP AusNet, Australia; Email: yogendra.vashishtha@sp-ausnet.com.au

video cameras to capture a continuous stream of digital images of overhead conductors and overhead line components. An obstacle to the widespread deployment of such an image capture system is the impracticability of manually inspecting the images returned by photographic surveys. The proposed system is capable of capturing multi-megapixel images at around 5 frames per second. Photos collected over the course of a single days survey can number in the hundreds of thousands and requires a team of engineers many days to inspect, signicantly increasing the cost. Manual inspection of digital images over long periods of time carries the risk of operator fatigue and lacks repeatability when conducted by different inspectors. Computerized image processing can vastly reduce the manual labour required to perform image inspections. From license plate recognition to ngerprint matching, image processing has been successfully used to transfer labour intensive tasks to the tireless computer. Therefore it is proposed that in parallel to the development of the image capture system an image processing capability be developed to automatically detect defects and assess asset condition. Through its ability to provide comprehensive assessment of aerial conductors and conductor ttings, the use of automatic image processing of aerial photography promises to revolutionize electricity network asset inspections in the near future. It is anticipated that our automated image processing capability will extend to images captured with other platforms, such as line inspection robots, unmanned aerial vehicles (UAV) and ground vehicles. Apart from improving the efciency of ofine transmission line inspection, the research presented here also opens up the possibilities of real time image processing for condition monitoring, repairs and vegetation management. II. AUTOMATIC IMAGE PROCESSING A. System overview The proposed image processing system is outlined in Figure 1. This paper details the Conductor Localization and Spacer Detection modules, including experimental results on real world aerial survey images. Note that the conductor localization module also performs preprocessing in converting high resolution survey video into lower resolution grayscale images. Both modules are written in C/C++ using OpenCV [1]. Dashed lines indicate planned future work. The user species the number of conductors he or she expects to see in the video images. Images where an unexpected number of conductors is found, such as detecting only 3 conductors for a quad-conductor line, will be returned by

IEEE INTERNATIONAL CONFERENCE ON APPLIED ROBOTICS FOR THE POWER INDUSTRY (CARPI 2010), MONTREAL, CANADA, OCTOBER, 2010

Digital Images from Aerial Survey Video

Conductor Localization

Spacer Detection

Other Component Detection Methods

Fig. 1.

Image processing system overview

our system for manual inspection. Only images where the expected number of conductors are localized will be passed onto the spacer detection module. The spacer detection results are intended for manual inspection. The goal is to drastically reduce the number of images that an engineer must inspect when he or she wishes to check a particular type of overhead line component. For example, in the experiments presented in Section III our test data contains over four thousand images but spacers are only seen in 211 video frames. Note that the image processing steps presented below are automatic and requires no user intervention apart from a priori specication of the number of expected conductors. B. Conductor localization We dene conductor localization as the automatic process of locating the conductor(s) in a transmission line aerial survey image. This process is the lynch pin of subsequent processing as many line components are physically attached to the conductor. Given that the camera eld of view used during image capture is quite narrow, conductor localization can be treated as the problem of detecting straight lines in a digital image. Image processing literature contains a plethora of line detection techniques, from Hough transform [2] to multi-scale line detectors using image pyramids [3]. After comparing the performance of several widely used techniques, we decided to apply a template matching approach for conductor localization. Let us rst examine the conductor localization results in Figure 2, which will provide some insight into why we chose a template matching method.

the large number of line features in the input image. Notice that some of the visual noise is actually supplied by other conductors in the background. There are also several line features oriented in parallel to the conductors. Despite the visual noise, our template-based approach is able to automatically localize the correct quad-bundle of transmission conductors. The results in Figure 2 help illustrate why we decided to use a template-based localization approach. Conductor detection approaches such as edge detection or Gabor ltering followed by Hough transform [4] runs the risk of detecting too many lines, which undermines the goal of automation as a way to reduce manual labour during inspection. The computational costs of Hough transform can also be prohibitive. Similarly, multi-scale approaches will return many false positives, unless we restrict their scale such that only lines of a similar width to the conductors are found. This is why we decided to use a template matching approach, where some tolerance in line width is provided through the use of a blurred template. Conductor localization is performed by sliding a line template vertically down the input image and matching it against the image pixel values. Matching is performed in grayscale using a template with the same width as the input image. To account for conductor orientation, the input image is rotated over multiple orientations with template matching performed for each orientation. The orientation that provides the strongest matching response tells us the overall conductor orientation. For each orientation, the template matching provides a vertical vector of Normalized Cross Correlation (NCC) values, the maxima of which represents the vertical coordinates of possible conductors. Non-maxima suppression is applied to the vector of NCC values to detect and localize the conductors. The maximum number of conductors that should be localized is specied by the user. Note that in most aerial survey videos, not all images contain the expected number of conductors. One or more conductors can be out of the cameras eld of view. For example, some survey images were captured when the camera is aimed at transmission towers resulting in no conductors being seen. A threshold on the NCC result is used to identify images were no conductors are present so that they can be returned for manual inspection. In addition, a dynamic threshold is applied on a frame-by-frame basis to nd images that do not contain the expected number of conductors. The threshold is calculated as a ratio of the maximum NCC peak for each image. These images are also returned for manual inspection. C. Spacer detection This section describes our spacer detection algorithm, which detects the presence of a quad-conductor spacer in an image and localizes the spacer by returning a bounding rectangle. The detection process begins by cropping and rotating the original image using the conductor localization results so that it only contains the conductors with some vertical padding. Then a Gabor lter [5] is applied to the cropped image to nd features spanning the set of conductors. From here on in, we shall refer to the results of Gabor ltering designed to nd spacers as

Fig. 2. Conductor localization by template matching (Left: Input image; Right: Conductor localization results in red)

The left image in Figure 2 illustrates some of the real world challenges faced by our conductor localization algorithm. Detecting the correct conductors is a non-trivial problem given

IEEE INTERNATIONAL CONFERENCE ON APPLIED ROBOTICS FOR THE POWER INDUSTRY (CARPI 2010), MONTREAL, CANADA, OCTOBER, 2010

the spacer Gabor image. Spacers are detected by looking for large clusters of pixels that respond strongly to Gabor ltering. An example illustrating the spacer detection steps is shown in Figure 3. One can imagine that spacer-like features in the background may cause false positives. To reduce false positives caused by background objects that lie across the conductors, we use the fact that a spacer disrupts the usually smooth contours of a conductor at the location where it attaches to the conductor. Figure 4 shows conductor-spacer joints detected automatically by our image processing system. Figure 4 also shows that one or more conductor-spacer joints can be missed due to occlusion and image noise. As such, our algorithm only requires the presence of one or more disruptions as sufcient proof that a spacer is present. Figure 5 shows an example where despite a strong spacer Gabor response, which would normally result in an incorrectly detected spacer, our algorithm automatically rejected this potential false positive due to the lack of conductor-spacer joints. III. E XPERIMENTAL RESULTS Our image processing system was tested on 6 video sequences of quad-conductor transmission lines containing a total of 4437 four-megapixel images. The same parameters are used across all six video sequences and all processing was carried out automatically. A. Conductor localization Conductor localization results were analyzed manually by going through the results frame-by-frame. Successful localization is dened as having the detected conductor lying on top of the actual conductor in a manner that is visually correct. As survey images are usually inspected manually, the authors feel this method of validation is apt. There were 4349 test images where all four quad-conductors are within the cameras eld of view. Our system was able to automatically localize all conductors in these images. However, 13 false positives were also detected along a short span of video due to a combination of insulators and background line features. Some false positive examples are shown in Figure 6. The remaining 75 video frames where one or more conductors were absent from the camera image were correctly returned for manual inspection. As only the false positives are incorrectly labelled, it can be said that our conductor localization method achieved an accuracy of 99.71%. B. Spacer detection The spacer detection module operated on 4362 images as 75 images were returned for manual inspection by the conductor localization step. Note that false positives from conductor localization were carried over to the spacer detection step. Again, we manually inspected the video frames to nd ground truth. In this case, we found 136 actual physical spacers, some of which are visible across multiple consecutive frames. The spacers can be seen in 211 images, including cases where a spacer is only partially visible.

Fig. 3. Spacer detection steps. From top to bottom: Input image, input after automatic cropping and rotation, spacer Gabor image, detection results

IEEE INTERNATIONAL CONFERENCE ON APPLIED ROBOTICS FOR THE POWER INDUSTRY (CARPI 2010), MONTREAL, CANADA, OCTOBER, 2010

Fig. 4. Detecting disruptions caused by conductor-spacer joints in order to reject false positives. For the sake of continuity, the same input image from Figure 3 is used to illustrate the process. Top: Cropped and rotated input image. Bottom: Conductor-spacer joint detection results as red squares

Our system was able to correctly detect spacers in 204 images, resulting in a hit rate of 96.68%. However, the system also returned 47 additional false positives. This equates to a false alarm rate of 1.13%. The majority of false positives can be attributed to three main causes. Firstly, other objects on the conductor such as dampers can appear like spacers in the Gabor ltered image. Secondly, frames where the conductors terminate or change appearance mid-frame can result in false spacers being detected. These frames tend to occur where the conductor arrives at a tower. These two cases of false positives are not of major concern as they are frames that will require manual inspection for separate reasons. Thirdly, unforunate congurations of background features can also result in a false positive. Background objects caused 10 of the 47 false positives and will be a focus of future research. False positive examples are shown in Figure 8. As mentioned earlier, there were 7 missed spacer images. Four missed images occurred because the spacer was only partially visible at the border of the image. These images can be seen in the rst row of Figure 9. Note that in all instances, the same physical spacer is found automatically by our system in the subsequent video frame. The remaining 3 missed spacers occurred because the conductors and the spacers are too far away from the camera. This meant our system cannot identify the spacers as it was not able to detect the conductor-spacer joints due to their small size. These images are presented in Figure 7.

Fig. 5. False positive spacer detection successfully prevented by checking for conductor-spacer joints. Top: Input image. Bottom: Spacer gabor

Fig. 7. Missed spacers caused by failure of conductor-joint detection due to larger than expected camera-conductor distance

C. Timing analysis Timing analysis was performed across all test images. Timing was conducted on a desktop PC with an Intel Xeon CPU (2.27 GHz) and 2GB of RAM. The C/C++ code was compiled using Microsoft Visual C++ 2008 compiler with optimization disabled (/Od) and running under Windows XP

Fig. 6.

Examples of false positive conductor localizations

IEEE INTERNATIONAL CONFERENCE ON APPLIED ROBOTICS FOR THE POWER INDUSTRY (CARPI 2010), MONTREAL, CANADA, OCTOBER, 2010

TABLE I AVERAGE P ROCESSING T IMES ( MS ) Preprocessing Conductor localization Spacer detection Total 52.6 204 16 272.6

R EFERENCES
[1] Opencv, http://opencv.willowgarage.com/wiki/. [2] R. O. Duda and P. E. Hart, Use of the hough transformation to detect lines and curves in pictures, Communications of the ACM, vol. 15, no. 1, pp. 1115, 1972. [3] T. Lindeberg, Edge detection and ridge detection with automatic scale selection, in IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, USA, June 1996. [4] C. Mu, J. Yu, Y. Feng, and J. Cai, Power lines extraction from aerial images based on gabor lter, in International Symposium on Spatial Analysis, Spatial-Temporal Data Modeling, and Data Mining, Y. Liu and X. Tang, Eds., vol. 7492, no. 1. SPIE, 2009, p. 74923P. [Online]. Available: http://link.aip.org/link/?PSI/7492/74923P/1 [5] J. Daugman, Two-dimensional spectral analysis of cortical receptive eld proles, Vision Research, vol. 20, no. 10, p. 84756, 1980.

Professional (Service Pack 2). The average processing times across the test image set are detailed in Table I. Preprocessing includes loading images from hard drive into memory as well as decompressing and resizing images for subsequent processing. Conductor localization and spacer detection includes all tasks described in Sections II-B and II-C respectively. The total processing time across all test data (4437 images) was roughly 20 minutes. Processing time can be reduced by enabling compiler optimizations and parallelizing computations given that images can be processed independently.

IV. C ONCLUSIONS AND FUTURE WORKS The conductor localization results clearly show that template matching is a robust means to identify line features of roughly-known width in aerial imagery across varying lighting conditions and backgrounds. However, ground truth locations of conductors is needed for more quantitative analyses of the localization results. Detailed comparisons against competing line detection methods is planned as future work. Despite the presence of false positives in spacer detection, the number of images returned by the system for manual inspection is an order of magnitude less than the quantity of test images. This promises large efciency gains for the manual inspection of overhead components. The number of false positives can be reduced by the rejection of dampers and tuning of parameters such as the number of expected conductor-spacer joints. A simple aspect ratio constraint on the width and height of detected spacers can also reject many false positives found in the test set. Computationally, the system is running near real time. Enabling compiler optimizations will further increase processing speed. The majority of computations is spent on template matching, which can be sped up through the use of FFTbased NCC as well as parallelization using multiple CPU cores or GPU architecture. Overall, the preliminary results suggest that image processing can help automate tedious and labour intensive tasks that are a regular part of condition assessment of overhead line components. Similar image processing techniques may also hold promise in automating surveys carried out via non-aerial means.

ACKNOWLEDGMENTS The authors thank Geoff Fairweather and David McLennan of Sp AusNet for making time to share their knowledge of the transmission network.

IEEE INTERNATIONAL CONFERENCE ON APPLIED ROBOTICS FOR THE POWER INDUSTRY (CARPI 2010), MONTREAL, CANADA, OCTOBER, 2010

Fig. 8. Examples of spacer detection false positives. Columns from left to right: False positives caused by dampers, mid-frame conductor termination and background features. Rows from top to bottom: Input image showing conductor-spacer joints, spacer Gabor image, detection results

Fig. 9.

Missed partially visible spacers (rst row) and the same spacer automatically detected in the subsequent video frame (second row)