Object Extraction in Data Mining Framework for Video Sequence

Prof. Manish Manoria Samrat Ashok Technological Institute, Vidisha,M.P.,India manishmanoria@rediffmail.com

Ms. Stuti Dave Samrat Ashok Technological Institute, Vidisha, M.P., India stutidave@rediffmail.com

Abstract - We are aware that the emerging video coding standard such as MPEG-4 enables various contentbased functionalities for multimedia applications. To support such functionalities and also to improve the coding efficiency, each frame of an image sequence is decomposed into video object planes (VOPs). Each VOP corresponds to a single moving object in the scene. This paper proposes an algorithm of key object(s) extraction from video database. Here we describe a framework of data mining by applying segmentation and enhancement based technique to the image and video data set, for the purpose of knowledge discovery. The objective of this particular segmentation algorithm is to segment the huge image and video data sets in order to extract important objects from them. This proposed method covers both spatial and temporal aspects of data mining. The video data set is parsed to create the image data set by preserving temporal information, then the background subtraction method is applied to get the moving object, and then double differencing operator is applied to get better result. The gamma enhancement function is applied with histogram equalization with a number of different parameters to improve the contrast of the resultant images. This method is thus able to detect moving object(s) with surprisingly good location capability, and eliminate noise, which occurs due to camera movements. The proposed approach is applied to a traffic video data set to demonstrate the segmentation for data mining of video database. KEYWORDS: Video mining, Video object segmentation, background subtraction, enhancement technique. 1 Introduction There is an enormous need for visual information management in the growing field of medical imaging and telemedicine, traffic monitoring etc. In particular, image understanding form image database queries and retrieval are major topics of research [1,2]. This newly emerging field combines aspects from databases, image processing and understanding, knowledge-based systems and context-based compression. Thus this growing amount of video information requires efficient management, and to support more efficient video database management, a video segmentation method has become necessary. Segmentation is one of the most challenging tasks in the field of video mining, since it lies at the base of virtually any scene analysis problem. The general approach that motivates the need for segmentation is that the problem of analyzing a complete scene can be subdivided into a set of simpler problems, of analyzing its primitives. Then, each primitive object in the scene can be processed independently [3][4]. This approach greatly simplifies many high-level vision problems, such as recognition, classification, scene analysis, etc. However, the ultimate goal of the segmentation process is to divide an image into the physical objects composing it, so that each region constitutes a semantically meaningful entity. In order to identify and track the temporal and relative spatial positions of objects in video sequences, it is necessary to have object-based representation of video data. For this purpose, attention has been devoted to segmenting video frames into regions such that each region, or a group of regions, corresponds to an object that is meaningful to human viewers [7, 8]. While most of the previous works [5] [6] are based on low-level global features, such as color histogram, shape and texture feature, our method focuses on obtaining object level segmentation; obtaining objects in each frame and their traces across the frames. There has been a large amount of researches [18, 19] to find object from video sequence when the segmentation problem is considered for a fixed camera domain, a usual technique is required to resolve the foreground

P21/1

objects i.e. background subtraction [9]. This involves the creation of a background model that is subtracted from the input image to create a difference image. The new difference image only contains objects or new features that have not yet appeared in the background. This method is especially suitable for video conferencing [10] and surveillance [11] applications, where the backgrounds remain still. Neri et al. [12] suggested a different segmentation technique based on change detection. The method assumes that the objects are moving over a static background, thus, if a moving camera is detected, then global motion compensation be applied first. Potential foreground regions are detected in a preliminary stage by applying a higher-order statistics (HOS) test to a group of inter-frame differences. The non-zero values in these difference frames are due to moving objects. Few references [13, 14] have addressed the problem of object segmentation and tracking using an active camera, they cannot able to give spatial information of moving object, which is very inefficient and non-flexible This method is thus able to detect moving object(s) with surprisingly good location capability. 2 Video Objects Extraction Figure 1 depicts the diagram of the proposed object extraction scheme. Prior to performing segmentation and extraction, a video database is created. It is a temporal data because it contains before-after sequence relationship that is first frame comes before second frame and second frame comes after first frame. Video parsing divides large video sequence in to frames or images.
Video file
Image Database Background Subtraction Object Detection Perform gamma correction

Create negative image

Improve contrast

Object extracted

Figure 1 diagram of the proposed object segmentation and extraction method The number of images/frames per second gives motion in video. Then reference frame of stationary component is created. The proposed background substraction method is applied which is able to detect moving object from each frame. 2.1 Background subtraction Background subtraction is a technique to remove nonmoving components from a video sequence. The main assumption for its application is that the camera remains stationary. The basic principle is to create a reference frame of the stationary components in the image. Step1: Once reference frame of stationary component is created, the reference frame is subtracted from any subsequent images. Those pixels resulting from new (moving) objects will generate a difference not equal to zero. Step2: In the video sequences the nonmoving objects were manually selected from the video data and then averaged together. Here accumulating and averaging images of the target area in some time interval construct a reference frame. The image sequence used consists of about 16 minutes of video from a traffic intersection with approximately constant lighting conditions. (The difference image is created by subtracting the reference frame from the current image) .The results are scaled by eq. 1. s = clog (1+|dij|) (1) Where dij is the value for the difference at pixel ij Step3: The double-difference operator (also called three-frame difference) has been applied to improve the performance of background subtraction. It is able to detect moving points with surprisingly good location

P21/2

capability, and eliminate noise, which occurs due to camera movements and able to filter very small moving objects in the scene. Hereafter, we sketch the double-difference operator. Eq.2 defines the difference-image. Dri (i,j) = [ Iri(i,j) - I r,i-1(i,j) ] (2)

Step 4: The double-difference image is obtained by eq.3 which perform a logical AND between pixels belonging to two subsequent difference-images, threshold by T 1, if (Dri+1 (i,j) >T ) ( Dri (i,j) > T ) D` Dri (i,j) = 0 otherwise (3)

The non-zero values in these difference frames are either due to noise or moving objects. The intensity of the resultant image is improved by gamma correction; here different perimeters are tried to obtain best result.

2(a) Reference frame

2(b) Frame 4

2(c) key object extracted from frame 4 Figure 2: Example result of background subtraction Figure 2(c) is the resultant image having key object occurring after applying the background substraction algorithm. Some approaches [17] or [12], which used background subtraction method, constructed the reference by accumulating and averaging images of the target area for some time interval. As mentioned above, this is not a robust technique as it is sensitive to intensity variations [15],[16]. The segmentation algorithms usually start with the gray value difference image between two consecutive frames. In this paper the computation of images has performed on gray level images because the gray level of the images or region is having uniform intensity. The intensity change in data flow can very well be observed in gray level. Then the gamma correction function is applied to improve the contrast of the resultant image as shown in figure 3. 3 Experimental Setup We have applied the proposed scheme to the traffic video sequence in our experiments. Their resolution is 195 x 255 pixels. The color video sequence consists of about 10 minutes of video from a traffic intersection with approximately constant lighting conditions. The color video clips used for this study were originally digitized in AVI format at 16 frames/second. The original video frames were of size 390 rows 510 columns, 24bit color. To reduce computation time, we made our test video clips by extracting frames from these originals at the rate of 4 frames per second. For simplicity and real-time processing purpose, we transform the color video frames to gray scale images and resize them to half of the original size (195 rows 255 columns). A small portion

P21/3

of the traffic video is used to illustrate how the proposed framework can be applied in various kinds (i.e., TV commercials, movie, and documentary, TV dramas, and animation). This process can be done in real-time or off-line. 3.1 Experiment Results The enhanced background subtraction method to the video sequences is able to remove nonmoving components from a video sequence. The main principle is to create a reference frame of the stationary components in the image. The reference frame is subtracted from any subsequent images. Those pixels resulting from new (moving) objects will generate a difference not equal to zero .The double-difference operator is used, which is able to detect moving points with amazingly good location capability, and is proved robust to noise due to camera movements and able to filter very small moving objects in the scene. By using this background subtraction method the accuracy of segmentation is increased and this thus provides support for real-time processing. The segmentation results for a few frames are shown in Figure 2 along with the original frames adjacent to them. Related work has been done based on image processing and rule-based reasoning. Here we applied the gamma correction and tried different perimeters to increase the contrast of the resultant image. This proposed framework, however can deal with more complex situations with more accuracy. Some of the objects, which are closely located to each other, are extracted in single segment. In Figure 2, the frame in the left most column (Figure 2(a)) is the background reference frame. The second column (Figure 2(b)) shows the original frame. The figure 2(c) shows the images after background subtraction. The extracted key object is shown in the third column (Figure 2(d)). Figure 3 shows the result gamma corrected images with parameters 0.3, 1.5, 3, 2.1. The result shown in figure 5 shows the improved intensity of the resultant frame compared with original one.

Figure 3 gamma corrected frame with parameters 0.3, 1.5, 3, 2.1

Figure 4 Histogram before Equalization (a) and After Equalization (b)

P21/4

4. Conclusion This paper introduces an algorithm for key object extraction for segmentation in video sequences. The main assumption underlying by proposed algorithm is that a motion characterizes physical objects that is separate from background. The algorithm is based on background substraction method. Experimental results demonstrate that proposed technique can successfully extract moving objects from video sequence. The boundaries of the extracted objects are accurate enough to place them in different frames. If there is insufficient motion, the algorithm is able to perform the correct segmentation and show the position of object in each frame. In order to extract the key object(s) from the video sequence, background subtraction techniques are employed. Using the background subtraction technique, both the efficiency of the segmentation process and the accuracy of the segmentation results are improved achieving more accurate video indexing and annotation. The existing schemes try to extract key frames from the video shot, but in this proposed scheme is able to extract key object(s) instead of key frame(s). Here we proposed a simple and efficient technique for object segmentation in video sequence. It is able to detect moving points with surprisingly good location capability, and eliminate noise, which occurs due to camera movements. It is easy to implement and fast to compute since it uses the segmentation result of the previous video frame to speed up the segmentation process of the current video frame. The proposed work extract small moving component(s) from image sequence and therefore this opens scope for possible future work in the area of doing all operation in image processing, and the global motion estimation and compensation. By considering more than one reference frame in the segmentation process, the motion can be estimated by comparing each frame with more than one reference frame so as to detect all the information of video scene. This work opens scope for futher study on data mining work, clusttering and association mining on video data set. 5. Acknowledgement This work was supported by all India council of technical education AICTE),New Delhi,India under RPS grant F.No.-8022/RID/NPROJ/RPS-14/2003-04. References 1. Jiawei Han, Data Mining Techniques, DBLP, SIGMOD, 1996, pp: 545 [Conf] 2. Michael C. Burl, Mining Patterns of Activity from Video Data, Department of Computer Science University of Colorado, Boulder, First SIAM International Conference on Data Mining, 2002. 3 R. Mech, M. Wollborn, "A Noise Robust Method for Segmentation of Moving Objects in Video Sequences," p. 2657, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP'97) Volume 4, 1997. 4 Xiaodong CAI, F.H.Ali and E.Stipidis An Integrated framework for Robust and Fast Automatic Video Segmentation Communication Research Group School of Engineering and Information Technology University of Sussex, Publication Date: Feb,2002,Volume,12,Issue:2,On page(s): [122-129]
P21-5

5 P.Stanchev, IASTED, Using image mining for image retrieval, Cancum, Maxico, 214-218, conf., CSE, May 19-21,2003 6 QBIC project: querying images by content, using color, texture, and shape Carlton W. Niblack, Ron Barber, Will Equitz, Myron D. Flickner, Eduardo H. Glasman, Dragutin Petkovic, Peter Yanker, Christos Faloutsos, and Gabriel Taubin , April 1993, pp. [173-187] Paper No. P21 7 Teuner, A.Pichler, Hosticka, B.J. Fraunhofer Inst. of Microelectron. Circuits & Syst., Duisburg; Unsupervised texture segmentation of images using tuned matched Gabor filters, Image Processing, IEEE Transactions on, Jun 1995, Volume: 4, Issue: 6 On page(s): 863-870 8 Jia-Yu Pan and Christos Faloutsos, "Geoplot": Spatial Data Mining on Video Libraries. In Proceedings of the Eleventh International conference on information and,Knowledge, Management, CIKM,2002, Mclean, VA (near Washington DC), November ,pp [4-9], 2002

P21/5

9 YuriA. Ivanov, Aaron, F.Bobick, JohnLiu,Fast Lighting Independent Background Subtraction,DBLP, International Journal of Computer Vision, 2000, v:37, n:2, pp:199-207 [Journal] 10 C.W. Lin, Y.-J. Chang, Y.-C. Chen, and M.-T. Sun, Implementation of a real time object-based virtual meeting system, in Proc. IEEE Int. Conf. Multimedia and Expo, pp. [565-568], August 2001, Tokyo, Japan. 11 I. Haritaoglu, D. Harwood, and L.S. Davis, W4: Who? When? Where? What? A real-time system for detecting and tracking people, in Proc. IEEE Int. Conf. Automatic Face and Gesture Recognition, pp. [222227],1998, Nara, Japan. 12 A.Neri, S. Colonnese, G. Russo, P. Talone, Automatic moving object and background separation, Signal Processing 66 (1998), pp. [219-232]. 13 Y. Ye, J. K. Tsotsos, K. Bennet, and E. Harley, Tracking a person with pre-recorded image database and a pan, tilt, and zoom camera, in Proc. IEEE Workshop on Visual Surveillance, pp.[10-17], 1998. 14 Hat, M. Saptharishi, and P. K. Khosla, Motion detection and segmentation using image mosaics, in Proc. IEEE Int. Conf. Multimedia and Expo, pp. 1577-1580, July 2000, NY, USA. 15 Cheung, C. Kamath, Robust Techniques for Back-ground Subtraction in Urban Traffic Video", In Video Comm and Image Proc., SPIE Electronic Imaging, San Jose 2004 16. Ismail Haritaoglu, David Harwood, Larry S. Davis: A Fast Background Scene Modeling and Maintenance for Outdoor Surveillance. ICPR 2000: 4179-4183 17. T. Meier and K.N. Ngan, Automatic segmentation based on Hausdorff object tracking,.Doc. ISO/IEC JTC1/SC29/WG11 MPEG97/m2238, July 1997. 18.T. Meier, and K.N. Ngan, Automatic segmentation of moving objects for video object plane generation. IEEE Trans. on Circuits and Systems for Video Technology, Vol. 8, No.5, September 1998.

P21/6