Home People Research Publications Demos
         
News Jobs Prospective
Students 		About Internal

Understanding Video of Crowded Environments

The automated monitoring and surveillance of crowded scenes is a remarkable challenge for current image and video understanding technology. It has application in areas such as homeland security, natural disaster prevention, research on insect behavior, and monitoring of animal populations, among others. It has recently acquired strong societal significance, due to the possibility of terrorist attacks on events involving large concentrations of people, a problem for which there are currently no effective solutions. This project lays the foundation for the technology that will enable the automated monitoring and surveillance of crowded scenes, by modeling their video as a visual texture that deforms itself in stochastic but predictable ways, in response to certain events. In particular, the project aims to produce 1) a suite of generative probabilistic models for the video produced by various types of crowded scenes, and optimal algorithms for the estimation of their parameters, 2) a family of classifiers that build on these models to design detectors of important events, 3) a collection of algorithms for crowd video stabilization, segmentation, and parsing, and 4) a large database of video examples, that will establish a common experimental framework for the evaluation of future research in the field. Educationally, the project will provide research opportunities to both undergraduate students and students of underrepresented backgrounds.

Recent
Results:
  • Results on PETS 2009 dataset. [demo]
  • Estimating crowd size in hour-long video sequence. [video]
  • Segmentation of an hour-long video of pedestrian traffic. [video]
    •
    Project/
    Results:
    Pedestrian Crowd Counting
    We estimate the size of moving crowds in a privacy preserving manner, i.e. without people models or tracking. The system first segments the crowd by its motion, extracts low-level features from each segment, and estimates the crowd count in each segment using a Gaussian process.
    [project | demo | PETS2009 demo]     		    

    Classification and Retrieval of Traffic Video
    We classify traffic congestion in video by representing the video as a dynamic texture, and classifying it using an SVM with a probabilistic kernel (the KL kernel). The resulting classifier is robust to noise and lighting changes.
    [project | demo]     		    

    Modeling video with Mixtures of Dynamic Textures
    We introduce the mixture of dynamic textures, which models a collection of video as samples from a set of dynamic textures. We use the model for video clustering and motion segmentation.
    [project | demo]     		    

    Databases: We have gathered several databases of crowded environments.

    database purpose link
    Highway Traffic classification [tgz 60MB]
    Motion Database classification [link Under Construction]
    Highway Traffic clustering [zip 42MB]
    Synthetic Video Textures   segmentation [zip 212MB]
    Pedestrian Crowds segmentation, counting   [zip 755MB | readme |
    CVPR annotations zip 2.6MB]

    Publications: Crowd Analysis

    Analysis of Crowded Scenes using Holistic Properties
    A. B. Chan, M. Morrow, and N. Vasconcelos
    In 11th IEEE Intl. Workshop on Performance Evaluation of Tracking and Surveillance (PETS 2009),
    Miami, June 2009.     © IEEE [pdf]

    Privacy Preserving Crowd Monitoring: Counting People without People Models or Tracking
    A. B. Chan, Z. S. J. Liang, and N. Vasconcelos.
    In, IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
    June 2008.     © IEEE [ps][pdf]

    Probabilistic Kernels for the Classification of Auto-regressive Visual Processes
    A. B. Chan and N. Vasconcelos,
    Proceedings of IEEE Conference on Computer Vision and Pattern Recognition,
    San Diego, June 2005.     © IEEE, [ps][pdf] (A longer version is available [ps][pdf]).

    Classification and Retrieval of Traffic Video using Auto-regressive Stochastic Processes
    A. B. Chan and N. Vasconcelos,
    Proceedings of 2005 IEEE Intelligent Vehicles Symposium,
    Las Vegas, June 2005.     © IEEE, [pdf].

     Dynamic Textures

    Layered Dynamic Textures
    A. B. Chan and N. Vasconcelos
    To appear in IEEE Trans. on Pattern Analysis and Machine Intelligence: Special Issue on Probabilistic Graphical Models in Computer Vision,
    2009.     © IEEE [ps][pdf]

    Variational Layered Dynamic Textures
    A. B. Chan and N. Vasconcelos
    In IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
    Miami, June 2009.     © IEEE [pdf]

    Modeling, clustering, and segmenting video with mixtures of dynamic textures
    A. B. Chan and N. Vasconcelos.
    IEEE Trans. on Pattern Analysis and Machine Intelligence,
    Vol. 30(5), pp. 909-926, May 2008.     [ps][pdf].

     Classifying Video with Kernel Dynamic Textures
    A. B. Chan and N. Vasconcelos
    Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
    Minneapolis, May 2007.     [ps][pdf]

    Layered Dynamic Textures
    A. B. Chan and N. Vasconcelos,
    Proceedings of Neural Information Processing Systems 18 (NIPS),
    pp. 203-210, Vancouver, December 2005.     [ps][pdf]

    Mixtures of Dynamic Textures
    A. B. Chan and N. Vasconcelos,
    In IEEE International Conference on Computer Vision, Proceedings
    October 2005.     © IEEE, [ps][pdf].

     The EM algorithm for mixtures of dynamic textures
    A. B. Chan and N. Vasconcelos,
    Technical Report SVCL-TR-2005-01    , March 2005.     [ps][pdf].

     Efficient Computation of the KL Divergence between Dynamic Textures
    A. B. Chan and N. Vasconcelos,
    Technical Report SVCL-TR-2004-02    , November 2004.     [ps][pdf]
    Contact: Antoni Chan, Nuno Vasconcelos

    



    © SVCL