Localized Trajectories for 2D and 3D Action Recognition

in Sensors (2019)

The Dense Trajectories concept is one of the most successful approaches in action recognition, suitable for scenarios involving a significant amount of motion. However, due to noise and background motion ... [more ▼]

The Dense Trajectories concept is one of the most successful approaches in action recognition, suitable for scenarios involving a significant amount of motion. However, due to noise and background motion, many generated trajectories are irrelevant to the actual human activity and can potentially lead to performance degradation. In this paper, we propose Localized Trajectories as an improved version of Dense Trajectories where motion trajectories are clustered around human body joints provided by RGB-D cameras and then encoded by local Bag-of-Words. As a result, the Localized Trajectories concept provides an advanced discriminative representation of actions. Moreover, we generalize Localized Trajectories to 3D by using the depth modality. One of the main advantages of 3D Localized Trajectories is that they describe radial displacements that are perpendicular to the image plane. Extensive experiments and analysis were carried out on five different datasets. [less ▲]

Pose Encoding for Robust Skeleton-Based Action Recognition

in CVPRW: Visual Understanding of Humans in Crowd Scene, Salt Lake City, Utah, June 18-22, 2018 (2018, June 18)

Some of the main challenges in skeleton-based action recognition systems are redundant and noisy pose transformations. Earlier works in skeleton-based action recognition explored different approaches for ... [more ▼]

Some of the main challenges in skeleton-based action recognition systems are redundant and noisy pose transformations. Earlier works in skeleton-based action recognition explored different approaches for filtering linear noise transformations, but neglect to address potential nonlinear transformations. In this paper, we present an unsupervised learning approach for estimating nonlinear noise transformations in pose estimates. Our approach starts by decoupling linear and nonlinear noise transformations. While the linear transformations are modelled explicitly the nonlinear transformations are learned from data. Subsequently, we use an autoencoder with L2-norm reconstruction error and show that it indeed does capture nonlinear noise transformations, and recover a denoised pose estimate which in turn improves performance significantly. We validate our approach on a publicly available dataset, NW-UCLA. [less ▲]

Deformation Based Curved Shape Representation

Doctoral thesis (2017)

Representation and modelling of an objects' shape is critical in object recognition, synthesis, tracking and many other applications in computer vision. As a result, there is a wide range of approaches in ... [more ▼]

Representation and modelling of an objects' shape is critical in object recognition, synthesis, tracking and many other applications in computer vision. As a result, there is a wide range of approaches in formulating representation space and quantifying the notion of similarity between shapes. A similarity metric between shapes is a basic building block in modelling shape categories, optimizing shape valued functionals, and designing a classifier. Consequently, any subsequent shape based computation is fundamentally dependent on the computational efficiency, robustness, and invariance to shape preserving transformations of the defined similarity metric. In this thesis, we propose a novel finite dimensional shape representation framework that leads to a computationally efficient, closed form solution, and noise tolerant similarity distance function. Several important characteristics of the proposed curved shape representation approach are discussed in relation to earlier works. Subsequently, two different solutions are proposed for optimal parameter estimation of curved shapes. Hence, providing two possible solutions for the point correspondence estimation problem between two curved shapes. Later in the thesis, we show that several statistical models can readily be adapted to the proposed shape representation framework for object category modelling. The thesis finalizes by exploring potential applications of the proposed curved shape representation in 3D facial surface and facial expression representation and modelling. [less ▲]

Deformation Based Curved Shape Representation

in IEEE Transactions on Pattern Analysis and Machine Intelligence (2017)

In this paper, we introduce a deformation based representation space for curved shapes in Rn. Given an ordered set of points sampled from a curved shape, the proposed method represents the set as an ... [more ▼]

In this paper, we introduce a deformation based representation space for curved shapes in Rn. Given an ordered set of points sampled from a curved shape, the proposed method represents the set as an element of a finite dimensional matrix Lie group. Variation due to scale and location are filtered in a preprocessing stage, while shapes that vary only in rotation are identified by an equivalence relationship. The use of a finite dimensional matrix Lie group leads to a similarity metric with an explicit geodesic solution. Subsequently, we discuss some of the properties of the metric and its relationship with a deformation by least action. Furthermore, invariance to reparametrization or estimation of point correspondence between shapes is formulated as an estimation of sampling function. Thereafter, two possible approaches are presented to solve the point correspondence estimation problem. Finally, we propose an adaptation of k-means clustering for shape analysis in the proposed representation space. Experimental results show that the proposed representation is robust to uninformative cues, e.g. local shape perturbation and displacement. In comparison to state of the art methods, it achieves a high precision on the Swedish and the Flavia leaf datasets and a comparable result on MPEG-7, Kimia99 and Kimia216 datasets. [less ▲]

Visual and human-interpretable feedback for assisting physical activity

in European Conference on Computer Vision (ECCV) Workshop on Assistive Computer Vision and Robotics Amsterdam, (2016)

Physical activity is essential for stroke survivors for recovering some autonomy in daily life activities. Post-stroke patients are initially subject to physical therapy under the supervision of a health ... [more ▼]

Physical activity is essential for stroke survivors for recovering some autonomy in daily life activities. Post-stroke patients are initially subject to physical therapy under the supervision of a health professional, but due to economical aspects, home based rehabilitation is eventually suggested. In order to support the physical activity of stroke patients at home, this paper presents a system for guiding the user in how to properly perform certain actions and movements. This is achieved by presenting feedback in form of visual information and human-interpretable messages. The core of the proposed approach is the analysis of the motion required for aligning body-parts with respect to a template skeleton pose, and how this information can be presented to the user in form of simple recommendations. Experimental results in three datasets show the potential of the proposed framework. [less ▲]

Interpreting Thermal 3D Models of Indoor Environments for Energy Efficiency

in Journal of Intelligent and Robotic Systems (2014)

In recent years 3D models of buildings are used in maintenance and inspection, preservation, and other building related applications. However, the usage of these models is limited because most models are ... [more ▼]

In recent years 3D models of buildings are used in maintenance and inspection, preservation, and other building related applications. However, the usage of these models is limited because most models are pure representations with no or little associated semantics. In this paper we present a pipeline of techniques used for interior interpretation, object detection, and adding energy related semantics to windows of a 3D thermal model. A sequence of algorithms is presented for building the fundamental semantics of a 3D model. Among other things, these algorithms enable the system to differentiate between objects in a room and objects that are part of the room, e.g. floor, windows. Subsequently, the thermal information is used to construct a stochastic mathematical model - namely Markov Random Field - of the temperature distribution of the detected windows. As a result, the MAP (Maximum a posteriori) framework is used to further label the windows as either open, closed or damaged based upon their temperature distribution. The experimental results showed the robustness of the techniques. Furthermore, a strategy to optimize the free parameters is described, in cases where there is ample training dataset. [less ▲]