Isolated rapid eye movement (REM) sleep behavior disorder (iRBD) is, in most cases, an early stage of Parkinson’s disease or related disorders. Diagnosis requires an overnight video-polysomnogram (vPSG), however, even for sleep experts, interpreting vPSG data is challenging. Using a 3D camera, automated analysis of movements has yielded high accuracy. We aimed to replicate and extend prior work using a conventional 2D camera. The dataset included 172 vPSG recordings from a clinical sleep center, 81 patients with iRBD and 91 non-RBD healthy controls. An optical flow computer vision algorithm automatically detected movements during REM sleep, from which features of rate, ratio, magnitude and velocity of movements, and ratio of immobility were extracted. Patients with iRBD exhibited an increased number of shorter movements and immobility periods. Accuracies for detecting iRBD ranged from 84.9% (with 2 features) to 87.2% (with 5 features). Combining all 5 features but only analyzing short (0.1-2 second duration) movements achieved the highest accuracy at 91.9%. This approach could be implemented in clinical sleep laboratories to facilitate and improve the diagnosis of iRBD. Read more

Drones or UAVs, equipped with different sensors, have been deployed in many places especially for urban traffic monitoring or last-mile delivery. It provides the ability to control the different aspects of traffic given real-time observations, an important pillar for the future of transportation and smart cities. With the increasing use of such machines, many previous state-of-the-art object detectors, who have achieved high performance on front-facing cameras, are being used on UAV datasets. When applied to high-resolution aerial images … Read more

Scene graph generation (SGG) methods extract relationships between objects. While most methods focus on improving top-down approaches, which build a scene graph based on detected objects from an off-the-shelf object detector, there is a limited amount of work on bottom-up approaches, which jointly detect objects and their relationships in a single stage. In this work, we present a novel bottom-up SGG approach by representing relationships using Composite Relationship Fields (CoRF). CoRF turns relationship detection into a dense regression and classification task, where each cell of the output feature map identifies surrounding objects and their relationships. Furthermore, we propose a refinement head that leverages Transformers for global scene reasoning, resulting in more meaningful relationship predictions. By combining both contributions, our method outperforms previous bottom-up methods on the Visual Genome dataset by 26% while preserving real-time performance. Read more

Visual scene understanding is a fundamental building block for autonomous agents operating in dynamic environments such as transportation systems. This thesis explores the progression of visual perception from object detection to scene-level relationship understanding, with applications in transportation. We first address the challenge of re-identifying agents across non-overlapping camera views, proposing a confidence-based approach for visual re-identification. We then extend perception to aerial imagery for traffic monitoring, developing methods that adapt to the wide range of object scales in drone-captured images. Finally, we move beyond individual object detection to relationship modeling between entities in a scene, proposing Composite Relationship Fields (CoRF) for scene graph generation. The thesis presents a cohesive vision for how autonomous systems can understand visual scenes at multiple levels of abstraction, from detecting individual objects to reasoning about their relationships. Read more

Transportation systems often rely on understanding the flow of vehicles or pedestrians. From traffic monitoring at the city scale, to commuters in train terminals, recent progress in sensing technology make it possible to use cameras to better understand the demand, i.e., better track moving agents (e.g., vehicles and pedestrians). Whether the cameras are mounted on drones, vehicles, or fixed in the built environments, they inevitably remain scattered … Read more

Autonomous vehicles rely on an accurate perception module. One of the fundamental challenges is to efficiently track pedestrians surrounding a vehicle to anticipate risky situations. Over the past decades, researchers have formulated the tracking problem as a data association one where they proposed various representations aiming for invariance to nuisances such as viewpoint changes, body deformation, object occlusion, and illumination changes. However, these methods still suffer … Read more