Vehicle Tailgating solution using Deep Sort and DETR

This project provides a tailored solution for detecting and monitoring vehicle tailgating behavior within designated regions of interest (ROI) in video footage. By integrating DETR (Detection Transformer) for object detection and Deep Sort for accurate tracking and reidentification, the system processes video feeds at a compute-efficient rate of 1 frame per second (fps). The approach enables the detection of critical events, such as tailgating and vehicle stoppage, through customizable parameters that adjust per camera, making the solution highly adaptable to diverse monitoring environments.

Year
2019

Real World Applications

1. Highway and Urban Traffic Management

  • Application: Integrate tailgating detection into existing highway and urban traffic management systems to monitor and identify vehicles following too closely. The system could alert authorities to high-risk zones and recurring incidents, enabling targeted interventions.

  • Business Case: By reducing tailgating-related incidents, municipalities can lower accident rates, improve traffic flow, and enhance public safety, which reduces the burden on emergency response resources and lowers road maintenance costs due to fewer accidents.

2. Smart City Safety Solutions

  • Application: As part of a smart city infrastructure, tailgating detection could be installed at intersections, toll gates, or pedestrian crossings to enhance road safety. The system could send real-time alerts to drivers or trigger warning signals to prevent collisions.

  • Business Case: Cities can leverage this data to identify high-risk behaviors, reduce accident rates in congested or vulnerable areas, and improve overall urban safety. Enhanced road safety can attract businesses, boost tourism, and improve quality of life, increasing a city’s economic value and reputation.

3. Fleet Management and Logistics

  • Application: Logistics companies and fleet managers could integrate tailgating detection to monitor and improve driver behavior within their fleets. Tailgating alerts would prompt drivers to maintain safer distances, reducing the likelihood of accidents and damage to cargo.

  • Business Case: By minimizing accidents, companies can decrease insurance premiums, reduce vehicle repair costs, and prevent costly delivery delays. Safer fleets also protect brand reputation and can help attract and retain clients focused on safe, reliable services.

4. Insurance Risk Assessment and Premium Adjustments

  • Application: Insurers could use tailgating data to assess driving behavior and adjust premiums based on driver risk. The system could be used to reward safe drivers or to track behaviors that might indicate a higher accident risk, such as tailgating in high-traffic areas.

  • Business Case: Using tailgating data for personalized premiums allows insurers to better match premiums with individual risk levels, enhancing profitability. Rewarding safe driving behavior could also lead to improved client loyalty and a competitive edge in the insurance market.

5. Automated Toll Systems and Access Control

  • Application: Tailgating detection can be added to automated toll systems to prevent vehicles from closely following others through toll barriers or entry points without paying. The system could trigger alarms or capture images for enforcement.

  • Business Case: For toll road operators, preventing toll evasion through tailgating saves revenue and improves system efficiency. Access control systems at private facilities, like gated communities or parking structures, can use tailgating detection to maintain security and control over entry points.

6. Public Transportation and Autonomous Vehicle Safety

  • Application: For public transportation vehicles and autonomous shuttles, tailgating detection can monitor the distance maintained by following vehicles. The system could communicate warnings to the tailgating vehicle or adjust the speed of the public transport vehicle to increase following distance.

  • Business Case: By enhancing safety around public transportation or autonomous fleets, cities and companies can reduce accident risks, build public trust in autonomous solutions, and support the rollout of smart and self-driving vehicles.

7. Data Analytics for Policy and Safety Studies

  • Application: Traffic safety researchers and policymakers could analyze aggregated tailgating data to identify risk patterns and evaluate the effectiveness of new safety regulations. Insights from tailgating detection data could shape future road safety initiatives.

  • Business Case: Data-driven policies can lead to more targeted and effective traffic safety measures. By identifying and mitigating risk patterns, this approach contributes to reducing accident rates, leading to social and economic benefits from improved public health and safety.

Architecture Components

  1. Video Ingestion

    • AWS S3: Raw video feeds are uploaded to an S3 bucket, which triggers processing workflows. S3 is also used for storing processed output, logs, and metadata.

    • Amazon S3 Event Notifications: Configured to invoke a Lambda function whenever a new video file is uploaded to the S3 bucket.

  2. Processing Pipeline

    • AWS Lambda: An initial Lambda function is triggered by S3 upload events, kicking off the video processing pipeline. It sends the video file’s metadata to AWS Step Functions for sequential task processing.

    • AWS Step Functions: Orchestrates the processing workflow, managing steps such as video pre-processing, object detection, tracking, and tailgating analysis. Step Functions can handle asynchronous tasks and errors gracefully.

  3. Object Detection and Tracking

    • Amazon SageMaker: The DETR model for object detection and Deep Sort for tracking are deployed as a SageMaker endpoint. This endpoint is invoked to process each frame at 1fps, using the uploaded video data from S3.

    • Amazon SQS: Used to queue processing jobs if multiple videos need simultaneous analysis, ensuring a scalable, event-driven workflow without overloading SageMaker resources.

    • AWS Lambda (Processing): After each frame is processed, a Lambda function handles the logic to track vehicles, apply ROI-based filtering, and update tailgating status.

  4. Parameter Configuration and Management

    • AWS Systems Manager Parameter Store: Stores configurable parameters (like ROI_OVERLAP_THRESHOLD, stop_second_threshold, etc.) per camera setup. Lambda functions retrieve these values during processing, ensuring flexibility and ease of updating parameters.

  5. Data Storage and Monitoring

    • Amazon DynamoDB: Stores tracking data, metadata on detected tailgating events, and historical logs for each processed video. This database serves as a record for further analysis and auditing.

    • Amazon CloudWatch: Monitors logs and metrics for the entire workflow, including Lambda invocations, Step Functions performance, and SageMaker endpoints. Alarms can be set up for error conditions or performance issues.

  6. Notification and Alerting (Optional)

    • Amazon SNS: Sends real-time notifications or alerts (e.g., SMS, email) when a tailgating incident is detected. SNS topics are subscribed by relevant stakeholders or integrated systems.

    • Amazon EventBridge: Can be set up to trigger additional workflows based on specific event patterns, like sending alerts only if the tailgating threshold is exceeded multiple times within a set period.

Tailgating for cars inside a region of interest

System Workflow

  1. Ingestion and Processing:

    • Video files are uploaded to S3, triggering a Lambda function to initiate the Step Functions workflow.

    • Step Functions manage each step in the processing pipeline, invoking SageMaker for object detection and tracking analysis.

  2. Detection and Tracking:

    • Frames are processed through SageMaker to identify vehicles and track them using Deep Sort. Tailgating conditions are evaluated based on parameters stored in Systems Manager Parameter Store.

  3. Storage and Monitoring:

    • Processed data and tailgating event logs are stored in DynamoDB, while logs and performance metrics are sent to CloudWatch for monitoring.

    • EventBridge and SNS provide alerting for real-time incidents detected.