Shutter Bugs

Posted on by Steven McDermott

Team Name

The Shutter Bugs

Timeline

Fall 2024 – Spring 2025

Students

  • Linus Kotinek – Computer Science
  • Carl Nguyen– Computer Science
  • Emilio Quinones – Computer Science
  • Hatim Piplodwala – Computer Science
  • Turza Saha – Computer Science

Abstract

The project aims to develop software that integrates modern computer vision for real-time event and object detection with cloud-based infrastructure for logging, analysis, and scalability while still maintaining or improving current performance levels of current NVRs. By incorporating these techniques and technologies, the system will offer a more efficient, intelligent, and secure video surveillance approach.

Background

Network video recorders (NVRs) are commonly used in video surveillance systems to provide a centralized system for video management. Many existing NVRs primarily focus on recording and storing video feeds from multiple cameras and offer remote access and basic motion detection. However, they are limited in accurate object/action detection capabilities for a more intricate real-time analysis and as a result, are also not as space efficient when integrating cloud-based storage if it is offered.

The integration of cloud-based storage allows for virtually unlimited storage (with a cost constraint), increased security of the stored data, and the ability to access the stored data from multiple locations or sites, making it a desirable component to have. However, there is still a cost behind the service, and video files are one of the more memory-intensive data types to store. As a result, users face the struggle of compromising between either saving costs or having enough recording data for maximum information keeping.

Integrating more advanced object/action detection models to solve this issue requires careful planning though since these models can be very processing power heavy.

Project Requirements

  • Set up and configure cameras to produce RTSP streams to allow for a wide range of data to be collected and high enough resolution for processing purposes
  • Manage several RTSP streams concurrently
  • Perform object and action detection on received RTSP streams
  • Process streams to keep only recordings specific to desired results
  • Generated a live written log of all detections
  • Enable cloud integration for detection results
  • Create a notification system to alert users of detected results
  • Create a GUI allowing for user interaction with the camera system
  • Create user settings to specify detection item(s), determine notification regularity, and allow for retrieval/deletion of files in the cloud
  • Maintain/improve performance levels of the NVR system

Design Constraints

  • Cost/economic: The cost of implementing the project should not exceed $800, which is the budget allocated to all CSE senior design teams by the CSE department. The amount of memory storage used and the amount of access with the cloud service should be limited or reduced to an acceptable manner to improve costs for the software’s users.
  • Interoperability: The project should be able to handle streams coming from any cameras capable of producing RTSP streams and be able to send notifications to emails coming from different services.
  • Maintainability: The project should be easily maintainable after its completion at the end of the final sprint. The physical hardware setup and configuration should be designed to be easily replicable in the case of unexpected damage occurring in the future. Software should be documented and organized in a manner that would allow for quick comprehension in case of future adjustments for adding new features, improving performance, or fixing bugs.
  • Schedule: The project must be completed within a limited timeframe. Scheduling conflicts must be resolved and allocated time for documentation purposes carefully planned so as not to interfere with time dedicated to developing the project. Project scope must be limited to what is required and compromises may have to be made on implementation methods to ensure project completion.
  • Usability: The project software should be easy for users to use without excessive documentation/research. It should also be able to perform data validation to prevent failure even when users misuse the software and handle long term use without facing significant performance loss.

Engineering Standards

  • Authentication & Encryption/Security Standards
    • The Federal Trade Commission (FTC) Act – Enforces privacy and security requirements under the FTC Act, particularly Section 5, which prohibits unfair or deceptive practices. This can apply to companies that fail to secure camera footage or personal data associated with the security cameras.
    • ISO/IEC 27001 – Provides a comprehensive framework for implementing information security controls to protect sensitive video data against breaches.
  • Common Engineering Standards
    • ISO/IEC 27017/27018 – Cloud-specific extensions to ISO 27001 that address security controls and privacy protection specifically for cloud services. These standards provide guidelines for securely implementing cloud storage for video data, addressing issues like data segregation, access management, and encryption in cloud environments.
    • IEEE 802.3 – Defines Ethernet standards for local area networks, establishing protocols for devices to communicate over wired connections at various speeds.
  • Programming/Web Dev Standards:
    • ISO/IEC 42010 – Establishes proper documentation of the system architecture, critical for a complex system integrating video processing, object detection, and cloud storage.
    • ISO/IEC 25010 – Defines quality characteristics for evaluating the software’s reliability, security, performance efficiency, and usability.
    • PEP 8 – Standard style guide for Python code which promotes readability and consistency across Python projects. This covers conventions for indentation, naming, line length, imports, whitespace and more.

System Overview

The Camera layer captures and transmits real-time video streams directly to the NVR layer, where they undergo processing. These cameras must be capable of providing RTSP streams of high enough resolution to later distinguish and categorize different entities within their field of vision. This layer ensures compatibility and seamless communication across the system by supporting protocols like RTSP. Key functions include streaming high-definition video to the NVR for processing.

The NVR layer takes the video streams from the cameras as input. Using this input, they feed the frames into an object detection model and/or a pose detection model which are then fed through the network interface into the livestream interface in UI. The processed streams are sent to the livestream interface to provide real-time object/action detection streaming to the user. The recorder interface formats the video streams into video files to be sent to the cloud storage with its associated logs to be available to the users. The network interface is used to send and receive data from different parts of the system through the Internet. We send the results through the Internet directly to the livestream interface in UI to provide real-time object detection streaming for the user.

The Cloud layer is responsible for securely storing, managing, and retrieving processed video recordings and detection data from the NVR system. This layer includes two primary subsystems: query management and storage. The query management subsystem handles the cloud queries made in the UI by the users. Users will have manual access to the cloud storage for retrieval/removal/download of detection logs with its associated video files for any detections that have been sent to the cloud. The storage subsystem handles the secure storage of video files and associated detection logs, ensuring data integrity. The data is organized in a structured manner to facilitate efficient retrieval, with video recordings and detection logs accessible to authorized users.

The UI layer serves as the user interface for the security camera NVR system which will provide essential features for the end users. The UI will include a livestream from all the cameras, customizable settings for notifications, detection preferences, and camera settings. Through cloud query features, users will be able to view and search detection logs, manually delete logs, and download specific logs and associated video files to their devices. Popup notifications will also be used to alert users of any detected objects.

Results

The project met the main requirements of our sponsor, which involved being able to process multiple RTSP streams for specific events and storing detection results into a cloud service for logging purposes. It was able to do so while improving upon the performance levels of the original NVR system by using optimization techniques such as hardware acceleration.

Future Work

In the future, the team hopes to:

  • Implement thermal imaging streams
  • Implement secure remote access to the system (app/web UI)
  • Include more action detection options for users
  • Make the cloud integration more robust and refined

Project Files

Project Charter
System Requirements Specification
Architectural Design Specification
Detailed Design Specification
Poster

References

Boto3 documentation — Boto3 Docs 1.16.56 documentation. (n.d.). Boto3.Amazonaws.com. https://boto3.amazonaws.com/v1/documentation/api/latest/index.html

OpenCV: OpenCV modules. (2024). Opencv.org. https://docs.opencv.org/3.4/

Qt for Python — Qt for Python. (n.d.). Doc.qt.io. https://doc.qt.io/qtforpython-6/

RLK8-1200D4-A. (2019). Reolink.com. https://reolink.com/us/product/rlk8-1200d4-a/

‌‌Ultralytics. (n.d.). Home. Docs.ultralytics.com. https://docs.ultralytics.com/#where-to-start

Steven McDermott

More Posts