Team Name
Interactive Living Stream
Timeline
Fall 2024 – Spring 2025
Students
- Bryan Ukeje – Software Engineering
- James Hofer – Computer Science
- Arham Ali – Software Engineering
- William Forbes – Software Engineering
Sponsor
USACE: Jason Knight – Outdoor Recreation Planner/Natural Resource Management Specialist at US Army Corps of Engineers, Tulsa District
Abstract
The Living Stream Project creates an interactive, aquatic life environment designed for
educational and public spaces such as visitor’s centers. Using a projector, a depth
sensor, and custom software, it simulates native fish species in their environment which
react to user movements.
Background
The Living Stream project is designed to create an interactive and educational
experience for both kids and adults. Traditional aquariums and exhibits often lack the
ability for visitors to actively engage with what they are seeing. This project changes
that by allowing people to interact with water and fish naturally. As people walk through
the projected water, they can see the fish react, either swimming away or staying still,
based on the type of fish. The native fish will stay while the other fishes will run away.
The goal is to teach visitors, especially children, how different aquatic species behave in
real life. The fish included in the project are native to Northeast Oklahoma, such as
Smallmouth Bass, Saugeye, Bluegill, and Muskie. The environment may also feature
turtles, frogs, and plants to make the experience feel more lifelike. By introducing these
elements, we hope to provide a fun yet informative way for children and adults to learn
about the aquatic environment.
This project is being developed in close partnership and effective communication with
the U.S. Army Corps of Engineers (USACE), whose primary focus is education. They
want to provide a tool that helps people learn about aquatic life in an engaging way. Our
team’s job is to make sure that the final product not only looks visually appealing but
also effectively meets the educational goals set by USACE. By focusing on both
interactivity and realism, we aim to deliver a tool that immerses visitors in an
educational experience, unlike typical nature exhibits.
Our team is working together with USACE through virtual meetings to keep the design
simple but effective. For example, we discussed using Unity for development because
it’s easier to work with for this project and is compatible with the computer system they
already have in place. Unity will allow us to incorporate real-time responses from the
environment, making it possible for visitors to trigger animations, such as water
splashes or fish movements when they walk through the projection area.
The area where the projection will happen is about 6 x 10 feet, and it will feature a
variety of natural elements like plants, fishes, and turtles displayed on the floor. We will
carefully consider lighting conditions to ensure the projection looks vibrant, even in a
room that allows some natural light. While the design will avoid unnecessary complexity,
it will still capture the attention of visitors through the natural behaviors of fish and other
aquatic life, making the experience both simple and memorable. The collaboration
between our team and USACE ensures that the final product balances educational
value with an engaging user experience.
Project Requirements
- Single projector overhead
- Short throw projector with low input lag
- Projection visibility must be clear for users to interact with
- Single depth sensor overhead
- Depth sensor must track multiple bodies within projected box
- Fish species native to Oklahoma area
- Create environment for fish (coral, rocks, plants, etc.)
- Create realistic fish movements
- Human and fish collision must create interaction
- Include water animations (ripples, splashes, etc.)
- Projection must fit within roughly 6×10 ft area
Design Constraints
- Constructability / Manufacturability: The software packaging must be simple
to deploy once delivered in the zip file. The zip should contain all necessary files.
The only possible concern would be the program file being too large to deliver, or
certain files not being able to be zipped. - Interoperability: During maintenance there is a possibility of changes impacting
the connectivity and effectiveness of the hardware systems. - Functionality: Hardware must continue to maintain efficient performance and
functionality after maintenance of different parts. - Schedule: The software components must be packaged and delivered on time to
meet project deadlines. - Usability: The process for extracting and running the software from the zip file
must be straight-forward. Ideally, the user should only need to extract the files
and run an executable, without needing to configure complex settings or
environment variables. Clear usage instructions should be included in the zip file.
Engineering Standards
- IEEE 830: Software requirements specification
- PEP 8: Python Programming Language
- ISO/IEC 21320-1: Core Zip File Format Requirements
- ISO/IEC 25010: Software performance quality
- ISO 23270: C# Programming Language
System Overview
The software architecture for this interactive, aquatic life simulation is designed in
layered sections within Unity for laptop hardware. Each layer and its subsystems are
designed to serve specific functions with inter-connectivity to create the entire system.
At the core, Unity receives depth sensor data from the Python app through the UDP
connection, which captures depth data from the Intel RealSense D435i sensor, enabling
real-time interactivity. This sensor data feeds into the simulation background processing
layer, which interprets and updates the 3D digital simulation including fish movement
and splash effects. Once the simulation is started, visual elements are rendered through
Unity’s HDRP Render Pipeline and sent to a projector for display. While the simulation is
active, the Play window will be displayed on the projector, having an option to let the
user stop the simulation at any time.
Results
The Living Stream project successfully delivers an interactive projected river scene where fish swim naturally and react to footsteps with water splashes and movement. We implemented a full end-to-end system with marker-based calibration, YOLO-based object tracking, and UDP communication, all managed through an easy run.bat setup. The final Unity scene executable includes custom built native fish using blender, and splash animations, and environmental visuals using Unity designed for public educational spaces.
Future Work
Our design meets the client’s essential goals by delivering an interactive projection that
reacts in real time as users walk through the stream, with water splashes and the
required native fish models moving away as needed. We learned that precise
calibration, consistent lighting, and sensor placement are crucial for accurate depth
sensing.
For future work, we plan to incorporate additional fish models. We appreciate the
support from our professor, our sponsors, and all team members who contributed to this
project.
Project Files
Project Charter
System Requirements Specification
Architectural Design Specification
Detailed Design Specification
Poster
References
- Body Tracking for iOS, Android, macOS, Windows, and Linux | LightBuzz — lightbuzz.com. https://lightbuzz.com/. [Accessed 24-09-2024].
- Dynamic Floor | Interactive Floor Projector | Breeze Creative — breezecreative.com. https://www.breezecreative.com/dynamic-floor. [Accessed 24-09-2024].
- LUMOplay | Interactive Floor and Wall Games — lumoplay.com.
https://www.lumoplay.com/. [Accessed 24-09-2024]. - MotionMagix⢠Interactive Wall & Floor Technology — touchmagix.com.
https://www.touchmagix.com/interactive-floor-interactive-wall. [Accessed 24-09-2024]. - Nuitrack Full Body Skeletal Tracking Software — nuitrack.com.
https://nuitrack.com/. [Accessed 24-09-2024]. - The ultimate mobile video coaching platform.
https://onform.com//. [Accessed 24-09 2024].
