Team Name
Melody Masters
Timeline
Summer 2023 – Fall 2023
Students
- Adrian Ramos
- Edgar Hernandez
- Benjamin Farmer
- Patrick Ferguson
- Angel Aguirre
Sponsor
Shawn Gieser – UTA Senior Instructor
Abstract
The objective of this project is to provide an easier way for new musicians to learn how to play instruments. The project has a react front end which allows a user to upload music sheets and have them displayed on their device. This will also be accompanied by our Firebase backend to store user data and previously uploaded music sheets. With this the user can play any music from their instrument and the Web-app will follow along and automatically turn the music sheet being displayed to the next page. This will allow the musician to keep their hands on their instrument and focus on playing.
Background
Performing music is a craft that takes many hours to complete effectively. One challenge of playing a new song is reading and playing the sheet music repeatedly to understand what spots in the songs are more challenging to play than others. Also, memorizing the start and end of pages allows a smoother transition to turning the page while still playing. This makes the time to learn new music longer than if one could just focus on the notes that need to be played.
SoundSync will be a web-app that will follow along with the music the user is playing and progress through the sheet music in the web-app to allow the user to keep playing the song they are on. This will allow users to play a song at their own pace, learning how to play and focusing solely on reading and playing music. Most apps now can write sheet music based on what is being played or play sheet music for the user to hear. These are good for improving a different task at hand. Our objective is unique in allowing users to play the music they want and not worry about having a booklet of music sheets with them or the annoyance of turning the sheet music to keep playing during their performance. Our sponsor explained his passion for music and learning new songs. This web-app would allow our sponsor and many other music lovers to enjoy learning new music without the hassle of carrying sheet music or turning pages while playing.
Project Requirements
Page Turning/Sliding on Page End – This feature will automatically turn the page of the music sheet being displayed so the user can keep playing without taking their hands off their instrument. It will also do this before the user reaches the end of the page so the page transition doesn’t cause a gap in time were the user is waiting to see the next note they need to play.
Sheet Music Scanning – The web-app will be able to scan music sheets uploaded from the user device. The web-app will then display these music sheets so the user can play a song and follow along with the music sheet.
Live Note Detection – The service will be able to recognize different notes being played by the user, so it knows when to turn the music sheet page being displayed.
Software Installation – SoundSync will be a web based application. The user will go to the website and begin playing their instrument.
Sheet Music Scanning/Generation Speed – This functionality will allow users to scan music sheets that will be generated as a playable online version for the user to follow along without the need for the physical copy and stored for future sessions.
Note highlighting that follows the user’s playing – This feature will highlight the current note the user is on so they know where they are in a song.
Account Functionality – The application will allow the user to sign up/sign into an account that will then act as a personal database/inventory of the user’s previously scanned sheet music as well as their past performance statistics.
User Tutorial Provided on Application Launch – When launched for the first time, the web-app will give the user a tutorial to show them how to navigate the it and use its features. It will be re-accessible afterwards if needed.
Proper Encryption of Account Information – User-sensitive information such as passwords will be appropriately encrypted.
System Overview
The SoundSync web-app is comprised of five major components working together to allow for a seamless detection of notes being played while keeping track of positioning on a music sheet. Starting with the database layer that will keep track of users information and sheet music images uploaded. We will be using Firebase for this task as it makes authenticating and storing data easy. The UI layer will be everything the user can see and interact with. This is essential for our users to be able to interact with the web-app, by signing in, uploading pictures of sheet music, and accessing the tracker for following along with the users progression through a song. The Sheet Music Trackers purpose is to take data from the Sheet Scanner and Note Recognition layer and display to the UI a synced recognition of notes being played by the user, and turning the music sheets page so the user can keep playing freely. The Sheet Scanner layer takes images the user inputs and scans the page of sheet music to identify key features for the scanner to create a formatted page in the web-app with all the important information used to playing a song. The Note Recognition layer processes the audio of the instrument played by the user to detect notes and sends them to the Tracker for further use in comparing the notes on the music sheet.
Results
The music page turning web-app successfully identified single notes on the music sheet with great accuracy. Through its algorithm, it recognized and displayed individual notes on the music sheet, enabling music page flow and page-turning functionality for musicians. The web-app’s precise identification of notes ensured a smooth transition between musical passages, giving the user experience by providing efficient and timely page turns based on the musician’s pace and performance. Users found the app incredibly useful for uninterrupted musical performances, as it detected and responded to each note, giving a smooth and consistent playing experience.
Future Work
Automatic Music Sheet Search – This will allow the user to scan a music sheet and check the database for previously uploaded music sheets to recommend to the user. If the user selects a match it will replace the user’s scans with the already created match to use.
Manual Page Turning With Pedal – Integrating a Bluetooth pedal to be implemented for manual page turning rather than automatic page turning. This would be for the preference of the user if they would like to be in control of the page turning or for if the detection of notes is incorrect while in the process of playing.
The integration of wider support for different instruments would be ideal for the users. The ideal music tracker will be able to support an abundance of instruments for all kinds of musicians.
Mobile app – The portability of having a mobile app so the user can have the app and is able to play at a moments notice. This will greatly provide exposure for SoundSync and allow many more users to have access to the service.
Project Files
Project Charter:
System Requirements Specification:
Architectural Design Specification:
Detailed Design Specification:
Poster:
References
Bluetooth. (n.d.). Specifications. Bluetooth® Technology Website. https://www.bluetooth.com/specifications/specs/
Google. (n.d.). Server-side encryption | firestore | google cloud. Google. https://cloud.google.com/firestore/docs/server-side-encryption
Ingber, J. (n.d.). A review of two sheet music apps: SM Music Reader and Sheet Music Scanner. The American Foundation for the Blind. https://www.afb.org/aw/23/5/17937
Joomla. (2020). Home. MusicReader. https://www.musicreader.net/en/
Library, T. S. L. (n.d.). Guides: Recording laws: Audio recording. Audio Recording – Recording Laws – Guides at Texas State Law Library. https://guides.sll.texas.gov/recording-laws/audio-recording
Lin, J. (2017, June 12). Pageflip Firefly Bluetooth/USB page turner review. The Gadgeteer. https://the-gadgeteer.com/2017/06/12/pageflip-firefly-bluetoothusb-page-turner-review/
OSHA. (n.d.). Department of Labor Logo United Statesdepartment of Labor. 1926.416 – General requirements. | Occupational Safety and Health Administration. https://www.osha.gov/laws-regs/regulations/standardnumber/1926/1926.416
Playscore. (2023, May 25). PlayScore 2 sheet music scanner. PlayScore. https://www.playscore.co/
Texas Government. (2023, May 2). Privacy and security policies. Privacy and Security Policies – Texas Parks & Wildlife Department. https://tpwd.texas.gov/site/policies/privacy-and-security-policies
Wilson, D. (n.d.). Understanding scanning requirements. docufi.com. https://www.docufi.com/how-to-guides/understanding-scanning-requirements
