Our colleagues at the University of Jaén recently participated in the European Researchers’ Night, where they gave a presentation called ‘Application to Generate Karaoke files using AI Description’. They discussed their use of the cutting-edge source separation techniques developed as part of the REPERTORIUM project to isolate and extract instrumental components from a mix in the context of karaoke tracks, revealing that this technology allows for the generation of a clean background track, free of vocals, which forms the basis for a karaoke file.
A karaoke experience isn’t just about the music – it’s also about displaying the lyrics in time with the song. For this, the team used Whisper to transcribe spoken and sung lyrics with high accuracy, adding precise time markers for each word or phrase. To further enhance the karaoke file by integrating the transcription of the sung notes, Basic Pitch is used to detect the pitch of the vocal performance and match it to the corresponding musical notes. This level of detail helps to provide an even more immersive experience for users who want to sing with accuracy or practice their vocal skills.
To demonstrate these innovations, the UJA team presented an interactive web environment where users can request the generation of karaoke tracks online. An audio file is uploaded, and the system automatically generates karaoke tracks in various formats compatible with UltraStar, a popular karaoke platform. With this tool, users can create karaoke tracks in minutes, whether for personal use, parties, or even for vocal training. The process is entirely automated, making it easy for anyone – regardless of technical skill – to generate high-quality karaoke files.
The result is a fully immersive, user-friendly experience that aligns lyrics and melodies with pinpoint precision. The system’s compatibility with multiple karaoke formats also ensures broad usability across various platforms, simplifying what was once a complex and time-consuming process, bringing karaoke creation into the hands of anyone with an internet connection. So, whether you’re a casual karaoke enthusiast or a serious vocalist, this platform offers a new level of convenience and customisation – and it began with REPERTORIUM!