AI-Powered Immersive Recording: REPERTORIUM Tests Spatial Audio with the Brandenburg State Orchestra Frankfurt

Picture of Repertorium Editorial Team
Repertorium Editorial Team
- 2 days ago

Exploring how artificial intelligence can transform the recording and experience of classical music

A groundbreaking recording session took place last weekend, as part of our project, with the Brandenburg State Orchestra Frankfurt (Oder) under conductor Simon Gaudenz, featuring bassoonist Hyunjin Kim.

The team recorded the Klassik am Sonntag concert at the Kleist Forum on Sunday, 26 October 2025, featuring works by Weber and Mendelssohn.

More about the concert: bsof.de

Exploring AI in Orchestral Recording

This session marks an important step in REPERTORIUM’s research into AI-assisted recording and spatial audio reconstruction. Using only a small number of carefully placed microphones, the project investigates how artificial intelligence can recreate a fully navigable sound field from minimal input.

The vision: listeners will soon be able to

  • Move virtually within the orchestra, positioning themselves beside the second clarinet or the timpani
  • Turn individual instruments up or down
  • Mute or solo specific parts and even play along themselves

While multi-track flexibility is common in pop music, bringing this level of control to live symphonic recording represents an entirely new artistic and technical frontier.

spatial audio - mics set up

Upcoming Interactive Audio Releases

In the coming weeks, selected excerpts from the Frankfurt (Oder) concert will appear on our website in an interactive audio player, allowing visitors to:

  • Solo, mute, or adjust the level of individual instruments and sections
  • Switch listening perspectives within the orchestra
  • Explore A/B examples that demonstrate the workflow — from the original recording to separated sources and the reconstructed sound field

These materials will make advances in sound-source separation and spatial reconstruction directly tangible for musicians and audiences alike.

Research, Education, and Artistic Applications

The recordings serve multiple purposes within the REPERTORIUM Project:

  • Research & Development: refining AI-driven systems for immersive classical-music listening
  • Education & Outreach: helping students, musicians, and producers understand how AI-based separation and reconstruction work using real orchestral material
  • Artistic Interaction: inviting creative engagement — for example, muting an orchestral line and playing it live as a personal practice or performance tool

Next Steps

A second test session will take place in November with the Lithuanian National Symphony Orchestra, continuing the evaluation and refinement of these methods.

By the end of the year, visitors to the REPERTORIUM platform will be able to explore these recordings first-hand — moving virtually through the orchestra and interacting with the music in ways that were previously impossible.

With Thanks

Our sincere thanks go to the Brandenburg State Orchestra FrankfurtSimon Gaudenz, and Hyunjin Kim for their openness and collaboration in this European initiative. Their partnership has made it possible to take another important step toward the future of immersive classical-music recording.

Sign up to our Newsletter to stay informed!

Get in touch if you have any questions!

The Project

REPERTORIUM uses AI to digitise ancient and classical manuscripts, preserve European musical heritage, and create state-of-the-art sound processing technologies, including metaverse-ready immersive audio. These technologies are the foundation of a general musical artificial intelligence that fully unleashes the powers of machine learning upon the domain of European classical heritage, advancing us towards a human-centred digital world.