The entertainment industry is harnessing technological advancements to offer increasingly immersive experiences, from performing arts to concerts to live sports. In a notable example, American rapper Eminem and Korean pop boy band NCT 127 hosted virtual concerts on Fortnite and Roblox, respectively, showcasing the industry’s embrace of innovative platforms.
Audiences worldwide could remotely access the metaverse concerts, which delivered special effects that were difficult to replicate in real life and games, such as hitting notes in time to the song performed as if they were playing Guitar Hero.
However, not everyone is keen on attending metaverse concerts as they cannot fully replicate the sensory experience and the crowd’s energy of offline performances. Mixed reality could help here. It could transform the in-person concert experience by combining elements of virtual reality (VR) and augmented reality (AR) to integrate digital content into the real world seamlessly.
Coldplay is among the musicians experimenting with mixed reality. The band’s Music of the Spheres World Tour(Mots) app allows fans to have AR aliens from its Higher Power music video dancing in their chosen environment (like in the concert venue or streets) and share videos or photos of it on social media.
An immersive and intimate experience
VR could also enable a new way of experiencing performing arts. Aventura, a Singapore-based creative tech studio, works with local dance companies to produce immersive shows that allow viewers to feel they are in the best seat in the house or watching a private performance.
See also: Alibaba anoints new chief in revamp of stalling commerce arm
Earlier this year, Avventura showcased five VR dance projects during the “PressPlay” biennial arts festival by the National Library Board Singapore. This resulted from the collaboration with Bhaskar’s Arts Academy, Singapore Chinese Dance Theatre and Maya Dance Theatre. A total of 172 people attended the showcase, and participants shared positive responses to the VR dance projects.
Since VR is still a nascent technology, developing VR dance performances is no easy feat. Over the Rainbow had the longest development of the five VR dance projects. The process took about six months — from engaging the Maya Dance Theatre to kickstart the creative process, to recording the dance performance, to developing and refining the spatial VR experience.
“This spatial VR experience demanded a comprehensive array of tasks including motion capture recording; the creation of digital avatars, costumes and environments; development in Unity (a 3D software development platform); and meticulous planning of physical props and spaces,” Christopher Chew, founder of Avventura, tells DigitalEdge.
See also: Break up Google? What’s at stake in antitrust action
He shares the general challenges of producing VR shows, too. “VR camera technology is still in its early days and is currently restricted in terms of choice and professional quality compared to conventional video cameras. This demands different shooting and lighting requirements. Moreover, live VR streaming is bandwidth-intensive and will be challenging at current Internet speeds — both upstream and downstream,” he says.
The good news is that he believes these will be resolved with the advent of 5G and as more advanced hardware and better price points are introduced in the next few years.
The human performer is also crucial for delivering an immersive and intimate VR experience. Chew explains: “The performer must perform for an audience of one instead of many. This requires appropriate eye contact and choreography that acknowledge where the audience is located. Most performing artists are used to performing on stage to an audience in a theatre. There needs to be a change in mindset and tweaks in stage directing and choreography to account for the need for the performance to be more personal.”
To develop a VR dance performance, Avventura had to conduct a motion capture recording and create digital avatars, costumes and environments, among other tasks. Photo: Avventura
Greener concerts
Although music lovers are delighted to have live concerts, tours and festivals back in full force, the events are heating the planet. The Tyndall Centre for Climate Change Research reveals that live concerts and performances produce 405,000 tonnes of greenhouse gas emissions yearly. Concert venues account for 34% of emissions, while audience travel contributes 33%.
To stay ahead of the latest tech trends, click here for DigitalEdge Section
Recognising this, Coldplay’s Mots app provides fans with insight into the planetary impact of their travel to and from the show, guiding them to make more sustainable choices. It uses SAP Business Technology Platform (BTP) to combine application development and automation, data and analytics, integration and artificial intelligence (AI) capabilities in one unified environment.
“BTP calculates the carbon footprint of the concertgoer’s journey and the number of trees to be planted to offset this. Additionally, SAP Analytics Cloud (SAC), running on BTP, aggregates and analyses the fan travel data gathered through the app. We helped Coldplay set up a SAC dashboard, which has a variety of tables and charts for valuable insights into fan travel,” shares Dr Kerstin Verzano, development lead of the Coldplay App and head of SAP Academy for Engineering at SAP SE.
He continues: “For instance, the Coldplay team can determine the proportion of fans who used public transport, the average number of people who carpooled, and how these factors vary between tour legs, countries and years. [In short,] SAP enables Coldplay to gain insights by powering the app and providing data in an aggregated way on SAP Analytics Cloud. The Coldplay team conducted the analysis, conclusions, and results achieved so far.”
Due to Coldplay’s proactive efforts to reduce its direct carbon emissions (from show production, freight, and band and crew travel), the Mots tour has produced 47% less carbon dioxide equivalent emissions than the band’s previous stadium tour in 2016 and 2017. This figure is expected to be higher if it includes the impact of the Mots app, which encourages fans to take more environmentally friendly commute options.
The Mots app offers an AR experience and enables fans to take more sustainable travel options to Coldplay concert. Photo: SAP and Coldplay
Uninterrupted real-time live sports
Technology has also made sports events more accessible by enabling sports competitions to be streamed live to fans worldwide. Fans expect uninterrupted and low-latency streaming of clear visuals to match the experience of those who are physically present at the sports event. This can be especially challenging in Formula One (F1), where every second counts.
“Viewers expect seamless, uninterrupted race coverage, regardless of external challenges. Tata Communications has built a bespoke low-latency and high-performance network worldwide to connect all F1 tracks back to the F1 remote operations centre at Biggin Hill in the UK,” says Dhaval Ponda, global head of the Media and Entertainment Business at Tata Communications.
He adds: “This network stands as one of the most resilient in the world, offering failover protection across multiple paths and high throughput capabilities to ensure minimal overheads and maximum utilisable bandwidth.
“Additionally, it boasts industry-leading monitoring and routing capabilities. We can seamlessly re-route data through real-time path-switching capabilities without impacting end-users even in weather or geological disruptions.”
Tata Communications is tapping AI to ensure continuous real-time transmission of every F1 race. “Through AI-powered analytics, we can continuously assess network performance, identify potential issues, and take proactive measures to optimise performance and ensure uninterrupted transmission. We’re also partnering Nvidia to establish a dedicated lab for AI/machine learning-based advanced network routing and cloud video production solutions [for F1],” says Ponda.
Every race weekend, more than 100 video feeds and over 250 audio channels are transferred between F1 tracks and the F1 remote operations centre in the UK via Tata Communications’ networks. Photo: Tata Communications and Formula One
Deepening fan engagement
F1 works with Salesforce to create highly personalised experiences to further strengthen fans’ connection to the sport. “There are 500 million F1 fans globally, yet less than 1% of them will attend a Grand Prix each year. Connected data and systems ensure that F1 can reach fans effectively and increase the accessibility and opportunity for fans to interact with the sport in their preferred format,” says Sujith Abraham, Salesforce’s senior vice president and general manager for Asean.
He explains that F1 employs Salesforce’s AI CRM platform, Einstein 1, to unify fan interactions from various sources, such as F1 TV, race-day pop-up stores and social media. This integration enables the seamless use of real-time fan data through Salesforce Data Cloud for personalised experiences.
“For instance, when fans attend an event through the F1 Paddock Club, F1 brand representatives can immediately know their names, favourite drivers and past purchases. They can also personalise follow-up communications based on their knowledge of each fan’s driver or merchandise preferences. By breaking down data silos and building a unified view of data across in-person and digital interactions, F1 can deliver more meaningful fan engagement and build fan loyalty,” says Abraham.
Beyond fan engagement, Salesforce is also helping F1 to achieve its sustainability goal. “With F1 fans expressing their expectations for responsible entertainment, sustainability remains paramount. F1 uses Net Zero Cloud to monitor its carbon footprint, gain actionable insights and strengthen its sustainability efforts. This ability to track, analyse and report on carbon emissions is integral to F1’s journey to net-zero by 2030,” he says.
With Salesforce, F1 has gained a unified view of data across in-person and digital interactions to deliver more meaningful fan engagement and build fan loyalty. Photo: Salesforce and Formula One
With ongoing technological advancements, the entertainment industry has a prime chance to leverage them, creatively offering fresh and enriched experiences. The only boundary? Our imagination.
