For viewers to fully enjoy content such as movies, sports games and live events, home TVs not only need excellent picture quality, but also impressive sound quality. Understanding this, industry leaders are striving for 3D audio development to add a deeper level of immersion to entertainment experiences – bringing exciting action scenes to life and capturing the quietest details of ambient noise.
Simply put, 3D audio makes listeners feel like they’re really in the middle of the action. Typically only supported in specific locations, such as movie theaters or recording studios, 3D audio is making its way into homes to give viewers a new way to experience their favorite content.
Samsung Electronics’ advanced research institute Samsung Research has been striving to popularize 3D audio since 2020. To this end, Samsung worked with Google to jointly develop Immersive Audio Model and Formats (IAMF) – an advanced 3D spatial audio technology adopted by Alliance for Open Media (AOM)1 in October 2023. Samsung Newsroom sat down with the team that developed IAMF to hear the story of how the technology came to life.
▲ The Visual Technology Team at Samsung Research led the development of a standard for audio technology. (From left) SungHee Hwang, JeongHoon Park and WooHyun Nam
A new way to enjoy 3D sound at home
Ambient noise is everywhere – from the sound of shoes scraping on the pavement to the gentle hum of cars driving down the road. While this noise may seem unnecessary in media, it is essential in adding depth and a sense of reality. 3D audio works to seamlessly blend ambient noise with conversations and other sound effects to create a more vivid and authentic entertainment experience.
“3D audio makes you feel like you’re really in the heart of the action by adjusting the strength, movement and vibrancy of the sound,” said WooHyun Nam of the Visual Technology Team at Samsung Research. “This way, viewers are able to enjoy a fuller sound that captures the 3D aspects of the world around us.”
▲ SungHee Hwang from Samsung Research’s Visual Technology Team
Despite the benefits of 3D audio, technological limitations have made it difficult to apply the technology to home audio devices. “3D audio information from content cannot be properly interpreted by home audio systems such as TV speakers or soundbars, resulting in a somewhat limited audio experience that lacks detail from the original content,” said SungHee Hwang of the Visual Technology Team at Samsung Research.
To address this, Samsung worked diligently with Google to develop an audio solution that would allow viewers to experience content audio as intended. “If 3D audio data can be read by device manufacturers, they can adjust the sound in audio devices – allowing for immersive audio experiences with standard TV speakers or soundbars at home,” said Nam. “By adjusting the sound to fit the home device environment, listeners can experience the sound exactly as the creator intended without any distortion or loss in quality.”
A uniform standard is required to send and receive audio data seamlessly between creators and device manufacturers. “Samsung and Google’s respective expertise in devices and content made the companies ideal partners to create IAMF technology,” said JeongHoon Park, Executive Vice President and head of the Visual Technology Team at Samsung Research. “By joining forces to develop this unprecedented technology, we are paving the way for consumers to enjoy 3D audio in their homes.”
Three characteristics of IAMF technology: Vertical sound, AI-based sound and customized sound
IAMF technology offers three different functions that enhance the sound experience.
▲ WooHyun Nam from Samsung Research’s Visual Technology Team
1. Ability to express sound vertically
Previous open source audio codecs only supported horizontal audio expression. With IAMF technology, sound can now be expressed vertically, so the sound becomes more and more multi-directional. “IAMF makes sound more realistic by allowing listeners to hear sound in front, behind or to both sides and also above or below them,” Nam said. “As such, when IAMF technology is applied to home TV speakers and soundbars, it allows listeners to hear sounds, such as birds flying overhead, on their TV at home.”
2. AI-based scene analysis and 3D sound effects
IAMF uses artificial intelligence and deep-learning technology to analyze scenes and emphasize certain aspects of the content – adjusting sound levels for more enhanced sound throughout the viewing experience. “In TV and film, there are certain scenes where the soundtrack or background music is the main focus,” Nam said. “IAMF will balance the sound in these cases. Likewise, the technology will fine-tune the sound when there is character dialogue so the listener can focus on the conversation.”
In addition, the IAMF technology provides optimal sound despite changes in the device environment. “By adjusting the scene analysis audio data according to the device environment, IAMF technology enables listeners to enjoy the content’s original sound quality on standard home televisions,” he added.
3. Highly customized sound
Users will be able to freely adjust the sound according to their preferences with IAMF technology. Whether viewers want to enhance sound effects from an action scene or enhance dialogue, IAMF gives them the flexibility to customize content audio for a more personalized experience.
▲ IAMF analyzes data from content and allows viewers to adjust and customize sound settings. When watching a sports game, users can directly choose whether to emphasize the commentator’s voice or the sounds of the game itself.
3D audio across the industry with open source IAMF
Open source is essential for creating a unified standard across the industry. IAMF is the first open source audio technology standard adopted by the AOM – meaning that both companies and independent content creators across the industry can access the technology and expand its use.
“To allow people to freely create content with 3D audio technology, related technology must be open to all,” Nam said. “Providing a complete open source framework for 3D audio, from creation to delivery and playback, will enable even more diverse audio content experiences in the future.”
In the same way, Park highlighted how the IAMF technology will have a big impact on the soundscape going forward. “Because we live in an era dominated by content creation, IAMF will help lead, expand and transform the 3D audio ecosystem,” he said.
Success achieved through collaboration
Research at the IAMF began in 2020 and took nearly four years to complete. Through a combination of persistence and hard work, the team was able to achieve success.
“The project involved many days of nonstop work, and there were times when we had to work at night because of the time difference between our office and Google’s,” Hwang said. “Despite these challenges, I was proud at the end. It was a great opportunity to work with developers from other countries.”
The teams developed a sense of camaraderie along the way and built a solid foundation of trust and understanding. “A Google employee mentioned how this partnership with Samsung has been the most enjoyable collaboration he had done so far,” Park recalled. “We were able to complete this project because our teams trusted each other. I am very proud of and grateful for the efforts of our team members and their cooperation.”
▲ JeongHoon Park, Corporate EVP and Head of the Visual Technology Team at Samsung Research
In addition, Park praised each team member’s unique skills and expertise that helped contribute to the creation of the IAMF. “WooHyun Nam was instrumental in proposing the technology development ideas, while SungHee Hwang organized and documented our team’s ideas in a clear manner,” he said.
A vision for ‘Samsung Sound’
After its release, Samsung’s research team realized that the standardization of 3D spatial sound marked a new era in audio technology.
“Thanks to the IAMF, we can start researching technologies that will open up a world of audio possibilities,” said Nam. In line with this, Samsung’s research team is currently developing an advanced version of IAMF technology that can be applied to various sectors such as mobile devices, metaverse, video games and more.
▲ With their combined efforts and tireless persistence, the visual technology team at Samsung Research was able to create a new standard for 3D audio.
Given the success of the IAMF, Samsung’s research team is motivated to create better audio technology for consumers. Each of the team members shared their expectations for the future of 3D audio.
Nam touched on his desire to create audio technology that is more immersive. “I want to create a more advanced 3D audio technology that makes users feel like they are really in the scene of a movie, TV show or live event,” he said. “I also hope to continue this research until 3D audio is applied to Samsung smartphones.”
More broadly, Hwang discussed how he hopes Samsung will be able to create unparalleled audio technology that will place it among the ranks of other leading audio companies. “My goal is to develop a technology that consumers will be able to easily distinguish as ‘Samsung Sound’ when they hear it,” he said. “I am optimistic that the IAMF standard is a stepping stone towards this dream.”
“My hope is that Samsung’s audio technology will enable consumers to enjoy an upgraded audio experience that is on par with current visual experiences,” said Park. “I hope that content creators will use Samsung’s audio technology to make 3D audio content available. other than that we aim to create a supportive environment that enables researchers to take on bold and exciting challenges, such as expanding Samsung’s audio technology.”
Technological standards such as IAMF will make entertainment more immersive for viewers across the board. Samsung’s research team is paving the way for innovations that will change the future of audio.
1 A non-profit industry consortium formed to develop and share technology for multimedia delivery, allowing creators to use the technology without worrying about cost. It is operated by an alliance of 38 companies, including Samsung Electronics, Google, Amazon, Apple and Meta.
