Sound Rendering(声音渲染)
合集下载
相关主题
- 1、下载文档前请自行甄别文档内容的完整性，平台不提供额外的编辑、内容补充、找答案等附加服务。
- 2、"仅部分预览"的文档,不可在线预览部分如存在完整性等问题,可反馈申请退款(可完整预览的文档不适用该条件!)。
- 3、如文档侵犯您的权益，请联系客服反馈,我们会尽快为您处理(人工客服工作时间：9:00-18:30)。
• Scientific & Engineering applications
• Room acoustic prediction • Electro-acoustic prediction • Simulation of engineering structures • Noise modeling
Modeling
Specular Reflection Scattering Diffraction Refraction Doppler Effect Attenuation Interference
Propagation
Late Reverberation Personalized HRTFs for 3D sound Digital Signal Processing
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Sound Rendering: Applications
• Interactive applications
• Computer gaming • Virtual environments & training • Music processing • User interfaces • Multimedia applications
Interpolation for DynamiБайду номын сангаас Scenes
3D Audio Rendering
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Sound Rendering 101: Simulation Pipeline
• Construct a 3D Model
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Graphical Rendering: Recent Developments
Visual rendering
▪ Rasterization with programmable shading ▪ Ray tracing
3. It may not “sound” real
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Sound Propagation
▪ Numerical Methods
▪ Solve Helmholtz Wave Equation ▪ Accurate computation ▪ Compute intensive (increases as fourth power of frequency) ▪ Not practical for interactive applications (yet!)
GPGPU & Many-core computing
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Acoustics vs. Graphics
▪ Speed: 340 m/s vs. 300,000,000 m/s
▪ Frequency: 20 to 20K Hz vs. RGB
Sound Synthesis & Propagation
1. As compared to visual rendering, sound rendering is rather primitive
2. Interactive applications:
♦ Use pre-recorded sound sources ♦ Fixed models of propagation
▪ Wavelength: 17m to 17cm vs. 700 to 400 nm
Specular Reflection Scattering Diffraction Refraction Doppler Effect Attenuation Interference
Propagation
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
• Create an impulse response from path data (time, strength, direction, spectrum)
• Convolve IR with anechoic recording or synthesized sounds
• Auralize
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Sound Rendering: Collaborators
• Faculty ♦ Prof. Ming C. Lin
• Students ♦ Lakulish Antani ♦ Anish Chandak ♦ Christian Lauterbach ♦ Will Moss ♦ Nikunj Raghuvanshi ♦ Zhimin Ren ♦ Micah Taylor ♦ Hengchin Yeh (Dongenk University) ♦ Yero Yeh
perception ▪ Improve the sense of immersion ▪ Development of auditory displays
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Sound Rendering
• Part of multi-modal interfaces • Harness the sense of hearing • Sound rendering can augment visual rendering
• Collision Detection and Proximity Queries • Motion Planning and Multi-agent simulations • Sound Synthesis and Propagation • Haptics and Interactive Applications • Interactive Ray Tracing • Physically-based Modeling and Animation • GPGP: General-Purpose Computations using Graphics Processors • Geometric and Solid Modeling • Walkthrough of Massive Models
Normalized Magnitude (dB)
©Arup Acoustics
6 0 -6 -12 -18 -24 -30 -36 -42 -48 -54 -60
0
100 200 300 400 500 600 700 800 900 1000
Time (ms)
• Simulate sound propagation to find valid paths from source(s) to receiver(s)
▪ Methods
▪ Image Source [Borish,1984] [Dalenback,1992] ▪ Ray Tracing [Krokstad,1968] [Kuttruff,1993] ▪ Beam Tracing [Funkhouser et al.1998] [Funkhouser et al. 1999] ▪ Phonon Tracing [Kapralos,2004] [Bertram,2005] ▪ Frustum Tracing [Lauterbach et al. 2007] [Chandak,et al. 2008][Taylor et al. 2009] ▪ Acoustic Radiance Transfer [Siltanen et al.,2007] [Siltanen et al. 2009] ▪ Conservative Frusta Tracing (CFT) [Chandak et al. 2009]
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Sound Rendering 101
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
©Arup Acoustics
Sound Rendering: An Overview
Current GPUs: Interactive performance
• Desktop applications • Handheld applications
The GPUs are perhaps the first commodity massively parallel processors
frequency
• Petaflop computation needed for simulating at medium-level (1K) frequencies
• Solving high-frequency wave equation is a challenging scientific problem
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Geometric Sound Propagation
▪ Geometric Propagation
▪ Ray-Approximation of Wave Equation ▪ High-frequency approximation ▪ Getting fast enough for interactive applications ▪ Highly dependent on the geometry details
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Sound Propagation: Challenges
• Sound propagation: solve the Wave equation • Complexity is roughly O(ω4+δ), where ω is the
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
What is Sound Rendering?
▪ To generate sound effects for simulated environments ▪ Sound provides spatial cues and improves scene
▪ Methods
▪ Finite Element Methods [Otsuru,2004] ▪ Boundary Element Methods [Ciskowski,1993] ▪ Finite Difference Time Domain [Kunz et al.1993] ▪ Digital Waveguide Mesh (DWM) [Savioja et al. 1994] ▪ Domain Decomposition [Raghuvanshi et al. 2008]
UNC GAMMA Group
• Faculty ♦ Ming C. Lin ♦ Dinesh Manocha Around 20-25 students and postdocs
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
UNC GAMMA Group: Current Projects
Interactive Sound Rendering: Why we need more cores
Dinesh Manocha
University of North Carolina Chapel Hill
/Sound
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Acoustic Geometry
-- surface simplification
Acoustic Material
-- absorption coefficient -- scattering coefficient
Source Modeling
-- area source -- emitting characteristics -- sound signal
• Room acoustic prediction • Electro-acoustic prediction • Simulation of engineering structures • Noise modeling
Modeling
Specular Reflection Scattering Diffraction Refraction Doppler Effect Attenuation Interference
Propagation
Late Reverberation Personalized HRTFs for 3D sound Digital Signal Processing
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Sound Rendering: Applications
• Interactive applications
• Computer gaming • Virtual environments & training • Music processing • User interfaces • Multimedia applications
Interpolation for DynamiБайду номын сангаас Scenes
3D Audio Rendering
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Sound Rendering 101: Simulation Pipeline
• Construct a 3D Model
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Graphical Rendering: Recent Developments
Visual rendering
▪ Rasterization with programmable shading ▪ Ray tracing
3. It may not “sound” real
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Sound Propagation
▪ Numerical Methods
▪ Solve Helmholtz Wave Equation ▪ Accurate computation ▪ Compute intensive (increases as fourth power of frequency) ▪ Not practical for interactive applications (yet!)
GPGPU & Many-core computing
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Acoustics vs. Graphics
▪ Speed: 340 m/s vs. 300,000,000 m/s
▪ Frequency: 20 to 20K Hz vs. RGB
Sound Synthesis & Propagation
1. As compared to visual rendering, sound rendering is rather primitive
2. Interactive applications:
♦ Use pre-recorded sound sources ♦ Fixed models of propagation
▪ Wavelength: 17m to 17cm vs. 700 to 400 nm
Specular Reflection Scattering Diffraction Refraction Doppler Effect Attenuation Interference
Propagation
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
• Create an impulse response from path data (time, strength, direction, spectrum)
• Convolve IR with anechoic recording or synthesized sounds
• Auralize
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Sound Rendering: Collaborators
• Faculty ♦ Prof. Ming C. Lin
• Students ♦ Lakulish Antani ♦ Anish Chandak ♦ Christian Lauterbach ♦ Will Moss ♦ Nikunj Raghuvanshi ♦ Zhimin Ren ♦ Micah Taylor ♦ Hengchin Yeh (Dongenk University) ♦ Yero Yeh
perception ▪ Improve the sense of immersion ▪ Development of auditory displays
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Sound Rendering
• Part of multi-modal interfaces • Harness the sense of hearing • Sound rendering can augment visual rendering
• Collision Detection and Proximity Queries • Motion Planning and Multi-agent simulations • Sound Synthesis and Propagation • Haptics and Interactive Applications • Interactive Ray Tracing • Physically-based Modeling and Animation • GPGP: General-Purpose Computations using Graphics Processors • Geometric and Solid Modeling • Walkthrough of Massive Models
Normalized Magnitude (dB)
©Arup Acoustics
6 0 -6 -12 -18 -24 -30 -36 -42 -48 -54 -60
0
100 200 300 400 500 600 700 800 900 1000
Time (ms)
• Simulate sound propagation to find valid paths from source(s) to receiver(s)
▪ Methods
▪ Image Source [Borish,1984] [Dalenback,1992] ▪ Ray Tracing [Krokstad,1968] [Kuttruff,1993] ▪ Beam Tracing [Funkhouser et al.1998] [Funkhouser et al. 1999] ▪ Phonon Tracing [Kapralos,2004] [Bertram,2005] ▪ Frustum Tracing [Lauterbach et al. 2007] [Chandak,et al. 2008][Taylor et al. 2009] ▪ Acoustic Radiance Transfer [Siltanen et al.,2007] [Siltanen et al. 2009] ▪ Conservative Frusta Tracing (CFT) [Chandak et al. 2009]
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Sound Rendering 101
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
©Arup Acoustics
Sound Rendering: An Overview
Current GPUs: Interactive performance
• Desktop applications • Handheld applications
The GPUs are perhaps the first commodity massively parallel processors
frequency
• Petaflop computation needed for simulating at medium-level (1K) frequencies
• Solving high-frequency wave equation is a challenging scientific problem
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Geometric Sound Propagation
▪ Geometric Propagation
▪ Ray-Approximation of Wave Equation ▪ High-frequency approximation ▪ Getting fast enough for interactive applications ▪ Highly dependent on the geometry details
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Sound Propagation: Challenges
• Sound propagation: solve the Wave equation • Complexity is roughly O(ω4+δ), where ω is the
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
What is Sound Rendering?
▪ To generate sound effects for simulated environments ▪ Sound provides spatial cues and improves scene
▪ Methods
▪ Finite Element Methods [Otsuru,2004] ▪ Boundary Element Methods [Ciskowski,1993] ▪ Finite Difference Time Domain [Kunz et al.1993] ▪ Digital Waveguide Mesh (DWM) [Savioja et al. 1994] ▪ Domain Decomposition [Raghuvanshi et al. 2008]
UNC GAMMA Group
• Faculty ♦ Ming C. Lin ♦ Dinesh Manocha Around 20-25 students and postdocs
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
UNC GAMMA Group: Current Projects
Interactive Sound Rendering: Why we need more cores
Dinesh Manocha
University of North Carolina Chapel Hill
/Sound
The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL
Acoustic Geometry
-- surface simplification
Acoustic Material
-- absorption coefficient -- scattering coefficient
Source Modeling
-- area source -- emitting characteristics -- sound signal