HEAR THE OCEAN

Immersive Virtual Soundscape

2022.7--2022.8

Project Members

Zhaorui Liu

Xiyao Jin

Xiaorong Shao

Qian Li

Heyu Gao

Zhirui Bian

Xinyue Xu

MY ROLE

Team Leader

TOOLS

Meta Oculus Quest 2

Ableton Live

Envelop for Live

TouchDesigner

Python

Godot

SURPPOTED BY

"Hear The Ocean" is a work of immersive generative art for real-time GPU particle swarm optimization. It uses the real-time GPU particle swarm optimization written by Compute Shader to simulate the behavior of krill in the ocean and real-time rendering in VR HMD in the form of artistic light dots. The real-time data (such as the center of gravity position, the average speed, etc.) obtained from the algorithm simulation are processed and used for the spatial audio design in VR. In terms of sound design, granular synthesis is used to create the subtle and crisp dreamy sound of the particle swarm. With the whale songs from scientific research data and other atmospheric sound effects simulating the marine environment, the soundscape of the virtual marine space is jointly formed.

Version 1.0

Version 2.0

Related Papers

Jin, X., Liu, Z., Shao, X., Li, H. & Li, Z. Self-adaptive Mixing System for Spatial Audio in Virtual Reality.[C]//Forum Acusticum, Torino, Italy, 2023.

Xingyu Lai, Zhaorui Liu, Xiyao Jin, Xiaorong Shao, Borou Yu, Jiajian Min, Mengying Zeng, Shuting Chen, Qiao Liang, Yi Ji, Pengcheng An, and Xueliang Li. Multi-sensory Interaction for Meditation.[C]//Proceedings of the Tenth International Symposium of Chinese CHI (Chinese CHI'22), Association for Computing Machinery, New York, NY, USA, 328–336.

Award