Audio emitter implementation in the KHR_Interactivity branch

[dqmachine]

I forked and branched off of the KHR_Interactivity branch to implement my audio changes since I need the changes in the interactivity branch and the interactivity PR is still open.

This supports KHR_Audio_Emitter. Resused parts of the audio example already in project where I could.

[Copilot]

Pull Request Overview

This PR implements the audio emitter functionality by introducing a new GLTF audio extension and integrating audio unit exporters for visual scripting. Key changes include:

• Adding the GltfAudioExtension class to serialize audio data.
• Creating several audio unit exporter classes (Play, Stop, Pause, UnPause) to support audio actions.
• Introducing a new audio export plugin and updating the export graph structure.

Reviewed Changes

Copilot reviewed 22 out of 32 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
Runtime/Scripts/Interactivity/Schema/GltfAudioExtension.cs	Implements the audio extension serialization for KHR_audio_emitter.
Editor/Scripts/Interactivity/VisualScriptingExport/VisualScriptingExportContext.cs	Modifies property setter for the exporter to allow public modification.
Editor/Scripts/Interactivity/VisualScriptingExport/UnitExporters/Audio/AudioSourceUnitExport.cs	Adds a new unit exporter for AudioSource with commented-out legacy code.
Editor/Scripts/Interactivity/VisualScriptingExport/UnitExporters/Audio/AudioSourceUnPauseNode.cs	Implements the unpause node exporter for audio sources.
Editor/Scripts/Interactivity/VisualScriptingExport/UnitExporters/Audio/AudioSourceStopNode.cs	Introduces the stop node exporter; note potential mismatched socket ID issue.
Editor/Scripts/Interactivity/VisualScriptingExport/UnitExporters/Audio/AudioSourcePlayNode.cs	Implements the play node exporter for audio sources.
Editor/Scripts/Interactivity/VisualScriptingExport/UnitExporters/Audio/AudioSourcePauseNode.cs	Implements the pause node exporter for audio sources.
Editor/Scripts/Interactivity/VisualScriptingExport/GltfAudioNodeHelper.cs	Provides helper methods for audio-related visual scripting node handling.
Editor/Scripts/Interactivity/VisualScriptingExport/GLTFAudioExportPlugin.cs	Introduces the export plugin for the audio emitter extension.
Editor/Scripts/Interactivity/VisualScriptingExport/ExportGraph.cs	Defines a container graph for managing exporter nodes.

Files not reviewed (10)

• Editor/Scripts/Interactivity/VisualScriptingExport/ExportGraph.cs.meta: Language not supported
• Editor/Scripts/Interactivity/VisualScriptingExport/GLTFAudioExportContext.cs.meta: Language not supported
• Editor/Scripts/Interactivity/VisualScriptingExport/GLTFAudioExportPlugin.cs.meta: Language not supported
• Editor/Scripts/Interactivity/VisualScriptingExport/GltfAudioNodeHelper.cs.meta: Language not supported
• Editor/Scripts/Interactivity/VisualScriptingExport/UnitExporters/Audio/AudioSourcePauseNode.cs.meta: Language not supported
• Editor/Scripts/Interactivity/VisualScriptingExport/UnitExporters/Audio/AudioSourcePlayNode.cs.meta: Language not supported
• Editor/Scripts/Interactivity/VisualScriptingExport/UnitExporters/Audio/AudioSourceStopNode.cs.meta: Language not supported
• Editor/Scripts/Interactivity/VisualScriptingExport/UnitExporters/Audio/AudioSourceUnPauseNode.cs.meta: Language not supported
• Editor/Scripts/Interactivity/VisualScriptingExport/UnitExporters/Audio/AudioSourceUnitExport.cs.meta: Language not supported
• Runtime/Scripts/Interactivity/Schema/GltfAudioExtension.cs.meta: Language not supported

[pfcDorn]

Hey,
I made some changes and implement (WIP) the KHR_audio_emitter Extension in a more generall way to use it with UnityGltf.
You can take a look at the KHR_audio_emitter PR

I take a look with your audio node implementations, i would not recommend to use any strings as an value input. Yes, it's currently in the interactivity extension (i should removed it), but it's not an official Gltf Interactivity Type. And i don't think you need it there. I would say the audiosource id/index should be enough as an Input, or declaring it in the config. So when you stop a audiosource with a specific id, all nodes which reference this emitter>audiosource id, should stop playing.

[dqmachine]

I agree. I put the string node output in as a placeholder for future potential audio node operations similar to animations. I will comment out the string output since it was a placeholder only.

Does you KHR_audio_emitter branch work within the visual scripting graph in Unity or is it separate like the example I saw a couple weeks ago?

[pfcDorn]

It's totally separate. It's just a plugin for the core UnityGltf. So it should work flawlessly with your interactivity implementation :)

[hybridherbst]

I think the general approach for adding entirely new interactivity features should be:

• adding import/export support for a particular extension independent of interactivity (e.g. audio, video)
• wiring it up with interactivity features through Unit exporters

[hybridherbst]

@dqmachine could you please clarify why you're now implementing a new GOOG_audio extension instead of basing on KHR_audio_emitter? Thanks!

If the idea is to differentiate from a ratified extension then it should be EXT_* and not GOOG_* since there's multple interested parties (at least UnityGLTF and you).

[dqmachine]

@dqmachine could you please clarify why you're now implementing a new GOOG_audio extension instead of basing on KHR_audio_emitter? Thanks!

If the idea is to differentiate from a ratified extension then it should be EXT_* and not GOOG_* since there's multple interested parties (at least UnityGLTF and you).

That was the intent, to have a simplified version of audio_emitter for now. Also, in case there were any upcoming changes.

Let me get back with you on the extension name after I talk with others involved.

[eugenecys]

We're proposing a stripped down version of the audio extensions as we have some features that we are planning for release that will most likely happen before KHR_audio_emitter is ratified, and prefer to have something stable so we won't run into compatibility issues should the KHR_audio_emitter spec changes.

The plan is to remove this once KHR_audio_emitter and KHR_audio_graph is ratified or close to it. Since only Google is proposing this and also to make it clearer it's not meant to be or replace the main audio extensions, GOOG_audio makes more sense.