Skip to content

DAVAI-37: Smooth sonification values in univariate dot plot #49

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

lublagg
Copy link
Collaborator

@lublagg lublagg commented May 29, 2025

DAVAI-37

This PR smooths out the transitions between bins by interpolating bin frequencies, when the graph sonification is of a univariate dot plot.

It also introduces some other changes due to issues I was noticing with the univariate playback. I think currently on the main branch, looping doesn't work with univariate dot plots.

One of the issues causing this was how we were previously using oscillator.start and oscillator.stop at different points in the code, rather than just depending on the Transport for starting, pausing, and stopping. This was because I didn't realize you could sync the oscillator to the transport. Now, by calling oscillator.current.sync, the oscillator's start is synced to the Transport's start.

Also, instead of calling scheduleOnce, the code now creates a new Tone.Part instance, which has a beginning synced to the Transport's start time, and an end synced to the durationRef. This is helpful because now, if we are looping, we no longer need to reschedule the tones at the end of a loop -- the transport just starts from the beginning, which the part is synced to, so everything plays as it should.

One thing is I'm not 100% sure this Part model will be compatible with when we want to step through the graphs in the future. But it seems like it actually might be more compatible than what we had before. But that is something to consider, although it feels a little bit beyond what I can figure out right now.

Also I moved some of the Tone.js code into two different hooks, to break up the component a bit, as it was feeling a bit unwieldy to me.

One thing that I've been thinking about, which is outside of the scope of this work but came up because of a comment left by Copilot, is that in the future we most likely will want to keep track of which graphs have ROI adornments created, and the positions of those adornments. I can imagine that when users are stepping through graphs, we might not want to re-set the position of an existing ROI every time a new graph is selected, and that if a previously selected graph is selected again, we'll want to know about its existing ROI position. Again, outside of the scope of this work, but just a note that I think somewhere down the line we'll have to incorporate that information in our models.

@lublagg lublagg requested a review from Copilot May 29, 2025 20:13
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements smoother sonification for univariate dot plots and refactors several sonification-related hooks and components to improve timing and state management. Key changes include:

  • Definition of new types (UnivariateNote and ScatterNote) in types.ts.
  • Introduction of the useTone and useSonificationScheduler hooks to manage Tone.js instance lifecycles and scheduling.
  • Refactoring of the GraphSonification component to integrate the new hooks, adjust ROI adornment behavior, and improve transport control.

Reviewed Changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
src/types.ts Added type definitions to support sonification notes.
src/hooks/use-tone.ts Created a custom hook to initialize, manage, and dispose Tone.js audio objects.
src/hooks/use-sonification-scheduler.ts Introduced scheduling logic for univariate and scatter plot sonification events.
src/components/graph-sonification.tsx Refactored the component to use the new hooks and consolidate ROI adornment logic.
src/components/graph-sonification-utils.ts Added helper functions for bin interpolation and ROI adornment creation.
mocks/tone.js Updated mocks for Tone.js objects to support the new dispose and sync methods.

@lublagg lublagg requested a review from emcelroy May 29, 2025 21:40
@lublagg lublagg changed the title DAVAI-137: Smooth sonification values in univariate dot plot DAVAI-37: Smooth sonification values in univariate dot plot May 29, 2025
Copy link
Contributor

@emcelroy emcelroy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good 👍 I just left one non-blocking comment about possibly improving some types.

Comment on lines +8 to +13
selectedGraph: any;
binValues: any;
pitchFractions: number[];
timeFractions: number[];
timeValues: number[];
primaryBounds: any;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be relatively easy to replace these instances of any with more specific types? I think selectedGraph could at least be ICODAPGraphModel, right?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants