Expression summaries not useful, and escaping from application mode

[RichCaloggero]

Expression summaries not useful, and escaping from application mode

The native accessibility built into MathJax is impressive! The navigation works better than MathPlayer, and I like the fact that the user has a  choice of speech rule sets.  It would be great to be able to retire good old MathPlayer and allow MathJax to handle all accessibility natively.

However, there are two major stumbling blocks which IMO must be overcome before this is truly useful in practice:

1. Currently, if you are reading a web page containing math with a
   screen reader (say NVDA in Firefox), because of the way the screen
   reader handles elements which have a role of "application", and because
   of the way the math is auraly collapsed by default, it is impossible to
   read math in context. For instance, if you had a sentence like "The
   derivative of 2x^2 is 2x", the screen reader would read something like
   the following when it encountered  it in a document: "The derivative of
   application is application". Even if the screen reader didn't hide the
   content behind the application role by default, MathJax is still only
   presenting an abbreviated form in speech at the top level. So, for
   instance, the top level of an equation might be "summation = power" (see
   the first equation following the math input box on asciimath.org).

IMO, MathJax should have a mode, which can be set via the menu, to allow
the expression to be fully spoken at top level, then you can drill down
via shift+space and arrows if you need to. Basically, reading math in context with current implementation is not possible, and this is a non-starter IMO. If you use assistive mathML and MathPlayer, you do get the ability to read math inline, and you can focus on any expression and drill down if desired.

2. Currently, pressing escape does back you out of the expression, but
   the expression still has browser focus. This means that the screen
   reader's document reading commands are still disabled due to the
   application role. I've been able to mitigate this by applying a global
   keydown handler on the document that when receiving an escape key and
   the element with focus is a MathJax container, blur() the element! This
   works, but it leaves the focus somewhat undefined, which may confuse users.

Rich Caloggero, MIT Adaptive Tech. Info Center

On 4/24/2019 11:37 AM, Mary J Ziegler wrote:
Hi Rich,

Perhaps I'm behind the times, but based on the lack of other replies, I'm not sure if anyone confirmed this for you or not?! Just now, I finally went through this message in detail on both Windows and Mac.

On Windows:
I, too, can navigate the MathJax on asciimath in the manner you describe (with explorer on/assistive math off), but can't do so as well with explorer inactive.
I, too, cannot navigate the same on edX - until I change the math renderer to Common HTML (as is on ascii math), in which case I can.
In both cases, when I turn off explorer, I'm at a loss to get into the detail of the math as when I press shift+space I end up in the context menu. Perhaps I need a different keyboard shortcut to get there with explorer off?

On Mac:
I keep running into the math getting collapsed and not knowing how to un-collapse it. We can look at that together. I can't sort out the keyboard shortcuts on VoiceOver to be sure if this is working or not. That said, it works for me when uncollapsed.

I need to go to my 12 noon event soon. See you in a bit!
Mary

-----Original Message-----
From: Rich Caloggero rjc@mit.edu
Sent: Thursday, April 18, 2019 1:45 PM
To: Jeff Witt jwitt@edx.org; Mary J Ziegler maryz@mit.edu
Cc: accessibility accessibility@mit.edu
Subject: MathJax Accessibility works without MathPlayer, but not on edX

http://asciimath.org/

MathJax is properly configured on this site. By default, it uses the configuration I suggested (using assistive math). See the MathJax menu by right clicking on an expression) and check out the "accessibility"
submenu).

Now, turn on the "explorer" from the accessibility menu, and after doing this, go back into the accessibility menu and be sure that "assistive math" is turned off; do not change anything else.

Now tab to an expression and press "shift+space". you will now be able to navigate the expression with up, down, left, and right arrows. My suspician is that this will work on MacOS as well as windows. I have not yet tested with jaws.

Another interesting thing is that MathPlayer is no longer needed. Even if you switch back to the "assistive math" mode and turn off the explorer, NVDA still reads the math, and I believe it processes it directly.

Can either of you verify any of this using the above URL?

I cannot get this to work on edX? What version of MathJax is in use on edX?  I have gotten it to work by downloading the latest version of MathJax (2.7.5) and using one of the supplied configuration files "AM_CHTML"; this processes asciimath input, and generates common HTML by default (but you can change the output format via the menu to whatever you want).

I will play around some more, and try to get it to work with TeX input (no reason it should not, but who knows)!

-- Rich

[zorkow]

Thanks for the feedback, and I am very happy to hear that you like the a11y extension.
There will be a number improvements and new features in the upcoming MathJax version 3 release wrt to accessibility, e.g., Nemeth output, clearspeak support, but also more flexibility and personalisation options. These might also help with some of the problems that you flag up.

1. I agree that the current behaviour of exposing only summaries on the top level via aria-labels is not optimal. At the time of the initial release the reasons for the design decision were firstly that initial user feedback was that lengthy math expressions could be very disruptive to the readability. And secondly to reduce rendering time on page load by generating initially shallow speech only. Since then
   SRE's performance has significantly improved and in v3 we can expose multiple alternative texts for any expression. Then it is only a question of which speech is exposed via the aria-label, which can be a user option. I could see at least three settings there: (1) summarise all, (2) summarise none, (3) summarise only display expressions as they tend to be very long.

2. I understand the UX that you aim for and I think it should be possible to achieve in the next version. But to manage expectations, in practice it is often difficult to predict if MathJax indeed achieves the desired behaviour on a client, as this depends very much on the combination of OS, Browser, and Screenreader (and often on their respective versions!). And testing all possible combinations is practically impossible. We simply don't have the resources.
   However, the more feedback we get from the community, the better we can hone the UX for all platforms.

[RichCaloggero]

Hi Volker. Thanx so much for your note. In the docs for the current accessibility extension, it mentions an option which is suposed to appear in the accessibility context menu in the explorer submenu, called "generation", which has the three options you discuss in your note. However, in the precompiled versions I've seen I never see this menu; I only get a checkbox to turn speech generation on / off.  Similarly for walker: docs say there should be several options for walker, but all I see is a chebox to turn it on / off. I tried rebuilding the extension, but I'm on windows and I'm not sure if SRE is being properly compiled; think it's grabbing precompiled SRC. Can you point me to a precompiled SRE which would work with MathJax that I could test with? As for my other issue, I think the solution would be to cause the escape key to not only back you out of the expression, but also force a blur of the element with role of application. This would turn off application mode for both NVDA and Jaws. Sometimes screen reader focus may move when blur is called, but that is a screen reader issue, not a MathJax issue and can be addressed by the screen reader authors or a screen reader add-on. Thanx for all your work on accessibility; it really makes a difference to many people!

…

-- Rich

On 5/23/2019 11:18 AM, Volker Sorge wrote: Thanks for the feedback, and I am very happy to hear that you like the a11y extension. There will be a number improvements and new features in the upcoming MathJax version 3 <https://github.com/mathjax/mathjax-v3> release wrt to accessibility, e.g., Nemeth output, clearspeak support, but also more flexibility and personalisation options. These might also help with some of the problems that you flag up. 1. I agree that the current behaviour of exposing only summaries on the top level via |aria-labels| is not optimal. At the time of the initial release the reasons for the design decision were firstly that initial user feedback was that lengthy math expressions could be very disruptive to the readability. And secondly to reduce rendering time on page load by generating initially shallow speech only. Since then SRE <https://github.com/zorkow/speech-rule-engine>'s performance has significantly improved and in v3 we can expose multiple alternative texts for any expression. Then it is only a question of which speech is exposed via the |aria-label|, which can be a user option. I could see at least three settings there: (1) summarise all, (2) summarise none, (3) summarise only display expressions as they tend to be very long. 2. I understand the UX that you aim for and I think it should be possible to achieve in the next version. But to manage expectations, in practice it is often difficult to predict if MathJax indeed achieves the desired behaviour on a client, as this depends very much on the combination of OS, Browser, and Screenreader (and often on their respective versions!). And testing all possible combinations is practically impossible. We simply don't have the resources. However, the more feedback we get from the community, the better we can hone the UX for all platforms. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#236?email_source=notifications&email_token=ABTLKH5EBFSNRBIFY4KJJSDPW2YWHA5CNFSM4HON23L2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWCR7MA#issuecomment-495263664>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABTLKH6NEUP7ISVZHAQIENDPW2YWHANCNFSM4HON23LQ>.

[RichCaloggero]

Building MathJax-a11y on windows

I'm building on win10, using WSL (windows subsystem for linux).

When attempting to build MathJax-a11y, I consistently get an error on line 94 of speech-rule-engine/src/common/system_external.js; patched as follows:

window.document.XPathResult = window.XPathResult || {};

Also, in the MathJax-a11y/gruntfile.js, leaving the branch set to 2.2.2 in the gitclone tasks options leads to make aborting with an error. It attempts to make target java. Commenting out the branch option, and patching speech-rule-engine/src/common/system_external.js as described above leads to a correct build.

Hope this is helpful.