Making "fake" simultaneous SLT test set from text-text test set

[bhaddow]

Hi

It's better to evaluate with realistic simultaneous SLT data, but there may be times when you want to take a text-text test set and use it as fake simul-SLT test data. Maybe for comparison to previous work, or because you want a specific language pair / domain. Is there already a script for this? Is it mainly a matter of generating fake timestamps, and putting things in the right format?

Barry

[obo]

I assume that for this usecase, you are not really interested in flicker or delay (because they would be bogus anyway), or is that the point?

If you are only interested in BLEU/friends, then SLTev can do this already now but there is still a horrible mess in documentation how exactly to as for MT-evaluation-only run and in what the file suffices should be.

[bhaddow]

Yes, I would want flicker and delay. The idea is to take a text-to-text test set, but assume that it is delivered incrementally. To create the time stamps we could (say) assume that every source word in the test set arrived at equal intervals. So if the first source line was "Resumption of the session .", it could be transformed to: P 0 1 Resumption P 0 2 Resumption of P 0 3 Resumption of the P 0 4 Resumption of the session C 0 5 Resumption of the session . or something like that. The "horrible mess in documentation" is why having a script would be nice. There's not much too it, but it would just get the files into the right place. On 08/02/2021 12:56, Ondrej Bojar wrote: This email was sent to you by someone outside the University. You should only click on links or attachments if you are certain that the email is genuine and the content is safe. I assume that for this usecase, you are not really interested in flicker or delay (because they would be bogus anyway), or is that the point? If you are only interested in BLEU/friends, then SLTev can do this already now but there is still a horrible mess in documentation how exactly to as for MT-evaluation-only run and in what the file suffices should be. — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub<#33 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAHSMG3GBS7Y7QKYWDQYISLS57NQ3ANCNFSM4XEVWQQA>. The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

[obo]

@mohammad2928 Has this been resolved?

[mohammad2928]

Yes, I have made fake data. But the increMT file was not created. I will make it soon.