User feedback

[jspaaks]

Notes from review session Nov 30, 2022

code state: https://github.com/ardc-fair-checklist/ssg/tree/ce660569c3bbedaf3f5a508113bf460f02e60b86

About landing page:

1. liked the simple style
2. maybe add simple animations on landing page, some eye candy
3. "I don't know what FAIR is" maybe add context with a link
4. call to action for each of the links (data/ and software/)
5. move logos to the bottom
   • rework vertical layout of landing page #53
6. consider adding a textual indication of the logo link targets
7. indicate what is going to happen when you click the software or data link, before you click the link; indicate time expectation management
8. repeat favicon logo in the site content for brand building

About the data route:

1. liked background
2. Something feels off about the title, maybe use "Self-assessment checklist for FAIR data"? ✔️ done
3. "or switch to the checklist for sofware instead" maybe replace with "if you want to check the FAIRness of your software, use this link."
4. expectation management on what is going to happen before users go through the form. Something like "We have questions in 4 categories, you'll see the bars at the bottom update as you go along"
5. aspect heading now just says Findable, consider adding context to explain what Findable means within FAIR
   • consider adding the per-principle snippets for FAIR data #55 (data)
6. reddish color of the 2nd progress bar suggests something is wrong
7. question 7: merge "No" and "Unsure" answers, they award the same points
8. what does Interoperable mean. Maybe add a context section under the H2 headings
9. question 9: some questions are difficult to answer because my option isn't there. Maybe add a field "Other"
10. Avoid overlaying the form content with the progress bars, it might be cleaner to have two rows, with the questions content div scrollable
11. After you complete the questions, add something about what's next, or some kind of closer message
12. Reviewer observes: "This checklist is useful but I didn't enjoy the process of filling it out"
13. How can I get help for a given question if something isn't clear; maybe add a "Read more..." section
14. target="_blank" on "Report an issue"
15. use dedicated issue template on the GitHub new issue link
    • consider linking the "Report an issue" button to a dedicated issue template #48
16. consider making the ARDC logo text large enough to be readable
17. Bug: I can sometimes select two things, related to going to GitHub in between. Happens on desktop + Arch Linux + Edge. Reproduce with: select option 0 on the first question, go to GitHub via link (in the same tab), click Back button, now select option 1 on the first question, now both options are checked.
    • Can select two radio buttons in one question #51

About the software route:

1. liked different color from data route
2. why can I get a badge for software but not data?
3. "Answer the N questions below" it's not telling me much to have a number there, just do "Answer the questions below" ✔️ done
4. Provide context on how this page helps a user

Reviewer filled out the form for https://github.com/NLeSC/python-template

(Numbers are question numbers)

1.  
2. confusing text, suggests you're doing it wrong when there is one component. Not fair if you went through the trouble of refactoring your code into being only a reusable library without any extras
3.  
4. "rich" metadata what does it mean?
5.  
6. difficult question, FAIRness of metadata is unclear what it means
7.  
8. what is "standardised" protocols?; if 7 is answered with no, hide questions 8 and 9, or at least 9. Also there is a typo in the possessive.
9.  
10. "software is no longer available" what does it mean? The phrasing could be SMARTer
11. difference between b and c, what happens if your software does both, for example export as CSV (generic) and as FIT (domain-specific)
12. what "data"?
13. what does "easy" mean? Also the type of documentation matters: e.g. developer docs v user docs. For option 3, consider replacing with something like: "Software is available in binary form and its documentation covers how to incorporate or extend"
14. maybe too complicated but what about portability between OSes, tool chains; phrasing of the answers does not cover the possible cases
15. What makes a license "standard"? What makes a license "machine readable"?
16. Reviewer suggests to not have this question, but instead only ask about technical / reproducibility aspects of provenance, not the project aspects. Reviewer doesn't see how project information is helping reproducibility.
17. Phrasing is confusing. Maybe split the answers into a group about dependencies and a group about other software.
18. "I feel like I'm solving a puzzle in every question" What is a standard? Confusing question. JHS: Maybe rephrase as "How much have domain-relevant practices been considered in writing the software?"

Reviewer observes: "checklist not a good fit for a package like python-template"

About "Get the badge" section

1. what do I do with the badge, maybe add text under heading
2. consider hiding the snippets behind a dropdown to not scare people with code

About progress bars:

1. Consider adding some kind of congratulations to the progress bars

Reviewer observes: "number of questions may be too many"