Can you please make a badge that shows the FAIR score of a project?

[RikDTJanssen]

code state: 313764c

Can you please make a badge that shows the FAIR score of a project? Now it only shows the link to the fairsoftwarechecklist.net website. It would be nicer if the scores were in the badge.

[tom-h]

Thanks for the feedback. That's a really good suggestion, and I can see why that would be an appealing feature in terms of displaying the badge on a project page. Given my further explanations below, we'd really appreciate your thoughts on how such a feature might be implemented.

There's a tricky balance to strike in implementing such a feature. It has been a conscious design decision to avoid reducing the FAIRness to a numerical score. That's why we give a visual indication with bars, instead of a number. In each question we know the relative "FAIRness" of a set of choices. For instance, we can say that some identifiers are better than others, and all are better than none. But there's no way to talk about the relative FAIRness of the different dimensions. For instance, are persistant identifiers more important and provenance metadata?

Not only do we want to suggest that any one action is more important than another, we also have the issue that in some cases it's a binary choice and in others a scale, and in others again it's an open ended task. Underpinning the tool there are in fact decisions like this that have been made (there has to be), but then we mask these decisions by not providing a numerical score.

Linking through to the answers was also a design decision. In reusing the software, we figured it would be more useful to know the answers to specific questions (especially if comparing software for instance) than to know an overall score.

So if the badge were to indicate FAIRness, it probably couldn't be a numerical score. So far it's bars when you click through to the page. What would you suggest?

[RikDTJanssen]

Thanks for your reply. I understand your considerations. Maybe this might work:

• Show four small (stacked) bars on the badge, for F, A, I, and R? Where the length corresponds to the score. The bars go from left to right in the badge. (I don't know how to draw that in this field.)
• Or something like F: ¾; A: ¾, I: ¼; R: ¼ in the badge, where e.g. ¼ means a score of one out of four.

[tom-h]

Great, thanks for that. My preference would be for bars over numbers. And One per letter makes sense (no action against any letter can be considered more or less FAIR than any other, IMHO)

[aaronkurz]

It could just be a larger badge, similar to the planet badge:

The bars are already percentages, so I guess they could be mapped to five distinct ranges (0-20, 20-40,...). Regarding the implementation, a naive approach would mean creating $5^3$ graphics and appropriately mapping them when generating the copy-able link. Alternatively there are tools out there that could be looked into: https://github.com/badges/awesome-badges

One quick SVG draft (I am thinking that reversing to red ---> green order would be better):

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<svg width="250" height="150" viewBox="0 0 250 150" xmlns="http://www.w3.org/2000/svg">
    <!-- Background -->
    <rect x="5" y="5" width="240" height="140" rx="10" ry="10" fill="#E6F3FF" stroke="#B0D0E8" stroke-width="2"/>
    
    <!-- Header -->
    <text x="15" y="30" font-family="Arial" font-size="18" font-weight="bold" fill="#003366">FAIR Software Checklist</text>
    
    <!-- Categories -->
    <text x="15" y="60" font-family="Arial" font-size="12" font-weight="bold" fill="#003366">Findable</text>
    <text x="15" y="80" font-family="Arial" font-size="12" font-weight="bold" fill="#003366">Accessible</text>
    <text x="15" y="100" font-family="Arial" font-size="12" font-weight="bold" fill="#003366">Interoperable</text>
    <text x="15" y="120" font-family="Arial" font-size="12" font-weight="bold" fill="#003366">Reusable</text>
    
    <!-- Score Bars with Fixed Sections -->
    <g>
        <!-- Findable -->
        <rect x="100" y="50" width="24" height="10" fill="green"/>
        <rect x="124" y="50" width="24" height="10" fill="lightgreen"/>
        <rect x="148" y="50" width="24" height="10" fill="yellow"/>
        <rect x="172" y="50" width="24" height="10" fill="orange"/>
        <rect x="196" y="50" width="24" height="10" fill="red"/>
        <circle cx="112" cy="55" r="5" fill="white" stroke="black" stroke-width="1"/>
    </g>
    
    <g>
        <!-- Accessible -->
        <rect x="100" y="70" width="24" height="10" fill="green"/>
        <rect x="124" y="70" width="24" height="10" fill="lightgreen"/>
        <rect x="148" y="70" width="24" height="10" fill="yellow"/>
        <rect x="172" y="70" width="24" height="10" fill="orange"/>
        <rect x="196" y="70" width="24" height="10" fill="red"/>
        <circle cx="136" cy="75" r="5" fill="white" stroke="black" stroke-width="1"/>
    </g>
    
    <g>
        <!-- Interoperable -->
        <rect x="100" y="90" width="24" height="10" fill="green"/>
        <rect x="124" y="90" width="24" height="10" fill="lightgreen"/>
        <rect x="148" y="90" width="24" height="10" fill="yellow"/>
        <rect x="172" y="90" width="24" height="10" fill="orange"/>
        <rect x="196" y="90" width="24" height="10" fill="red"/>
        <circle cx="160" cy="95" r="5" fill="white" stroke="black" stroke-width="1"/>
    </g>
    
    <g>
        <!-- Reusable -->
        <rect x="100" y="110" width="24" height="10" fill="green"/>
        <rect x="124" y="110" width="24" height="10" fill="lightgreen"/>
        <rect x="148" y="110" width="24" height="10" fill="yellow"/>
        <rect x="172" y="110" width="24" height="10" fill="orange"/>
        <rect x="196" y="110" width="24" height="10" fill="red"/>
        <circle cx="184" cy="115" r="5" fill="white" stroke="black" stroke-width="1"/>
    </g>
</svg>

[RikDTJanssen]

Yes, that would be cool, thanks! I would like to suggest to make the badge a bit more compact, so can be used next to badges like https://github.com/badges/awesome-badges. Also I would like to suggest not using the color red, because some people might see it as "wrong" while probably we'd rather like to encourage people to get a higher score. Thanks!

[aaronkurz]

@RikDTJanssen I can also imagine a badge similar to the others with four colorful areas instead of one for the scores, where the score percentage is again divided into four sections, but this time from A - E. But then for me it is hard to add also the names of the categories for space reasons... it could look something like this:

Regarding the colors: I think red is also okay? It will push people to get it green and motivate them to follow best practices? An alternative could be to make it some gradient from blue to green, where blue is then more neutral. But I think that makes it less intuitive?

SVG:

<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" width="285" height="40" viewBox="0 0 285 40">
    <!-- Left Label -->
    <rect x="0" y="0" width="125" height="40" fill="#4A4A4A"/>
    <text x="10" y="25" font-family="Arial" font-size="16" fill="white">FAIR Software</text>
    
    <!-- F Section -->
    <rect x="125" y="0" width="40" height="40" fill="#4CAF50"/>
    <text x="140" y="25" font-family="Arial" font-size="16" fill="white">A</text>

    <!-- A Section -->
    <rect x="165" y="0" width="40" height="40" fill="#FFEB3B"/>
    <text x="180" y="25" font-family="Arial" font-size="16" fill="black">C</text>
    
    <!-- I Section -->
    <rect x="205" y="0" width="40" height="40" fill="#8BC34A"/>
    <text x="220" y="25" font-family="Arial" font-size="16" fill="white">B</text>
  
    <!-- R Section -->
    <rect x="245" y="0" width="40" height="40" fill="#F44336"/>
    <text x="260" y="25" font-family="Arial" font-size="16" fill="white">E</text>
</svg>

[RikDTJanssen]

Thanks, I like this one also. And maybe the blue to green gradient is better.

How about making a "stacked bar" (going from bottom to top) in e.g. the green A square in your example above? Assuming every of F, A, I, and R have 4 possible values, if someone scores a 1 for F, then show an small bottom line (fill it for 1/4), if a person scores 2 than fill it half, if a 3 then 3/4, etc.

The 4 squares in your example above represent from left to right F, A, I, and R, so maybe a category label is not required.

And, scores like A to E are used in some countries only, not in all (not in mine). I really have to rephrase "A" as being "the best" in my mind before it makes sense to me. Maybe a number or letter is not necessary when there is a stacked bar or what you suggested in #96 (comment).

[aaronkurz]

@RikDTJanssen With stacks I would suggest separating the percentages in 4 bars since it is then more visible; something like this maybe? There should maybe be some dividers so that the separate areas are more clear when the stack hight is equal among the categories or a category is at a hard 0%.

<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" width="246" height="40" viewBox="0 0 246 40">

    <rect x="0" y="0" width="125" height="40" fill="#4A4A4A"/>
    <text x="10" y="25" font-family="Arial" font-size="16" fill="white">FAIR Software</text>
    
    <g transform="translate(125, 0)">
        <rect width="30" height="10" y="30" fill="#64B5F6"/>
        <rect width="30" height="10" y="20" fill="#42A5F5"/>
        <rect width="30" height="10" y="10" fill="#1E88E5"/>
        <rect width="30" height="10" y="0" fill="#186EBA"/>
    </g>
    
    <g transform="translate(155, 20)">
        <rect width="30" height="10" y="10" fill="#64B5F6"/>
        <rect width="30" height="10" y="0" fill="#42A5F5"/>
    </g>
    
    <g transform="translate(185, 10)">
        <rect width="30" height="10" y="20" fill="#64B5F6"/>
        <rect width="30" height="10" y="10" fill="#42A5F5"/>
        <rect width="30" height="10" y="0" fill="#1E88E5"/>
    </g>
    
    <g transform="translate(215, 30)">
        <rect width="30" height="10" y="0" fill="#64B5F6"/>
    </g>

</svg>

[RikDTJanssen]

Yes, exactly! That is wonderful. Seeing this, maybe it is even possible to superimpose the F, A, I, and R on the stacked bars? Thanks!

[aaronkurz]

@RikDTJanssen That is a possibility, but I worry it will be a bit messy? Maybe we wait for some input from @jspaaks, @sverhoeven or @tom-h.

Regarding the implementation (after final design), I could then...

1. Write up a script generating svg graphics for all combinations
2. Add dynamic linking to the suitable graphic for the badge links

[RikDTJanssen]

Sure, no problem! And many thanks!

[jspaaks]

Hey folks,
thanks for the nice suggestions and draft designs for possible badges. I just wanted to let you know that both @tom-h and myself have moved on from this project so it's unlikely that we'll be able to do any work on this.

Your best bet is probably one of the community managers (@c-martinez perhaps) at my former place of work (Netherlands eScienceCenter), but be advised they are currently going through a reorganization, including big cuts to aforementioned department, so might be best to temper your expectations.

Best regards,
-Jurriaan

[tom-h]

Yep, as @jspaaks says, and I'm no longer at the ARDC. On the ARDC side, @orchid00 and Nick Jenkins would be appropriate to query about this work.

[c-martinez]

Thanks @jspaaks for looping me in, and thanks @RikDTJanssen and @aaronkurz for your enthusiasm in contributing to this project.

First impression, I like the bars (and I agree, superimposing the letters would make it more clear what the bars refer to).

In principle, I would say that even if ARDC and Netherlands eScience Center were the initiators of this effort, it is open for contribution from the community.

Having said that, I have a couple of questions:

• How do we see these badges being generated? Would they be generated on the flight (you mentioned a script, but that would need to run somewhere)? Or, because they are a limited number of possible combinations we can pre-generate them and host them somewhere?
• How do we make sure that the badge SVG always matches with the FAIR score parameters (the ?f=31&a=32113&i=32121&r=123 part of the URL)?

[aaronkurz]

@c-martinez Thanks for your feedback and input! Then I would make sure to add the superimposition.

Regarding the questions:

1. Since the number of combinations is limited, they could be pre-generated and hosted similar to the current badge.svg.
2. I did not fully make sure yet, but I am pretty certain (since the overall percentage is calculated for each category too) that it is possible to create a mapping function that takes the answer keys for each category and returns something like [1, 2, 1, 4], meaning f has 1 bar slice, a has 2 etc.; then, it could just be statically mapped to the correct pre-computed graphic, e.g, badge-1-2-1-4.svg.

So in essence, the addition would be a range of $5^3 = 125$ (mapping the percentages currently shown on the website to 25% ranges for the four bars and the option of zero bars) svg graphics that are hosted similarly as before and a function that maps to the right badge for the copy-able snippets.

Maybe I would like to just check regarding the mapping, if it is okay as below:

• 0% -> 0 bars
• ]0% - 25%] -> 1 bar
• ]25% - 50%] -> 2 bars
• ]50% - 75%] -> 3 bars
• ]75% - 100%] -> 4 bars

Regarding the implementation, I could do that and create a PR from a fork. I am a bit busy these weeks but I could probably get around to it in the second half of April! :)

[c-martinez]

Pre-generated badges sounds like a good alternative. We need to be a bit careful that we do not accidentally destroy an information when mapping from URL's to badges to answers. For example I am worried about cases like

https://fairsoftwarechecklist.net/v0.2?f=30&a=32113&i=32322&r=133

and

https://fairsoftwarechecklist.net/v0.2?f=01&a=32113&i=32322&r=133

both will visually look the same (F=50%), but will map to different answers.

I guess the main risk would be if someone changes updates their answers and changes the badge but not the URL or the other way around. (small risk, but still though I should share it).

[aaronkurz]

@c-martinez Yes that is true, but in the end, if the user just copies the whole snippet it would be correctly mapped. I could include a sentence warning of that - that the whole snipped needs to be copied when changing answers. But of course it would be "trickable". But I guess in the end this is not meant as a certificate, but more a motivation for developers and for providing more visibility to the concept of FAIR (research) software.

[jspaaks]

Also, be aware that you may need to host the badge files under a versioned route like v0.2/, such that later versions of the checklist may still opt to use, say,

1. a different number of states per FAIR letter (e.g. [0, 10, ... ,100] instead of [0, 25, ... , 100],
2. different branding,
3. or perhaps an altogether different system of retrieving badges,

without interfering with existing links to v0.2 badges.