Skip to content

Conversion of ml/h to itself yields floating point/rounding inaccuracy #65

Open
@floriandejonckheere

Description

@floriandejonckheere

I've noticed that for some values of mL/h, converting to itself yields an inaccurate result:

Unitwise(4, "mL/h").convert_to("mL/h")
=> #<Unitwise::Measurement value=3.999999993 unit=mL/h>

Oddly enough, this doesn't happen for all values, e.g. 0-3, 5 and 10 yield the correct result. And it seems that mL/h is the only unit where this happens: L, dL, hL, s, min, h in any variation seem to be correct:

[:mL, :L, :dL, :hL]
  .product([:ms, :s, :min, :h])
  .map { |n, d| [n, d].join("/") }
  .index_with { |unit| (0..1000).select { |i| Unitwise(i, unit).convert_to(unit).value != i }.count }

=> {"mL/ms"=>0,
 "mL/s"=>0,
 "mL/min"=>0,
 "mL/h"=>981,
 "L/ms"=>0,
 "L/s"=>0,
 "L/min"=>0,
 "L/h"=>0,
 "dL/ms"=>0,
 "dL/s"=>0,
 "dL/min"=>0,
 "dL/h"=>0,
 "hL/ms"=>0,
 "hL/s"=>0,
 "hL/min"=>0,
 "hL/h"=>0}

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions