Skip to content

Forgetting is not fixed in the latest release? #291

Open
@Newspaper8459

Description

@Newspaper8459

In metrics.py, there was a problem in the range, where it previously ranged from 0 to k-2. This had apparently been fixed in the past, and I can check that in this commit.

Below is the current code I can find in the repository:

def forgetting(all_preds, all_targets, all_tasks):
    ...

    for j in range(k - 1):
        # Accuracy on task j after learning current task k
        a_kj = _get_R_ij(k - 1, j, all_preds, all_targets, all_tasks)
        # Best previous accuracy on task j
        max_a_lj = max(
            _get_R_ij(l, j, all_preds, all_targets, all_tasks)
            for l in range(k - 1)
            if l >= j
        )
    ...

However, it hasn't been fixed in the latest release, which is 1.2.7.
Below is the code in the latest release 1.2.7.

def forgetting(all_preds, all_targets, all_tasks):
    ...
    for j in range(k - 2):
        # Accuracy on task j after learning current task k
        a_kj = _get_R_ij(k - 1, j, all_preds, all_targets, all_tasks)
        # Best previous accuracy on task j
        max_a_lj = max(_get_R_ij(l, j, all_preds, all_targets, all_tasks) for l in range(k - 2) if l >= j)
        f += max_a_lj - a_kj  # We want this results to be as low as possible
    ...

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions