Hide “Improvement available” for Topic versions assessed on an outdated dataset
Request type
Improvement / UI logic
Problem
The “Improvement available” option may currently be displayed for an older Topic version whose accuracy was calculated using an outdated dataset.
A historical accuracy score may have been satisfactory when that version was originally assessed. However, after new conversations are added, this score no longer represents the version’s performance on the current dataset.
Displaying “Improvement available” in this situation is misleading because it suggests that the old result is still relevant and can be used as the basis for further improvement.
Current behavior
An older Topic version may continue to display:
its historical accuracy score;
the “Improvement available” option;
even though that version has not been reassessed using the current dataset.
As a result, users may compare accuracy scores calculated on different datasets or attempt to improve a version based on outdated assessment results.
Expected behavior
The “Improvement available” option should be displayed only for Topic versions that have been assessed using the current dataset.
If a Topic version was assessed using an older dataset version, the “Improvement available” option should not be displayed for that version.
After the user reassesses the Topic version using the current dataset, the system should use the new result to determine whether “Improvement available” should be displayed.
Proposed logic
The system should compare the dataset used for the version’s most recent assessment with the current dataset:
Assessment dataset version = Current dataset version
If the dataset versions match, the system may display “Improvement available” according to the existing improvement logic.
If the dataset versions do not match, the option must remain hidden.
The validation should use the dataset version or dataset snapshot identifier rather than only the number of conversations.
This is important because:
automatically generated review sets usually increase the dataset in fixed batches;
manually created review sets may contain any number of conversations selected by the user.
Acceptance criteria
“Improvement available” is hidden for Topic versions assessed on an outdated dataset.
After reassessment on the current dataset, the option is displayed according to the existing improvement logic.
The behavior works for both automatic and manually created review sets.
Log in to comment and vote
No comments yet
Be the first to share your thoughts.