Test case minimizer #456
Labels
No labels
Compat/Breaking
Kind
Bad merge
Kind
Bug
Kind
Documentation
Kind
Enhancement
Kind
Feature
Kind
New language
Kind
Security
Kind
Testing
Priority
Critical
Priority
High
Priority
Low
Priority
Medium
Reviewed
Confirmed
Reviewed
Duplicate
Reviewed
Invalid
Reviewed
Won't Fix
Status
Abandoned
Status
Blocked
Status
Need More Info
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference: mergiraf/mergiraf#456
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
When we have an example of a bad merge (or any other test case, for instance one that triggers a panic), it is often useful to minimize it, so that we can more easily debug it, and potentially add it to our test suite once the bug is fixed.
It's generally not too hard to do it manually, but it would still be interesting to have a helper to automatically minimize those examples. Such sorts of heuristics are often used in combination with fuzzers (#450).
I'm thinking something like this could work. It would take as inputs:
And then it would repeatedly
And it would stop when it doesn't progress anymore (say, reaches a threshold of unsuccessful reduction attempts on the current test case).
Oh this would be amazing! I'd argue that manual minimization is actually pretty daunting at times, because of how easy it is to slip from a conflict that requires structural resolution (which is what reproduces the big at hand) to one that's solvable by a line-based merge
I had a go at it… I'm not sure how useful it will be in practice outside of fuzzing contexts, because the work of manually minimizing gets replaced by the work of writing a script which defines the conditions under which we want to minimize the test case, which can get non-trivial I would say.