ctest failures invisible
I’m trying to resurrect !3029 (closed) on which progress seems to have stalled. I think I’ve solved all the failures except a single unit test in the ctest suite. You can see the failure here, https://gitlab.com/graphviz/graphviz/-/jobs/3899795706. As observable, the failure output is quite unhelpful.
Running the equivalent steps locally, I can reproduce the failure. Locally I get no output at all, not even the cryptic “suppressions used…” text. Toggling --output-on-failure and $CTEST_OUTPUT_ON_FAILURE has no effect. Trying --verbose and --extra-verbose alters the output, but still provides no information about the test failure.
Bypassing ctest altogether and running the unit test directly finally gets us the failure information:
$ env LD_LIBRARY_PATH=${PREFIX}/lib ./tests/test_edge_node_overlap_all_node_shapes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
test_edge_node_overlap_all_node_shapes is a Catch v2.13.8 host application.
Run with -? for options
-------------------------------------------------------------------------------
Overlap all node shapes
-------------------------------------------------------------------------------
/tmp/tmp.npCMjCmmIL/graphviz/tests/test_edge_node_overlap_all_node_shapes.cpp:10
...............................................................................
/tmp/tmp.npCMjCmmIL/graphviz/tests/test_edge_node_overlap_utilities.cpp:223: FAILED:
CHECK( head_node_edge_overlap <= check_options.max_node_edge_overlap + check_options.svg_rounding_error * 2 )
with expansion:
1.1804219191 <= 0.51
with messages:
Node shape: plain
Head node overlap:
width: 8.400
height: 1.180
===============================================================================
test cases: 1 | 0 passed | 1 failed
assertions: 362 | 361 passed | 1 failed
Does anyone know how we get this level of information to actually show in CI? I was under the impression --output-on-failure did this, but it seems kind of useless. I am unsure if our problem here is due to ctest, catch2, or some interaction between the two. I am also wondering if this is some kind of stderr buffering or line discipline issue.