[go: up one dir, main page]

ctest failures invisible

I’m trying to resurrect !3029 (closed) on which progress seems to have stalled. I think I’ve solved all the failures except a single unit test in the ctest suite. You can see the failure here, https://gitlab.com/graphviz/graphviz/-/jobs/3899795706. As observable, the failure output is quite unhelpful.

Running the equivalent steps locally, I can reproduce the failure. Locally I get no output at all, not even the cryptic “suppressions used…” text. Toggling --output-on-failure and $CTEST_OUTPUT_ON_FAILURE has no effect. Trying --verbose and --extra-verbose alters the output, but still provides no information about the test failure.

Bypassing ctest altogether and running the unit test directly finally gets us the failure information:

$ env LD_LIBRARY_PATH=${PREFIX}/lib ./tests/test_edge_node_overlap_all_node_shapes

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
test_edge_node_overlap_all_node_shapes is a Catch v2.13.8 host application.
Run with -? for options

-------------------------------------------------------------------------------
Overlap all node shapes
-------------------------------------------------------------------------------
/tmp/tmp.npCMjCmmIL/graphviz/tests/test_edge_node_overlap_all_node_shapes.cpp:10
...............................................................................

/tmp/tmp.npCMjCmmIL/graphviz/tests/test_edge_node_overlap_utilities.cpp:223: FAILED:
  CHECK( head_node_edge_overlap <= check_options.max_node_edge_overlap + check_options.svg_rounding_error * 2 )
with expansion:
  1.1804219191 <= 0.51
with messages:
  Node shape: plain
  Head node overlap:
    width:  8.400
    height: 1.180

===============================================================================
test cases:   1 |   0 passed | 1 failed
assertions: 362 | 361 passed | 1 failed

Does anyone know how we get this level of information to actually show in CI? I was under the impression --output-on-failure did this, but it seems kind of useless. I am unsure if our problem here is due to ctest, catch2, or some interaction between the two. I am also wondering if this is some kind of stderr buffering or line discipline issue.