Skip to content

Incorrect output with -R flag #386

@palage4a

Description

@palage4a

I get strange output if i run tests with -R flag and one of the tests fails. See repro.

Repro
Install luatest:

tt rocks install luatest

Version:

$ cat .rocks/share/tarantool/luatest/VERSION.lua
return '1.0.1'

Create a test:

local t = require('luatest')

local g = t.group()

local max_count = 5
local counter = 0

g.test_repro = function()
    counter = counter + 1
    t.assert_lt(counter, max_count)
end

Output:

$ .rocks/bin/luatest ./test/repro_test.lua -R 5
Tarantool version is 3.1.0-0-g663f509a2
....F

Failed tests:
-------------

1) repro.test_repro
...agecha/code/palage4a/luatest-repro///test/repro_test.lua:10: Assertion failed: 5 < 5
stack traceback:
        ...agecha/code/palage4a/luatest-repro///test/repro_test.lua:10: in function 'repro.test_repro'
        ...
        [C]: in function 'xpcall'
2) repro.test_repro
...agecha/code/palage4a/luatest-repro///test/repro_test.lua:10: Assertion failed: 5 < 5
stack traceback:
        ...agecha/code/palage4a/luatest-repro///test/repro_test.lua:10: in function 'repro.test_repro'
        ...
        [C]: in function 'xpcall'
3) repro.test_repro
...agecha/code/palage4a/luatest-repro///test/repro_test.lua:10: Assertion failed: 5 < 5
stack traceback:
        ...agecha/code/palage4a/luatest-repro///test/repro_test.lua:10: in function 'repro.test_repro'
        ...
        [C]: in function 'xpcall'
4) repro.test_repro
...agecha/code/palage4a/luatest-repro///test/repro_test.lua:10: Assertion failed: 5 < 5
stack traceback:
        ...agecha/code/palage4a/luatest-repro///test/repro_test.lua:10: in function 'repro.test_repro'
        ...
        [C]: in function 'xpcall'
5) repro.test_repro
...agecha/code/palage4a/luatest-repro///test/repro_test.lua:10: Assertion failed: 5 < 5
stack traceback:
        ...agecha/code/palage4a/luatest-repro///test/repro_test.lua:10: in function 'repro.test_repro'
        ...
        [C]: in function 'xpcall'
Ran 5 tests in 0.007 seconds, 0 succeeded, 5 failed

Failed tests:

repro.test_repro
repro.test_repro
repro.test_repro
repro.test_repro
repro.test_repro

Expected behavior:

  • Marks only failed tests as failed;
  • Shows different test results for each failed test;

Actual behavior:

  • Marks all tests as failed;
  • Shows tests results from the first failed test for all failed tests;

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions