test-runner: Assert stack trace is stripped when used within `test`

[Ethan-Arrowood]

Version

20.3.1

Platform

Darwin Ethans-MBP 22.5.0 Darwin Kernel Version 22.5.0: Thu Jun 8 22:22:20 PDT 2023; root:xnu-8796.121.3~7/RELEASE_ARM64_T6000 arm64

Subsystem

test-runner

What steps will reproduce the bug?

Clone: https://github.com/Ethan-Arrowood/node-test-runner-assert-stack-trace-issue

Run: npm run expected

Then: npm run actual

See difference.

Expected:

❯ npm run expected

> expected
> node expected.js

AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:

0 !== 1

    at err (file:///Users/ethanarrowood/Documents/github/node-test-runner-assert-stack-trace/expected.js:5:16)
    at file:///Users/ethanarrowood/Documents/github/node-test-runner-assert-stack-trace/expected.js:11:9
    at ModuleJob.run (node:internal/modules/esm/module_job:192:25)
    at async DefaultModuleLoader.import (node:internal/modules/esm/loader:246:24)
    at async loadESM (node:internal/process/esm_loader:40:7)
    at async handleMainPromise (node:internal/modules/run_main:66:12)

Actual:

❯ npm run actual

> actual
> node --test --test-reporter=./reporter.cjs test.js

Error [ERR_TEST_FAILURE]: Expected values to be strictly equal:

0 !== 1

How often does it reproduce? Is there a required condition?

Any assert failure within a test block causes issue.

What is the expected behavior? Why is that the expected behavior?

I expect to get the complete stack trace from assert in my test reporter

What do you see instead?

Incomplete stack trace

Additional information

No response

[Ethan-Arrowood]

Also happens with

describe('tests', () => {
    it('should fail', () => {
        assert.strictEqual(0, 1);
    })
})

[Ethan-Arrowood]

I think it might be this line:

node/lib/internal/errors.js

Line 1612 in c301404

hideInternalStackFrames(this);

causing the problem

[Ethan-Arrowood]

But also, this may not be a bug and is instead working as intended. I've noticed that the original error is available as error.cause. This solves the original issue; albeit a little confusing. Maybe all that is necessary is a documentation update.

[cjihrig]

The ERR_TEST_FAILURE error type is mostly a wrapper that the test runner uses. The built in reporter unwraps it here. Your reporter should probably do something similar.

Changing the error type to unwrap it in core would be a breaking change at this point (but could technically still be done). I think the docs are wrong too, as they currently state:

The error thrown by the test.

But this appears to be the wrapped error.

[Ethan-Arrowood]

Yeah sounds good. I'm at the same conclusion now myself after tinkering with it in core a bit. I will send a docs update to close out this issue