Proposal: Make title not optional and having two tests with the same title should fail the tests

[jfmengels]

I'm proposing two breaking changes, that would aim users to have a better test suite (or at least better documented in a way)

Making the title of a test mandatory

I can barely see why there should not be one. I can only imagine the case where the file only has one test in a file whose name is self-explanatory, but even then, an additional title can't hurt. In all other cases (like having multiple tests without a title), you're having "undocumented" tests, which is not great for your test suite. I see having a mandatory title as a feature rather than an inconvenience. (And if we do, we should probably forbid "" as the test title).

Having two tests with the same title should fail the tests

• having two tests in a file (or group, in the future) share the same title should fail the tests
• (should we implement groups) having two groups in a file or inside another group share the same title should fail the tests

The reasoning is like this: If I have two tests with the same title, I have probably copy-pasted a test without updating the title, or I have two tests that do not describe very well what they are testing. Enforcing different titles for every test should help have more descriptive tests. We already attempt to enforce that in the AVA ESLint plugin, but it would be done more effectively in the runner.

Should the title be kept optional, we would probably ignore tests without title, or accept at most one.

(I made the same proposal in an other test runner, and thought I'd propose it here too. Titles in that test runner are mandatory too by the way)

Let me know what you think :)

[novemberborn]

👍 on making titles mandatory.

Not sure about forcing titles to be unique. I run into this from time to time when using macros, though perhaps that's only because the linter cannot resolve the macro.

[jfmengels]

I run into this from time to time when using macros, though perhaps that's only because the linter cannot resolve the macro.

The linter has problems knowing whether two title expressions are similar, if they are a bit complex and have different contexts. That's why we don't report errors as soon as it's more complicated than a simple string.
The runner on the other hand would know the exact value, and doesn't have that problem. That's why I think it's a better idea to have this implemented in the runner than in the linter (not that we'd remove the rule, it can be still useful when working in the editor).

[ThomasBem]

I think both of these suggestions are valid and good improvements.

We actually had some trouble with duplicated test titles a while ago. It turned out that when we wrote our first test for a new react component. We always ended up calling it 'Render to DOM' by default. None really thought about it and it just happened out of habit.

Then at some point we where making a new component and our 'Render to DOM' test just kept on failing. It was a real simple test and we could not figure out what was wrong.

Turned out some other code had broken a previous 'Render to DOM' test in a different component and we just didn't see that it wasn't the one we where looking at...

[sindresorhus]

I'm a big 👍 on requiring unique test titles. I think that's a great change.

I would also be ok with requiring a test title for multiple tests, but I kinda like the ability to not have a title when you only have one test though. I use this a lot in my tiny modules. Example. Although, I'm happy to be convinced otherwise.

[ThomasBem]

I would like to do some work on this unless any of you guys have already started?

[jfmengels]

I kinda like the ability to not have a title when you only have one test though

Absolutely. Though in the case of your example, I would probably have split the tests up and added titles like the following:

test('should return false if process.env.TERM_PROGRAM !== "Hyper"', t => {
	process.env.TERM_PROGRAM = undefined;
	t.false(m());
});

test('should return true if process.env.TERM_PROGRAM === "Hyper"', t => {
	process.env.TERM_PROGRAM = 'Hyper';
	t.true(m());
});

// and more tests with the version argument...

...and I think that it would make the test a little easier/faster to understand, but that's my style of writing tests.

If you don't get the message of what you're testing across, you are making things harder for future readers. Having a test title is IMO one of the best ways to explain what you are testing. When the test implementation is well written and explicit, readers can get a good feeling of what is tested, but they almost necessarily need to read the code. When the test fails in your terminal/CI, having a good title show up should give you a good hint.

Anyway, I think that a test can only be made clearer by having a title, and that allowing an optional title will incite the lazier users to just not add a title. If you really don't feel like thinking of a title for a single-test function, you're always free to give it a bogus one, like the name of the tested function (test('is-hyper', t => {})), which is not that big of a deal IMO.

@ThomasBem I don't think that anybody has started on this yet. I was planning to go at it at some point, but if you feel like it, please do :)

[ThomasBem]

Anyway, I think that a test can only be made clearer by having a title, and that allowing an optional title will incite the lazier users to just not add a title. If you really don't feel like thinking of a title for a single-test function, you're always free to give it a bogus one, like the name of the tested function (test('is-hyper', t => {})), which is not that big of a deal IMO.

I agree with this. I also think that once people are forced to write a title, in many cases they will actually write something sensible. The difference between just writing 'Test' and writing something like 'Throws if no type is supplied' is pretty negligible once you have to write something.

@jfmengels I will give a shot then! Think its a good improvement and probably something I can figure out how to do :)

[novemberborn]

I would also be ok with requiring a test title for multiple tests, but I kinda like the ability to not have a title when you only have one test though. I use this a lot in my tiny modules. Example. Although, I'm happy to be convinced otherwise.

@sindresorhus, that seems like an unnecessary exception. More work on our side for little gain on the user's side.

@ThomasBem I think we have sufficient consensus on this feature. I'm adding the assigned label, so it's all yours. At the risk of being excessively clear, please note that the titles should be unique within a test file, not across all files.

[sindresorhus]

that seems like an unnecessary exception. More work on our side for little gain on the user's side.

👍 Many good arguments for why a test title makes sense here, so sure.

[sindresorhus]

Note to self: We need to adjust the test-title ESLint rule accordingly.