Running w/out Jest prevents consistent filenames

[therealpecus]

Hi!

We have over 6000 tests to run and are avoiding Jest since it adds an unbereable overhead.
Running without Jest prevents consistent test names, since the testId is automatically added.

The responsibility of the test name being unique should be in the hands of the implementator for manual tests, since changing the order of tests invalidates them (the original snapshot would not be found).

Is there a quick workaround?

Tx,
M

[NimaSoroush]

Hi @therealpecus, I am not sure what would be the best approach for unique ID generation. There is already an incremental test ID generation in place but I think for anything more sophisticated you need to have your own ID generator as part of your test and pass the test name to Differencify somehow

e.g.

const Differencify = require('differencify');
const differencify = new Differencify(GlobalOptions);

const prefix = 'test name';
let ID = 0;
const testNameGenerator = () => {
  const testName = `${testName} ${ID + 1}`
  return testName;
};

(async () => {
  await differencify
    .init({ testName: testNameGenerator() })
    .launch()
    .newPage()
    .setViewport({ width: 1600, height: 1200 })
    .goto('https://github.com/NimaSoroush/differencify')
    .waitFor(1000)
    .screenshot()
    .toMatchSnapshot()
    .result((result) => {
      console.log(result); // Prints true or false
    })
    .close()
    .end();

  await differencify
    .init({ testName: testNameGenerator() })
    .launch()
    .newPage()
    .setViewport({ width: 1600, height: 1200 })
    .goto('https://github.com/NimaSoroush/differencify')
    .waitFor(1000)
    .screenshot()
    .toMatchSnapshot()
    .result((result) => {
      console.log(result); // Prints true or false
    })
    .close()
    .end();
})();

[therealpecus]

Hi @NimaSoroush

thanks for your reply. There is an issue with the current code, whereas the internal testId is always added when tests are run outside Jest.

See this relevant snippet:

differencify/src/target.js

Lines 232 to 240 in 5ec7eef

	if (this.testConfig.isJest && !this.testConfig.testNameProvided) {
	this.testConfig.testName = this.testId
	? `${this.testStats.currentTestName} ${this.testId}`
	: this.testStats.currentTestName;
	} else {
	this.testConfig.testName = this.testId
	? `${this.testConfig.testName} ${this.testId}`
	: this.testConfig.testName;
	}

The else adds the testId even when the testConfig.testName has been provided. It happens even when testId is undefined, because of:

differencify/src/index.js

Line 60 in 5ec7eef

this.testId += 1;

which returns NaN which in turns evaluates to true.

Running unchained allows to bypass this test. We already generate unique test names for each test, and pass it as a config to init():

const asyncTest = async ([vid, breakpoint, filename, width, height, title], current = 'N/A', total = 'N/A') => {
  // avoid Differencify chaining to ensure test filenames are not polluted by ids
  const target = differencify.init({ testName: `${vid}-${width}`, chain: false });
  const page = await target.newPage();
  await page.setViewport({ width, height });
  await page.goto(`http://localhost:3000/_variants/${theme}/${filename}.html`, { waitUntil: 'networkidle2' });
  const image = await page.screenshot();

  // override filenames including a test counter
  target.testConfig.isJest = false;
  differencify.testId = undefined;
  target.testConfig.testId = undefined;

  const test = await target.toMatchSnapshot(image);
  console.info(
    `[${String(current).padStart(String(total).length)}/${total}] ${
      test ? '✅' : '❌'
    }  ${title} 'http://localhost:3000/_variants/${theme}/${filename}.html' (${breakpoint})`
  );
  await page.close();
};

We solved it by explicitely dropping out of Jest, and nullying both the global testId and test specific testId.

[NimaSoroush]

Hi @therealpecus , Sorry for the confusion. Yes, you are right and this needs to be fixed. Are you happy putting a fix for this? otherwise, I will prioritize a fix for at some point

[therealpecus]

Sure! I'll make a PR.