Skip to content

test: Enable specifying flaky tests on fips#16329

Closed
komawar wants to merge 1 commit intonodejs:masterfrom
komawar:enable-flaky-on-fips
Closed

test: Enable specifying flaky tests on fips#16329
komawar wants to merge 1 commit intonodejs:masterfrom
komawar:enable-flaky-on-fips

Conversation

@komawar
Copy link
Contributor

@komawar komawar commented Oct 20, 2017

Adds a way to mark a specified test as 'flaky' on fips compliant
systems.

Earlier, the tools/test.py script supported only 'mode',
'system' and 'arch' for test environment specification. This limits the
ability to specify the behavior of tests and setting pre-determined
behavior of the same on other types of systems. As an example, the
feature request below indicates the need to specify certain tests as
'flaky' on fips compliant systems. It hints at future possibility of a
shared library, which in turn may need a specifier for running tests.

This commit introduces a new item in the env dict, called type
which defaults to simple type. It also adds an optional command
line argument --type, which inputs strings. Current functionality
extends to setting simple or fips for this type variable.
However, extending it to further uses is rather simple by adding "if"
conditions at appropriate places in the tools/test.py script.

Fixes: #14746

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines
Affected core subsystem(s)
  • build
  • doc
  • test

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python PRs and issues that require attention from people who are familiar with Python. test Issues and PRs related to the tests. tools Issues and PRs related to the tools directory.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature Request: mark tests flaky on FIPS

9 participants