RHTAP nightly tests failing on nodejs-basic sample

[michael-valdron]

Which area this feature is related to?

/kind bug

Which area this bug is related to?

/area registry

What versions of software are you using?

Go project

Operating System and version: N/A

Go Pkg Version: 1.18

Bug Summary

Describe the bug:

Nightly testing of the community devfile registry stacks/samples on RHTAP is failing when testing the nodejs-basic.

To Reproduce:

Run the RHTAP test suite on the registry content.

Expected behavior

Test suite should run without failure status.

Any logs, error output, screenshots etc? Provide the devfile that sees this bug, if applicable

See for full log: https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-devfile-registry-main-v4.12-rhtap-nightly-devfile-compatibility/1707002224158183424

INFO[2023-09-27T13:13:47Z] Ran for 1h13m20s                             
ERRO[2023-09-27T13:13:47Z] Some steps failed:                           
ERRO[2023-09-27T13:13:47Z] 
  * could not run steps: step rhtap-nightly-devfile-compatibility failed: "rhtap-nightly-devfile-compatibility" test steps failed: "rhtap-nightly-devfile-compatibility" pod "rhtap-nightly-devfile-compatibility-devfile-rhtap-nightly" failed: could not watch pod: the pod ci-op-y5dwnzzm/rhtap-nightly-devfile-compatibility-devfile-rhtap-nightly failed after 22m34s (failed containers: test): ContainerFailed one or more containers exited
Container test exited with code 1, reason Error
---
son: Running
  PipelineRun devfile-sample-tv789 reason: Running
  PipelineRun devfile-sample-tv789 reason: Running
  PipelineRun devfile-sample-tv789 reason: Running
  PipelineRun devfile-sample-tv789 reason: Completed
  [FAILED] in [AfterAll] - /tmp/tmp.8s3irlKe7O/registry/tests/rhtap/rhtap_test.go:61 @ 09/27/23 12:55:49.29
  << Timeline
  [FAILED] Expected success, but got an error:
      <*errors.errorString | 0xc0003e6cc0>: 
      timed out waiting for the condition
      {
          s: "timed out waiting for the condition",
      }
  In [AfterAll] at: /tmp/tmp.8s3irlKe7O/registry/tests/rhtap/rhtap_test.go:61 @ 09/27/23 12:55:49.29 

Additional context

Any workaround?

Unknown at this time.

Suggestion on how to fix the bug

Unknown at this time.

[michael-valdron]

Mostly likely the result of devfile/registry#205 changes, there is a mismatch between the nodejs-basic base image and the parent nodejs base image:

Sample Image Ref: https://github.com/nodeshift-starters/devfile-sample/blob/cb64992eebf5c18900d283a9bca08b4ab5db2874/Dockerfile#L14
Parent Image Ref: https://github.com/devfile/registry/blob/f6f5fba56ceb559181ec91af68ea15c322b9d785/stacks/nodejs/2.1.1/devfile.yaml#L22

[thepetk]

Created a PR to rollback the updates introduced by devfile/registry#205

[michael-valdron]

Nightly tests are still unverified, will re-open until they are.

[thepetk]

Looking further into the issue I see that this update inside the sample might have caused the issue (merged 11 Aug).

Also I see in the job history section that we also have successful runs for this job. I can identify as the job task failing the rhtap-nightly-devfile-compatibility. The first time I can see this failing is https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-devfile-registry-main-v4.12-rhtap-nightly-devfile-compatibility/1696311725915115520 (date: 19 Aug - no records existing before that point so we may have even older failures)

I think the best approach to fix this incompatibility issue is to:

• Try to merge the changes introduced by PR#196
• Remove the ignore rules from the renovate.json

cc @michael-valdron

[michael-valdron]

Nightly test still failing: https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-devfile-registry-main-v4.12-rhtap-nightly-devfile-compatibility/1707407999657578496

Keeping open..