Skip to content

argo cli submit needs retries #14106

@tooptoop4

Description

@tooptoop4

Pre-requisites

  • I have double-checked my configuration
  • I have tested with the :latest image tag (i.e. quay.io/argoproj/workflow-controller:latest) and can confirm the issue still exists on :latest. If not, I have explained why, in detail, in my description below.
  • I have searched existing issues and could not find a match for this bug
  • I'd like to contribute the fix myself (see contributing guide)

What happened? What did you expect to happen?

using argo cli to submit workflow sometimes get:

level=fatal msg="Failed to submit workflow: rpc error: code = AlreadyExists desc = Operation cannot be fulfilled on resourcequotas \"redact\": the object has been modified; please apply your changes to the latest version and try again"

return fmt.Errorf("Failed to submit workflow: %v", err)
}
if err = printWorkflow(created, common.GetFlags{Output: cliOpts.Output}); err != nil {
return err
}
return common.WaitWatchOrLog(ctx, serviceClient, namespace, []string{created.Name}, *cliOpts)
}
func submitWorkflows(ctx context.Context, serviceClient workflowpkg.WorkflowServiceClient, namespace string, workflows []wfv1.Workflow, submitOpts *wfv1.SubmitOpts, cliOpts *common.CliSubmitOpts) error {
if err := validateOptions(workflows, submitOpts, cliOpts); err != nil {
return err
}
if len(workflows) == 0 {
return errors.New("No Workflow found in given files")
}
var workflowNames []string
for _, wf := range workflows {
if wf.Namespace == "" {
// This is here to avoid passing an empty namespace when using --server-dry-run
wf.Namespace = namespace
}
err := util.ApplySubmitOpts(&wf, submitOpts)
if err != nil {
return err
}
if cliOpts.Priority != nil {
wf.Spec.Priority = cliOpts.Priority
}
options := &metav1.CreateOptions{}
if submitOpts.DryRun {
options.DryRun = []string{"All"}
}
created, err := serviceClient.CreateWorkflow(ctx, &workflowpkg.WorkflowCreateRequest{
Namespace: wf.Namespace,
Workflow: &wf,
ServerDryRun: submitOpts.ServerDryRun,
CreateOptions: options,
})
if err != nil {
return fmt.Errorf("Failed to submit workflow: %v", err)

Version(s)

3.4.11

Paste a minimal workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflow that uses private images.

n/a

Logs from the workflow controller

kubectl logs -n argo deploy/workflow-controller | grep ${workflow}

n/a

Logs from in your workflow's wait container

kubectl logs -n argo -c wait -l workflows.argoproj.io/workflow=${workflow},workflow.argoproj.io/phase!=Succeeded

n/a

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions