First Application deploy to Swarm worker fails with unauthorized / No such image, retry succeeds

[joker791]

To Reproduce

1. Add a private registry in Dokploy

2. Prepare a Docker Swarm cluster with at least 2 nodes:

   • manager
   • worker

3. Create an Application in Dokploy:

   • service type: Application
   • use any normal app source that Dokploy can build and deploy
   • configure it to deploy to the Swarm cluster

4. In the Advanced settings:

   • select the private registry in Advanced / Cluster Settings

5. Trigger the first deploy.

6. Observe that the task scheduled to the worker node fails to pull the image.
   On the worker node (dokploy-cluster02), Docker daemon logs show errors like:

   • unauthorized: authentication required
   • pull access denied
   • No such image

7. Without changing registry credentials, trigger deploy again.

8. Observe that the worker node can now pull the same image successfully and the service starts correctly.

Current vs. Expected behavior

Expected behavior

The first deploy should succeed on the worker node without requiring a second deploy.

Actual behavior

The first deploy fails on the worker with registry authentication errors, but a later deploy succeeds.

Provide environment information

Operating System:
  OS: Ubuntu 24.04
  Arch: amd64
Dokploy version: 0.28.6
VPS Provider: Local VM
What applications/services are you trying to deploy?
  Staic Web

Which area(s) are affected? (Select all that apply)

Application

Are you deploying the applications where Dokploy is installed or on a remote server?

Both

Additional context

• This issue happens with Application deployments
• Manual Swarm CLI test works on the first attempt:
  docker service create --with-registry-auth ...
• This suggests the issue is likely in Dokploy’s Application deployment path rather than Docker Swarm itself.

Will you send a PR to fix it?

No

[dosubot]

Hi @joker791! I'm Dosu and I’m helping the dokploy team.

This is a confirmed bug in Dokploy. You've identified it correctly - the Application deployment path is missing a docker login call before creating/updating Swarm services.

Root Cause: The mechanizeDockerContainer function only passes authconfig to the Dockerode API, but Docker Swarm requires credentials to be cached in ~/.docker/config.json on the manager node to distribute auth to worker nodes. The authconfig parameter alone is insufficient.

Interestingly, the rollback code already has the fix - it explicitly calls docker login before service updates with a comment explaining:

"The authconfig in CreateServiceOptions alone is not sufficient — Docker Swarm also relies on the daemon's cached credentials (~/.docker/config.json) to distribute auth to nodes."

Related issues: #1998, #3111, #3403, #3861

Workaround: Manually run docker login on the manager node before your first deploy, or trigger a second deploy (as you've discovered).

To reply, just mention @dosu.

---

Docs are dead. Just use Dosu.

[joker791]

@dosu Thanks for the analysis.

I tested the suggested workaround, but it does not resolve the issue in my case.

What I have already verified:

1. I already ran docker login manually before the first deploy.
2. I also tested with manual login on all relevant nodes, not just the manager.
3. The first Dokploy Application deploy still fails on the worker with:
   • unauthorized: authentication required
   • pull access denied
   • No such image

[dosubot]

Thanks for testing that - I appreciate the additional details. Let me dig deeper into what's happening.

After further investigation, I found something important: the docker login that happens during builds is only executed on the build server (manager node) to push images. Even though you ran docker login manually on all nodes, there may be a gap in how Dokploy propagates credentials when creating the Swarm service.

The authconfig parameter passed to the Dockerode API is supposed to distribute credentials to workers, but the codebase itself acknowledges that this mechanism can fail under certain conditions.

Could you provide more details?

1. What type of private registry are you using? (Docker Hub, AWS ECR, Harbor, self-hosted, etc.)
2. Is your registry using a non-standard port or path?
3. When you run docker login manually, are you using the exact same registry URL that Dokploy uses? (Check the registryUrl in Dokploy's registry configuration)
4. What do the worker node Docker daemon logs show immediately when the pull fails? (journalctl -u docker -f on the worker during deploy)

This could be a credential format mismatch, registry URL inconsistency, or a Docker daemon credential caching issue specific to certain registry types. The fact that retry works but fresh manual login doesn't suggests the issue might be more nuanced than the standard authentication gap.

To reply, just mention @dosu.

---

Share context across your team and agents. Try Dosu.

[joker791]

@dosu Thanks for digging deeper.

Here are the additional details:

1. Registry type
   I tested with both:

   • Aliyun Container Registry
   • Harbor

   So this does not seem specific to one registry vendor.

2. Registry port / path
   No non-standard port is used.
   No special/custom path setup is involved.

3. Registry URL consistency
   Yes — the manual docker login uses the exact same registry URL configured in Dokploy.

4. Worker node Docker daemon logs during the failed first deploy
   After keeping only the key lines, the worker shows:

   level=error msg="Not continuing with pull after error" error="errors:
   denied: requested access to the resource is denied
   unauthorized: authentication required"
   
   level=error msg="pulling image failed" error="pull access denied for <registry>/<repo>/<image>, repository does not exist or may require 'docker login': denied: requested access to the resource is denied" module=node/agent/taskmanager
   
   level=error msg="fatal task error" error="No such image: <registry>/<repo>/<image>:latest" module=node/agent/taskmanager
   
   

If I trigger Deploy again in Dokploy immediately after this failure, the image pull succeeds and the service starts normally.

[dosubot]

Thanks for the detailed logs - this confirms the root cause. The issue is deeper than just missing docker login - it's a Dockerode API limitation.

Why docker login on all nodes doesn't help: Docker Swarm requires the manager to actively distribute credentials to workers during service operations. This is done via the --with-registry-auth CLI flag, which tells Docker to encrypt and send the manager's credentials to workers alongside the task. However, Dockerode (the Node.js Docker library Dokploy uses) doesn't support this flag.

The mechanizeDockerContainer function only passes authconfig to the Dockerode API, which authenticates the manager's API request but doesn't trigger credential distribution to workers.

Why the second deploy works: During the first deploy, credentials get cached on the manager. The second attempt may succeed due to:

• Tasks getting scheduled on the manager node (which has credentials)
• Partial image layer caching from the first attempt
• Lucky timing/scheduling

This is probabilistic and unreliable.

The fix would require Dokploy to replace Dockerode's createService()/service.update() calls with shell commands that support --with-registry-auth - similar to how Compose deployments already work, which correctly uses this flag.

Workaround: After the first deploy fails, you can manually run on the manager:

docker service update --with-registry-auth --force <service-name>

This forces Docker to distribute the cached credentials to workers. Related discussion: #3111.

To reply, just mention @dosu.

---

Docs are dead. Just use Dosu.

[joker791]

@dosu Thanks, but I want to clarify one important point:

The second Dokploy deploy is not probabilistic in my testing.

It is 100% reproducible:

• if the first deploy creates the Swarm service but fails to pull on the worker
• then running Deploy again in Dokploy
• will consistently succeed and the worker can pull the image normally

So I do not think this is explained by:

• lucky scheduling
• manager-side task placement
• partial image layer caching
• or timing

This looks much more deterministic:

• initial service creation path fails to propagate registry auth correctly
• subsequent deploys succeed once the service already exists

That makes me suspect the problem is specifically related to the difference between:

• the create path for a brand new Swarm service
• and the update path for an existing Swarm service

[dosubot]

You're absolutely right - this is deterministic, not probabilistic. I found the root cause.

The bug: There's an API inconsistency in Dockerode that Dokploy's code doesn't account for:

• service.update() auto-extracts authconfig from the options object ✅
• docker.createService() does NOT extract authconfig when called with a single argument ❌

Looking at the code:

// UPDATE path - authconfig is auto-extracted from spread settings
await service.update({ ...settings, /* ... */ });

// CREATE path - authconfig in settings is IGNORED!
await docker.createService(settings);

The fix requires using the two-argument signature for createService():

await docker.createService(settings.authconfig, settings);

Why your pattern is 100% reproducible:

1. First deploy calls createService(settings) → authconfig ignored → service is created but tasks fail to pull
2. Service still exists in Swarm (even with failing tasks)
3. Second deploy: service.inspect() succeeds → takes UPDATE path → service.update() properly extracts authconfig → works

This exact fix is in PR #3784 and PR #3863, but they haven't been merged yet. This confirms your issue and the fix is known - it just needs to land in a release.

To reply, just mention @dosu.

---

Docs are dead. Just use Dosu.

[joker791]

@dosu Thanks, this matches my testing.

Glad the root cause is identified. I’ll wait for the new release with the fix.

[dosubot]

Glad we could pinpoint it! Your detailed testing (especially noting it was 100% reproducible, not random) was really helpful in narrowing down the CREATE vs UPDATE path difference.

In the meantime, if you need a workaround, triggering a second deploy immediately after the first failure should consistently work since the service will exist and take the UPDATE path. Not ideal, but functional until the fix lands.

To reply, just mention @dosu.

---

Docs are dead. Just use Dosu.