Skip to content

fix(drizzle): surface connection errors in beginTransaction instead of hanging forever#16220

Merged
AlessioGr merged 5 commits intomainfrom
fix/drizzle-transaction-freeze
Apr 14, 2026
Merged

fix(drizzle): surface connection errors in beginTransaction instead of hanging forever#16220
AlessioGr merged 5 commits intomainfrom
fix/drizzle-transaction-freeze

Conversation

@AlessioGr
Copy link
Copy Markdown
Member

@AlessioGr AlessioGr commented Apr 9, 2026

When beginTransaction fails to establish a database connection (e.g. due to network issues or expired AWS IAM tokens), the function hangs forever instead of throwing an error.

This happens because drizzle.transaction() rejects when the connection fails, but the old .catch(() => { // swallow }) silently ate the error. Since the transaction callback never ran, transactionReady() was never called, leaving the outer await permanently suspended. Callers never receive an error and the operation never completes.

The fix replaces the swallowed .catch with one that rejects the outer promise, so the existing try/catch block can log the real error and re-throw it to callers.

What happened in practice

In practice, this issue resulted in the job queue to randomly stop processing jobs, if a previous cron hangs during beginTransaction. The only way to get it to start processing jobs again was to restart the node process.


Comment thread test/database/int.spec.ts Outdated
})
}

it('should throw error when beginTransaction fails to connect', async () => {
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before this change, this test failed on postgres/sqlite through timing out

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 9, 2026

📦 esbuild Bundle Analysis for payload

This analysis was generated by esbuild-bundle-analyzer. 🤖

Meta File Out File Size (raw) Note
packages/next/meta_index.json esbuild/index.js 984.34 KB ✅ No change
packages/payload/meta_index.json esbuild/index.js 1.38 MB ✅ No change
packages/payload/meta_shared.json esbuild/exports/shared.js 191.27 KB ✅ No change
packages/richtext-lexical/meta_client.json esbuild/exports/client_optimized/index.js 288.93 KB ⚠️ +8.36 KB (+3.0%)
packages/ui/meta_client.json esbuild/exports/client_optimized/index.js 1.18 MB ✅ No change
packages/ui/meta_shared.json esbuild/exports/shared_optimized/index.js 16.32 KB ✅ No change
Largest paths These visualization shows top 20 largest paths in the bundle.

Meta file: packages/next/meta_index.json, Out file: esbuild/index.js

Path Size
../../node_modules ${{\color{Goldenrod}{ ████████████████████▌ }}}$ 82.4%, 807.63 KB
dist/views/Version ${{\color{Goldenrod}{ █▎ }}}$ 5.3%, 51.49 KB
dist/views/Dashboard ${{\color{Goldenrod}{ ▌ }}}$ 2.2%, 21.37 KB
dist/views/Document ${{\color{Goldenrod}{ ▍ }}}$ 1.7%, 16.59 KB
dist/views/List ${{\color{Goldenrod}{ ▎ }}}$ 1.2%, 11.38 KB
dist/views/Root ${{\color{Goldenrod}{ ▏ }}}$ 0.9%, 9.03 KB
dist/views/Versions ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 6.17 KB
dist/views/API ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 6.13 KB
dist/elements/Nav ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 5.96 KB
dist/views/Account ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 5.55 KB
dist/elements/DocumentHeader ${{\color{Goldenrod}{ ▏ }}}$ 0.5%, 4.81 KB
dist/views/Login ${{\color{Goldenrod}{ }}}$ 0.4%, 4.40 KB
dist/layouts/Root ${{\color{Goldenrod}{ }}}$ 0.3%, 3.20 KB
dist/views/ForgotPassword ${{\color{Goldenrod}{ }}}$ 0.3%, 3.13 KB
dist/views/CreateFirstUser ${{\color{Goldenrod}{ }}}$ 0.3%, 2.81 KB
dist/templates/Default ${{\color{Goldenrod}{ }}}$ 0.3%, 2.64 KB
dist/views/BrowseByFolder ${{\color{Goldenrod}{ }}}$ 0.3%, 2.61 KB
dist/views/CollectionFolders ${{\color{Goldenrod}{ }}}$ 0.2%, 2.44 KB
dist/views/ResetPassword ${{\color{Goldenrod}{ }}}$ 0.2%, 2.40 KB
dist/views/Logout ${{\color{Goldenrod}{ }}}$ 0.2%, 1.94 KB
(other) ${{\color{Goldenrod}{ ████▍ }}}$ 17.6%, 172.03 KB

Meta file: packages/payload/meta_index.json, Out file: esbuild/index.js

Path Size
../../node_modules ${{\color{Goldenrod}{ █████████████████▏ }}}$ 68.6%, 942.29 KB
dist/fields/hooks ${{\color{Goldenrod}{ ▊ }}}$ 3.2%, 44.01 KB
dist/collections/operations ${{\color{Goldenrod}{ ▋ }}}$ 2.9%, 39.93 KB
dist/versions/migrations ${{\color{Goldenrod}{ ▎ }}}$ 1.3%, 18.50 KB
dist/auth/operations ${{\color{Goldenrod}{ ▎ }}}$ 1.1%, 15.63 KB
dist/fields/config ${{\color{Goldenrod}{ ▎ }}}$ 1.0%, 14.16 KB
dist/globals/operations ${{\color{Goldenrod}{ ▎ }}}$ 1.0%, 13.28 KB
dist/utilities/configToJSONSchema.js ${{\color{Goldenrod}{ ▎ }}}$ 1.0%, 13.13 KB
dist/queues/operations ${{\color{Goldenrod}{ ▏ }}}$ 0.9%, 12.69 KB
dist/fields/validations.js ${{\color{Goldenrod}{ ▏ }}}$ 0.8%, 10.54 KB
dist/bin/generateImportMap ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 8.91 KB
dist/collections/config ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 8.37 KB
dist/config/orderable ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 8.00 KB
dist/index.js ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 7.69 KB
dist/uploads/fetchAPI-multipart ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 7.67 KB
dist/database/migrations ${{\color{Goldenrod}{ ▏ }}}$ 0.5%, 7.54 KB
dist/config/sanitize.js ${{\color{Goldenrod}{ ▏ }}}$ 0.5%, 6.26 KB
dist/collections/endpoints ${{\color{Goldenrod}{ ▏ }}}$ 0.5%, 6.23 KB
dist/auth/strategies ${{\color{Goldenrod}{ }}}$ 0.4%, 5.50 KB
dist/queues/config ${{\color{Goldenrod}{ }}}$ 0.4%, 5.31 KB
(other) ${{\color{Goldenrod}{ ███████▊ }}}$ 31.4%, 431.34 KB

Meta file: packages/payload/meta_shared.json, Out file: esbuild/exports/shared.js

Path Size
../../node_modules ${{\color{Goldenrod}{ ███████████████████▊ }}}$ 79.4%, 148.89 KB
dist/fields/validations.js ${{\color{Goldenrod}{ █▍ }}}$ 5.6%, 10.54 KB
dist/config/orderable ${{\color{Goldenrod}{ ▍ }}}$ 1.7%, 3.13 KB
dist/fields/baseFields ${{\color{Goldenrod}{ ▍ }}}$ 1.5%, 2.79 KB
dist/utilities/deepCopyObject.js ${{\color{Goldenrod}{ ▎ }}}$ 1.4%, 2.54 KB
dist/auth/cookies.js ${{\color{Goldenrod}{ ▏ }}}$ 0.8%, 1.55 KB
dist/utilities/flattenTopLevelFields.js ${{\color{Goldenrod}{ ▏ }}}$ 0.8%, 1.42 KB
dist/fields/config ${{\color{Goldenrod}{ ▏ }}}$ 0.7%, 1.28 KB
dist/utilities/getVersionsConfig.js ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 1.04 KB
dist/utilities/flattenAllFields.js ${{\color{Goldenrod}{ ▏ }}}$ 0.5%, 943 B
dist/folders/utils ${{\color{Goldenrod}{ ▏ }}}$ 0.5%, 916 B
dist/utilities/unflatten.js ${{\color{Goldenrod}{ }}}$ 0.4%, 779 B
dist/utilities/sanitizeUserDataForEmail.js ${{\color{Goldenrod}{ }}}$ 0.4%, 713 B
dist/utilities/getFieldPermissions.js ${{\color{Goldenrod}{ }}}$ 0.3%, 651 B
dist/collections/config ${{\color{Goldenrod}{ }}}$ 0.3%, 570 B
dist/bin/generateImportMap ${{\color{Goldenrod}{ }}}$ 0.3%, 561 B
dist/auth/sessions.js ${{\color{Goldenrod}{ }}}$ 0.3%, 525 B
dist/fields/getFieldPaths.js ${{\color{Goldenrod}{ }}}$ 0.3%, 485 B
dist/utilities/getSafeRedirect.js ${{\color{Goldenrod}{ }}}$ 0.2%, 423 B
dist/utilities/deepMerge.js ${{\color{Goldenrod}{ }}}$ 0.2%, 413 B
(other) ${{\color{Goldenrod}{ █████▏ }}}$ 20.6%, 38.72 KB

Meta file: packages/richtext-lexical/meta_client.json, Out file: esbuild/exports/client_optimized/index.js

Path Size
dist/features/blocks ${{\color{Goldenrod}{ ███▏ }}}$ 12.7%, 36.34 KB
dist/lexical/plugins ${{\color{Goldenrod}{ ██▊ }}}$ 11.4%, 32.65 KB
dist/lexical/ui ${{\color{Goldenrod}{ ██▏ }}}$ 8.5%, 24.36 KB
dist/features/experimental_table ${{\color{Goldenrod}{ ██ }}}$ 8.3%, 23.70 KB
dist/packages/@lexical ${{\color{Goldenrod}{ █▋ }}}$ 6.6%, 18.99 KB
dist/features/link ${{\color{Goldenrod}{ █▌ }}}$ 6.4%, 18.24 KB
dist/features/toolbars ${{\color{Goldenrod}{ █▍ }}}$ 5.6%, 16.08 KB
dist/features/upload ${{\color{Goldenrod}{ █▏ }}}$ 4.8%, 13.77 KB
dist/features/textState ${{\color{Goldenrod}{ ▉ }}}$ 3.9%, 11.08 KB
dist/features/relationship ${{\color{Goldenrod}{ ▊ }}}$ 3.2%, 9.03 KB
dist/lexical/utils ${{\color{Goldenrod}{ ▊ }}}$ 3.0%, 8.49 KB
dist/features/converters ${{\color{Goldenrod}{ ▋ }}}$ 2.9%, 8.36 KB
dist/features/debug ${{\color{Goldenrod}{ ▋ }}}$ 2.6%, 7.40 KB
dist/utilities/fieldsDrawer ${{\color{Goldenrod}{ ▋ }}}$ 2.5%, 7.15 KB
dist/lexical/config ${{\color{Goldenrod}{ ▍ }}}$ 1.8%, 5.08 KB
dist/features/lists ${{\color{Goldenrod}{ ▍ }}}$ 1.8%, 5.00 KB
dist/features/format ${{\color{Goldenrod}{ ▎ }}}$ 1.2%, 3.46 KB
dist/lexical/LexicalEditor.js ${{\color{Goldenrod}{ ▎ }}}$ 1.1%, 3.23 KB
dist/field/Field.js ${{\color{Goldenrod}{ ▎ }}}$ 1.0%, 2.80 KB
dist/lexical/nodes ${{\color{Goldenrod}{ ▏ }}}$ 0.9%, 2.66 KB
(other) ${{\color{Goldenrod}{ █████████████████████▊ }}}$ 87.3%, 249.28 KB

Meta file: packages/ui/meta_client.json, Out file: esbuild/exports/client_optimized/index.js

Path Size
../../node_modules ${{\color{Goldenrod}{ ████████████▎ }}}$ 49.3%, 579.12 KB
dist/elements/FolderView ${{\color{Goldenrod}{ ▋ }}}$ 2.5%, 29.38 KB
dist/elements/BulkUpload ${{\color{Goldenrod}{ ▌ }}}$ 2.4%, 28.24 KB
dist/elements/WhereBuilder ${{\color{Goldenrod}{ ▍ }}}$ 1.5%, 17.38 KB
dist/views/Edit ${{\color{Goldenrod}{ ▍ }}}$ 1.5%, 17.30 KB
dist/forms/Form ${{\color{Goldenrod}{ ▎ }}}$ 1.4%, 15.91 KB
dist/fields/Relationship ${{\color{Goldenrod}{ ▎ }}}$ 1.3%, 15.79 KB
dist/elements/Table ${{\color{Goldenrod}{ ▎ }}}$ 1.3%, 15.77 KB
dist/fields/Upload ${{\color{Goldenrod}{ ▎ }}}$ 1.2%, 14.22 KB
dist/fields/Blocks ${{\color{Goldenrod}{ ▎ }}}$ 1.2%, 13.90 KB
dist/elements/QueryPresets ${{\color{Goldenrod}{ ▏ }}}$ 0.9%, 10.36 KB
dist/elements/PublishButton ${{\color{Goldenrod}{ ▏ }}}$ 0.8%, 9.11 KB
dist/providers/Folders ${{\color{Goldenrod}{ ▏ }}}$ 0.7%, 8.46 KB
dist/elements/HTMLDiff ${{\color{Goldenrod}{ ▏ }}}$ 0.7%, 8.38 KB
dist/elements/ListHeader ${{\color{Goldenrod}{ ▏ }}}$ 0.7%, 8.06 KB
dist/fields/Array ${{\color{Goldenrod}{ ▏ }}}$ 0.7%, 7.73 KB
dist/views/CollectionFolder ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 7.50 KB
dist/views/List ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 7.36 KB
dist/elements/ReactSelect ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 7.33 KB
dist/elements/LivePreview ${{\color{Goldenrod}{ ▏ }}}$ 0.6%, 7.03 KB
(other) ${{\color{Goldenrod}{ ████████████▋ }}}$ 50.7%, 595.21 KB

Meta file: packages/ui/meta_shared.json, Out file: esbuild/exports/shared_optimized/index.js

Path Size
dist/graphics/Logo ${{\color{Goldenrod}{ █████ }}}$ 20.0%, 3.12 KB
../../node_modules ${{\color{Goldenrod}{ ████▎ }}}$ 17.0%, 2.65 KB
dist/graphics/Icon ${{\color{Goldenrod}{ ██▍ }}}$ 9.8%, 1.52 KB
dist/utilities/formatDocTitle ${{\color{Goldenrod}{ ██▏ }}}$ 8.5%, 1.32 KB
dist/providers/TableColumns ${{\color{Goldenrod}{ █▍ }}}$ 5.5%, 862 B
dist/utilities/groupNavItems.js ${{\color{Goldenrod}{ █▎ }}}$ 5.2%, 814 B
dist/utilities/getGlobalData.js ${{\color{Goldenrod}{ █▏ }}}$ 4.9%, 762 B
dist/utilities/api.js ${{\color{Goldenrod}{ █▏ }}}$ 4.8%, 756 B
dist/elements/Translation ${{\color{Goldenrod}{ ▊ }}}$ 3.2%, 493 B
dist/utilities/handleTakeOver.js ${{\color{Goldenrod}{ ▋ }}}$ 2.8%, 440 B
dist/utilities/traverseForLocalizedFields.js ${{\color{Goldenrod}{ ▋ }}}$ 2.6%, 399 B
dist/elements/withMergedProps ${{\color{Goldenrod}{ ▌ }}}$ 2.2%, 339 B
dist/utilities/getVisibleEntities.js ${{\color{Goldenrod}{ ▌ }}}$ 2.1%, 329 B
dist/utilities/getNavGroups.js ${{\color{Goldenrod}{ ▍ }}}$ 1.9%, 301 B
dist/elements/WithServerSideProps ${{\color{Goldenrod}{ ▍ }}}$ 1.5%, 232 B
dist/utilities/handleGoBack.js ${{\color{Goldenrod}{ ▎ }}}$ 1.2%, 180 B
dist/fields/mergeFieldStyles.js ${{\color{Goldenrod}{ ▎ }}}$ 1.0%, 159 B
dist/utilities/handleBackToDashboard.js ${{\color{Goldenrod}{ ▎ }}}$ 1.0%, 152 B
dist/forms/Form ${{\color{Goldenrod}{ ▏ }}}$ 0.9%, 147 B
dist/utilities/abortAndIgnore.js ${{\color{Goldenrod}{ ▏ }}}$ 0.9%, 146 B
(other) ${{\color{Goldenrod}{ ████████████████████ }}}$ 80.0%, 12.51 KB
Details

Next to the size is how much the size has increased or decreased compared with the base branch of this PR.

  • ‼️: Size increased by 20% or more. Special attention should be given to this.
  • ⚠️: Size increased in acceptable range (lower than 20%).
  • ✅: No change or even downsized.
  • 🗑️: The out file is deleted: not found in base branch.
  • 🆕: The out file is newly found: will be added to base branch.

@DanRibbens DanRibbens requested a review from r1tsuu April 9, 2026 13:17
Copy link
Copy Markdown
Member

@r1tsuu r1tsuu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some failing tests but I think it's expected, just need to suppress them

Comment thread test/database/int.spec.ts
Comment thread test/database/int.spec.ts
@AlessioGr AlessioGr requested a review from r1tsuu April 9, 2026 22:05
@AlessioGr AlessioGr enabled auto-merge (squash) April 9, 2026 22:05
@AlessioGr AlessioGr merged commit 1e1c591 into main Apr 14, 2026
166 checks passed
@AlessioGr AlessioGr deleted the fix/drizzle-transaction-freeze branch April 14, 2026 16:58
@github-actions
Copy link
Copy Markdown
Contributor

🚀 This is included in version v3.83.0

milamer pushed a commit to milamer/payload that referenced this pull request Apr 20, 2026
…f hanging forever (payloadcms#16220)

When `beginTransaction` fails to establish a database connection (e.g.
due to network issues or expired AWS IAM tokens), the function hangs
forever instead of throwing an error.

This happens because `drizzle.transaction()` rejects when the connection
fails, but the old `.catch(() => { // swallow })` silently ate the
error. Since the transaction callback never ran, `transactionReady()`
was never called, leaving the outer `await` permanently suspended.
Callers never receive an error and the operation never completes.

The fix replaces the swallowed `.catch` with one that rejects the outer
promise, so the existing `try/catch` block can log the real error and
re-throw it to callers.

## What happened in practice

In practice, this issue resulted in the job queue to randomly stop
processing jobs, if a previous cron hangs during `beginTransaction`. The
only way to get it to start processing jobs again was to restart the
node process.

---
- To see the specific tasks where the Asana app for GitHub is being
used, see below:
  - https://app.asana.com/0/0/1213983836266037
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants