Celery Release Notes

What's Changed

• Add a smoke test workflow that runs against py-amqp, Kombu and Billiard main by @GangEunzzang in https://github.com/celery/celery/pull/10676
• fix(backends/couchbase): missing document raises instead of returning None by @simpleqt in https://github.com/celery/celery/pull/10680
• Preserve propagate in nested native result joins by @GangEunzzang in https://github.com/celery/celery/pull/10689
• Chore(deps): Update boto3 requirement from >=1.43.93 to >=1.43.98 by @dependabot[bot] in https://github.com/celery/celery/pull/10695
• Fix cache directory ownership in Docker builds by @phanky1 in https://github.com/celery/celery/pull/10707
• Modify apt-get install to add libmemcached-dev and libev-dev by @auvipy in https://github.com/celery/celery/pull/10709
• Chore(deps): Update click requirement from <9.0,>=8.1.2 to >=8.5.0,<9.0 by @dependabot[bot] in https://github.com/celery/celery/pull/10696
• Chore(deps): Update tox requirement from >=4.61.4 to ==4.64.3 by @dependabot[bot] in https://github.com/celery/celery/pull/10684
• fix: add missing spaces in three backend exception messages by @simpleqt in https://github.com/celery/celery/pull/10650
• docs: fix the malformed celery[tblib] bundle term in README by @simpleqt in https://github.com/celery/celery/pull/10651
• Fix escaped percent signs in node formatting by @jakezwang in https://github.com/celery/celery/pull/10663 …

What's Changed

• Fix Django worker recursion bug + defensive checks for pool_cls.module by @maycuatroi1 in https://github.com/celery/celery/pull/10048
• Update elasticsearch requirement from <=9.1.2 to <=9.2.1 by @dependabot[bot] in https://github.com/celery/celery/pull/10053
• Revert "Update elasticsearch requirement from <=9.1.2 to <=9.2.1" by @auvipy in https://github.com/celery/celery/pull/10054
• Docs: Update user_preload_options example to use click. by @jorsyk in https://github.com/celery/celery/pull/10056
• Fix invalid configuration key "bootstrap_servers" in Kafka demo by @jorsyk in https://github.com/celery/celery/pull/10060
• Fix broken images on PyPI page by @Timour-Ilyas in https://github.com/celery/celery/pull/10066
• Remove broken reference. in https://github.com/celery/celery/pull/10071
• Removed --dist=loadscope from smoke tests by @Nusnus in https://github.com/celery/celery/pull/10073
• Docs: Clarify task_retry signal args may be None by @GangEunzzang in https://github.com/celery/celery/pull/10076
• Update example for Django by @sbc-khacnha in https://github.com/celery/celery/pull/10081
• Update CI matrix to exclude Python 3.9 by @auvipy in https://github.com/celery/celery/pull/9980
• Update default Python versions in Integration test by @auvipy in https://github.com/celery/celery/pull/10088
• Update default Python versions in smoke-tests.yml by @auvipy in https://github.com/celery/celery/pull/10089 …

What's Changed

• Fix Django worker recursion bug + defensive checks for pool_cls.module by @maycuatroi1 in https://github.com/celery/celery/pull/10048
• Docs: Update user_preload_options example to use click. by @jorsyk in https://github.com/celery/celery/pull/10056
• Fix invalid configuration key "bootstrap_servers" in Kafka demo by @jorsyk in https://github.com/celery/celery/pull/10060
• Fix broken images on PyPI page by @Timour-Ilyas in https://github.com/celery/celery/pull/10066
• Remove broken reference. by @sueannioanis in https://github.com/celery/celery/pull/10071
• Removed --dist=loadscope from smoke tests by @Nusnus in https://github.com/celery/celery/pull/10073
• Docs: Clarify task_retry signal args may be None by @GangEunzzang in https://github.com/celery/celery/pull/10076
• Update example for Django by @sbc-khacnha in https://github.com/celery/celery/pull/10081
• Make tests compatible with pymongo >= 4.16 by @cjwatson in https://github.com/celery/celery/pull/10074
• fix: source install of cassandra-driver by @Izzette in https://github.com/celery/celery/pull/10105
• fix: register task cross-reference role in Sphinx extension by @veeceey in https://github.com/celery/celery/pull/10100
• fix: avoid cycle detection in native delayed delivery by @Izzette in https://github.com/celery/celery/pull/10095
• fix(asynpool): avoid AttributeError when proc lacks _sentinel_poll by @mriddle in https://github.com/celery/celery/pull/10086 …

What's Changed

• Fix recursive WorkController instantiation in DjangoWorkerFixup + AttributeError when pool_cls is a string by @bruunotrindade in https://github.com/celery/celery/pull/10045
• Bugfix: Revoked tasks now immediately update backend status to REVOKED by @Nusnus in https://github.com/celery/celery/pull/9869
• Prepare for release: v5.6.2 by @Nusnus in https://github.com/celery/celery/pull/10049

New Contributors

• @bruunotrindade made their first contribution in https://github.com/celery/celery/pull/10045

Full Changelog: https://github.com/celery/celery/compare/v5.6.1...v5.6.2

What's Changed

• Fix Redis Sentinel ACL authentication support by @anthonykuzmich7 in https://github.com/celery/celery/pull/10013
• Fix: Broker heartbeats not sent during graceful shutdown by @weetster in https://github.com/celery/celery/pull/9986
• docs #5410 -- Document confirm_publish broker transport option by @JaeHyuckSa in https://github.com/celery/celery/pull/10016
• close DB pools only in prefork mode by @petrprikryl in https://github.com/celery/celery/pull/10020
• Fix: Avoid unnecessary Django database connection creation during cleanup by @snopoke in https://github.com/celery/celery/pull/10015
• reliable prefork detection by @petrprikryl in https://github.com/celery/celery/pull/10023
• better coverage by @petrprikryl in https://github.com/celery/celery/pull/10029
• Docs: clarify result_extended vs periodic task metadata and show headers["periodic_task_name"] example by @SpaceShaman in https://github.com/celery/celery/pull/10030
• Stop importing pytest_subtests by @cjwatson in https://github.com/celery/celery/pull/10032
• Only use exceptiongroup backport for Python < 3.11 by @cjwatson in https://github.com/celery/celery/pull/10033
• Prepare for release: v5.6.1 by @Nusnus in https://github.com/celery/celery/pull/10037

New Contributors

• @anthonykuzmich7 made their first contribution in https://github.com/celery/celery/pull/10013
• @weetster made their first contribution in https://github.com/celery/celery/pull/9986 …

Celery v5.6.0 is now available.

Key Highlights

See What's new in Celery 5.6 for a complete overview or read the main highlights below.

Python 3.9 Minimum Version

Celery 5.6.0 drops support for Python 3.8 (EOL). The minimum required Python version is now 3.9. Users still on Python 3.8 must upgrade their Python version before upgrading to Celery 5.6.0.

Additionally, this release includes initial support for Python 3.14.

SQS: Reverted to pycurl from urllib3

The switch from pycurl to urllib3 for the SQS transport (introduced in Celery 5.5.0 via Kombu) has been reverted due to critical issues affecting SQS users.

Contributed by @auvipy in https://github.com/celery/celery/pull/9620.

Security Fix: Broker Credential Leak Prevention

Fixed a security issue where broker URLs containing passwords were being logged in plaintext by the delayed delivery mechanism. Broker credentials are now properly sanitized in all log output.

Contributed by @giancarloromeo in https://github.com/celery/celery/pull/9997.

Memory Leak Fixes

Two significant memory leaks have been fixed in this release:

Exception Handling Memory Leak: Fixed a critical memory leak in task exception handling that was particularly severe on Python 3.11+ due to enhanced traceback data. The fix properly breaks reference cycles in tracebacks to allow garbage collection. …

Celery v5.6.0 Release Candidate 2 is now available for testing. Please help us test this version and report any issues.

What's Changed

• Remove Python 4.0 version condition for pytest dependencies by @ulgens in https://github.com/celery/celery/pull/9993
• Sanitize broker URL in delayed delivery logs (avoid leaking credentials) by @giancarloromeo in https://github.com/celery/celery/pull/9997
• Don't fail task on timeout during cold shutdown by @Nusnus in https://github.com/celery/celery/pull/9678
• Add Py39-314t to CI by @Nusnus in https://github.com/celery/celery/pull/9999
• asynpool: Don't return from inside a finally block by @torarvid in https://github.com/celery/celery/pull/10000
• Prepare for (pre) release: v5.6.0rc2 by @Nusnus in https://github.com/celery/celery/pull/10005

New Contributors

• @ulgens made their first contribution in https://github.com/celery/celery/pull/9993
• @giancarloromeo made their first contribution in https://github.com/celery/celery/pull/9997
• @torarvid made their first contribution in https://github.com/celery/celery/pull/10000

Full Changelog: https://github.com/celery/celery/compare/v5.6.0rc1...v5.6.0rc2

Celery v5.6.0 Release Candidate 1 is now available for testing. Please help us test this version and report any issues.

What's Changed

• Add support for Django Connection pool by @petrprikryl in https://github.com/celery/celery/pull/9953
• Pin tblib to ==3.1.0 by @kernelshard in https://github.com/celery/celery/pull/9967
• fix(worker): continue to attempt to bind other queues after a native delayed delivery binding failure has occurred by @Izzette in https://github.com/celery/celery/pull/9959
• Handle UnpicklingError in persistent scheduler initialization by @pakallis in https://github.com/celery/celery/pull/9952
• Bug Fix: Nested Chords Fail When Using django-celery-results with a Redis Backend by @kumuthu53 in https://github.com/celery/celery/pull/9950
• Add support pymongo 4.12 by @kidoz in https://github.com/celery/celery/pull/9665
• Make tests compatible with pymongo >= 4.14 by @cjwatson in https://github.com/celery/celery/pull/9968
• tblib updated from 3.1.0 to 3.2.0 by @MehrazRumman in https://github.com/celery/celery/pull/9970
• Fix remaining function typing and docstring by @wcislo-saleor in https://github.com/celery/celery/pull/9971
• Fix regex pattern in version parsing and remove duplicate entry in all by @sagar1343 in https://github.com/celery/celery/pull/9978
• Bump Kombu to v5.6.0 and removed <5.7 limit on kombu by @Nusnus in https://github.com/celery/celery/pull/9981 …

Celery v5.6.0 Beta 2 is now available for testing. Please help us test this version and report any issues.

What's Changed

• GitHub Actions: Test on Python 3.14 release candidate 2 by @cclauss in https://github.com/celery/celery/pull/9891
• Update pypy to python 3.11 by @cclauss in https://github.com/celery/celery/pull/9896
• Feature: Add support credential_provider to Redis Backend by @alaminopu in https://github.com/celery/celery/pull/9879
• Celery.timezone: try tzlocal.get_localzone() before using LocalTimezone by @woutdenolf in https://github.com/celery/celery/pull/9862
• Run integration tests on Python 3.14 by @cclauss in https://github.com/celery/celery/pull/9903
• Fix arithmetic overflow for MSSQL result backend by @vladborovtsov in https://github.com/celery/celery/pull/9904
• Add documentation for task_id param for apply_async function by @sumo1998 in https://github.com/celery/celery/pull/9906
• Support redis client name by @mdalp in https://github.com/celery/celery/pull/9900
• Bump Kombu to v5.6.0rc1 by @Nusnus in https://github.com/celery/celery/pull/9918
• Fix broker connection retry attempt counter in the error log by @sumo1998 in https://github.com/celery/celery/pull/9911
• fix: restrict disable-prefetch feature to Redis brokers only by @Dkhodos in https://github.com/celery/celery/pull/9919
• fix(): preserve group order in replaced signature by @simonepozzoli-pix4d in https://github.com/celery/celery/pull/9910 …

Celery v5.6.0 Beta 1 is now available for testing. Please help us test this version and report any issues.

What's Changed

• docs: mention of json serializer recursive reference message size blowup (#5000) by @chrisbarber in https://github.com/celery/celery/pull/9743
• docs: typo in canvas.rst by @chrisbarber in https://github.com/celery/celery/pull/9744
• Makes _on_retry return a float as required to be used as errback on retry_over_time by @maccinza in https://github.com/celery/celery/pull/9741
• Update canvas.rst doc calculation order for callback by @auvipy in https://github.com/celery/celery/pull/9758
• Updated Blacksmith logo by @Nusnus in https://github.com/celery/celery/pull/9763
• Made the Sponsors logos link to their website by @Nusnus in https://github.com/celery/celery/pull/9764
• add missing cloudamqp logo by @auvipy in https://github.com/celery/celery/pull/9767
• Improve sponsor visibility by @auvipy in https://github.com/celery/celery/pull/9768
• fix: (#9773) task_id must not be empty with chain as body of a chord by @dfm88 in https://github.com/celery/celery/pull/9774
• Update setup.py to fix deprecation warning by @auvipy in https://github.com/celery/celery/pull/9771
• Adds integration test for chord_unlock bug when routed to quorum/topic queue by @maccinza in https://github.com/celery/celery/pull/9766 …

What's Changed

• make the tests run on python 3.13 for gcs backend by @auvipy in https://github.com/celery/celery/pull/9677
• Added DeepWiki to README by @Nusnus in https://github.com/celery/celery/pull/9683
• Limit redis to <=v5.2.1 to match Kombu by @Nusnus in https://github.com/celery/celery/pull/9693
• Use EX_OK instead of literal zero by @Nusnus in https://github.com/celery/celery/pull/9684
• Make wheel metadata reproducible by @cjwatson in https://github.com/celery/celery/pull/9687
• let celery install from kombu dependencies for better align by @auvipy in https://github.com/celery/celery/pull/9696
• Fix stamping documentation to clarify stamped_headers key is optional in visitor methods by @Nusnus in https://github.com/celery/celery/pull/9697
• Support apply_async without queue argument on quorum queues by @bityob in https://github.com/celery/celery/pull/9686
• Updated rabbitmq doc about using quorum queues with task routes by @bityob in https://github.com/celery/celery/pull/9707
• Add: Dumper Unit Test by @yiyu0x in https://github.com/celery/celery/pull/9711
• Add unit test for event.group_from by @rogerforlife in https://github.com/celery/celery/pull/9709
• refactor: add beat_cron_starting_deadline documentation warning by @alirafiei75 in https://github.com/celery/celery/pull/9712
• fix: resolve issue #9569 by supporting distinct broker transport options for workers by @yiyu0x in https://github.com/celery/celery/pull/9695 …

What's Changed

• Fix calculating remaining time across DST changes by @imrivera in https://github.com/celery/celery/pull/9669
• Remove setup_logger from COMPAT_MODULES by @rconradharris in https://github.com/celery/celery/pull/9668
• Fix mongodb bullet and fix github links in contributions section by @jabberwock404 in https://github.com/celery/celery/pull/9672
• Prepare for release: v5.5.2 by @Nusnus in https://github.com/celery/celery/pull/9675

New Contributors

• @imrivera made their first contribution in https://github.com/celery/celery/pull/9669
• @rconradharris made their first contribution in https://github.com/celery/celery/pull/9668
• @jabberwock404 made their first contribution in https://github.com/celery/celery/pull/9672

Full Changelog: https://github.com/celery/celery/compare/v5.5.1...v5.5.2

Celery v5.5.0 is now available.

Key Highlights

See What's new in Celery 5.5 for a complete overview or read the main highlights below.

Redis Broker Stability Improvements

Long-standing disconnection issues with the Redis broker have been identified and resolved in Kombu 5.5.0, which is included with this release. These improvements significantly enhance stability when using Redis as a broker.

Additionally, the Redis backend now has better exception handling with the new exception_safe_to_retry feature, which improves resilience during temporary Redis connection issues. See Redis backend settings for complete documentation.

Contributed by @drienkop in #9614.

pycurl replaced with urllib3

Replaced the pycurl dependency with urllib3.

We're monitoring the performance impact of this change and welcome feedback from users who notice any significant differences in their environments.

Contributed by @spawn-guy in Kombu #2134 and integrated in Celery via #9526.

RabbitMQ Quorum Queues Support …

Celery v5.5.0 Release Candidate 5 is now available for testing. Please help us test this version and report any issues.

Key Highlights

See What’s new in Celery 5.5 (Immunity) or read the main highlights below.

Using Kombu 5.5.0rc3

The minimum required Kombu version has been bumped to 5.5.0. Kombu is currently at 5.5.0rc3.

Complete Quorum Queues Support

A completely new ETA mechanism was developed to allow full support with RabbitMQ Quorum Queues.

After upgrading to this version, please share your feedback on the quorum queues support.

Relevant Issues: #9207, #6067

• New documentation.
• New broker_native_delayed_delivery_queue_type configuration option.

New support for Google Pub/Sub transport

After upgrading to this version, please share your feedback on the Google Pub/Sub transport support.

Relevant Issues: #9351

Python 3.13 Improved Support

Additional dependencies have been migrated successfully to Python 3.13, including Kombu and py-amqp.

Soft Shutdown …

Celery v5.5.0 Release Candidate 4 is now available for testing. Please help us test this version and report any issues.

Key Highlights

See What’s new in Celery 5.5 (Immunity) or read the main highlights below.

Using Kombu 5.5.0rc2

The minimum required Kombu version has been bumped to 5.5.0. Kombu is current at 5.5.0rc2.

Complete Quorum Queues Support

A completely new ETA mechanism was developed to allow full support with RabbitMQ Quorum Queues.

After upgrading to this version, please share your feedback on the quorum queues support.

Relevant Issues: #9207, #6067

• New documentation.
• New broker_native_delayed_delivery_queue_type configuration option.

New support for Google Pub/Sub transport

After upgrading to this version, please share your feedback on the Google Pub/Sub transport support.

Relevant Issues: #9351

Python 3.13 Improved Support

Additional dependencies have been migrated successfully to Python 3.13, including Kombu and py-amqp.

Soft Shutdown …

Celery v5.5.0 Release Candidate 3 is now available for testing. Please help us test this version and report any issues.

Key Highlights

See What’s new in Celery 5.5 (Immunity) or read the main highlights below.

Using Kombu 5.5.0rc2

The minimum required Kombu version has been bumped to 5.5.0. Kombu is current at 5.5.0rc2.

Complete Quorum Queues Support

A completely new ETA mechanism was developed to allow full support with RabbitMQ Quorum Queues.

After upgrading to this version, please share your feedback on the quorum queues support.

Relevant Issues: #9207, #6067

• New documentation.
• New broker_native_delayed_delivery_queue_type configuration option.

New support for Google Pub/Sub transport

After upgrading to this version, please share your feedback on the Google Pub/Sub transport support.

Relevant Issues: #9351

Python 3.13 Improved Support

Additional dependencies have been migrated successfully to Python 3.13, including Kombu and py-amqp.

Soft Shutdown …

Celery v5.5.0 Release Candidate 2 is now available for testing. Please help us test this version and report any issues.

Key Highlights

See What’s new in Celery 5.5 (Immunity) or read the main highlights below.

Using Kombu 5.5.0rc2

The minimum required Kombu version has been bumped to 5.5.0. Kombu is current at 5.5.0rc2.

Complete Quorum Queues Support

A completely new ETA mechanism was developed to allow full support with RabbitMQ Quorum Queues.

After upgrading to this version, please share your feedback on the quorum queues support.

Relevant Issues: #9207, #6067

• New documentation.
• New broker_native_delayed_delivery_queue_type configuration option.

New support for Google Pub/Sub transport

After upgrading to this version, please share your feedback on the Google Pub/Sub transport support.

Relevant Issues: #9351

Python 3.13 Improved Support

Additional dependencies have been migrated successfully to Python 3.13, including Kombu and py-amqp.

Previous Pre-release Highlights

Python 3.13 Initial Support …

Celery v5.5.0 Release Candidate 1 is now available for testing. Please help us test this version and report any issues.

Key Highlights

See What’s new in Celery 5.5 (Immunity) or read the main highlights below.

Python 3.13 Initial Support

This release introduces the initial support for Python 3.13 with Celery.

After upgrading to this version, please share your feedback on the Python 3.13 support.

Soft Shutdown

The soft shutdown is a new mechanism in Celery that sits between the warm shutdown and the cold shutdown. It sets a time limited "warm shutdown" period, during which the worker will continue to process tasks that are already running. After the soft shutdown ends, the worker will initiate a graceful cold shutdown, stopping all tasks and exiting. …

Celery v5.5.0 Beta 4 is now available for testing. Please help us test this version and report any issues.

Key Highlights

Python 3.13 Initial Support

This release introduces the initial support for Python 3.13 with Celery.

After upgrading to this version, please share your feedback on the Python 3.13 support.

Previous Pre-release Highlights

Soft Shutdown

The soft shutdown is a new mechanism in Celery that sits between the warm shutdown and the cold shutdown. It sets a time limited "warm shutdown" period, during which the worker will continue to process tasks that are already running. After the soft shutdown ends, the worker will initiate a graceful cold shutdown, stopping all tasks and exiting.

The soft shutdown is disabled by default, and can be enabled by setting the new configuration option worker_soft_shutdown_timeout. If a worker is not running any task when the soft shutdown initiates, it will skip the warm shutdown period and proceed directly to the cold shutdown unless the new configuration option worker_enable_soft_shutdown_on_idle is set to True. This is useful for workers that are idle, waiting on ETA tasks to be executed that still want to enable the soft shutdown…