Apache DolphinScheduler Brings Major Enhancements and Performance Upgrades in Latest Release

Recently, Apache DolphinScheduler released version 3.3.0 alpha. The 3.3.0 alpha version introduces breakthrough improvements in task management, fault tolerance, and scalability, offering a more stable and flexible solution for large-scale workflow scheduling. However, please note that this is an alpha version—upgrade with caution.

Try it now:

• Source code download: https://github.com/apache/dolphinscheduler/archive/refs/tags/3.3.0-alpha.zip
• Upgrade guide: Please refer to the community documentation

This update includes numerous DSIPs (DolphinScheduler Improvement Proposals), Features (new functionalities), and Improvements. Below is a summary of the release highlights.

• For full update details, check the Release Note: https://github.com/apache/dolphinscheduler/releases/tag/3.3.0-alpha

DSIP (DolphinScheduler Improvement Proposals)

The following are structural improvement proposals included in the DolphinScheduler 3.3.0 alpha version, bringing systematic changes to core functionalities, architecture, and performance:

✅ Enhanced Connection Center Support

• DSIP-19 series: Added support for Zeppelin, SageMaker, K8s connections (by @xdu-chenrj)
• DSIP-27: Java task plugin improvements (by @ailiujiarui)

🧾 Audit Logs and Remote Logs Optimization

• DSIP-24 / 25 / 26 / 57: Upgraded architecture for remote and audit logs with rollback support (by @pegasas, @qingwli, @SbloodyS)

🔨 Master/Task Execution Logic Optimization

• DSIP-44 / 54 / 56 / 61 / 67 / 65 / 73 / 87: Optimizations from thread pool scheduling, task execution abstraction, to scheduling command mechanisms (by @ruanwenjun)

💡 User Experience Optimization

• DSIP-50 / 51 / 59 / 60 / 62 / 68 / 72 / 78: Removed redundant plugins/modules and unified UI structure (by @SbloodyS)

🧰 Other Improvements

• DSIP-75: Automatically add license headers (by @ruanwenjun)
• DSIP-70: Removed upgrade code for versions 1.x/2.x (by @SbloodyS)
• DSIP-38: Optimized plugin upload/download and dependencies (by @SbloodyS)

Feature (New Features)

🎧 Monitoring and Alerts

• Added Listener mechanism (#14981)
• Supported Prometheus AlertManager, Aliyun Voice, HTTP test sending (#15079, #15248, #15163)

🧪 New Plugins and Integrations

• Supported OceanBase DataX tasks (#16281)
• Supported EMR Serverless Spark (#16126)
• Supported param passing for Dinky task Prepare Params (#15947)

🧱 Resource and Permission Management

• UI now displays resource paths; added Worker Group permission management (#16590, #15600)

🗂 Storage Extensions

• Added Tencent Cloud COS storage plugin (#16565)

📄 Documentation Enhancements

• Security model documentation and standardized security naming (#16609, #16587)

Improvement (Optimizations)

⚙ Core Modules and Performance

• Master/Worker thread pools, Zookeeper timeout optimization, RPC retry mechanisms, and dependency task handling improvements (multiple PRs by @ruanwenjun)

🧪 Test Coverage Improvements

• Enhanced UT and E2E test coverage and filled in missing test cases (numerous PRs by @abzymeinsjtu)

🧹 Cleanup and Refactoring

• Removed redundant code, optimized logging, and standardized variable naming (multiple PRs by @SbloodyS, @caicancai, @Gallardot)

📦 Deployment/Documentation Enhancements

• Fixed documentation typos, added Helm config details, optimized Docker Hub usage and installation flow (multiple PRs by @SbloodyS, @Gallardot)

Bug Fix

The following modules were the focus of bug fixes during this release cycle:

• Core Scheduling Module:Fixed issues like sub-workflows continuing to run while in PAUSE state (#16564), dependency task status parsing errors (#16599), incorrect global parameter passing (#16376), and scheduling anomalies for conditional dependency tasks in WAITING_DEPEND state (#16232), improving stability and correctness.

• Plugin-Related Issues:Fixed a NullPointerException in the Elasticsearch plugin (#16107) and potential failures in multi-threaded file downloads in the file plugin (#16063), enhancing plugin robustness and compatibility.

• Data Source Configuration Issues:Addressed incorrect parsing of data source parameters (#15874) and resolved MySQL connection failures in some JDBC versions (#16376), improving connection adaptability.

• Frontend Display Issues:Fixed a naming error in the DATASOURCE_CATALOG permission module that caused UI display issues (#16590), and resolved discrepancies between task state display and backend data (#16594), improving frontend usability.

• Test Case Fixes:Fixed E2E test failures caused by faulty plugin dependencies (#15892) and addressed instability in some unit tests under specific environments (#16120), enhancing test system reliability.

Document (Documentation Updates)

This release includes updates across installation guides, contributor documentation, and configuration references:

• Installation and Deployment Docs:Updated Helm installation instructions, clarified deployment steps, and documented Chart parameter configs for easier setup (#16604).

• Contributor Documentation:Improved contribution guidelines with new PR title conventions and submission flowcharts to help new contributors participate more effectively (#16294, #16125).

• Configuration and Usage Docs:Added details for task dependencies, data source configurations, and alert strategies (#15972, #16097, #16407), improving readability and accuracy.

Chore (Miscellaneous Optimizations)

Chore updates focus on CI processes, module cleanup, and partial refactoring:

• CI/CD Process Optimization: Introduced a Mock Server to simulate scheduling execution, accelerating E2E test speed and improving stability (#16030); added compatibility for scheduling failures under multi-task concurrency (#16139); simplified CI logic and E2E execution commands (#16643).
• Module Cleanup and Refactoring :Removed unused old modules, plugins, and invalid test code, simplifying the codebase and improving clarity and maintainability (#15973, #16588).

Contributors

This release was made possible by the collaboration of community developers. Special thanks to the following core contributors (partial list):

• Ruan Wenjun (PMC Member): Led architectural design and core feature development.
• Thanks to Gallardot, weideguo, and a total of 71 contributors (GitHub IDs below):

@Blue-Fox, @ZarakiKenpachi, @zhangyongqin, @ironnight, @liuchuo, @yehfan, @CaIShunfeng, @yuxiaoyang-cn, @kongchongming, @MasahiroZhang, @wuxuqiang, @gaoqiang173, @ckeeper97, @twocolds, @hvhye, @liujx0429, @lingkong-jiankong, @jiangm2019, @izukayy, @derryyy, @serpywa, @2rinn, @ilyaszq, @weidaokun, @codenavie, @peijie-liu, @mario27547, @yassirlr, @mingxiaojun, @lixunxd, @wmm2019, @Shawn1997, @nobida, @Chen-lina, @hzytong, @jiangang-huawei, @zhangchaofan, @dongchunlin, @HXT-DAO, @zhaozhichao, @YucongZhu, @onakobayashi, @sdh2110, @laitinglin, @wumomo-npm, @tang-xi, @Maxwell-nz, @bigfrog828, @flarv, @buqiong, @peicongl, @xiayuheng, @kokaokami, @ztust-hsy, @ChengYu-Cloud, @sxt5699, @0d7000, @suxianglong, @one-man-team, @zhouyi0410, @tianxuyuan, @heguanghu, @LotharHuber, @chongxiao-qw, @mcz1988, @windzhe, @linhong-ua, @sunkiming, @nothedream, @kenbliu, @klovejun, @ywq1024, @jianghaitao