Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(core): Reduce large relay message payloads to protect Redis #12342

Open
wants to merge 2 commits into
base: offload-manual-executions-to-workers
Choose a base branch
from

Conversation

ivov
Copy link
Contributor

@ivov ivov commented Dec 23, 2024

Summary

It is possible for a manual execution in scaling mode to cause so much data to flow through Redis that it becomes unresponsive. Specifically, the nodeExecuteAfter and executionFinished events may carry arbitrarily large payloads.

In this PR we replace payloads above a 5 MB limit with placeholders that signal the UI to direct the user to view the full payload in execution history instead. Events with payloads below the size limit continue to be sent and displayed as usual.

In future, we can override these placeholders with the full execution data fetched from the DB at the end of the execution. Since we are already planning to refactor execution data handling on the client in the near future, that change is out of scope for this PR.

Testing

  • Start a main in scaling mode with OFFLOAD_MANUAL_EXECUTIONS_TO_WORKERS=true
  • Start a worker
  • Monitor Redis: docker exec -it <container> redis-cli monitor
  • Start an ad hoc server (see Slack) at localhost:3000 serving a 30 MB payload
  • Import workflow (see Slack), run it manually and call the test webhook
  • Verify that worker is processing manual execution and relaying to main
  • Verify that items after the Fetch large payload node have been trimmed
  • Verify in execution history that those items are displayed in full

Recording: https://share.cleanshot.com/GCrCsPXw

Related Linear tickets, Github issues, and Community forum posts

https://linear.app/n8n/issue/PAY-2403/reduce-large-relay-message-payloads-to-protect-redis

Review / Merge checklist

  • PR title and summary are descriptive.
  • Docs updated or follow-up ticket created
  • Tests included
  • PR labeled with release/backport

@n8n-assistant n8n-assistant bot added core Enhancement outside /nodes-base and /editor-ui n8n team Authored by the n8n team ui Enhancement in /editor-ui or /design-system labels Dec 23, 2024
Copy link

codecov bot commented Dec 23, 2024

Codecov Report

Attention: Patch coverage is 25.58140% with 32 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
packages/cli/src/push/index.ts 24.00% 19 Missing ⚠️
packages/editor-ui/src/components/RunData.vue 23.52% 13 Missing ⚠️

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Enhancement outside /nodes-base and /editor-ui n8n team Authored by the n8n team ui Enhancement in /editor-ui or /design-system
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant