Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pipeline.upsert() does not pass parallelism_config when updating pipeline #4384

Closed
1 task
amuzlera opened this issue Jan 6, 2025 · 1 comment
Closed
1 task
Labels
bug This issue is a confirmed bug.

Comments

@amuzlera
Copy link

amuzlera commented Jan 6, 2025

Describe the bug

In the upsert function of the Pipeline class in the pipeline.py file, the parallelism_config parameter is not passed when updating an existing pipeline. This causes the parallelism configuration to not be applied during the pipeline update.

Regression Issue

  • Select this option if this issue appears to be a regression.

Expected Behavior

The parallelism_config parameter should be passed to the update function so that the parallelism configuration is correctly applied during the pipeline update.

Current Behavior

Currently, parallelism_config is only passed to the create function, but not to the update function, resulting in the omission of the parallelism configuration during the pipeline update.

Reproduction Steps

from sagemaker.workflow.pipeline import Pipeline
from sagemaker.workflow.pipeline_context import PipelineSession
from sagemaker.workflow.parallelism_config import ParallelismConfiguration

pipeline_session = PipelineSession()

pipeline = Pipeline(
name="test-pipeline",
sagemaker_session=pipeline_session,
)

role_arn = "arn:aws:iam::123456789012:role/SageMakerRole"
parallelism_config = ParallelismConfiguration(max_parallel_execution_steps=10)

pipeline.upsert(role_arn=role_arn, parallelism_config=parallelism_config)

Call the upsert method again to update the pipeline

This should pass the parallelism_config, but it does not

pipeline.upsert(role_arn=role_arn, parallelism_config=parallelism_config)

Possible Solution

No response

Additional Information/Context

response = self.update(role_arn, description, parallelism_config) # Include parallelism_config when updateing

SDK version used

1.28.54

Environment details (OS name and version, etc.)

OS Name and Version: Windows 11

@amuzlera amuzlera added bug This issue is a confirmed bug. needs-triage This issue or PR still needs to be triaged. labels Jan 6, 2025
@nateprewitt
Copy link
Contributor

Hi @amuzlera, this looks like you may be looking for the Amazon SageMaker SDK. It looks like this issue may have been discussed previously in aws/sagemaker-python-sdk#4017 and appears to have been addressed. If you update to the latest version of the Amazon SageMaker SDK and are still experiencing issues, we'd recommend opening an issue in their repo for further assistance. Thanks!

@nateprewitt nateprewitt removed the needs-triage This issue or PR still needs to be triaged. label Jan 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue is a confirmed bug.
Projects
None yet
Development

No branches or pull requests

2 participants