Skip to content

Commit

Permalink
[SPARK-49057][SQL][TESTS][FOLLOWUP] Handle _LEGACY_ERROR_TEMP_2235
Browse files Browse the repository at this point in the history
…error case

### What changes were proposed in this pull request?

This PR aims to fix a flaky test by handling `_LEGACY_ERROR_TEMP_2235`(multiple failures exception) in addition to the single exception.

### Why are the changes needed?

After merging
- #47533

The following failures were reported multiple times in the PR and today.
- https://github.com/apache/spark/actions/runs/11358629880/job/31593568476
- https://github.com/apache/spark/actions/runs/11367718498/job/31621128680
- https://github.com/apache/spark/actions/runs/11360602982/job/31598792247
```
[info] - SPARK-47148: AQE should avoid to submit shuffle job on cancellation *** FAILED *** (6 seconds, 92 milliseconds)
[info]   "Multiple failures in stage materialization." did not contain "coalesce test error" (AdaptiveQueryExecSuite.scala:939)
```

The root cause is that `AdaptiveSparkPlanExec.cleanUpAndThrowException` throws two types of exceptions. When there are multiple errors, `_LEGACY_ERROR_TEMP_2235` is thrown. We need to handle this too in the test case.

https://github.com/apache/spark/blob/bcfe62b9988f9b00c23de0b71acc1c6170edee9e/sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala#L843-L850

https://github.com/apache/spark/blob/bcfe62b9988f9b00c23de0b71acc1c6170edee9e/sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryExecutionErrors.scala#L1916-L1921

### Does this PR introduce _any_ user-facing change?

No, this is a test-only change.

### How was this patch tested?

Pass the CIs.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #48498 from dongjoon-hyun/SPARK-49057.

Authored-by: Dongjoon Hyun <dongjoon@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
  • Loading branch information
dongjoon-hyun committed Oct 16, 2024
1 parent f860af6 commit 31a4117
Showing 1 changed file with 2 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -936,7 +936,8 @@ class AdaptiveQueryExecSuite
val error = intercept[SparkException] {
joined.collect()
}
assert(error.getMessage() contains "coalesce test error")
assert((Seq(error) ++ Option(error.getCause) ++ error.getSuppressed()).exists(
e => e.getMessage() != null && e.getMessage().contains("coalesce test error")))

val adaptivePlan = joined.queryExecution.executedPlan.asInstanceOf[AdaptiveSparkPlanExec]

Expand Down

0 comments on commit 31a4117

Please sign in to comment.