Skip to content

Commit

Permalink
[SPARK-45682][CORE][FOLLOW-UP] Restore console progress bar in Spark …
Browse files Browse the repository at this point in the history
…shells

### What changes were proposed in this pull request?

This PR is a followup of #43573 that restore the console progress bar in Spark shells.
Previously it was `CR + bar + CR` but mistakenly it was fixed as `s"${CR}bar$CR"`.

Before:

```
>>> spark.range(10000).repartition(4000).count()
bar
```

After:

```
>>> spark.range(10000).repartition(4000).count()
[Stage 5:==============>                                     (1121 + 16) / 4000]
```

### Why are the changes needed?

To have the proper working console bar. This is a regression.

### Does this PR introduce _any_ user-facing change?

Not yet because the main change has not been released out yet.

### How was this patch tested?

Manually tested:

```bash
./bin/pyspark
spark.range(10000).repartition(4000).count()
```

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #43900 from HyukjinKwon/SPARK-45682.

Authored-by: Hyukjin Kwon <gurwls223@apache.org>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
  • Loading branch information
HyukjinKwon committed Nov 20, 2023
1 parent 70bb931 commit baff56d
Showing 1 changed file with 1 addition and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ private[spark] class ConsoleProgressBar(sc: SparkContext) extends Logging {
// only refresh if it's changed OR after 1 minute (or the ssh connection will be closed
// after idle some time)
if (bar != lastProgressBar || now - lastUpdateTime > 60 * 1000L) {
System.err.print(s"${CR}bar$CR")
System.err.print(s"$CR$bar$CR")
lastUpdateTime = now
}
lastProgressBar = bar
Expand Down

0 comments on commit baff56d

Please sign in to comment.