Skip to content

Commit

Permalink
[MINOR][DOCS] Correct Python Spark Connect documentation about pip in…
Browse files Browse the repository at this point in the history
…stallation

### What changes were proposed in this pull request?

This PR fixes the Spark Connect documentation from `pyspark==3.5.0` to `pyspark[connect]==3.5.0`; otherwise it will fail to execute the example as is because of missing dependencies. This is sort of a followup of SPARK-44867.

https://github.com/apache/spark/blob/d31c8596cd714766892d1395e30358bd1cd3cb84/python/setup.py#L325-L332

### Why are the changes needed?

To guide users about using Spark Connect

### Does this PR introduce _any_ user-facing change?

Yes, this fixes the user-facing documentation for Python Spark Connect.

### How was this patch tested?

Manually checked with Markdown editor.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #43919 from HyukjinKwon/SPARK-44867-followup.

Authored-by: Hyukjin Kwon <gurwls223@apache.org>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
  • Loading branch information
HyukjinKwon committed Nov 21, 2023
1 parent f1a3a97 commit df1280c
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions docs/spark-connect-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -279,11 +279,11 @@ The connection may also be programmatically created using _SparkSession#builder_

<div data-lang="python" markdown="1">

First, install PySpark with `pip install pyspark==3.5.0` or if building a packaged PySpark application/library,
First, install PySpark with `pip install pyspark[connect]==3.5.0` or if building a packaged PySpark application/library,
add it your setup.py file as:
{% highlight python %}
install_requires=[
'pyspark==3.5.0'
'pyspark[connect]==3.5.0'
]
{% endhighlight %}

Expand Down

0 comments on commit df1280c

Please sign in to comment.