-
Notifications
You must be signed in to change notification settings - Fork 318
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Enabling configuring IP Address for Dotnet Backend #537
base: main
Are you sure you want to change the base?
Enabling configuring IP Address for Dotnet Backend #537
Conversation
Can we not update the usage, but let the user set the environment variable? |
...scala/microsoft-spark-2.3.x/src/main/scala/org/apache/spark/deploy/dotnet/DotnetRunner.scala
Outdated
Show resolved
Hide resolved
@suhsteve Can you help reviewing this PR? |
src/scala/microsoft-spark-2.3.x/src/main/scala/org/apache/spark/api/dotnet/DotnetBackend.scala
Outdated
Show resolved
Hide resolved
...scala/microsoft-spark-2.3.x/src/main/scala/org/apache/spark/deploy/dotnet/DotnetRunner.scala
Outdated
Show resolved
Hide resolved
...scala/microsoft-spark-2.3.x/src/main/scala/org/apache/spark/deploy/dotnet/DotnetRunner.scala
Outdated
Show resolved
Hide resolved
@Niharikadutta Any updates on this one? We are having trouble debugging our .Net for Spark application with docker container setup and this seems to fix the issue. |
Can we get an update on this PR @Niharikadutta? What are the security concerns @imback82 ? If it's user-configurable and defaults to the most secure, how is that any different than an ASP.NET server allowing 0.0.0.0 when specified explicitly by the user. The general guidance for WSL2 is to run Linux based things you want to connect to with 0.0.0.0. We wouldn't be doing the override in anything other than a debug/test scenario, so it would be secure by default. |
@Macromullet , @ZavezX we are working on getting this PR merged asap, thank you for your patience. |
@suhsteve, @elvaliuliuliu Could you please help review? |
This PR gives the user the ability to specify a particular IP address they want the Dotnet backend process to spin up on while submitting the .NET for Apache Spark application.
The user can do that by setting the following environment variables:
DOTNET_SPARK_BACKEND_IP_ADDRESS
DOTNET_SPARK_CALLBACK_SERVER_IP_ADDRESS
This PR fixes #419