Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NoSuchMethodErrors on cidr_match queries #115

Open
mstair opened this issue Apr 30, 2023 · 2 comments
Open

NoSuchMethodErrors on cidr_match queries #115

mstair opened this issue Apr 30, 2023 · 2 comments
Assignees

Comments

@mstair
Copy link

mstair commented Apr 30, 2023

Getting following error on attempts to run cidr_match queries against following CREATE TABLE security_log (user_id STRING, remote_host_ip STRING). Cluster is 11.3 LTS ML (includes Apache Spark 3.3.0, Scala 2.12)

from dbl_transpiler import spl

display(spl("""
index=security_log remote_host_ip=109.177.0.0/16
"""))

...

Py4JJavaError: An error occurred while calling z:com.databricks.labs.transpiler.spl.Transpiler.toDataFrame.
: java.lang.NoSuchMethodError: org.apache.spark.sql.catalyst.expressions.Cast$.apply$default$4()Z
	at org.apache.spark.sql.CidrMatch.aton(CidrMatch.scala:43)
	at org.apache.spark.sql.CidrMatch.ipAddress(CidrMatch.scala:24)
	at org.apache.spark.sql.CidrMatch.cidrMatch(CidrMatch.scala:22)
@nfx nfx self-assigned this May 8, 2023
@mstair
Copy link
Author

mstair commented May 23, 2023

For completeness, I tried to run on DB 12.0/12.2/13.0 clusters and got the same error.

@mstair
Copy link
Author

mstair commented Jul 13, 2023

Same error on 13.2 cluster also. @nfx any update?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants