Skip to content

Commit

Permalink
Merge pull request #9 from databricks-industry-solutions/update-spark…
Browse files Browse the repository at this point in the history
…-3.5.0

updating spark versions
  • Loading branch information
zavoraad authored Feb 18, 2025
2 parents b05263e + 77c54ab commit 72ece74
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 3 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ The goal of this repo is not to represent every permutation of a json schema ->

## Running

Built for **Spark 3.3.0 & Scala 2.12** Please make sure your cluster matches these versions
Latest updated build for **Spark 3.5.0 & Scala 2.12** Please make sure your cluster matches these versions.

```scala
import com.databricks.industry.solutions.json2spark._
Expand Down
5 changes: 3 additions & 2 deletions build.sbt
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
lazy val scala212 = "2.12.8"
lazy val sparkVersion = sys.env.getOrElse("SPARK_VERSION", "3.3.0")
lazy val sparkVersion = sys.env.getOrElse("SPARK_VERSION", "3.5.0")

ThisBuild / scalaVersion := sys.env.getOrElse("SCALA_VERSION", scala212)
ThisBuild / organization := "com.databricks"
ThisBuild / organizationName := "Databricks, Inc."

lazy val sparkDependencies = Seq(
"com.databricks" % "databricks-jdbc" % "2.7.1",
"org.apache.spark" %% "spark-catalyst" % sparkVersion,
"org.apache.spark" %% "spark-core" % sparkVersion,
"org.apache.spark" %% "spark-mllib" % sparkVersion,
Expand Down Expand Up @@ -38,7 +39,7 @@ assembly / assemblyShadeRules := Seq(
lazy val core = (project in file("."))
.settings(
name := "json2spark-schema",
version := "0.0.3",
version := "0.0.4",
scalacOptions += "-target:jvm-1.8",
libraryDependencies ++= coreDependencies ++ projectDependencies ,
assemblyJarName := s"${name.value}-${version.value}_assembly.jar",
Expand Down

0 comments on commit 72ece74

Please sign in to comment.