Skip to content

Commit 3e5227e

Browse files
authored
Updated quick start guide with Gradle example (#4)
1 parent 96a6cae commit 3e5227e

File tree

1 file changed

+49
-10
lines changed

1 file changed

+49
-10
lines changed

docs/quick-start-guide.md

Lines changed: 49 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -34,17 +34,18 @@ fun main() {
3434
}
3535
```
3636

37-
## Building the application with Maven
38-
37+
## Building the application
3938
Because Kotlin Spark API is not part of the official Apache Spark distribution yet, it is not enough to add Spark
40-
as a dependency in your pom.xml file.
39+
as a dependency to your build file.
4140
You need to:
4241
- Add Spark as a dependency
4342
- Add Kotlin Spark API as a dependency
4443
- Add Kotlin Standard Library as a dependency
4544

4645
When packaging your project into a jar file, you need to explicitly include Kotlin Spark API and Kotlin Standard Library
47-
dependencies, for example, using `maven-shade-plugin`.
46+
dependencies. Here you can find an example of building your application with Maven, and with Gradle.
47+
48+
### Building the application with Maven
4849

4950
Here's what the `pom.xml` looks like for this example:
5051
```xml
@@ -148,17 +149,55 @@ Here's what the project structure should look like:
148149

149150
Now you can package the application using Maven:
150151
`mvn package`
151-
152-
When done, you can execute the packaged application with `./bin/spark-submit`:
153152

154-
`YOUR_SPARK_HOME/bin/spark-submit --class "SimpleApp" --master local YOUR_PROJECT/target/kotlin-spark-example-1.0-SNAPSHOT.jar`
153+
### Building the application with Gradle
155154

156-
This example is also available as a [GitHub repo](https://github.com/MKhalusova/kotlin-spark-example), feel free to give it a try.
155+
Here's what the `build.gradle` looks like for this example:
157156

157+
```
158+
plugins {
159+
id 'org.jetbrains.kotlin.jvm' version '1.3.72'
160+
id 'com.github.johnrengelman.shadow' version '5.2.0'
161+
}
158162
163+
group = 'org.example'
164+
version = '1.0-SNAPSHOT'
159165
160-
166+
repositories {
167+
mavenCentral()
168+
maven { url = 'https://jitpack.io' }
169+
}
161170
171+
dependencies {
172+
// Kotlin stdlib
173+
implementation 'org.jetbrains.kotlin:kotlin-stdlib:1.3.72'
174+
// Kotlin Spark API
175+
implementation 'com.github.JetBrains.kotlin-spark-api:kotlin-spark-api:0.1.0'
176+
// Apache Spark
177+
compileOnly 'org.apache.spark:spark-sql_2.12:3.0.0-preview2'
178+
}
179+
180+
compileKotlin {
181+
kotlinOptions.jvmTarget = '1.8'
182+
}
183+
184+
shadowJar {
185+
dependencies {
186+
exclude(dependency {
187+
it.moduleGroup == 'org.apache.spark' || it.moduleGroup == "org.scala-lang"
188+
})
189+
}
190+
}
191+
```
192+
193+
Now you can package the application using Gradle:
194+
`gradle shadowJar`
162195

163196

164-
197+
## Executing the application with spark-submit
198+
199+
Once you have your jar, you can execute the packaged application with `./bin/spark-submit`:
200+
201+
`YOUR_SPARK_HOME/bin/spark-submit --class "SimpleApp" --master local [path to your jar]`
202+
203+
This example is also available as a [GitHub repo](https://github.com/MKhalusova/kotlin-spark-example), feel free to give it a try.

0 commit comments

Comments
 (0)