Skip to content

Latest commit

 

History

History
8 lines (8 loc) · 377 Bytes

README.md

File metadata and controls

8 lines (8 loc) · 377 Bytes

SparkTuning

An cost-based parameter tuning tool for Spark framework

How to run

  1. compile and package code/spark-1.4.0
  2. install code/spark-1.4.0 to your cluster
  3. shrink input data and sumbit an application to spark cluster
  4. feed the application runtime logs to code/WhatIf, get the best config
  5. rerun the application with the best config and the whole input data