Task 3.1 -  Advanced Analytics using PySpark .ipynb

{"cells":[{"cell_type":"markdown","metadata":{"id":"JRkv266S972W"},"source":["#Task 3: Advanced Analytics using PySpark\n","By Abdulwahab Adeshina Abdulliameed \\\n","Task 3.1) Analyze and Interpret Big Data\n","`In this note, we will Analyze and Interpret Big Data using PySpark.`\n"]},{"cell_type":"markdown","metadata":{"id":"82nnS1p_J8vA"},"source":["**Setting Up PySpark in Colab**"]},{"cell_type":"code","execution_count":1,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":6669,"status":"ok","timestamp":1648899570917,"user":{"displayName":"Abdulwahab Abdulliameed","userId":"10732377996348789858"},"user_tz":-60},"id":"lNBpRPQXJPg9","outputId":"2ed5e72e-eb9f-46ad-e33c-20db70d72cf6"},"outputs":[{"output_type":"stream","name":"stdout","text":["\r0% [Working]\r            \rHit:1 http://archive.ubuntu.com/ubuntu bionic InRelease\n","\r0% [Waiting for headers] [Connecting to security.ubuntu.com (185.125.190.36)] [\r                                                                               \rGet:2 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]\n","Get:3 https://cloud.r-project.org/bin/linux/ubuntu bionic-cran40/ InRelease [3,626 B]\n","Ign:4 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64  InRelease\n","Get:5 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]\n","Get:6 http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu bionic InRelease [15.9 kB]\n","Ign:7 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64  InRelease\n","Get:8 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB]\n","Get:9 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64  Release [696 B]\n","Hit:10 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64  Release\n","Get:11 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64  Release.gpg [836 B]\n","Get:12 https://cloud.r-project.org/bin/linux/ubuntu bionic-cran40/ Packages [80.8 kB]\n","Hit:13 http://ppa.launchpad.net/cran/libgit2/ubuntu bionic InRelease\n","Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [29.8 kB]\n","Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [2,264 kB]\n","Get:17 http://ppa.launchpad.net/deadsnakes/ppa/ubuntu bionic InRelease [15.9 kB]\n","Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [918 kB]\n","Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [3,132 kB]\n","Get:20 http://archive.ubuntu.com/ubuntu bionic-backports/universe amd64 Packages [12.9 kB]\n","Get:21 http://archive.ubuntu.com/ubuntu bionic-backports/main amd64 Packages [12.2 kB]\n","Get:22 http://ppa.launchpad.net/graphics-drivers/ppa/ubuntu bionic InRelease [21.3 kB]\n","Get:23 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64  Packages [950 kB]\n","Get:24 http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu bionic/main Sources [1,830 kB]\n","Get:25 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,486 kB]\n","Get:26 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [2,693 kB]\n","Get:27 http://ppa.launchpad.net/c2d4u.team/c2d4u4.0+/ubuntu bionic/main amd64 Packages [937 kB]\n","Get:28 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [884 kB]\n","Get:29 http://ppa.launchpad.net/deadsnakes/ppa/ubuntu bionic/main amd64 Packages [45.3 kB]\n","Get:30 http://ppa.launchpad.net/graphics-drivers/ppa/ubuntu bionic/main amd64 Packages [44.3 kB]\n","Fetched 15.6 MB in 3s (4,503 kB/s)\n","Reading package lists... Done\n"]}],"source":["#Update system packages\n","!sudo apt-get update"]},{"cell_type":"code","execution_count":2,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":31283,"status":"ok","timestamp":1648899611937,"user":{"displayName":"Abdulwahab Abdulliameed","userId":"10732377996348789858"},"user_tz":-60},"id":"u7saEvNpKXEy","outputId":"90b7f472-525d-46dc-beef-65d14e919e9a"},"outputs":[{"output_type":"stream","name":"stdout","text":["Reading package lists... Done\n","Building dependency tree       \n","Reading state information... Done\n","The following additional packages will be installed:\n","  default-jdk-headless openjdk-11-jdk openjdk-11-jdk-headless openjdk-11-jre\n","  openjdk-11-jre-headless\n","Suggested packages:\n","  openjdk-11-demo openjdk-11-source visualvm libnss-mdns fonts-dejavu-extra\n","  fonts-ipafont-gothic fonts-ipafont-mincho fonts-wqy-microhei\n","  | fonts-wqy-zenhei fonts-indic\n","Recommended packages:\n","  libatk-wrapper-java-jni fonts-dejavu-extra\n","The following NEW packages will be installed:\n","  default-jdk default-jdk-headless openjdk-11-jdk\n","The following packages will be upgraded:\n","  openjdk-11-jdk-headless openjdk-11-jre openjdk-11-jre-headless\n","3 upgraded, 3 newly installed, 0 to remove and 79 not upgraded.\n","Need to get 260 MB of archives.\n","After this operation, 1,610 kB of additional disk space will be used.\n","Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 openjdk-11-jre amd64 11.0.14.1+1-0ubuntu1~18.04 [174 kB]\n","Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 openjdk-11-jdk-headless amd64 11.0.14.1+1-0ubuntu1~18.04 [221 MB]\n","Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 openjdk-11-jre-headless amd64 11.0.14.1+1-0ubuntu1~18.04 [37.2 MB]\n","Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 default-jdk-headless amd64 2:1.11-68ubuntu1~18.04.1 [1,132 B]\n","Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 openjdk-11-jdk amd64 11.0.14.1+1-0ubuntu1~18.04 [1,544 kB]\n","Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 default-jdk amd64 2:1.11-68ubuntu1~18.04.1 [1,092 B]\n","Fetched 260 MB in 5s (52.1 MB/s)\n","debconf: unable to initialize frontend: Dialog\n","debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 76, <> line 6.)\n","debconf: falling back to frontend: Readline\n","debconf: unable to initialize frontend: Readline\n","debconf: (This frontend requires a controlling tty.)\n","debconf: falling back to frontend: Teletype\n","dpkg-preconfigure: unable to re-open stdin: \n","(Reading database ... 156210 files and directories currently installed.)\n","Preparing to unpack .../0-openjdk-11-jre_11.0.14.1+1-0ubuntu1~18.04_amd64.deb ...\n","Unpacking openjdk-11-jre:amd64 (11.0.14.1+1-0ubuntu1~18.04) over (11.0.14+9-0ubuntu2~18.04) ...\n","Preparing to unpack .../1-openjdk-11-jdk-headless_11.0.14.1+1-0ubuntu1~18.04_amd64.deb ...\n","Unpacking openjdk-11-jdk-headless:amd64 (11.0.14.1+1-0ubuntu1~18.04) over (11.0.14+9-0ubuntu2~18.04) ...\n","Preparing to unpack .../2-openjdk-11-jre-headless_11.0.14.1+1-0ubuntu1~18.04_amd64.deb ...\n","Unpacking openjdk-11-jre-headless:amd64 (11.0.14.1+1-0ubuntu1~18.04) over (11.0.14+9-0ubuntu2~18.04) ...\n","Selecting previously unselected package default-jdk-headless.\n","Preparing to unpack .../3-default-jdk-headless_2%3a1.11-68ubuntu1~18.04.1_amd64.deb ...\n","Unpacking default-jdk-headless (2:1.11-68ubuntu1~18.04.1) ...\n","Selecting previously unselected package openjdk-11-jdk:amd64.\n","Preparing to unpack .../4-openjdk-11-jdk_11.0.14.1+1-0ubuntu1~18.04_amd64.deb ...\n","Unpacking openjdk-11-jdk:amd64 (11.0.14.1+1-0ubuntu1~18.04) ...\n","Selecting previously unselected package default-jdk.\n","Preparing to unpack .../5-default-jdk_2%3a1.11-68ubuntu1~18.04.1_amd64.deb ...\n","Unpacking default-jdk (2:1.11-68ubuntu1~18.04.1) ...\n","Setting up openjdk-11-jre-headless:amd64 (11.0.14.1+1-0ubuntu1~18.04) ...\n","Setting up openjdk-11-jdk-headless:amd64 (11.0.14.1+1-0ubuntu1~18.04) ...\n","Setting up default-jdk-headless (2:1.11-68ubuntu1~18.04.1) ...\n","Setting up openjdk-11-jre:amd64 (11.0.14.1+1-0ubuntu1~18.04) ...\n","Setting up openjdk-11-jdk:amd64 (11.0.14.1+1-0ubuntu1~18.04) ...\n","update-alternatives: using /usr/lib/jvm/java-11-openjdk-amd64/bin/jconsole to provide /usr/bin/jconsole (jconsole) in auto mode\n","Setting up default-jdk (2:1.11-68ubuntu1~18.04.1) ...\n","Processing triggers for hicolor-icon-theme (0.17-2) ...\n","Processing triggers for mime-support (3.60ubuntu1) ...\n"]}],"source":["#install Java\n","!sudo apt install default-jdk"]},{"cell_type":"code","execution_count":3,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":12052,"status":"ok","timestamp":1648899637272,"user":{"displayName":"Abdulwahab Abdulliameed","userId":"10732377996348789858"},"user_tz":-60},"id":"di3lVZJjKgSD","outputId":"03ad7683-3caf-48f4-e3e0-944a887845bd"},"outputs":[{"output_type":"stream","name":"stdout","text":["--2022-04-02 11:40:25--  https://archive.apache.org/dist/spark/spark-3.0.3/spark-3.0.3-bin-hadoop2.7.tgz\n","Resolving archive.apache.org (archive.apache.org)... 138.201.131.134, 2a01:4f8:172:2ec5::2\n","Connecting to archive.apache.org (archive.apache.org)|138.201.131.134|:443... connected.\n","HTTP request sent, awaiting response... 200 OK\n","Length: 220400553 (210M) [application/x-gzip]\n","Saving to: ‘spark-3.0.3-bin-hadoop2.7.tgz’\n","\n","spark-3.0.3-bin-had 100%[===================>] 210.19M  28.0MB/s    in 8.2s    \n","\n","2022-04-02 11:40:34 (25.6 MB/s) - ‘spark-3.0.3-bin-hadoop2.7.tgz’ saved [220400553/220400553]\n","\n","spark-3.0.3-bin-hadoop2.7/\n","spark-3.0.3-bin-hadoop2.7/NOTICE\n","spark-3.0.3-bin-hadoop2.7/kubernetes/\n","spark-3.0.3-bin-hadoop2.7/kubernetes/tests/\n","spark-3.0.3-bin-hadoop2.7/kubernetes/tests/worker_memory_check.py\n","spark-3.0.3-bin-hadoop2.7/kubernetes/tests/py_container_checks.py\n","spark-3.0.3-bin-hadoop2.7/kubernetes/tests/pyfiles.py\n","spark-3.0.3-bin-hadoop2.7/kubernetes/dockerfiles/\n","spark-3.0.3-bin-hadoop2.7/kubernetes/dockerfiles/spark/\n","spark-3.0.3-bin-hadoop2.7/kubernetes/dockerfiles/spark/entrypoint.sh\n","spark-3.0.3-bin-hadoop2.7/kubernetes/dockerfiles/spark/bindings/\n","spark-3.0.3-bin-hadoop2.7/kubernetes/dockerfiles/spark/bindings/R/\n","spark-3.0.3-bin-hadoop2.7/kubernetes/dockerfiles/spark/bindings/R/Dockerfile\n","spark-3.0.3-bin-hadoop2.7/kubernetes/dockerfiles/spark/bindings/python/\n","spark-3.0.3-bin-hadoop2.7/kubernetes/dockerfiles/spark/bindings/python/Dockerfile\n","spark-3.0.3-bin-hadoop2.7/kubernetes/dockerfiles/spark/Dockerfile\n","spark-3.0.3-bin-hadoop2.7/jars/\n","spark-3.0.3-bin-hadoop2.7/jars/jackson-xc-1.9.13.jar\n","spark-3.0.3-bin-hadoop2.7/jars/commons-digester-1.8.jar\n","spark-3.0.3-bin-hadoop2.7/jars/api-util-1.0.0-M20.jar\n","spark-3.0.3-bin-hadoop2.7/jars/hive-vector-code-gen-2.3.7.jar\n","spark-3.0.3-bin-hadoop2.7/jars/derby-10.12.1.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/commons-beanutils-1.9.4.jar\n","spark-3.0.3-bin-hadoop2.7/jars/httpcore-4.4.12.jar\n","spark-3.0.3-bin-hadoop2.7/jars/hadoop-yarn-api-2.7.4.jar\n","spark-3.0.3-bin-hadoop2.7/jars/scala-library-2.12.10.jar\n","spark-3.0.3-bin-hadoop2.7/jars/parquet-format-2.4.0.jar\n","spark-3.0.3-bin-hadoop2.7/jars/kryo-shaded-4.0.2.jar\n","spark-3.0.3-bin-hadoop2.7/jars/xercesImpl-2.12.0.jar\n","spark-3.0.3-bin-hadoop2.7/jars/commons-logging-1.1.3.jar\n","spark-3.0.3-bin-hadoop2.7/jars/okio-1.15.0.jar\n","spark-3.0.3-bin-hadoop2.7/jars/commons-compiler-3.0.16.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jdo-api-3.0.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/spire-macros_2.12-0.17.0-M1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/arrow-memory-0.15.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/JLargeArrays-1.5.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jsp-api-2.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/logging-interceptor-3.12.6.jar\n","spark-3.0.3-bin-hadoop2.7/jars/javax.servlet-api-3.1.0.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jcl-over-slf4j-1.7.30.jar\n","spark-3.0.3-bin-hadoop2.7/jars/hive-cli-2.3.7.jar\n","spark-3.0.3-bin-hadoop2.7/jars/apacheds-i18n-2.0.0-M15.jar\n","spark-3.0.3-bin-hadoop2.7/jars/parquet-common-1.10.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/stax-api-1.0.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jackson-dataformat-yaml-2.10.0.jar\n","spark-3.0.3-bin-hadoop2.7/jars/algebra_2.12-2.0.0-M2.jar\n","spark-3.0.3-bin-hadoop2.7/jars/spark-graphx_2.12-3.0.3.jar\n","spark-3.0.3-bin-hadoop2.7/jars/hadoop-annotations-2.7.4.jar\n","spark-3.0.3-bin-hadoop2.7/jars/guice-servlet-3.0.jar\n","spark-3.0.3-bin-hadoop2.7/jars/kubernetes-client-4.9.2.jar\n","spark-3.0.3-bin-hadoop2.7/jars/spire-util_2.12-0.17.0-M1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jakarta.activation-api-1.2.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/spark-network-shuffle_2.12-3.0.3.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jline-2.14.6.jar\n","spark-3.0.3-bin-hadoop2.7/jars/breeze_2.12-1.0.jar\n","spark-3.0.3-bin-hadoop2.7/jars/metrics-jvm-4.1.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/dropwizard-metrics-hadoop-metrics2-reporter-0.1.2.jar\n","spark-3.0.3-bin-hadoop2.7/jars/spark-launcher_2.12-3.0.3.jar\n","spark-3.0.3-bin-hadoop2.7/jars/machinist_2.12-0.6.8.jar\n","spark-3.0.3-bin-hadoop2.7/jars/scala-compiler-2.12.10.jar\n","spark-3.0.3-bin-hadoop2.7/jars/commons-pool-1.5.4.jar\n","spark-3.0.3-bin-hadoop2.7/jars/arrow-format-0.15.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/spark-mllib_2.12-3.0.3.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jta-1.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/spark-yarn_2.12-3.0.3.jar\n","spark-3.0.3-bin-hadoop2.7/jars/istack-commons-runtime-3.0.8.jar\n","spark-3.0.3-bin-hadoop2.7/jars/metrics-graphite-4.1.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/hadoop-hdfs-2.7.4.jar\n","spark-3.0.3-bin-hadoop2.7/jars/commons-collections-3.2.2.jar\n","spark-3.0.3-bin-hadoop2.7/jars/audience-annotations-0.5.0.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jpam-1.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jakarta.annotation-api-1.3.5.jar\n","spark-3.0.3-bin-hadoop2.7/jars/xmlenc-0.52.jar\n","spark-3.0.3-bin-hadoop2.7/jars/api-asn1-api-1.0.0-M20.jar\n","spark-3.0.3-bin-hadoop2.7/jars/py4j-0.10.9.jar\n","spark-3.0.3-bin-hadoop2.7/jars/hadoop-mapreduce-client-common-2.7.4.jar\n","spark-3.0.3-bin-hadoop2.7/jars/spire-platform_2.12-0.17.0-M1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/avro-1.8.2.jar\n","spark-3.0.3-bin-hadoop2.7/jars/spark-core_2.12-3.0.3.jar\n","spark-3.0.3-bin-hadoop2.7/jars/curator-client-2.7.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/antlr4-runtime-4.7.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/javax.inject-1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/spark-kubernetes_2.12-3.0.3.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jersey-media-jaxb-2.30.jar\n","spark-3.0.3-bin-hadoop2.7/jars/commons-lang3-3.9.jar\n","spark-3.0.3-bin-hadoop2.7/jars/javax.jdo-3.2.0-m3.jar\n","spark-3.0.3-bin-hadoop2.7/jars/hadoop-auth-2.7.4.jar\n","spark-3.0.3-bin-hadoop2.7/jars/commons-io-2.4.jar\n","spark-3.0.3-bin-hadoop2.7/jars/objenesis-2.5.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/paranamer-2.8.jar\n","spark-3.0.3-bin-hadoop2.7/jars/cats-kernel_2.12-2.0.0-M4.jar\n","spark-3.0.3-bin-hadoop2.7/jars/flatbuffers-java-1.9.0.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jersey-server-2.30.jar\n","spark-3.0.3-bin-hadoop2.7/jars/stream-2.9.6.jar\n","spark-3.0.3-bin-hadoop2.7/jars/datanucleus-api-jdo-4.2.4.jar\n","spark-3.0.3-bin-hadoop2.7/jars/gson-2.2.4.jar\n","spark-3.0.3-bin-hadoop2.7/jars/xml-apis-1.4.01.jar\n","spark-3.0.3-bin-hadoop2.7/jars/apacheds-kerberos-codec-2.0.0-M15.jar\n","spark-3.0.3-bin-hadoop2.7/jars/hadoop-mapreduce-client-shuffle-2.7.4.jar\n","spark-3.0.3-bin-hadoop2.7/jars/spark-hive_2.12-3.0.3.jar\n","spark-3.0.3-bin-hadoop2.7/jars/hive-jdbc-2.3.7.jar\n","spark-3.0.3-bin-hadoop2.7/jars/hive-exec-2.3.7-core.jar\n","spark-3.0.3-bin-hadoop2.7/jars/opencsv-2.3.jar\n","spark-3.0.3-bin-hadoop2.7/jars/spark-sketch_2.12-3.0.3.jar\n","spark-3.0.3-bin-hadoop2.7/jars/avro-mapred-1.8.2-hadoop2.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jsr305-3.0.0.jar\n","spark-3.0.3-bin-hadoop2.7/jars/univocity-parsers-2.9.0.jar\n","spark-3.0.3-bin-hadoop2.7/jars/macro-compat_2.12-1.1.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/commons-dbcp-1.4.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jersey-container-servlet-core-2.30.jar\n","spark-3.0.3-bin-hadoop2.7/jars/orc-core-1.5.10.jar\n","spark-3.0.3-bin-hadoop2.7/jars/hk2-api-2.6.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/parquet-jackson-1.10.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/mesos-1.4.0-shaded-protobuf.jar\n","spark-3.0.3-bin-hadoop2.7/jars/chill-java-0.9.5.jar\n","spark-3.0.3-bin-hadoop2.7/jars/spark-tags_2.12-3.0.3-tests.jar\n","spark-3.0.3-bin-hadoop2.7/jars/automaton-1.11-8.jar\n","spark-3.0.3-bin-hadoop2.7/jars/commons-lang-2.6.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jackson-annotations-2.10.0.jar\n","spark-3.0.3-bin-hadoop2.7/jars/hk2-utils-2.6.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/spark-network-common_2.12-3.0.3.jar\n","spark-3.0.3-bin-hadoop2.7/jars/spark-repl_2.12-3.0.3.jar\n","spark-3.0.3-bin-hadoop2.7/jars/velocity-1.5.jar\n","spark-3.0.3-bin-hadoop2.7/jars/htrace-core-3.1.0-incubating.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jul-to-slf4j-1.7.30.jar\n","spark-3.0.3-bin-hadoop2.7/jars/JTransforms-3.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/json4s-ast_2.12-3.6.6.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jersey-client-2.30.jar\n","spark-3.0.3-bin-hadoop2.7/jars/httpclient-4.5.6.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jetty-sslengine-6.1.26.jar\n","spark-3.0.3-bin-hadoop2.7/jars/spark-kvstore_2.12-3.0.3.jar\n","spark-3.0.3-bin-hadoop2.7/jars/spire_2.12-0.17.0-M1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jersey-hk2-2.30.jar\n","spark-3.0.3-bin-hadoop2.7/jars/commons-math3-3.4.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/xbean-asm7-shaded-4.15.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jakarta.validation-api-2.0.2.jar\n","spark-3.0.3-bin-hadoop2.7/jars/parquet-encoding-1.10.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/snappy-java-1.1.8.2.jar\n","spark-3.0.3-bin-hadoop2.7/jars/slf4j-log4j12-1.7.30.jar\n","spark-3.0.3-bin-hadoop2.7/jars/hadoop-mapreduce-client-app-2.7.4.jar\n","spark-3.0.3-bin-hadoop2.7/jars/zstd-jni-1.4.4-3.jar\n","spark-3.0.3-bin-hadoop2.7/jars/slf4j-api-1.7.30.jar\n","spark-3.0.3-bin-hadoop2.7/jars/hive-shims-0.23-2.3.7.jar\n","spark-3.0.3-bin-hadoop2.7/jars/guava-14.0.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/stax-api-1.0-2.jar\n","spark-3.0.3-bin-hadoop2.7/jars/zjsonpatch-0.3.0.jar\n","spark-3.0.3-bin-hadoop2.7/jars/hive-storage-api-2.7.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/shapeless_2.12-2.3.3.jar\n","spark-3.0.3-bin-hadoop2.7/jars/kubernetes-model-common-4.9.2.jar\n","spark-3.0.3-bin-hadoop2.7/jars/oro-2.0.8.jar\n","spark-3.0.3-bin-hadoop2.7/jars/hadoop-yarn-client-2.7.4.jar\n","spark-3.0.3-bin-hadoop2.7/jars/libfb303-0.9.3.jar\n","spark-3.0.3-bin-hadoop2.7/jars/core-1.1.2.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jersey-container-servlet-2.30.jar\n","spark-3.0.3-bin-hadoop2.7/jars/datanucleus-rdbms-4.1.19.jar\n","spark-3.0.3-bin-hadoop2.7/jars/super-csv-2.2.0.jar\n","spark-3.0.3-bin-hadoop2.7/jars/compress-lzf-1.0.3.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jackson-module-paranamer-2.10.0.jar\n","spark-3.0.3-bin-hadoop2.7/jars/aopalliance-1.0.jar\n","spark-3.0.3-bin-hadoop2.7/jars/osgi-resource-locator-1.0.3.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jackson-module-scala_2.12-2.10.0.jar\n","spark-3.0.3-bin-hadoop2.7/jars/hive-shims-2.3.7.jar\n","spark-3.0.3-bin-hadoop2.7/jars/hadoop-yarn-server-common-2.7.4.jar\n","spark-3.0.3-bin-hadoop2.7/jars/json-1.8.jar\n","spark-3.0.3-bin-hadoop2.7/jars/antlr-runtime-3.5.2.jar\n","spark-3.0.3-bin-hadoop2.7/jars/threeten-extra-1.5.0.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jetty-6.1.26.jar\n","spark-3.0.3-bin-hadoop2.7/jars/spark-tags_2.12-3.0.3.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jackson-module-jaxb-annotations-2.10.0.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jersey-common-2.30.jar\n","spark-3.0.3-bin-hadoop2.7/jars/aircompressor-0.10.jar\n","spark-3.0.3-bin-hadoop2.7/jars/lz4-java-1.7.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/hadoop-client-2.7.4.jar\n","spark-3.0.3-bin-hadoop2.7/jars/activation-1.1.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/shims-0.7.45.jar\n","spark-3.0.3-bin-hadoop2.7/jars/commons-crypto-1.1.0.jar\n","spark-3.0.3-bin-hadoop2.7/jars/spark-mesos_2.12-3.0.3.jar\n","spark-3.0.3-bin-hadoop2.7/jars/libthrift-0.12.0.jar\n","spark-3.0.3-bin-hadoop2.7/jars/orc-mapreduce-1.5.10.jar\n","spark-3.0.3-bin-hadoop2.7/jars/HikariCP-2.5.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/generex-1.0.2.jar\n","spark-3.0.3-bin-hadoop2.7/jars/okhttp-3.12.6.jar\n","spark-3.0.3-bin-hadoop2.7/jars/breeze-macros_2.12-1.0.jar\n","spark-3.0.3-bin-hadoop2.7/jars/leveldbjni-all-1.8.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jaxb-runtime-2.3.2.jar\n","spark-3.0.3-bin-hadoop2.7/jars/commons-httpclient-3.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jackson-core-asl-1.9.13.jar\n","spark-3.0.3-bin-hadoop2.7/jars/scala-xml_2.12-1.2.0.jar\n","spark-3.0.3-bin-hadoop2.7/jars/hk2-locator-2.6.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/hadoop-common-2.7.4.jar\n","spark-3.0.3-bin-hadoop2.7/jars/curator-recipes-2.7.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jackson-mapper-asl-1.9.13.jar\n","spark-3.0.3-bin-hadoop2.7/jars/hive-common-2.3.7.jar\n","spark-3.0.3-bin-hadoop2.7/jars/xz-1.5.jar\n","spark-3.0.3-bin-hadoop2.7/jars/ST4-4.0.4.jar\n","spark-3.0.3-bin-hadoop2.7/jars/hive-shims-scheduler-2.3.7.jar\n","spark-3.0.3-bin-hadoop2.7/jars/spark-unsafe_2.12-3.0.3.jar\n","spark-3.0.3-bin-hadoop2.7/jars/parquet-hadoop-1.10.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/chill_2.12-0.9.5.jar\n","spark-3.0.3-bin-hadoop2.7/jars/commons-configuration-1.6.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jackson-databind-2.10.0.jar\n","spark-3.0.3-bin-hadoop2.7/jars/spark-hive-thriftserver_2.12-3.0.3.jar\n","spark-3.0.3-bin-hadoop2.7/jars/scala-reflect-2.12.10.jar\n","spark-3.0.3-bin-hadoop2.7/jars/joda-time-2.10.5.jar\n","spark-3.0.3-bin-hadoop2.7/jars/minlog-1.3.0.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jakarta.ws.rs-api-2.1.6.jar\n","spark-3.0.3-bin-hadoop2.7/jars/orc-shims-1.5.10.jar\n","spark-3.0.3-bin-hadoop2.7/jars/aopalliance-repackaged-2.6.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/commons-compress-1.20.jar\n","spark-3.0.3-bin-hadoop2.7/jars/bonecp-0.8.0.RELEASE.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jodd-core-3.5.2.jar\n","spark-3.0.3-bin-hadoop2.7/jars/hadoop-yarn-server-web-proxy-2.7.4.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jackson-core-2.10.0.jar\n","spark-3.0.3-bin-hadoop2.7/jars/protobuf-java-2.5.0.jar\n","spark-3.0.3-bin-hadoop2.7/jars/commons-net-3.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/hadoop-mapreduce-client-jobclient-2.7.4.jar\n","spark-3.0.3-bin-hadoop2.7/jars/datanucleus-core-4.1.17.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jackson-jaxrs-1.9.13.jar\n","spark-3.0.3-bin-hadoop2.7/jars/janino-3.0.16.jar\n","spark-3.0.3-bin-hadoop2.7/jars/hive-shims-common-2.3.7.jar\n","spark-3.0.3-bin-hadoop2.7/jars/javolution-5.5.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/transaction-api-1.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/spark-streaming_2.12-3.0.3.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jackson-datatype-jsr310-2.10.3.jar\n","spark-3.0.3-bin-hadoop2.7/jars/spark-mllib-local_2.12-3.0.3.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jaxb-api-2.2.2.jar\n","spark-3.0.3-bin-hadoop2.7/jars/commons-cli-1.2.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jakarta.inject-2.6.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/metrics-jmx-4.1.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/arrow-vector-0.15.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jakarta.xml.bind-api-2.3.2.jar\n","spark-3.0.3-bin-hadoop2.7/jars/commons-codec-1.10.jar\n","spark-3.0.3-bin-hadoop2.7/jars/spark-catalyst_2.12-3.0.3.jar\n","spark-3.0.3-bin-hadoop2.7/jars/hadoop-yarn-common-2.7.4.jar\n","spark-3.0.3-bin-hadoop2.7/jars/curator-framework-2.7.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/arpack_combined_all-0.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/hive-metastore-2.3.7.jar\n","spark-3.0.3-bin-hadoop2.7/jars/hive-beeline-2.3.7.jar\n","spark-3.0.3-bin-hadoop2.7/jars/metrics-core-4.1.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/hive-llap-common-2.3.7.jar\n","spark-3.0.3-bin-hadoop2.7/jars/netty-all-4.1.47.Final.jar\n","spark-3.0.3-bin-hadoop2.7/jars/javassist-3.25.0-GA.jar\n","spark-3.0.3-bin-hadoop2.7/jars/scala-collection-compat_2.12-2.1.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/spark-sql_2.12-3.0.3.jar\n","spark-3.0.3-bin-hadoop2.7/jars/parquet-column-1.10.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/jetty-util-6.1.26.jar\n","spark-3.0.3-bin-hadoop2.7/jars/kubernetes-model-4.9.2.jar\n","spark-3.0.3-bin-hadoop2.7/jars/avro-ipc-1.8.2.jar\n","spark-3.0.3-bin-hadoop2.7/jars/guice-3.0.jar\n","spark-3.0.3-bin-hadoop2.7/jars/commons-text-1.6.jar\n","spark-3.0.3-bin-hadoop2.7/jars/json4s-scalap_2.12-3.6.6.jar\n","spark-3.0.3-bin-hadoop2.7/jars/metrics-json-4.1.1.jar\n","spark-3.0.3-bin-hadoop2.7/jars/snakeyaml-1.24.jar\n","spark-3.0.3-bin-hadoop2.7/jars/json4s-jackson_2.12-3.6.6.jar\n","spark-3.0.3-bin-hadoop2.7/jars/ivy-2.4.0.jar\n","spark-3.0.3-bin-hadoop2.7/jars/hadoop-mapreduce-client-core-2.7.4.jar\n","spark-3.0.3-bin-hadoop2.7/jars/json4s-core_2.12-3.6.6.jar\n","spark-3.0.3-bin-hadoop2.7/jars/pyrolite-4.30.jar\n","spark-3.0.3-bin-hadoop2.7/jars/log4j-1.2.17.jar\n","spark-3.0.3-bin-hadoop2.7/jars/hive-serde-2.3.7.jar\n","spark-3.0.3-bin-hadoop2.7/jars/scala-parser-combinators_2.12-1.1.2.jar\n","spark-3.0.3-bin-hadoop2.7/jars/RoaringBitmap-0.7.45.jar\n","spark-3.0.3-bin-hadoop2.7/jars/zookeeper-3.4.14.jar\n","spark-3.0.3-bin-hadoop2.7/data/\n","spark-3.0.3-bin-hadoop2.7/data/mllib/\n","spark-3.0.3-bin-hadoop2.7/data/mllib/sample_lda_data.txt\n","spark-3.0.3-bin-hadoop2.7/data/mllib/sample_libsvm_data.txt\n","spark-3.0.3-bin-hadoop2.7/data/mllib/sample_svm_data.txt\n","spark-3.0.3-bin-hadoop2.7/data/mllib/sample_multiclass_classification_data.txt\n","spark-3.0.3-bin-hadoop2.7/data/mllib/sample_linear_regression_data.txt\n","spark-3.0.3-bin-hadoop2.7/data/mllib/sample_lda_libsvm_data.txt\n","spark-3.0.3-bin-hadoop2.7/data/mllib/sample_fpgrowth.txt\n","spark-3.0.3-bin-hadoop2.7/data/mllib/sample_binary_classification_data.txt\n","spark-3.0.3-bin-hadoop2.7/data/mllib/sample_isotonic_regression_libsvm_data.txt\n","spark-3.0.3-bin-hadoop2.7/data/mllib/iris_libsvm.txt\n","spark-3.0.3-bin-hadoop2.7/data/mllib/sample_movielens_data.txt\n","spark-3.0.3-bin-hadoop2.7/data/mllib/als/\n","spark-3.0.3-bin-hadoop2.7/data/mllib/als/test.data\n","spark-3.0.3-bin-hadoop2.7/data/mllib/als/sample_movielens_ratings.txt\n","spark-3.0.3-bin-hadoop2.7/data/mllib/sample_kmeans_data.txt\n","spark-3.0.3-bin-hadoop2.7/data/mllib/pic_data.txt\n","spark-3.0.3-bin-hadoop2.7/data/mllib/images/\n","spark-3.0.3-bin-hadoop2.7/data/mllib/images/origin/\n","spark-3.0.3-bin-hadoop2.7/data/mllib/images/origin/kittens/\n","spark-3.0.3-bin-hadoop2.7/data/mllib/images/origin/kittens/54893.jpg\n","spark-3.0.3-bin-hadoop2.7/data/mllib/images/origin/kittens/not-image.txt\n","spark-3.0.3-bin-hadoop2.7/data/mllib/images/origin/kittens/DP802813.jpg\n","spark-3.0.3-bin-hadoop2.7/data/mllib/images/origin/kittens/29.5.a_b_EGDP022204.jpg\n","spark-3.0.3-bin-hadoop2.7/data/mllib/images/origin/kittens/DP153539.jpg\n","spark-3.0.3-bin-hadoop2.7/data/mllib/images/origin/license.txt\n","spark-3.0.3-bin-hadoop2.7/data/mllib/images/origin/multi-channel/\n","spark-3.0.3-bin-hadoop2.7/data/mllib/images/origin/multi-channel/BGRA.png\n","spark-3.0.3-bin-hadoop2.7/data/mllib/images/origin/multi-channel/BGRA_alpha_60.png\n","spark-3.0.3-bin-hadoop2.7/data/mllib/images/origin/multi-channel/grayscale.jpg\n","spark-3.0.3-bin-hadoop2.7/data/mllib/images/origin/multi-channel/chr30.4.184.jpg\n","spark-3.0.3-bin-hadoop2.7/data/mllib/images/partitioned/\n","spark-3.0.3-bin-hadoop2.7/data/mllib/images/partitioned/cls=multichannel/\n","spark-3.0.3-bin-hadoop2.7/data/mllib/images/partitioned/cls=multichannel/date=2018-01/\n","spark-3.0.3-bin-hadoop2.7/data/mllib/images/partitioned/cls=multichannel/date=2018-01/BGRA.png\n","spark-3.0.3-bin-hadoop2.7/data/mllib/images/partitioned/cls=multichannel/date=2018-01/BGRA_alpha_60.png\n","spark-3.0.3-bin-hadoop2.7/data/mllib/images/partitioned/cls=multichannel/date=2018-02/\n","spark-3.0.3-bin-hadoop2.7/data/mllib/images/partitioned/cls=multichannel/date=2018-02/grayscale.jpg\n","spark-3.0.3-bin-hadoop2.7/data/mllib/images/partitioned/cls=multichannel/date=2018-02/chr30.4.184.jpg\n","spark-3.0.3-bin-hadoop2.7/data/mllib/images/partitioned/cls=kittens/\n","spark-3.0.3-bin-hadoop2.7/data/mllib/images/partitioned/cls=kittens/date=2018-01/\n","spark-3.0.3-bin-hadoop2.7/data/mllib/images/partitioned/cls=kittens/date=2018-01/not-image.txt\n","spark-3.0.3-bin-hadoop2.7/data/mllib/images/partitioned/cls=kittens/date=2018-01/29.5.a_b_EGDP022204.jpg\n","spark-3.0.3-bin-hadoop2.7/data/mllib/images/partitioned/cls=kittens/date=2018-02/\n","spark-3.0.3-bin-hadoop2.7/data/mllib/images/partitioned/cls=kittens/date=2018-02/54893.jpg\n","spark-3.0.3-bin-hadoop2.7/data/mllib/images/partitioned/cls=kittens/date=2018-02/DP802813.jpg\n","spark-3.0.3-bin-hadoop2.7/data/mllib/images/partitioned/cls=kittens/date=2018-02/DP153539.jpg\n","spark-3.0.3-bin-hadoop2.7/data/mllib/images/license.txt\n","spark-3.0.3-bin-hadoop2.7/data/mllib/ridge-data/\n","spark-3.0.3-bin-hadoop2.7/data/mllib/ridge-data/lpsa.data\n","spark-3.0.3-bin-hadoop2.7/data/mllib/kmeans_data.txt\n","spark-3.0.3-bin-hadoop2.7/data/mllib/streaming_kmeans_data_test.txt\n","spark-3.0.3-bin-hadoop2.7/data/mllib/pagerank_data.txt\n","spark-3.0.3-bin-hadoop2.7/data/mllib/gmm_data.txt\n","spark-3.0.3-bin-hadoop2.7/data/graphx/\n","spark-3.0.3-bin-hadoop2.7/data/graphx/users.txt\n","spark-3.0.3-bin-hadoop2.7/data/graphx/followers.txt\n","spark-3.0.3-bin-hadoop2.7/data/streaming/\n","spark-3.0.3-bin-hadoop2.7/data/streaming/AFINN-111.txt\n","spark-3.0.3-bin-hadoop2.7/R/\n","spark-3.0.3-bin-hadoop2.7/R/lib/\n","spark-3.0.3-bin-hadoop2.7/R/lib/sparkr.zip\n","spark-3.0.3-bin-hadoop2.7/R/lib/SparkR/\n","spark-3.0.3-bin-hadoop2.7/R/lib/SparkR/tests/\n","spark-3.0.3-bin-hadoop2.7/R/lib/SparkR/tests/testthat/\n","spark-3.0.3-bin-hadoop2.7/R/lib/SparkR/tests/testthat/test_basic.R\n","spark-3.0.3-bin-hadoop2.7/R/lib/SparkR/DESCRIPTION\n","spark-3.0.3-bin-hadoop2.7/R/lib/SparkR/profile/\n","spark-3.0.3-bin-hadoop2.7/R/lib/SparkR/profile/shell.R\n","spark-3.0.3-bin-hadoop2.7/R/lib/SparkR/profile/general.R\n","spark-3.0.3-bin-hadoop2.7/R/lib/SparkR/INDEX\n","spark-3.0.3-bin-hadoop2.7/R/lib/SparkR/Meta/\n","spark-3.0.3-bin-hadoop2.7/R/lib/SparkR/Meta/features.rds\n","spark-3.0.3-bin-hadoop2.7/R/lib/SparkR/Meta/links.rds\n","spark-3.0.3-bin-hadoop2.7/R/lib/SparkR/Meta/nsInfo.rds\n","spark-3.0.3-bin-hadoop2.7/R/lib/SparkR/Meta/package.rds\n","spark-3.0.3-bin-hadoop2.7/R/lib/SparkR/Meta/Rd.rds\n","spark-3.0.3-bin-hadoop2.7/R/lib/SparkR/Meta/hsearch.rds\n","spark-3.0.3-bin-hadoop2.7/R/lib/SparkR/help/\n","spark-3.0.3-bin-hadoop2.7/R/lib/SparkR/help/SparkR.rdb\n","spark-3.0.3-bin-hadoop2.7/R/lib/SparkR/help/aliases.rds\n","spark-3.0.3-bin-hadoop2.7/R/lib/SparkR/help/SparkR.rdx\n","spark-3.0.3-bin-hadoop2.7/R/lib/SparkR/help/AnIndex\n","spark-3.0.3-bin-hadoop2.7/R/lib/SparkR/help/paths.rds\n","spark-3.0.3-bin-hadoop2.7/R/lib/SparkR/R/\n","spark-3.0.3-bin-hadoop2.7/R/lib/SparkR/R/SparkR.rdb\n","spark-3.0.3-bin-hadoop2.7/R/lib/SparkR/R/SparkR.rdx\n","spark-3.0.3-bin-hadoop2.7/R/lib/SparkR/R/SparkR\n","spark-3.0.3-bin-hadoop2.7/R/lib/SparkR/NAMESPACE\n","spark-3.0.3-bin-hadoop2.7/R/lib/SparkR/html/\n","spark-3.0.3-bin-hadoop2.7/R/lib/SparkR/html/00Index.html\n","spark-3.0.3-bin-hadoop2.7/R/lib/SparkR/html/R.css\n","spark-3.0.3-bin-hadoop2.7/R/lib/SparkR/worker/\n","spark-3.0.3-bin-hadoop2.7/R/lib/SparkR/worker/worker.R\n","spark-3.0.3-bin-hadoop2.7/R/lib/SparkR/worker/daemon.R\n","spark-3.0.3-bin-hadoop2.7/README.md\n","spark-3.0.3-bin-hadoop2.7/RELEASE\n","spark-3.0.3-bin-hadoop2.7/yarn/\n","spark-3.0.3-bin-hadoop2.7/yarn/spark-3.0.3-yarn-shuffle.jar\n","spark-3.0.3-bin-hadoop2.7/LICENSE\n","spark-3.0.3-bin-hadoop2.7/sbin/\n","spark-3.0.3-bin-hadoop2.7/sbin/stop-mesos-shuffle-service.sh\n","spark-3.0.3-bin-hadoop2.7/sbin/start-master.sh\n","spark-3.0.3-bin-hadoop2.7/sbin/spark-config.sh\n","spark-3.0.3-bin-hadoop2.7/sbin/start-history-server.sh\n","spark-3.0.3-bin-hadoop2.7/sbin/start-slaves.sh\n","spark-3.0.3-bin-hadoop2.7/sbin/spark-daemon.sh\n","spark-3.0.3-bin-hadoop2.7/sbin/stop-mesos-dispatcher.sh\n","spark-3.0.3-bin-hadoop2.7/sbin/start-mesos-shuffle-service.sh\n","spark-3.0.3-bin-hadoop2.7/sbin/slaves.sh\n","spark-3.0.3-bin-hadoop2.7/sbin/stop-history-server.sh\n","spark-3.0.3-bin-hadoop2.7/sbin/start-thriftserver.sh\n","spark-3.0.3-bin-hadoop2.7/sbin/stop-thriftserver.sh\n","spark-3.0.3-bin-hadoop2.7/sbin/start-slave.sh\n","spark-3.0.3-bin-hadoop2.7/sbin/start-all.sh\n","spark-3.0.3-bin-hadoop2.7/sbin/stop-slave.sh\n","spark-3.0.3-bin-hadoop2.7/sbin/spark-daemons.sh\n","spark-3.0.3-bin-hadoop2.7/sbin/stop-slaves.sh\n","spark-3.0.3-bin-hadoop2.7/sbin/stop-all.sh\n","spark-3.0.3-bin-hadoop2.7/sbin/start-mesos-dispatcher.sh\n","spark-3.0.3-bin-hadoop2.7/sbin/stop-master.sh\n","spark-3.0.3-bin-hadoop2.7/examples/\n","spark-3.0.3-bin-hadoop2.7/examples/src/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/r/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/r/ml/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/r/ml/survreg.R\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/r/ml/glm.R\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/r/ml/prefixSpan.R\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/r/ml/powerIterationClustering.R\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/r/ml/lda.R\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/r/ml/kstest.R\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/r/ml/isoreg.R\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/r/ml/ml.R\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/r/ml/naiveBayes.R\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/r/ml/mlp.R\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/r/ml/als.R\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/r/ml/kmeans.R\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/r/ml/svmLinear.R\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/r/ml/logit.R\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/r/ml/randomForest.R\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/r/ml/gbt.R\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/r/ml/decisionTree.R\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/r/ml/gaussianMixture.R\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/r/ml/bisectingKmeans.R\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/r/ml/fpm.R\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/r/dataframe.R\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/r/RSparkSQLExample.R\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/r/data-manipulation.R\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/r/streaming/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/r/streaming/structured_network_wordcount.R\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/SparkPageRank.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/SkewedGroupByTest.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/AccumulatorMetricsTest.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/HdfsTest.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/Word2VecExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/DeveloperApiExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/MinHashLSHExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/IndexToStringExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/BisectingKMeansExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/CorrelationExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/MulticlassLogisticRegressionWithElasticNetExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/VectorSizeHintExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/NGramExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/BucketedRandomProjectionLSHExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/BinarizerExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/RandomForestExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/StandardScalerExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/LogisticRegressionExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/FeatureHasherExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/OneVsRestExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/StopWordsRemoverExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/LinearSVCExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/GaussianMixtureExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/MaxAbsScalerExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/GBTExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/IsotonicRegressionExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/ModelSelectionViaCrossValidationExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/EstimatorTransformerParamExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/DecisionTreeRegressionExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/RandomForestRegressorExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/PrefixSpanExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/KMeansExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/ModelSelectionViaTrainValidationSplitExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/LogisticRegressionSummaryExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/MultilayerPerceptronClassifierExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/PCAExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/DCTExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/FPGrowthExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/ElementwiseProductExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/GeneralizedLinearRegressionExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/CountVectorizerExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/MinMaxScalerExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/ChiSquareTestExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/LogisticRegressionWithElasticNetExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/FMRegressorExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/VectorSlicerExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/LinearRegressionExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/StringIndexerExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/GradientBoostedTreeRegressorExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/ALSExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/DecisionTreeClassificationExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/PolynomialExpansionExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/FMClassifierExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/UnaryTransformerExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/ImputerExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/TokenizerExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/DecisionTreeExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/GradientBoostedTreeClassifierExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/PowerIterationClusteringExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/SQLTransformerExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/SummarizerExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/DataFrameExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/BucketizerExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/TfIdfExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/AFTSurvivalRegressionExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/QuantileDiscretizerExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/OneHotEncoderExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/PipelineExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/InteractionExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/LDAExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/RFormulaExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/VectorIndexerExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/VectorAssemblerExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/NormalizerExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/ChiSqSelectorExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/NaiveBayesExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/RandomForestClassifierExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/LinearRegressionWithElasticNetExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ml/RobustScalerExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/LogQuery.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/SparkRemoteFileTest.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/ExceptionHandlingTest.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/pythonconverters/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/pythonconverters/AvroConverters.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/BroadcastTest.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/SparkHdfsLR.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/Word2VecExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/GradientBoostedTreesRunner.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/RandomRDDGeneration.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/PMMLModelExportExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/BisectingKMeansExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/DecisionTreeRunner.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/MultiLabelMetricsExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/CorrelationsExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/LBFGSExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/TFIDFExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/StandardScalerExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/LogisticRegressionWithLBFGSExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/StreamingTestExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/BinaryClassification.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/DenseKMeans.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/GradientBoostingClassificationExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/RankingMetricsExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/GaussianMixtureExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/MultivariateSummarizer.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/IsotonicRegressionExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/PCAOnSourceVectorExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/Correlations.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/HypothesisTestingExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/DecisionTreeRegressionExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/AssociationRulesExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/StreamingLogisticRegression.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/SVDExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/PrefixSpanExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/KMeansExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/SparseNaiveBayes.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/AbstractParams.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/BinaryClassificationMetricsExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/SampledRDDs.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/FPGrowthExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/ElementwiseProductExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/RecommendationExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/TallSkinnySVD.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/HypothesisTestingKolmogorovSmirnovTestExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/GradientBoostingRegressionExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/StreamingKMeansExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/CosineSimilarity.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/RandomForestRegressionExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/StreamingLinearRegressionExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/StratifiedSamplingExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/DecisionTreeClassificationExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/TallSkinnyPCA.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/RandomForestClassificationExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/SimpleFPGrowth.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/LatentDirichletAllocationExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/KernelDensityEstimationExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/PowerIterationClusteringExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/SVMWithSGDExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/LDAExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/MovieLensALS.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/NormalizerExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/MulticlassMetricsExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/ChiSqSelectorExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/SummaryStatisticsExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/PCAOnRowMatrixExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/mllib/NaiveBayesExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/SparkTC.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/DriverSubmissionTest.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/MultiBroadcastTest.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/SparkLR.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/SparkPi.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/graphx/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/graphx/ConnectedComponentsExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/graphx/LiveJournalPageRank.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/graphx/AggregateMessagesExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/graphx/SSSPExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/graphx/SynthBenchmark.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/graphx/TriangleCountingExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/graphx/PageRankExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/graphx/ComprehensiveExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/graphx/Analytics.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/SparkKMeans.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/streaming/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/streaming/clickstream/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/streaming/clickstream/PageViewGenerator.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/streaming/clickstream/PageViewStream.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/streaming/DirectKafkaWordCount.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/streaming/HdfsWordCount.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/streaming/DirectKerberizedKafkaWordCount.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/streaming/RecoverableNetworkWordCount.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/streaming/NetworkWordCount.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/streaming/QueueStream.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/streaming/SqlNetworkWordCount.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/streaming/CustomReceiver.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/streaming/RawNetworkGrep.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/streaming/StatefulNetworkWordCount.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/streaming/StreamingExamples.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/LocalLR.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/LocalKMeans.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/GroupByTest.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/LocalALS.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/sql/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/sql/SimpleTypedAggregator.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/sql/UserDefinedUntypedAggregation.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/sql/hive/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/sql/hive/SparkHiveExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/sql/RDDRelation.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/sql/SparkSQLExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/sql/SQLDataSourceExample.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/sql/streaming/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/sql/streaming/StructuredKerberizedKafkaWordCount.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/sql/streaming/StructuredNetworkWordCount.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/sql/streaming/StructuredNetworkWordCountWindowed.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/sql/streaming/StructuredKafkaWordCount.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/sql/streaming/StructuredSessionization.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/sql/UserDefinedTypedAggregation.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/sql/UserDefinedScalar.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/SimpleSkewedGroupByTest.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/LocalFileLR.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/LocalPi.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/DFSReadWriteTest.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scala/org/apache/spark/examples/SparkALS.scala\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/resources/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/resources/people.json\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/resources/users.avro\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/resources/people.csv\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/resources/users.parquet\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/resources/users.orc\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/resources/dir1/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/resources/dir1/file1.parquet\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/resources/dir1/dir2/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/resources/dir1/dir2/file2.parquet\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/resources/dir1/file3.json\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/resources/user.avsc\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/resources/full_user.avsc\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/resources/kv1.txt\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/resources/people.txt\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/resources/employees.json\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaStringIndexerExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaChiSqSelectorExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaLabeledDocument.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaLogisticRegressionWithElasticNetExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaPolynomialExpansionExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaTfIdfExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaBucketizerExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaLogisticRegressionSummaryExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaSummarizerExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaQuantileDiscretizerExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaBinarizerExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaOneHotEncoderExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaStandardScalerExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaDCTExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaDecisionTreeRegressionExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaRandomForestClassifierExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaOneVsRestExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaGeneralizedLinearRegressionExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaGaussianMixtureExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaWord2VecExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaNormalizerExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaLDAExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaCountVectorizerExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaMultilayerPerceptronClassifierExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaMinMaxScalerExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaCorrelationExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaNGramExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaImputerExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaMinHashLSHExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaVectorSizeHintExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaFMClassifierExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaFeatureHasherExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaModelSelectionViaTrainValidationSplitExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaSQLTransformerExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaKMeansExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaVectorAssemblerExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaGradientBoostedTreeRegressorExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaPrefixSpanExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaFMRegressorExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaNaiveBayesExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaMulticlassLogisticRegressionWithElasticNetExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaModelSelectionViaCrossValidationExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaDocument.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaGradientBoostedTreeClassifierExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaRandomForestRegressorExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaAFTSurvivalRegressionExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaIsotonicRegressionExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaElementwiseProductExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaALSExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaBucketedRandomProjectionLSHExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaVectorSlicerExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaTokenizerExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaRFormulaExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaEstimatorTransformerParamExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaPowerIterationClusteringExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaBisectingKMeansExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaMaxAbsScalerExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaDecisionTreeClassificationExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaPCAExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaPipelineExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaStopWordsRemoverExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaIndexToStringExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaFPGrowthExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaVectorIndexerExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaInteractionExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaChiSquareTestExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaRobustScalerExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaLinearRegressionWithElasticNetExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/ml/JavaLinearSVCExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/JavaWordCount.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/JavaHdfsLR.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/JavaStatusTrackerDemo.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/mllib/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/mllib/JavaChiSqSelectorExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/mllib/JavaHypothesisTestingKolmogorovSmirnovTestExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/mllib/JavaRandomForestRegressionExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/mllib/JavaSimpleFPGrowth.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/mllib/JavaRandomForestClassificationExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/mllib/JavaMulticlassClassificationMetricsExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/mllib/JavaKernelDensityEstimationExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/mllib/JavaSummaryStatisticsExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/mllib/JavaDecisionTreeRegressionExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/mllib/JavaGradientBoostingClassificationExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/mllib/JavaGaussianMixtureExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/mllib/JavaStratifiedSamplingExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/mllib/JavaSVDExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/mllib/JavaAssociationRulesExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/mllib/JavaALS.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/mllib/JavaKMeansExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/mllib/JavaSVMWithSGDExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/mllib/JavaPrefixSpanExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/mllib/JavaNaiveBayesExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/mllib/JavaLogisticRegressionWithLBFGSExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/mllib/JavaLBFGSExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/mllib/JavaGradientBoostingRegressionExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/mllib/JavaRankingMetricsExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/mllib/JavaIsotonicRegressionExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/mllib/JavaElementwiseProductExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/mllib/JavaCorrelationsExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/mllib/JavaPowerIterationClusteringExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/mllib/JavaBisectingKMeansExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/mllib/JavaDecisionTreeClassificationExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/mllib/JavaPCAExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/mllib/JavaRecommendationExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/mllib/JavaLatentDirichletAllocationExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/mllib/JavaHypothesisTestingExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/mllib/JavaMultiLabelClassificationMetricsExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/mllib/JavaBinaryClassificationMetricsExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/mllib/JavaStreamingTestExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/JavaPageRank.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/JavaLogQuery.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/streaming/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/streaming/JavaStatefulNetworkWordCount.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/streaming/JavaCustomReceiver.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/streaming/JavaRecord.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/streaming/JavaNetworkWordCount.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/streaming/JavaDirectKerberizedKafkaWordCount.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/streaming/JavaDirectKafkaWordCount.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/streaming/JavaSqlNetworkWordCount.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/streaming/JavaRecoverableNetworkWordCount.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/streaming/JavaQueueStream.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/JavaSparkPi.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/sql/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/sql/JavaUserDefinedScalar.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/sql/hive/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/sql/hive/JavaSparkHiveExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/sql/JavaUserDefinedTypedAggregation.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/sql/JavaSQLDataSourceExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/sql/JavaSparkSQLExample.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/sql/JavaUserDefinedUntypedAggregation.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/sql/streaming/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/sql/streaming/JavaStructuredKafkaWordCount.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/sql/streaming/JavaStructuredNetworkWordCountWindowed.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/sql/streaming/JavaStructuredSessionization.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/sql/streaming/JavaStructuredNetworkWordCount.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/sql/streaming/JavaStructuredKerberizedKafkaWordCount.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/java/org/apache/spark/examples/JavaTC.java\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scripts/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/scripts/getGpusResources.sh\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/kmeans.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/dct_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/count_vectorizer_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/chisq_selector_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/gradient_boosted_tree_regressor_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/tf_idf_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/cross_validator.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/linear_regression_with_elastic_net.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/normalizer_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/fm_regressor_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/polynomial_expansion_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/pipeline_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/generalized_linear_regression_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/multilayer_perceptron_classification.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/bisecting_k_means_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/chi_square_test_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/train_validation_split.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/stopwords_remover_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/linearsvc.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/lda_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/random_forest_regressor_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/vector_assembler_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/word2vec_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/string_indexer_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/decision_tree_classification_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/index_to_string_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/bucketizer_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/vector_size_hint_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/vector_indexer_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/tokenizer_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/robust_scaler_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/standard_scaler_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/kmeans_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/power_iteration_clustering_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/naive_bayes_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/pca_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/aft_survival_regression.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/min_max_scaler_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/elementwise_product_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/dataframe_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/n_gram_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/rformula_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/one_vs_rest_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/als_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/onehot_encoder_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/vector_slicer_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/logistic_regression_with_elastic_net.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/sql_transformer.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/multiclass_logistic_regression_with_elastic_net.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/summarizer_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/quantile_discretizer_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/feature_hasher_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/imputer_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/bucketed_random_projection_lsh_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/estimator_transformer_param_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/gradient_boosted_tree_classifier_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/logistic_regression_summary_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/decision_tree_regression_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/fm_classifier_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/gaussian_mixture_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/fpgrowth_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/random_forest_classifier_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/correlation_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/min_hash_lsh_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/binarizer_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/prefixspan_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/max_abs_scaler_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/isotonic_regression_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/ml/interaction_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/logistic_regression.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/als.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/word2vec.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/correlations_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/gradient_boosting_classification_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/kernel_density_estimation_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/hypothesis_testing_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/svd_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/kmeans.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/latent_dirichlet_allocation_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/tf_idf_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/correlations.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/random_forest_regression_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/normalizer_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/logistic_regression.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/multi_label_metrics_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/gradient_boosting_regression_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/random_forest_classification_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/streaming_linear_regression_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/sampled_rdds.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/bisecting_k_means_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/logistic_regression_with_lbfgs_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/k_means_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/streaming_k_means_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/word2vec_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/multi_class_metrics_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/decision_tree_classification_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/summary_statistics_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/binary_classification_metrics_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/standard_scaler_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/power_iteration_clustering_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/gaussian_mixture_model.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/regression_metrics_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/naive_bayes_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/elementwise_product_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/linear_regression_with_sgd_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/pca_rowmatrix_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/hypothesis_testing_kolmogorov_smirnov_test_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/random_rdd_generation.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/decision_tree_regression_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/gaussian_mixture_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/recommendation_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/fpgrowth_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/ranking_metrics_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/svm_with_sgd_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/isotonic_regression_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/mllib/stratified_sampling_example.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/wordcount.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/status_api_demo.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/pagerank.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/sort.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/transitive_closure.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/avro_inputformat.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/pi.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/streaming/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/streaming/queue_stream.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/streaming/stateful_network_wordcount.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/streaming/network_wordjoinsentiments.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/streaming/sql_network_wordcount.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/streaming/network_wordcount.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/streaming/hdfs_wordcount.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/streaming/recoverable_network_wordcount.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/sql/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/sql/datasource.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/sql/hive.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/sql/arrow.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/sql/streaming/\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/sql/streaming/structured_network_wordcount_windowed.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/sql/streaming/structured_network_wordcount.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/sql/streaming/structured_kafka_wordcount.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/sql/basic.py\n","spark-3.0.3-bin-hadoop2.7/examples/src/main/python/parquet_inputformat.py\n","spark-3.0.3-bin-hadoop2.7/examples/jars/\n","spark-3.0.3-bin-hadoop2.7/examples/jars/spark-examples_2.12-3.0.3.jar\n","spark-3.0.3-bin-hadoop2.7/examples/jars/scopt_2.12-3.7.1.jar\n","spark-3.0.3-bin-hadoop2.7/conf/\n","spark-3.0.3-bin-hadoop2.7/conf/slaves.template\n","spark-3.0.3-bin-hadoop2.7/conf/metrics.properties.template\n","spark-3.0.3-bin-hadoop2.7/conf/fairscheduler.xml.template\n","spark-3.0.3-bin-hadoop2.7/conf/log4j.properties.template\n","spark-3.0.3-bin-hadoop2.7/conf/spark-defaults.conf.template\n","spark-3.0.3-bin-hadoop2.7/conf/spark-env.sh.template\n","spark-3.0.3-bin-hadoop2.7/bin/\n","spark-3.0.3-bin-hadoop2.7/bin/sparkR.cmd\n","spark-3.0.3-bin-hadoop2.7/bin/sparkR\n","spark-3.0.3-bin-hadoop2.7/bin/spark-submit\n","spark-3.0.3-bin-hadoop2.7/bin/pyspark2.cmd\n","spark-3.0.3-bin-hadoop2.7/bin/spark-class\n","spark-3.0.3-bin-hadoop2.7/bin/pyspark.cmd\n","spark-3.0.3-bin-hadoop2.7/bin/spark-submit2.cmd\n","spark-3.0.3-bin-hadoop2.7/bin/load-spark-env.cmd\n","spark-3.0.3-bin-hadoop2.7/bin/spark-sql\n","spark-3.0.3-bin-hadoop2.7/bin/docker-image-tool.sh\n","spark-3.0.3-bin-hadoop2.7/bin/find-spark-home.cmd\n","spark-3.0.3-bin-hadoop2.7/bin/load-spark-env.sh\n","spark-3.0.3-bin-hadoop2.7/bin/pyspark\n","spark-3.0.3-bin-hadoop2.7/bin/spark-shell.cmd\n","spark-3.0.3-bin-hadoop2.7/bin/spark-shell2.cmd\n","spark-3.0.3-bin-hadoop2.7/bin/spark-submit.cmd\n","spark-3.0.3-bin-hadoop2.7/bin/beeline.cmd\n","spark-3.0.3-bin-hadoop2.7/bin/find-spark-home\n","spark-3.0.3-bin-hadoop2.7/bin/spark-class.cmd\n","spark-3.0.3-bin-hadoop2.7/bin/sparkR2.cmd\n","spark-3.0.3-bin-hadoop2.7/bin/beeline\n","spark-3.0.3-bin-hadoop2.7/bin/spark-class2.cmd\n","spark-3.0.3-bin-hadoop2.7/bin/spark-sql.cmd\n","spark-3.0.3-bin-hadoop2.7/bin/run-example\n","spark-3.0.3-bin-hadoop2.7/bin/spark-shell\n","spark-3.0.3-bin-hadoop2.7/bin/run-example.cmd\n","spark-3.0.3-bin-hadoop2.7/bin/spark-sql2.cmd\n","spark-3.0.3-bin-hadoop2.7/python/\n","spark-3.0.3-bin-hadoop2.7/python/.gitignore\n","spark-3.0.3-bin-hadoop2.7/python/run-tests-with-coverage\n","spark-3.0.3-bin-hadoop2.7/python/pylintrc\n","spark-3.0.3-bin-hadoop2.7/python/MANIFEST.in\n","spark-3.0.3-bin-hadoop2.7/python/README.md\n","spark-3.0.3-bin-hadoop2.7/python/test_coverage/\n","spark-3.0.3-bin-hadoop2.7/python/test_coverage/coverage_daemon.py\n","spark-3.0.3-bin-hadoop2.7/python/test_coverage/conf/\n","spark-3.0.3-bin-hadoop2.7/python/test_coverage/conf/spark-defaults.conf\n","spark-3.0.3-bin-hadoop2.7/python/test_coverage/sitecustomize.py\n","spark-3.0.3-bin-hadoop2.7/python/run-tests.py\n","spark-3.0.3-bin-hadoop2.7/python/setup.py\n","spark-3.0.3-bin-hadoop2.7/python/test_support/\n","spark-3.0.3-bin-hadoop2.7/python/test_support/userlibrary.py\n","spark-3.0.3-bin-hadoop2.7/python/test_support/hello/\n","spark-3.0.3-bin-hadoop2.7/python/test_support/hello/sub_hello/\n","spark-3.0.3-bin-hadoop2.7/python/test_support/hello/sub_hello/sub_hello.txt\n","spark-3.0.3-bin-hadoop2.7/python/test_support/hello/hello.txt\n","spark-3.0.3-bin-hadoop2.7/python/test_support/userlib-0.1.zip\n","spark-3.0.3-bin-hadoop2.7/python/test_support/SimpleHTTPServer.py\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/people.json\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/people_array.json\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/people_array_utf16le.json\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/text-test.txt\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/ages.csv\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/parquet_partitioned/\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/parquet_partitioned/year=2015/\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/parquet_partitioned/year=2015/month=10/\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/parquet_partitioned/year=2015/month=10/day=26/\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/parquet_partitioned/year=2015/month=10/day=26/.part-r-00005.gz.parquet.crc\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/parquet_partitioned/year=2015/month=10/day=26/part-r-00005.gz.parquet\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/parquet_partitioned/year=2015/month=10/day=25/\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/parquet_partitioned/year=2015/month=10/day=25/.part-r-00002.gz.parquet.crc\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/parquet_partitioned/year=2015/month=10/day=25/.part-r-00004.gz.parquet.crc\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/parquet_partitioned/year=2015/month=10/day=25/part-r-00004.gz.parquet\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/parquet_partitioned/year=2015/month=10/day=25/part-r-00002.gz.parquet\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/parquet_partitioned/year=2015/month=9/\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/parquet_partitioned/year=2015/month=9/day=1/\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/parquet_partitioned/year=2015/month=9/day=1/.part-r-00007.gz.parquet.crc\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/parquet_partitioned/year=2015/month=9/day=1/part-r-00007.gz.parquet\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/parquet_partitioned/_metadata\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/parquet_partitioned/year=2014/\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/parquet_partitioned/year=2014/month=9/\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/parquet_partitioned/year=2014/month=9/day=1/\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/parquet_partitioned/year=2014/month=9/day=1/.part-r-00008.gz.parquet.crc\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/parquet_partitioned/year=2014/month=9/day=1/part-r-00008.gz.parquet\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/parquet_partitioned/_common_metadata\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/parquet_partitioned/_SUCCESS\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/orc_partitioned/\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/orc_partitioned/b=1/\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/orc_partitioned/b=1/c=1/\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/orc_partitioned/b=1/c=1/part-r-00000-829af031-b970-49d6-ad39-30460a0be2c8.orc\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/orc_partitioned/b=1/c=1/.part-r-00000-829af031-b970-49d6-ad39-30460a0be2c8.orc.crc\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/orc_partitioned/b=0/\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/orc_partitioned/b=0/c=0/\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/orc_partitioned/b=0/c=0/part-r-00000-829af031-b970-49d6-ad39-30460a0be2c8.orc\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/orc_partitioned/b=0/c=0/.part-r-00000-829af031-b970-49d6-ad39-30460a0be2c8.orc.crc\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/orc_partitioned/_SUCCESS\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/ages_newlines.csv\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/streaming/\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/streaming/text-test.txt\n","spark-3.0.3-bin-hadoop2.7/python/test_support/sql/people1.json\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/tests/\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/tests/test_rddbarrier.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/tests/test_worker.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/tests/test_serializers.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/tests/test_util.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/tests/test_rdd.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/tests/__init__.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/tests/test_broadcast.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/tests/test_appsubmit.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/tests/test_profiler.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/tests/test_pin_thread.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/tests/test_shuffle.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/tests/test_join.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/tests/test_taskcontext.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/tests/test_context.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/tests/test_readwrite.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/tests/test_conf.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/tests/test_daemon.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/testing/\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/testing/mlutils.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/testing/__init__.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/testing/mllibutils.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/testing/utils.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/testing/sqlutils.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/testing/streamingutils.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/accumulators.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/rddsampler.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/tests/\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/tests/test_algorithms.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/tests/test_evaluation.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/tests/test_util.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/tests/test_feature.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/tests/test_pipeline.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/tests/test_wrapper.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/tests/__init__.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/tests/test_tuning.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/tests/test_persistence.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/tests/test_param.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/tests/test_training_summary.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/tests/test_linalg.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/tests/test_image.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/tests/test_stat.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/tests/test_base.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/functions.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/tuning.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/pipeline.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/base.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/feature.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/__init__.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/stat.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/image.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/classification.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/recommendation.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/regression.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/param/\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/param/_shared_params_code_gen.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/param/__init__.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/param/shared.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/tree.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/fpm.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/wrapper.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/clustering.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/common.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/linalg/\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/linalg/__init__.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/evaluation.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/ml/util.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/find_spark_home.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/heapq3.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/serializers.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/java_gateway.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/traceback_utils.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/conf.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/__init__.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/mllib/\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/mllib/tests/\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/mllib/tests/test_algorithms.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/mllib/tests/test_streaming_algorithms.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/mllib/tests/test_util.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/mllib/tests/test_feature.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/mllib/tests/__init__.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/mllib/tests/test_linalg.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/mllib/tests/test_stat.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/mllib/feature.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/mllib/__init__.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/mllib/classification.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/mllib/recommendation.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/mllib/regression.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/mllib/tree.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/mllib/fpm.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/mllib/random.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/mllib/stat/\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/mllib/stat/distribution.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/mllib/stat/__init__.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/mllib/stat/KernelDensity.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/mllib/stat/test.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/mllib/stat/_statistics.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/mllib/clustering.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/mllib/common.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/mllib/linalg/\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/mllib/linalg/__init__.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/mllib/linalg/distributed.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/mllib/evaluation.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/mllib/util.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/resultiterable.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/profiler.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/statcounter.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/join.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/daemon.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/rdd.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/context.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/cloudpickle.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/version.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/resource.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/files.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/worker.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/shell.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/streaming/\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/streaming/tests/\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/streaming/tests/test_listener.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/streaming/tests/test_kinesis.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/streaming/tests/__init__.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/streaming/tests/test_dstream.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/streaming/tests/test_context.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/streaming/dstream.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/streaming/__init__.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/streaming/kinesis.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/streaming/listener.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/streaming/context.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/streaming/util.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/status.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/tests/\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/tests/test_functions.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/tests/test_readwriter.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/tests/test_utils.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/tests/test_pandas_grouped_map.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/tests/test_dataframe.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/tests/test_pandas_map.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/tests/test_udf.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/tests/test_streaming.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/tests/__init__.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/tests/test_serde.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/tests/test_pandas_udf_window.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/tests/test_group.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/tests/test_pandas_udf.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/tests/test_pandas_cogrouped_map.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/tests/test_pandas_udf_grouped_agg.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/tests/test_pandas_udf_scalar.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/tests/test_catalog.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/tests/test_datasources.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/tests/test_pandas_udf_typehints.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/tests/test_types.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/tests/test_column.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/tests/test_context.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/tests/test_conf.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/tests/test_arrow.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/tests/test_session.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/pandas/\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/pandas/functions.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/pandas/serializers.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/pandas/__init__.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/pandas/typehints.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/pandas/map_ops.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/pandas/types.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/pandas/group_ops.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/pandas/utils.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/pandas/conversion.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/functions.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/readwriter.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/catalog.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/window.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/udf.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/conf.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/__init__.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/session.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/column.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/group.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/context.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/types.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/dataframe.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/avro/\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/avro/functions.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/avro/__init__.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/utils.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/sql/streaming.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/shuffle.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/taskcontext.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/_globals.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/broadcast.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/util.py\n","spark-3.0.3-bin-hadoop2.7/python/pyspark/storagelevel.py\n","spark-3.0.3-bin-hadoop2.7/python/.coveragerc\n","spark-3.0.3-bin-hadoop2.7/python/docs/\n","spark-3.0.3-bin-hadoop2.7/python/docs/index.rst\n","spark-3.0.3-bin-hadoop2.7/python/docs/conf.py\n","spark-3.0.3-bin-hadoop2.7/python/docs/pyspark.ml.rst\n","spark-3.0.3-bin-hadoop2.7/python/docs/pyspark.rst\n","spark-3.0.3-bin-hadoop2.7/python/docs/_templates/\n","spark-3.0.3-bin-hadoop2.7/python/docs/_templates/layout.html\n","spark-3.0.3-bin-hadoop2.7/python/docs/_static/\n","spark-3.0.3-bin-hadoop2.7/python/docs/_static/pyspark.css\n","spark-3.0.3-bin-hadoop2.7/python/docs/_static/copybutton.js\n","spark-3.0.3-bin-hadoop2.7/python/docs/_static/pyspark.js\n","spark-3.0.3-bin-hadoop2.7/python/docs/pyspark.mllib.rst\n","spark-3.0.3-bin-hadoop2.7/python/docs/make2.bat\n","spark-3.0.3-bin-hadoop2.7/python/docs/pyspark.streaming.rst\n","spark-3.0.3-bin-hadoop2.7/python/docs/pyspark.sql.rst\n","spark-3.0.3-bin-hadoop2.7/python/docs/make.bat\n","spark-3.0.3-bin-hadoop2.7/python/docs/Makefile\n","spark-3.0.3-bin-hadoop2.7/python/docs/pyspark.resource.rst\n","spark-3.0.3-bin-hadoop2.7/python/lib/\n","spark-3.0.3-bin-hadoop2.7/python/lib/PY4J_LICENSE.txt\n","spark-3.0.3-bin-hadoop2.7/python/lib/py4j-0.10.9-src.zip\n","spark-3.0.3-bin-hadoop2.7/python/lib/pyspark.zip\n","spark-3.0.3-bin-hadoop2.7/python/run-tests\n","spark-3.0.3-bin-hadoop2.7/python/setup.cfg\n","spark-3.0.3-bin-hadoop2.7/licenses/\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-respond.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-sbt-launch-lib.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-antlr.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-dagre-d3.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-pyrolite.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-sorttable.js.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-janino.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-protobuf.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-jquery.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-scopt.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-netlib.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-d3.min.js.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-graphlib-dot.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-AnchorJS.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-datatables.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-pmml-model.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-paranamer.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-jakarta-ws-rs-api\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-dnsjava.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-jakarta.xml.bind-api.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-jakarta-annotation-api\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-CC0.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-jodd.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-f2j.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-heapq.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-machinist.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-javolution.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-modernizr.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-spire.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-leveldbjni.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-join.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-zstd-jni.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-slf4j.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-arpack.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-jsp-api.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-JTransforms.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-JLargeArrays.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-bootstrap.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-reflectasm.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-javassist.html\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-zstd.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-json-formatter.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-matchMedia-polyfill.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-scala.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-jakarta.activation-api.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-automaton.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-javax-transaction-transaction-api.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-jaxb-runtime.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-minlog.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-mustache.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-xmlenc.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-jline.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-istack-commons-runtime.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-py4j.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-vis-timeline.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-re2j.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-kryo.txt\n","spark-3.0.3-bin-hadoop2.7/licenses/LICENSE-cloudpickle.txt\n"]}],"source":["# Download Apache Spark and extract\n","!wget https://archive.apache.org/dist/spark/spark-3.0.3/spark-3.0.3-bin-hadoop2.7.tgz\n","!tar xvf spark-3.0.3-bin-hadoop2.7.tgz"]},{"cell_type":"code","execution_count":4,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":55797,"status":"ok","timestamp":1648899703934,"user":{"displayName":"Abdulwahab Abdulliameed","userId":"10732377996348789858"},"user_tz":-60},"id":"COCOvti5KySL","outputId":"b630ebc5-bb2e-48b7-a3a0-4049603ff448"},"outputs":[{"output_type":"stream","name":"stdout","text":["Collecting pyspark\n","  Downloading pyspark-3.2.1.tar.gz (281.4 MB)\n","\u001b[K     |████████████████████████████████| 281.4 MB 34 kB/s \n","\u001b[?25hCollecting py4j==0.10.9.3\n","  Downloading py4j-0.10.9.3-py2.py3-none-any.whl (198 kB)\n","\u001b[K     |████████████████████████████████| 198 kB 46.8 MB/s \n","\u001b[?25hBuilding wheels for collected packages: pyspark\n","  Building wheel for pyspark (setup.py) ... \u001b[?25l\u001b[?25hdone\n","  Created wheel for pyspark: filename=pyspark-3.2.1-py2.py3-none-any.whl size=281853642 sha256=44ea0932ee215b15cab624cadeb23dbe505e0457be3a6636ce498d940962aa53\n","  Stored in directory: /root/.cache/pip/wheels/9f/f5/07/7cd8017084dce4e93e84e92efd1e1d5334db05f2e83bcef74f\n","Successfully built pyspark\n","Installing collected packages: py4j, pyspark\n","Successfully installed py4j-0.10.9.3 pyspark-3.2.1\n"]}],"source":["#Install pyspark\n","!pip install -q findspark\n","!pip install pyspark"]},{"cell_type":"code","execution_count":5,"metadata":{"id":"AnE6ejahLDYa","executionInfo":{"status":"ok","timestamp":1648899710070,"user_tz":-60,"elapsed":4,"user":{"displayName":"Abdulwahab Abdulliameed","userId":"10732377996348789858"}}},"outputs":[],"source":["#setting the environment path \n","import os\n","os.environ[\"JAVA_HOME\"] = \"/usr/lib/jvm/java-11-openjdk-amd64\"\n","os.environ[\"SPARK_HOME\"] = \"/content/spark-3.0.3-bin-hadoop2.7\""]},{"cell_type":"code","execution_count":6,"metadata":{"id":"QzRaOphTLKrL","executionInfo":{"status":"ok","timestamp":1648899733502,"user_tz":-60,"elapsed":7267,"user":{"displayName":"Abdulwahab Abdulliameed","userId":"10732377996348789858"}}},"outputs":[],"source":["#Running a local SparkSession\n","import findspark\n","findspark.init()\n","from pyspark.sql import SparkSession\n","spark = SparkSession.builder.master(\"local[*]\").getOrCreate()"]},{"cell_type":"code","execution_count":7,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":1133,"status":"ok","timestamp":1648899739953,"user":{"displayName":"Abdulwahab Abdulliameed","userId":"10732377996348789858"},"user_tz":-60},"id":"V6UKIfaBwy1R","outputId":"b16c8b94-3c03-4b4c-f17c-5382a3955fbd"},"outputs":[{"output_type":"stream","name":"stdout","text":["3.0.3\n"]}],"source":["# Check the pyspark version\n","import pyspark\n","print(pyspark.__version__)"]},{"cell_type":"code","execution_count":8,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":220},"executionInfo":{"elapsed":18,"status":"ok","timestamp":1648899743558,"user":{"displayName":"Abdulwahab Abdulliameed","userId":"10732377996348789858"},"user_tz":-60},"id":"KB-iQ0l0fMkU","outputId":"a5a8585c-40e5-4cda-c9a0-6bc927a57b99"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["<pyspark.sql.session.SparkSession at 0x7fbdcd4b0e10>"],"text/html":["\n","            <div>\n","                <p><b>SparkSession - in-memory</b></p>\n","                \n","        <div>\n","            <p><b>SparkContext</b></p>\n","\n","            <p><a href=\"http://d5508e2694da:4040\">Spark UI</a></p>\n","\n","            <dl>\n","              <dt>Version</dt>\n","                <dd><code>v3.0.3</code></dd>\n","              <dt>Master</dt>\n","                <dd><code>local[*]</code></dd>\n","              <dt>AppName</dt>\n","                <dd><code>pyspark-shell</code></dd>\n","            </dl>\n","        </div>\n","        \n","            </div>\n","        "]},"metadata":{},"execution_count":8}],"source":["#Pyspark shell\n","spark"]},{"cell_type":"code","execution_count":9,"metadata":{"id":"TqHTb6K-Dc29","executionInfo":{"status":"ok","timestamp":1648899748562,"user_tz":-60,"elapsed":423,"user":{"displayName":"Abdulwahab Abdulliameed","userId":"10732377996348789858"}}},"outputs":[],"source":["#Running a local SparkSession\n","import findspark\n","findspark.init()\n","from pyspark.sql import SparkSession\n","spark = SparkSession.builder\\\n",".master(\"local\")\\\n",".appName(\"Colab\")\\\n",".config('spark.ui.port', '4050')\\\n",".getOrCreate()"]},{"cell_type":"code","execution_count":10,"metadata":{"id":"7wVS5hGWUmdN","executionInfo":{"status":"ok","timestamp":1648899760263,"user_tz":-60,"elapsed":497,"user":{"displayName":"Abdulwahab Abdulliameed","userId":"10732377996348789858"}}},"outputs":[],"source":["from pyspark.sql import SparkSession\n","# Numerical analysis package\n","import numpy as np\n","# Package to work with data\n","import pandas as pd\n","# Package for Matplotlib Pyplot\n","# Pyplot is a collection of functions in the popular visualization package Matplotlib.\n","import matplotlib as mpl\n","import matplotlib.pyplot as plt\n","# Package for Seaborn\n","# Seaborn is a Python data visualization library built on top of Matplotlib.\n","import seaborn as sns\n","from pyspark.ml.linalg import DenseMatrix, Vectors\n","from pyspark.ml.stat import Correlation\n","import json\n","import pyspark.sql.functions as F\n","import pyspark.sql\n","from pyspark.sql.functions import col, skewness, kurtosis\n","from pyspark.context import SparkContext\n","from pyspark.sql.functions import * \n","from pyspark.sql.functions import isnan, when, count, col\n","from pyspark.sql.functions import when\n","from pyspark.sql.functions import UserDefinedFunction\n","from pyspark.sql.functions import from_unixtime, unix_timestamp\n","from pyspark.sql.types import StringType\n","from datetime import date, timedelta, datetime\n","import time"]},{"cell_type":"markdown","metadata":{"id":"OVr_jzwa_j_q"},"source":["**Connect/Mount Google Drive to Colab**"]},{"cell_type":"code","execution_count":11,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":40705,"status":"ok","timestamp":1648899805557,"user":{"displayName":"Abdulwahab Abdulliameed","userId":"10732377996348789858"},"user_tz":-60},"id":"UqwaPjGCQWCS","outputId":"1bafb580-5181-4c69-d2ed-d6f01681f899"},"outputs":[{"output_type":"stream","name":"stdout","text":["Mounted at /content/drive\n"]}],"source":["#Connect/Mount Google Drive to Colab\n","from google.colab import drive\n","drive.mount('/content/drive')"]},{"cell_type":"markdown","metadata":{"id":"dc46wz78ADHt"},"source":["**Define the UNSW-NB15 Dataframe**"]},{"cell_type":"code","execution_count":12,"metadata":{"id":"koboL_Ic-Wya","executionInfo":{"status":"ok","timestamp":1648899847686,"user_tz":-60,"elapsed":34029,"user":{"displayName":"Abdulwahab Abdulliameed","userId":"10732377996348789858"}}},"outputs":[],"source":["#Define the UNSW-NB15 Dataframe\n","UNSWNB15 = spark.read.csv('/content/drive/MyDrive/UEL-UNICAF/Module 3: Big Data Analytics/Assessment/UNSW-NB15.csv', inferSchema=True )"]},{"cell_type":"code","execution_count":13,"metadata":{"id":"iBI8fkBGLpLk","executionInfo":{"status":"ok","timestamp":1648899859096,"user_tz":-60,"elapsed":3216,"user":{"displayName":"Abdulwahab Abdulliameed","userId":"10732377996348789858"}}},"outputs":[],"source":["#Rename the Columns Header\n","UNSWNB15 = UNSWNB15.withColumnRenamed('_c0','srcip') \\\n",".withColumnRenamed('_c1','sport') \\\n",".withColumnRenamed('_c2','dstip') \\\n",".withColumnRenamed('_c3','dsport') \\\n",".withColumnRenamed('_c4','proto') \\\n",".withColumnRenamed('_c5','state') \\\n",".withColumnRenamed('_c6','dur') \\\n",".withColumnRenamed('_c7','sbytes') \\\n",".withColumnRenamed('_c8','dbytes') \\\n",".withColumnRenamed('_c9','sttl') \\\n",".withColumnRenamed('_c10','dttl') \\\n",".withColumnRenamed('_c11','sloss') \\\n",".withColumnRenamed('_c12','dloss') \\\n",".withColumnRenamed('_c13','service') \\\n",".withColumnRenamed('_c14','Sload') \\\n",".withColumnRenamed('_c15','Dload') \\\n",".withColumnRenamed('_c16','Spkts') \\\n",".withColumnRenamed('_c17','Dpkts') \\\n",".withColumnRenamed('_c18','swin') \\\n",".withColumnRenamed('_c19','dwin') \\\n",".withColumnRenamed('_c20','stcpb') \\\n",".withColumnRenamed('_c21','dtcpb') \\\n",".withColumnRenamed('_c22','smeansz') \\\n",".withColumnRenamed('_c23','dmeansz') \\\n",".withColumnRenamed('_c24','trans_depth') \\\n",".withColumnRenamed('_c25','res_bdy_len') \\\n",".withColumnRenamed('_c26','Sjit') \\\n",".withColumnRenamed('_c27','Djit') \\\n",".withColumnRenamed('_c28','Stime') \\\n",".withColumnRenamed('_c29','Ltime') \\\n",".withColumnRenamed('_c30','Spkt') \\\n",".withColumnRenamed('_c31','Dpkt') \\\n",".withColumnRenamed('_c32','tcprtt') \\\n",".withColumnRenamed('_c33','synack') \\\n",".withColumnRenamed('_c34','ackdat') \\\n",".withColumnRenamed('_c35','is_sm_ips_ports') \\\n",".withColumnRenamed('_c36','ct_state_ttl') \\\n",".withColumnRenamed('_c37','ct_flw_http_mthd') \\\n",".withColumnRenamed('_c38','is_ftp_login') \\\n",".withColumnRenamed('_c39','ct_ftp_cmd') \\\n",".withColumnRenamed('_c40','ct_srv_src') \\\n",".withColumnRenamed('_c41','ct_srv_dst') \\\n",".withColumnRenamed('_c42','ct_dst_ltm') \\\n",".withColumnRenamed('_c43','ct_src_ ltm') \\\n",".withColumnRenamed('_c44','ct_src_dport_ltm') \\\n",".withColumnRenamed('_c45','ct_dst_sport_ltm') \\\n",".withColumnRenamed('_c46','ct_dst_src_ltm') \\\n",".withColumnRenamed('_c47','attack_cat') \\\n",".withColumnRenamed('_c48','Label') \\"]},{"cell_type":"markdown","metadata":{"id":"PRHrtpRhAM1U"},"source":["**Working with PySpark**"]},{"cell_type":"code","execution_count":14,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":501,"status":"ok","timestamp":1648899865708,"user":{"displayName":"Abdulwahab Abdulliameed","userId":"10732377996348789858"},"user_tz":-60},"id":"4asIc4ys2jR-","outputId":"a172c78a-a4d7-4429-bf0c-5841ac7cb811"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["['srcip',\n"," 'sport',\n"," 'dstip',\n"," 'dsport',\n"," 'proto',\n"," 'state',\n"," 'dur',\n"," 'sbytes',\n"," 'dbytes',\n"," 'sttl',\n"," 'dttl',\n"," 'sloss',\n"," 'dloss',\n"," 'service',\n"," 'Sload',\n"," 'Dload',\n"," 'Spkts',\n"," 'Dpkts',\n"," 'swin',\n"," 'dwin',\n"," 'stcpb',\n"," 'dtcpb',\n"," 'smeansz',\n"," 'dmeansz',\n"," 'trans_depth',\n"," 'res_bdy_len',\n"," 'Sjit',\n"," 'Djit',\n"," 'Stime',\n"," 'Ltime',\n"," 'Spkt',\n"," 'Dpkt',\n"," 'tcprtt',\n"," 'synack',\n"," 'ackdat',\n"," 'is_sm_ips_ports',\n"," 'ct_state_ttl',\n"," 'ct_flw_http_mthd',\n"," 'is_ftp_login',\n"," 'ct_ftp_cmd',\n"," 'ct_srv_src',\n"," 'ct_srv_dst',\n"," 'ct_dst_ltm',\n"," 'ct_src_ ltm',\n"," 'ct_src_dport_ltm',\n"," 'ct_dst_sport_ltm',\n"," 'ct_dst_src_ltm',\n"," 'attack_cat',\n"," 'Label']"]},"metadata":{},"execution_count":14}],"source":["#Show Columns\n","UNSWNB15.columns"]},{"cell_type":"code","execution_count":15,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":320,"status":"ok","timestamp":1648899873446,"user":{"displayName":"Abdulwahab Abdulliameed","userId":"10732377996348789858"},"user_tz":-60},"id":"hXqmEGunDz_q","outputId":"5dc341b7-2bfd-4f67-8536-a2b1fc3876d8"},"outputs":[{"output_type":"stream","name":"stdout","text":["root\n"," |-- srcip: string (nullable = true)\n"," |-- sport: integer (nullable = true)\n"," |-- dstip: string (nullable = true)\n"," |-- dsport: integer (nullable = true)\n"," |-- proto: string (nullable = true)\n"," |-- state: string (nullable = true)\n"," |-- dur: double (nullable = true)\n"," |-- sbytes: integer (nullable = true)\n"," |-- dbytes: integer (nullable = true)\n"," |-- sttl: integer (nullable = true)\n"," |-- dttl: integer (nullable = true)\n"," |-- sloss: integer (nullable = true)\n"," |-- dloss: integer (nullable = true)\n"," |-- service: string (nullable = true)\n"," |-- Sload: double (nullable = true)\n"," |-- Dload: double (nullable = true)\n"," |-- Spkts: integer (nullable = true)\n"," |-- Dpkts: integer (nullable = true)\n"," |-- swin: integer (nullable = true)\n"," |-- dwin: integer (nullable = true)\n"," |-- stcpb: long (nullable = true)\n"," |-- dtcpb: long (nullable = true)\n"," |-- smeansz: integer (nullable = true)\n"," |-- dmeansz: integer (nullable = true)\n"," |-- trans_depth: integer (nullable = true)\n"," |-- res_bdy_len: integer (nullable = true)\n"," |-- Sjit: double (nullable = true)\n"," |-- Djit: double (nullable = true)\n"," |-- Stime: integer (nullable = true)\n"," |-- Ltime: integer (nullable = true)\n"," |-- Spkt: double (nullable = true)\n"," |-- Dpkt: double (nullable = true)\n"," |-- tcprtt: double (nullable = true)\n"," |-- synack: double (nullable = true)\n"," |-- ackdat: double (nullable = true)\n"," |-- is_sm_ips_ports: integer (nullable = true)\n"," |-- ct_state_ttl: integer (nullable = true)\n"," |-- ct_flw_http_mthd: integer (nullable = true)\n"," |-- is_ftp_login: integer (nullable = true)\n"," |-- ct_ftp_cmd: integer (nullable = true)\n"," |-- ct_srv_src: integer (nullable = true)\n"," |-- ct_srv_dst: integer (nullable = true)\n"," |-- ct_dst_ltm: integer (nullable = true)\n"," |-- ct_src_ ltm: integer (nullable = true)\n"," |-- ct_src_dport_ltm: integer (nullable = true)\n"," |-- ct_dst_sport_ltm: integer (nullable = true)\n"," |-- ct_dst_src_ltm: integer (nullable = true)\n"," |-- attack_cat: string (nullable = true)\n"," |-- Label: integer (nullable = true)\n","\n"]}],"source":["#Display Schema\n","UNSWNB15.printSchema()"]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":24,"status":"ok","timestamp":1648626806561,"user":{"displayName":"Abdulwahab Abdulliameed","userId":"10732377996348789858"},"user_tz":-60},"id":"oLUjbHSDQBBh","outputId":"fc4dc236-360f-4427-8bf4-3107c7d65286"},"outputs":[{"data":{"text/plain":["2539739"]},"execution_count":17,"metadata":{},"output_type":"execute_result"}],"source":["#Count all records/rows\n","UNSWNB15.count()"]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":22,"status":"ok","timestamp":1648626806562,"user":{"displayName":"Abdulwahab Abdulliameed","userId":"10732377996348789858"},"user_tz":-60},"id":"FzsO-MGXzOx4","outputId":"066546e9-f7a1-4521-e14e-5be74c9e1dbe"},"outputs":[{"name":"stdout","output_type":"stream","text":["+----------+-----+-------------+------+-----+-----+------------+------+------+----+----+-----+-----+--------+---------+---------+-----+-----+----+----+----------+----------+-------+-------+-----------+-----------+----------+---------+----------+----------+------------+------------+------------+------------+------------+---------------+------------+----------------+------------+----------+----------+----------+----------+-----------+----------------+----------------+--------------+----------+-----+\n","|     srcip|sport|        dstip|dsport|proto|state|         dur|sbytes|dbytes|sttl|dttl|sloss|dloss| service|    Sload|    Dload|Spkts|Dpkts|swin|dwin|     stcpb|     dtcpb|smeansz|dmeansz|trans_depth|res_bdy_len|      Sjit|     Djit|     Stime|     Ltime|        Spkt|        Dpkt|      tcprtt|      synack|      ackdat|is_sm_ips_ports|ct_state_ttl|ct_flw_http_mthd|is_ftp_login|ct_ftp_cmd|ct_srv_src|ct_srv_dst|ct_dst_ltm|ct_src_ ltm|ct_src_dport_ltm|ct_dst_sport_ltm|ct_dst_src_ltm|attack_cat|Label|\n","+----------+-----+-------------+------+-----+-----+------------+------+------+----+----+-----+-----+--------+---------+---------+-----+-----+----+----+----------+----------+-------+-------+-----------+-----------+----------+---------+----------+----------+------------+------------+------------+------------+------------+---------------+------------+----------------+------------+----------+----------+----------+----------+-----------+----------------+----------------+--------------+----------+-----+\n","|59.166.0.3|56716|149.171.126.8|   143|  tcp|  FIN|  0.82546002|  7812| 16236|  31|  29|   30|   32|       -| 75090.25|156111.73|  122|  126| 255| 255|2751097753|2748686736|     64|    129|          0|          0| 445.25928| 474.9451|1421970774|1421970775|   6.8190908|    6.599896|5.9700001E-4|4.6899999E-4|     1.28E-4|              0|           0|               0|           0|         0|         2|         7|         1|          4|               1|               1|             1|      null|    0|\n","|59.166.0.0|43467|149.171.126.6| 49729|  tcp|  FIN|    0.101815|  4238| 65628|  31|  29|    7|   30|       -|328438.84|5087030.5|   72|   74| 255| 255| 961515433|3225510659|     59|    887|          0|          0|       0.0|91.579567|1421970775|1421970775|    1.429493|    1.387192|      6.8E-4|5.4600002E-4|     1.34E-4|              0|           0|               0|           0|         0|         7|         4|         1|          6|               1|               1|             1|      null|    0|\n","|59.166.0.5|41289|149.171.126.2|  9574|  tcp|  FIN| 0.044002999|  2750| 29104|  31|  29|    7|   17|       -|488693.97|5181101.5|   44|   48| 255| 255|3291096757|1191410228|     63|    606|          0|          0| 78.126968|62.206562|1421970775|1421970775|    1.014977|  0.92583001|     0.00125|     4.85E-4|     7.65E-4|              0|           0|               0|           0|         0|         3|         5|         3|          3|               1|               1|             1|      null|    0|\n","|59.166.0.9|43785|149.171.126.0|  6881|  tcp|  FIN|   2.7908299| 10476|395734|  31|  29|   16|  143|       -|29863.518|1130840.8|  180|  320| 255| 255|3934392726|3961690324|     58|   1237|          0|          0| 2707.4927| 2018.976|1421970772|1421970775|   15.589459|   8.7470121|6.8400003E-4|5.3199998E-4|1.5199999E-4|              0|           0|               0|           0|         0|        11|         4|         3|          2|               1|               1|             1|      null|    0|\n","|59.166.0.8|40691|149.171.126.9|  6881|  tcp|  FIN|   2.6335001| 13350|548216|  31|  29|   21|  197|       -|40381.238|1661560.6|  232|  438| 255| 255|   1518931|  18267719|     58|   1252|          0|          0| 718.33679|500.57288|1421970773|1421970775|   11.399026|   6.0251832|     6.19E-4|     4.89E-4|      1.3E-4|              0|           0|               0|           0|         0|        16|         7|         7|          1|               1|               1|             1|      null|    0|\n","|59.166.0.3|20393|149.171.126.3|  5190|  tcp|  FIN|    0.115048|  1958|  2308|  31|  29|    6|    6|       -|129963.15|153814.06|   22|   24| 255| 255|3646899201|3651364285|     89|     96|          0|          0| 435.26627|417.08563|1421970775|1421970775|    5.460381|    4.976913|7.0999999E-4|     5.73E-4|     1.37E-4|              0|           0|               0|           0|         0|         2|         6|         1|          4|               1|               1|             1|      null|    0|\n","|59.166.0.7|19792|149.171.126.0|    53|  udp|  CON|    0.003362|   146|   178|  31|  29|    0|    0|     dns|173706.13| 211778.7|    2|    2|   0|   0|         0|         0|     73|     89|          0|          0|       0.0|      0.0|1421970775|1421970775|       0.011|0.0060000001|         0.0|         0.0|         0.0|              0|           0|               0|           0|         0|         3|         2|         3|          3|               3|               1|             1|      null|    0|\n","|59.166.0.3|14382|149.171.126.9|  3354|  tcp|  FIN|  0.45305201|   424|  8824|  31|  29|    1|    4|ftp-data| 6551.124| 142835.7|    8|   12| 255| 255|2206905053|3307670308|     53|    735|          0|          0| 3906.7949|3074.6694|1421970775|1421970775|   64.671288|   41.134998|      6.8E-4|5.5900001E-4|     1.21E-4|              0|           0|               0|           0|         0|         4|         6|         7|          4|               1|               1|             2|      null|    0|\n","|59.166.0.9|37074|149.171.126.2|    53|  udp|  CON|    0.001088|   146|   178|  31|  29|    0|    0|     dns|536764.69|654411.75|    2|    2|   0|   0|         0|         0|     73|     89|          0|          0|       0.0|      0.0|1421970775|1421970775|       0.001|0.0089999996|         0.0|         0.0|         0.0|              0|           0|               0|           0|         0|         2|         5|         3|          2|               1|               1|             1|      null|    0|\n","|59.166.0.7|12569|149.171.126.5|    53|  udp|  CON|9.6899999E-4|   146|   178|  31|  29|    0|    0|     dns|602683.19|734778.13|    2|    2|   0|   0|         0|         0|     73|     89|          0|          0|       0.0|      0.0|1421970775|1421970775|0.0099999998|       0.003|         0.0|         0.0|         0.0|              0|           0|               0|           0|         0|         3|         1|         2|          3|               3|               1|             1|      null|    0|\n","|59.166.0.1|12792|149.171.126.7|    53|  udp|  CON|0.0010629999|   146|   178|  31|  29|    0|    0|     dns|549388.56| 669802.5|    2|    2|   0|   0|         0|         0|     73|     89|          0|          0|       0.0|      0.0|1421970780|1421970780|0.0020000001|       0.003|         0.0|         0.0|         0.0|              0|           0|               0|           0|         0|         5|         3|         2|          4|               3|               1|             1|      null|    0|\n","|59.166.0.0|63414|149.171.126.9| 10330|  tcp|  FIN|  0.26501101|  8928|   320|  31|  29|    4|    1|ftp-data|250283.94|8060.0425|   14|    6| 255| 255|2386904726|3699988372|    638|     53|          0|          0| 1725.2916|71.464249|1421970775|1421970776|   20.385462|   52.644802|7.7500002E-4|6.3299999E-4|     1.42E-4|              0|           0|               0|           0|         0|         3|         6|         6|          5|               1|               1|             2|      null|    0|\n","|59.166.0.1|33555|149.171.126.3|  6881|  tcp|  FIN|  0.51712799|  1540|  1644|  31|  29|    4|    4|       -|22338.764|  24025.0|   16|   18| 255| 255|1741520309|3943579644|     96|     91|          0|          0| 2036.1301| 51.91766|1421970776|1421970776|   34.433331|   30.388353|6.5399997E-4|5.2200002E-4|     1.32E-4|              0|           0|               0|           0|         0|         4|         6|         5|          7|               4|               1|             4|      null|    0|\n","|59.166.0.8|10867|149.171.126.8|   111|  udp|  CON|0.0053389999|   568|   312|  31|  29|    0|    0|       -|638321.81|350627.47|    4|    4|   0|   0|         0|         0|    142|     78|          0|          0| 1.7430201| 1.757632|1421970776|1421970776|        1.24|    1.252333|         0.0|         0.0|         0.0|              0|           0|               0|           0|         0|        16|         7|         5|          5|               1|               1|             4|      null|    0|\n","|59.166.0.8|12411|149.171.126.8|  1715|  udp|  CON|    0.001739|   512|   304|  31|  29|    0|    0|       -|1766532.5|1048878.6|    4|    4|   0|   0|         0|         0|    128|     76|          0|          0|0.63026798| 0.318434|1421970776|1421970776|  0.44966701|    0.227667|         0.0|         0.0|         0.0|              0|           0|               0|           0|         0|        16|         7|         5|          5|               1|               1|             4|      null|    0|\n","|59.166.0.8|46725|149.171.126.2|    53|  udp|  CON|    0.001018|   146|   178|  31|  29|    0|    0|     dns|573673.88|699410.63|    2|    2|   0|   0|         0|         0|     73|     89|          0|          0|       0.0|      0.0|1421970776|1421970776|       0.011|       0.011|         0.0|         0.0|         0.0|              0|           0|               0|           0|         0|         2|         5|         3|          5|               1|               1|             1|      null|    0|\n","|59.166.0.1|51562|149.171.126.4|    53|  udp|  CON|    0.001044|   146|   178|  31|  29|    0|    0|     dns|559386.94|681992.31|    2|    2|   0|   0|         0|         0|     73|     89|          0|          0|       0.0|      0.0|1421970776|1421970776|       0.011|       0.003|         0.0|         0.0|         0.0|              0|           0|               0|           0|         0|         2|         3|         5|          7|               2|               1|             3|      null|    0|\n","|59.166.0.3|48838|149.171.126.2|    53|  udp|  CON|9.8699995E-4|   146|   178|  31|  29|    0|    0|     dns| 591692.0|721377.94|    2|    2|   0|   0|         0|         0|     73|     89|          0|          0|       0.0|      0.0|1421970776|1421970776|0.0080000004|       0.003|         0.0|         0.0|         0.0|              0|           0|               0|           0|         0|         2|         5|         3|          3|               1|               1|             1|      null|    0|\n","|59.166.0.0|16907|149.171.126.9|    21|  tcp|  FIN|   2.2547121|  2934|  3740|  31|  29|   11|   15|     ftp|10211.503|13025.166|   52|   54| 255| 255| 241515551|2584135680|     56|     69|          0|          0| 3141.3708| 99.93412|1421970773|1421970776|   44.203648|   42.531734|6.6100003E-4|5.2499998E-4|     1.36E-4|              0|           0|               0|           0|         0|         1|         3|         6|          5|               1|               1|             2|      null|    0|\n","|59.166.0.0| 1915|149.171.126.4| 32945|  tcp|  FIN| 0.051220998|  2854| 29104|  31|  29|    7|   17|       -|436071.16|4450987.0|   46|   48| 255| 255|1921515932|1974066994|     62|    606|          0|          0| 78.226654|76.387978|1421970776|1421970776|   1.1294219|   1.0781699|      8.1E-4|     5.43E-4|     2.67E-4|              0|           0|               0|           0|         0|         7|         2|         5|          5|               1|               1|             2|      null|    0|\n","+----------+-----+-------------+------+-----+-----+------------+------+------+----+----+-----+-----+--------+---------+---------+-----+-----+----+----+----------+----------+-------+-------+-----------+-----------+----------+---------+----------+----------+------------+------------+------------+------------+------------+---------------+------------+----------------+------------+----------+----------+----------+----------+-----------+----------------+----------------+--------------+----------+-----+\n","only showing top 20 rows\n","\n"]}],"source":["#Show Dataframe UNSWNB15\n","UNSWNB15.show()"]},{"cell_type":"markdown","source":["**Descriptive Statistics**"],"metadata":{"id":"eyz13XfBBHbv"}},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":16047,"status":"ok","timestamp":1648626951544,"user":{"displayName":"Abdulwahab Abdulliameed","userId":"10732377996348789858"},"user_tz":-60},"id":"5fqzX9iDXNsa","outputId":"0fc9d738-b8b9-4062-d97e-1037ea057de9"},"outputs":[{"name":"stdout","output_type":"stream","text":["+-------+-----------+------------------+-----------+------------------+-------+-------+------------------+-----------------+------------------+-----------------+------------------+-----------------+------------------+-------+--------------------+-----------------+-----------------+------------------+------------------+------------------+--------------------+--------------------+------------------+------------------+-------------------+------------------+------------------+------------------+--------------------+--------------------+------------------+-----------------+--------------------+--------------------+-------------------+--------------------+-------------------+-------------------+-------------------+--------------------+------------------+------------------+-----------------+-----------------+-----------------+-----------------+------------------+----------+-------------------+\n","|summary|      srcip|             sport|      dstip|            dsport|  proto|  state|               dur|           sbytes|            dbytes|             sttl|              dttl|            sloss|             dloss|service|               Sload|            Dload|            Spkts|             Dpkts|              swin|              dwin|               stcpb|               dtcpb|           smeansz|           dmeansz|        trans_depth|       res_bdy_len|              Sjit|              Djit|               Stime|               Ltime|              Spkt|             Dpkt|              tcprtt|              synack|             ackdat|     is_sm_ips_ports|       ct_state_ttl|   ct_flw_http_mthd|       is_ftp_login|          ct_ftp_cmd|        ct_srv_src|        ct_srv_dst|       ct_dst_ltm|      ct_src_ ltm| ct_src_dport_ltm| ct_dst_sport_ltm|    ct_dst_src_ltm|attack_cat|              Label|\n","+-------+-----------+------------------+-----------+------------------+-------+-------+------------------+-----------------+------------------+-----------------+------------------+-----------------+------------------+-------+--------------------+-----------------+-----------------+------------------+------------------+------------------+--------------------+--------------------+------------------+------------------+-------------------+------------------+------------------+------------------+--------------------+--------------------+------------------+-----------------+--------------------+--------------------+-------------------+--------------------+-------------------+-------------------+-------------------+--------------------+------------------+------------------+-----------------+-----------------+-----------------+-----------------+------------------+----------+-------------------+\n","|  count|    2539739|           2539739|    2539739|           2539739|2539739|2539739|           2539739|          2539739|           2539739|          2539739|           2539739|          2539739|           2539739|2539739|             2539739|          2539739|          2539739|           2539739|           2539739|           2539739|             2539739|             2539739|           2539739|           2539739|            2539739|           2539739|           2539739|           2539739|             2539739|             2539739|           2539739|          2539739|             2539739|             2539739|            2539739|             2539739|            2539739|            2539739|            2539739|             2539739|           2539739|           2539739|          2539739|          2539739|          2539739|          2539739|           2539739|    321283|            2539739|\n","|   mean|       null| 30536.93108583205|       null|11235.096788685767|   null|   null|0.6588634003800393|4340.072263330996| 36432.01132478574|62.78149802007214|   30.770440978384|5.164547223159545|16.331423819534212|   null| 3.694927939363408E7|2451158.440857539| 33.2925678583508|42.731821655689814|150.10695075360107|149.76404347060858|1.2618542900429437E9|1.2619194806546414E9|124.25270588828222|276.70543193611627|0.08326328020320198| 4242.632264575218|1589.1543650105623| 730.1640026779703|1.4232606949103932E9|1.4232606956744623E9| 193.0465504924596|78.83432357570544|0.006181224255109452|0.003287993858563316|0.00289323039543641|0.001651744529654425|0.26118471228736495|0.11009123378425893|0.01735335796316078|0.020560380417042855| 9.207911915358231| 8.989882818667587|6.439727074317479|6.901640286659378|4.642571933572702|3.593031803661715|  6.84655746121944|      null| 0.1265023689442104|\n","| stddev|       null|20441.216792342224|       null| 18438.20083577165|   null|   null|13.925767633289283|56409.39812286232|161105.30400824756|74.62669956848524|42.851922189518724|22.51836838779722|56.597890353205116|   null|1.1860413786386509E8|4225032.897503491|76.28775585678378|121.50842004704197|125.47913122211004|125.54062192608511| 1.422045057823864E9| 1.422157469789883E9|151.92108772283464| 335.6231650135929|0.35002455674771116|47503.383153168135|16911.377637309888|3438.7574708801694|  1134456.4636742047|  1134456.2648463438|2778.5830034350847|1433.278071549331|  0.0461613828526482|0.025937245439328835|0.02394810919858883|0.040608089328964335| 0.6831310323762818| 0.5564518805021055|0.13346510417617913|  0.1843730431164465|10.837080948770966|10.822813466163238|8.162329925209063|8.205340175616495|8.478001123292248|6.174757636883334|11.258797786324628|      null|0.33241474561750184|\n","|    min|10.40.170.2|                 0|10.40.170.2|                 0|    3pc|    ACC|               0.0|                0|                 0|                0|                 0|                0|                 0|      -|                 0.0|              0.0|                0|                 0|                 0|                 0|                   0|                   0|                 0|                 0|                  0|                 0|               0.0|               0.0|          1421927377|          1421927414|               0.0|              0.0|                 0.0|                 0.0|                0.0|                   0|                  0|                  0|                  0|                   0|                 1|                 1|                1|                1|                1|                1|                 1|  Analysis|                  0|\n","|    max| 59.166.0.9|             65535| 59.166.0.9|             65535|   zero|     no|         8786.6377|         14355774|          14657531|              255|               254|             5319|              5507|    ssl|         5.9880003E9|       1.287619E8|            10646|             11018|               255|               255|          4294958913|          4294953724|              1504|              1500|                172|           6558056|         1483830.9|         781221.13|          1424262068|          1424262069|         84371.492|         59485.32|           10.037506|           4.5252719|          5.5122342|                   1|                  6|                 36|                  4|                   8|                67|                67|               67|               67|               67|               60|                67|     Worms|                  1|\n","+-------+-----------+------------------+-----------+------------------+-------+-------+------------------+-----------------+------------------+-----------------+------------------+-----------------+------------------+-------+--------------------+-----------------+-----------------+------------------+------------------+------------------+--------------------+--------------------+------------------+------------------+-------------------+------------------+------------------+------------------+--------------------+--------------------+------------------+-----------------+--------------------+--------------------+-------------------+--------------------+-------------------+-------------------+-------------------+--------------------+------------------+------------------+-----------------+-----------------+-----------------+-----------------+------------------+----------+-------------------+\n","\n"]}],"source":["#Describe/Summary Statistics of UNSWNB15 Dataframe\n","UNSWNB15.describe().show()"]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":8457,"status":"ok","timestamp":1648626959999,"user":{"displayName":"Abdulwahab Abdulliameed","userId":"10732377996348789858"},"user_tz":-60},"id":"SXDwoY7F8jY5","outputId":"6191f5e4-008a-41ea-927b-48ac27c1bcf1"},"outputs":[{"name":"stdout","output_type":"stream","text":["+-------+-----------------+------------------+-----------------+------------------+\n","|summary|           sbytes|            dbytes|            Spkts|             Dpkts|\n","+-------+-----------------+------------------+-----------------+------------------+\n","|  count|          2539739|           2539739|          2539739|           2539739|\n","|   mean|4340.072263330996| 36432.01132478574| 33.2925678583508|42.731821655689814|\n","| stddev|56409.39812286232|161105.30400824756|76.28775585678378|121.50842004704197|\n","|    min|                0|                 0|                0|                 0|\n","|    max|         14355774|          14657531|            10646|             11018|\n","+-------+-----------------+------------------+-----------------+------------------+\n","\n"]}],"source":["# Descriptive/Summary statistics of the 4 columns (numeric column)\n","# sbytes - Source bytes, \n","# dbytes - Destination bytes,\n","# Spkts - Source Packets and \n","# Dpkts - Destination Packets\n","UNSWNB15.describe('sbytes', 'dbytes', 'Spkts','Dpkts').show()"]},{"cell_type":"markdown","source":["**Correlation**"],"metadata":{"id":"9odB0c85BZWE"}},{"cell_type":"code","execution_count":16,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":9642,"status":"ok","timestamp":1648899937308,"user":{"displayName":"Abdulwahab Abdulliameed","userId":"10732377996348789858"},"user_tz":-60},"id":"2s4GgMNho_D9","outputId":"66973566-e39a-479a-d833-64c902e659b0"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["0.04702553278490288"]},"metadata":{},"execution_count":16}],"source":["#Correlation of sbytes and dbytes column.\n","UNSWNB15.stat.corr('sbytes','dbytes')"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"I4M2xu5EU1gU"},"outputs":[],"source":["#Register the DataFrame as a temporary table in the SQLContext.\n","UNSWNB15.registerTempTable(\"UNSWNB15\")"]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":384,"status":"ok","timestamp":1648626960720,"user":{"displayName":"Abdulwahab Abdulliameed","userId":"10732377996348789858"},"user_tz":-60},"id":"FQnpV4rHVtdQ","outputId":"dec5e672-423d-40f8-9deb-f33bb0c4bfb2"},"outputs":[{"name":"stdout","output_type":"stream","text":["+----------+-----+------+-----+--------+\n","|     srcip|sport|sbytes|Spkts| service|\n","+----------+-----+------+-----+--------+\n","|59.166.0.3|56716|  7812|  122|       -|\n","|59.166.0.0|43467|  4238|   72|       -|\n","|59.166.0.5|41289|  2750|   44|       -|\n","|59.166.0.9|43785| 10476|  180|       -|\n","|59.166.0.8|40691| 13350|  232|       -|\n","|59.166.0.3|20393|  1958|   22|       -|\n","|59.166.0.7|19792|   146|    2|     dns|\n","|59.166.0.3|14382|   424|    8|ftp-data|\n","|59.166.0.9|37074|   146|    2|     dns|\n","|59.166.0.7|12569|   146|    2|     dns|\n","|59.166.0.1|12792|   146|    2|     dns|\n","|59.166.0.0|63414|  8928|   14|ftp-data|\n","|59.166.0.1|33555|  1540|   16|       -|\n","|59.166.0.8|10867|   568|    4|       -|\n","|59.166.0.8|12411|   512|    4|       -|\n","|59.166.0.8|46725|   146|    2|     dns|\n","|59.166.0.1|51562|   146|    2|     dns|\n","|59.166.0.3|48838|   146|    2|     dns|\n","|59.166.0.0|16907|  2934|   52|     ftp|\n","|59.166.0.0| 1915|  2854|   46|       -|\n","+----------+-----+------+-----+--------+\n","only showing top 20 rows\n","\n"]}],"source":["spark.sql(\"select srcip, sport, sbytes, Spkts, service from UNSWNB15\").show()"]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":11699,"status":"ok","timestamp":1648626972419,"user":{"displayName":"Abdulwahab Abdulliameed","userId":"10732377996348789858"},"user_tz":-60},"id":"uji7joJFg52E","outputId":"f7d29444-6ee8-46ac-dfd7-5658357fcaf7"},"outputs":[{"name":"stdout","output_type":"stream","text":["+-----+\n","|SRCIP|\n","+-----+\n","|   43|\n","+-----+\n","\n"]}],"source":["spark.sql(\"select COUNT(DISTINCT srcip) AS SRCIP from UNSWNB15\").show()"]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"elapsed":19072,"status":"ok","timestamp":1648627011918,"user":{"displayName":"Abdulwahab Abdulliameed","userId":"10732377996348789858"},"user_tz":-60},"id":"r9oj7Ur2kyk_","outputId":"801f4aaf-9505-4795-f032-03cb7496ee5e"},"outputs":[{"name":"stdout","output_type":"stream","text":["+--------------+-------------------+\n","|      SourceIP|Numberofconnections|\n","+--------------+-------------------+\n","|    59.166.0.4|             197959|\n","|    59.166.0.1|             197680|\n","|    59.166.0.5|             197626|\n","|    59.166.0.2|             197550|\n","|    59.166.0.0|             197528|\n","|    59.166.0.3|             195953|\n","|    59.166.0.9|             190187|\n","|    59.166.0.6|             189419|\n","|    59.166.0.8|             189341|\n","|    59.166.0.7|             189059|\n","|  175.45.176.1|             127995|\n","|  175.45.176.3|             118670|\n","|  175.45.176.0|              82999|\n","|149.171.126.18|              81680|\n","|149.171.126.15|              44883|\n","|149.171.126.14|              41047|\n","|  175.45.176.2|              32026|\n","|149.171.126.10|              30365|\n","|149.171.126.12|              16082|\n","|    10.40.85.1|               4012|\n","+--------------+-------------------+\n","only showing top 20 rows\n","\n"]}],"source":["spark.sql (\"SELECT srcip AS SourceIP, COUNT(label) AS Numberofconnections FROM UNSWNB15 GROUP BY srcip ORDER BY Numberofconnections DESC \").show()"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"tsvSfU7qD39I"},"outputs":[],"source":["df = spark.sql (\"SELECT srcip AS SourceIP, COUNT(label) AS Numberofconnections FROM UNSWNB15 GROUP BY srcip ORDER BY Numberofconnections DESC LIMIT 10 \").show()"]},{"cell_type":"code","execution_count":null,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":295},"executionInfo":{"elapsed":46847,"status":"ok","timestamp":1648716917452,"user":{"displayName":"Abdulwahab Abdulliameed","userId":"10732377996348789858"},"user_tz":-60},"id":"nLWkqqGP0Gxs","outputId":"d7efc508-ff15-4816-92c1-c905df53b616"},"outputs":[{"output_type":"display_data","data":{"text/plain":["<Figure size 432x288 with 1 Axes>"],"image/png":"iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de3zcdZ3v8dc79yZNm7ZJS2kbSi+ogEJrBbkout6QdWHP8QKsqKysnLMrHBcvB3VdVDxnXZddj3ddVA7gDRVc7SKKx1VkFVAqBaQoUAtt00JvNL0mbZN8zh+/3ySTySSZtJ2ZJPN+Ph7zyO/ynd98cpvPfC+/71cRgZmZVa6qcgdgZmbl5URgZlbhnAjMzCqcE4GZWYVzIjAzq3BOBGZmFc6JwCY1SR+R9PUiXXuNpJcV49pZrxGSlqTbX5L090V4jR9JetvRvq5NHE4EVhSSzpZ0j6Rdkp6V9CtJLyp3XIWS9GNJ1+Y5foGkZyTVRMRJEXFXqWKKiP8eER87kmvkS4wR8dqIuOnIorOJzInAjjpJ04Dbgc8CM4F5wEeBA+WMa4xuAi6RpJzjbwG+ERE9ZYjJrCicCKwYTgCIiG9FRG9EdEXETyLiYQBJiyX9TNIOSdslfUNSS+bJkp6S9D5JD0vaJ+mrkuakTRh7JP1U0oy07MK0+eRySZslPS3pvcMFJunFaU2lU9JDIzTtfB+YBbwk67kzgNcBN2fF+cp0+zRJqyTtlrRF0ifT4y+T1JETQ+7z7k3jeVrS5yTVDRP7jZL+V7r975L2Zj36JF2anvu0pI1pLL+V9JL0+LnAB4EL0+c8lB6/S9JfpdtVkj4kab2krZJuljQ952f9Nkkb0t/d3w33s7aJw4nAiuFxoFfSTZJem3nTziLg48CxwPOABcBHcsq8HngVSVL5M+BHJG9ibSR/t/8jp/zLgaXAq4GrM2+0g15Umgf8EPhfJDWV9wK3SWrLLRsRXcB3gLdmHX4T8IeIeCjP9/xp4NMRMQ1YnD63EL3AVUArcAbwCuBvRntSRPxZREyNiKnAG4FngP9IT98PnEryPX4T+K6khoj4MfAPwLfT556S59KXpo+XA4uAqcDncsqcDTwnjfUaSc8r8Hu1cWpCJgJJN6SfVh4poOz/kfRg+nhcUmcpYqxkEbGb5M0igC8D2yStlDQnPb82Iv5fRByIiG3AJ4Fzci7z2YjYEhGbgP8Efh0RqyOiG/g3YFlO+Y9GxL6I+B3wf4GL84R2CXBHRNwREX0R8f+AVcB5w3wrNwFvkNSQ7r81PZbPIWCJpNaI2BsR9w1TbpCI+G1E3BcRPRHxFPCvDP1ZDEvSCWlMb4qIjek1vx4RO9Jr/gtQT/LGXYg3A5+MiHURsRf4AHCRpJqsMh9Na3kPAQ8B+RKKTSATMhEANwLnFlIwIq6KiFMj4lSSNuvvFTMwS0TE7yPi0oiYD5xM8un/UwBpM88tkjZJ2g18neQTcbYtWdtdefan5pTfmLW9Pn29XMcBb0ybYTrTDwVnA3OH+R5+CWwH/lzSYuA0kk/Y+VxGUnv5g6T7Jb1umHKDSDpB0u1pB/Rukk/suT+L4Z47HfgB8KE01szx90r6vZKO+k5geqHXJPm5rc/aXw/UAHOyjj2Ttb2fob8Lm2AmZCKIiLuBZ7OPpe3OP07bRP9T0nPzPPVi4FslCdL6RcQfSJL3yemhfyCpLTw/bUq5hKS56EgsyNpuBzbnKbMR+FpEtGQ9miLiH0e47s0kNYFLgDsjYku+QhHxRERcDMwGPgHcKqkJ2Ac0ZspJqiZp3sr4IvAHYGn6s/ggBfwsJFWRJKWfR8T1WcdfAvxPkmasGRHRAuzKuuZo0w1vJkmYGe1AD4MTsU0yEzIRDON64MqIeCFJ2+8Xsk9KOg44HvhZGWKrKJKeK+k9kuan+wtIknCmuaQZ2AvsStvt33cUXvbvJTVKOgn4S+Dbecp8HfgzSa+RVC2pIe3MnT/CdW8GXgm8g+GbhZB0iaS2iOgDMs2PfST9JQ2S/lRSLfAhkqaajGZgN7A3/fDy14V9u/xvoAl4V87xZpI37m1AjaRrgGlZ57cAC9NEks+3gKskHS9pKgN9Ch4lNYlNikSQ/sGeSdIp9iBJO2tudf8i4NaI6C11fBVoD3A68GtJ+0gSwCPAe9LzHwWWk3xS/SFHp7nuF8Bakg7Tf46In+QWSNvQLyD51L2NpIbwPkb4P0jb7e8hedNdOcLrnwuskbSXpOP4orQdfRdJ5+9XgE0kNYTsUUTvBf6C5Gf2ZfInsHwuBl4M7MwaOfRm4E7gxyQJaD3QzeBms++mX3dIeiDPdW8AvgbcDTyZPv/KAmOyCUoTdWEaSQuB2yPiZCXj1h+LiLxtvWn51cA7I+KeEoVoJZD+HTwJ1PpTq9nhmRQ1gnSUypOS3gigRP9IhrTKPQO4t0whmpmNWxMyEUj6Fsmb+nMkdUi6jGTY22XpTTJrSJoAMi4CbomJWv0xMyuiCds0ZGZmR8eErBGYmdnRUzN6kfGltbU1Fi5cWO4wzMwmlN/+9rfbI2LIdCowARPBwoULWbVqVbnDMDObUCStH+6cm4bMzCqcE4GZWYVzIjAzq3ATro8gn0OHDtHR0UF3d/eQcw0NDcyfP5/a2toyRGZmNv5NikTQ0dFBc3MzCxcuRFkrC0YEO3bsoKOjg+OPP76MEZqZjV+Tommou7ubWbNmDUoCAJKYNWtW3pqCmZklJkUiAIYkgdGOm5lZYtIkgkK98xsP8IW71pY7DDOzcaPiEsEPf/c0//Tjx8odhpnZuDFpEsFwk+d5Uj0zs5FNikTQ0NDAjh07hrzpZ0YNNTQ0lCkyM7Pxb1IMH50/fz4dHR1s27ZtyLnMfQRmZpbfpEgEtbW1vk/AzOwwTYqmITMzO3xFSwSSbpC0VdIjo5R7kaQeSW8oVixmZja8YtYIbgTOHamApGrgE8BPihiHmZmNoGiJICLuBp4dpdiVwG3A1mLFYWZmIytbH4GkecB/Ab5YQNnLJa2StCrfyCAzMzt85ews/hRwdUT0jVYwIq6PiBURsaKtLe+Sm2ZmdpjKOXx0BXBLOilcK3CepJ6I+H4ZYzIzqzhlSwQR0T/wX9KNwO1OAmZmpVe0RCDpW8DLgFZJHcCHgVqAiPhSsV7XzMzGpmiJICIuHkPZS4sVh5mZjcx3FpuZVTgnAjOzCudEYGZW4ZwIzMwqnBOBmVmFcyIwM6twFZUIvH6xmdlQFZUIevucCMzMclVUIjjU60RgZparohLB/oM95Q7BzGzcqahEsHP/oXKHYGY27lRUItix90C5QzAzG3cqKhFs33uw3CGYmY07FZUInt3nGoGZWa6KSgSuEZiZDVVRiWD9jn3lDsHMbNypqESw4dn95Q7BzGzcqahEsGW3+wjMzHIVLRFIukHSVkmPDHP+zZIelvQ7SfdIOqVYsWRs2+NEYGaWq5g1ghuBc0c4/yRwTkQ8H/gYcH0RYwHgYG9fsV/CzGzCKebi9XdLWjjC+Xuydu8D5hcrFjMzG9546SO4DPjRcCclXS5plaRV27ZtK2FYZmaTX9kTgaSXkySCq4crExHXR8SKiFjR1tZWuuDMzCpA0ZqGCiHpBcBXgNdGxI5yxmJmVqnKViOQ1A58D3hLRDxerjjMzCpd0WoEkr4FvAxoldQBfBioBYiILwHXALOAL0gC6ImIFcWKx8zM8ivmqKGLRzn/V8BfFev1zcysMGXvLDYzs/JyIjAzq3BOBGZmFc6JwMyswjkRmJlVOCcCM7MK50RgZlbhnAjMzCqcE4GZWYVzIjAzq3BOBGZmFc6JwMyswjkRmJlVOCcCM7MKN6ZEIKlK0rRiBWNmZqU3aiKQ9E1J0yQ1AY8Aj0p6X/FDMzOzUiikRnBiROwG/hz4EXA88JaiRmVmZiVTSCKolVRLkghWRsQhIIoblpmZlUohieBfgaeAJuBuSccBu0d7kqQbJG2V9Mgw5yXpM5LWSnpY0vKxBG5mZkfHqIkgIj4TEfMi4rxIrAdeXsC1bwTOHeH8a4Gl6eNy4IsFXNPMzI6yQjqL50j6qqQfpfsnAm8b7XkRcTfw7AhFLgBuTpPLfUCLpLkFxm1mZkdJIU1DNwJ3Asem+48Df3sUXnsesDFrvyM9NoSkyyWtkrRq27ZtR+Glzcwso5BE0BoR3wH6ACKiB+gtalQ5IuL6iFgRESva2tpK+dJmZpNeIYlgn6RZpCOFJL0Y2HUUXnsTsCBrf356zMzMSqimgDLvBlYCiyX9CmgD3ngUXnslcIWkW4DTgV0R8fRRuK6ZmY1BIYlgDXAO8BxAwGMU1sn8LeBlQKukDuDDQC1ARHwJuAM4D1gL7Af+cuzhm5nZkSokEdwbEctJEgIAkh4ARhz3HxEXj3I+gHcWEqSZmRXPsIlA0jEko3imSFpGUhsAmAY0liA2MzMrgZFqBK8BLiXpxP1k1vHdwAeLGJOZmZXQsIkgIm4CbpL0+oi4rYQxmZlZCRUyfPQ6SddJel7RozEzs5IrJBGcQnI38Vcl3Zfe5evFaczMJolCJp3bExFfjogzgatJhoE+LekmSUuKHqGZmRVVIfcDVEs6X9K/AZ8C/gVYBPw7yb0AZmY2gRVyH8ETwM+B6yLinqzjt0p6aXHCMjOzUikkEbwgIvbmOxER/+Mox2NmZiU2bNOQpAZJbwP+JF1N7GpJt0v6tKTWEsZoZmZFNFIfwc3Aq4G3A3cB7cDngD0kaxSYmdkkMFLT0IkRcbKkGqAjIs5Jj/9Y0kMliM3MzEpgpBrBQehfiGZzzrmSLkxjZmbFM1KNYL6kz5BMNpfZJt3Pu6SkmZlNPCMlgvdlba/KOZe7b2ZmE9Rok86ZmdkkV8hcQ2ZmNolVZCKYM62+3CGYmY0bRU0Eks6V9JiktZLen+d8u6SfS1ot6WFJ5xUznowXzG8pxcuYmU0Io04xIakNeAewMLt8RLx9lOdVA58HXgV0APdLWhkRj2YV+xDwnYj4oqQTSSaxWzjG72HMTj52erFfwsxswihkrqEfAP8J/JSx3T9wGrA2ItYBSLoFuADITgRBsgYywHSG3q9QFEtmTy3Fy5iZTQiFJILGiLj6MK49D9iYtd8BnJ5T5iPATyRdCTQBr8x3IUmXA5cDtLe3H0Yog7VOrTvia5iZTRaF9BHcXsS2+4uBGyNiPnAe8DVJQ2KKiOsjYkVErGhrazviF22qLyT/mZlVhkISwbtIkkG3pD3pY3cBz9sELMjan58ey3YZ8B2AiLgXaACKPrNpTbWK/RJmZhNGIUtVNkdEVUQ0pNvNEVHImsX3A0slHS+pDrgIWJlTZgPwCgBJzyNJBNvG9i2MXU2VE4GZWUZBbSSSzgcyq5HdFRG3j/aciOiRdAVwJ1AN3BARayRdC6yKiJXAe4AvS7qKpOP40oiIw/lGxqK6qiJvnzAzy6uQ4aP/CLwI+EZ66F2SzoqID4z23Ii4g5x1jSPimqztR4GzxhTxUVAt1wjMzDIKqRGcB5waEX0Akm4CVgOjJoLxynnAzGxAoW0k2bfiTvi7sdxZbGY2oJAawceB1ZJ+TrIWwUuBIdNFTCRuGjIzGzBqIoiIb0m6i6SfAODqiHimqFEVmZwIzMz6Dds0JOm56dflwFySO4M7gGPTYxOW84CZ2YCRagTvJpnW4V/ynAvgT4oSUQlUOROYmfUbaYWyy9PN10ZEd/Y5SQ1FjarInAbMzAYUMmrongKPTRiuEZiZDRi2RiDpGJIZRKdIWsbAB+lpQGMJYise5wEzs34j9RG8BriUZLK4T2Yd3wN8sIgxFZ2nGjIzGzBSH8FNwE2SXh8Rt5UwpqLz8FEzswGF3Edwm6Q/BU4imR00c/zaYgZWTE4DZmYDRu0slvQl4ELgSpL30DcCxxU5rqJyZ7GZ2YBCRg2dGRFvBXZGxEeBM4ATihtWcTkPmJkNKCQRdKVf90s6FjhEcqfxhOVEYGY2oJBJ526X1AJcBzxAclfxV4oaVZHJvQRmZv0K6Sz+WLp5m6TbgYaI2FXcsIrLw0fNzAYUulTlmcDCTHlJRMTNRYyrqDx81MxsQCGjhr4G/DNwNslU1C8CVhRycUnnSnpM0lpJedcwkPQmSY9KWiPpm2OI/bC5RmBmNqCQGsEK4MSxLiovqRr4PPAqkumr75e0Ml2nOFNmKcmSl2dFxE5Js8fyGofLNQIzswGFjBp6BDjmMK59GrA2ItZFxEHgFuCCnDLvAD4fETsBImLrYbyOmZkdgUJqBK3Ao5J+AxzIHIyI80d53jxgY9Z+B3B6TpkTACT9CqgGPhIRP869kKTLSdZGoL29vYCQzcysUIUkgo8U+fWXAi8jmdzubknPj4jO7EIRcT1wPcCKFSvG1ERlZmYjK2T46C8O89qbgAVZ+/PTY9k6gF9HxCHgSUmPkySG+w/zNc3MbIxGWrP4l+nXPZJ2Zz32SNpdwLXvB5ZKOl5SHXARsDKnzPdJagNIaiVpKlp3GN+HmZkdppGmoT47/dp8OBeOiB5JVwB3krT/3xARayRdC6yKiJXpuVdLehToBd4XETsO5/XMzOzwjNo0JOlrEfGW0Y7lExF3AHfkHLsmazuAd6cPMzMrg0KGj56UvSOpBnhhccIxM7NSG6mP4AOS9gAvyO4fALYAPyhZhGZmVlTDJoKI+HjaP3BdRExLH80RMSsiPlDCGM3MrIgKaRq6XVITgKRLJH1S0oReoczMzAYUkgi+SLIozSnAe4A/AhN25lEzMxuskETQk47uuQD4XER8HjisIaVmZjb+FDLFxB5JHwDeArxEUhVQW9ywzMysVAqpEVxIMtnc2yPiGZKpIq4ralRmZlYyoyaC9M3/NqA+PbQd+LdiBmVmZqVTyApl7wBuBf41PTSPZI4gMzObBAppGnoncBawGyAingBKspKYmZkVXyGJ4EC6whjQP8WE1wQwM5skCkkEv5D0QWCKpFcB3wX+vbhhmZlZqRSSCN4PbAN+B/w3ktlEP1TMoIotuS3CzMygsBXK+iR9H/h+RGwrQUxF9+T2fSxqm1ruMMzMxoWRZh+VpI9I2g48BjwmaZuka4Z7zkTxq7Xbyx2Cmdm4MVLT0FUko4VeFBEzI2ImcDpwlqSrShJdkfzSicDMrN9IieAtwMUR8WTmQESsAy4B3lrswIrp3j/uoLfP/QRmZjByIqiNiCEfndN+goLmGpJ0rqTHJK2V9P4Ryr1eUkhaUch1j9Tu7h4e2bSrFC9lZjbujZQIDh7mOQAkVQOfB14LnAhcLOnEPOWagXcBvx7tmkeTm4fMzBIjJYJTspaozH7sAZ5fwLVPA9ZGxLr0hrRbSKayzvUx4BNA95ijP0zPPabZHcZmZqmRlqqszlqiclrOcpWFNA3NAzZm7Xekx/pJWg4siIgfjnQhSZdLWiVp1bZtRz6C9ewlraxav5PuQ71HfC0zs4mukBvKiiJd1+CTJKuejSgiro+IFRGxoq2t7Yhf+6ylrRzs6WPVUzuP+FpmZhNdMRPBJmBB1v789FhGM3AycJekp4AXAytL0WF82sKZ1FTJ/QRmZhQ3EdwPLJV0vKQ64CJgZeZkROyKiNaIWBgRC4H7gPMjYlURYwKgqb6G5e0z3E9gZkYRE0FE9ABXAHcCvwe+ExFrJF0r6fxivW6hzlrSyiObd9G5f9QBUGZmk1pR+wgi4o6IOCEiFkfE/06PXRMRK/OUfVkpagMAh3r7OHvpLCKSm8vMzCpZ2TqLy2n/wV5eML+Fprpq9xOYWcWryETQdbCX2uoqXrxolvsJzKziVWQi2HewB0j6CZ7asZ+OnfvLHJGZWflUTCLIXoym62ByI9nZS1sBuGet+wnMrHJVTCI40NPXv70/TQRLZ0+lrbne/QRmVtEqJhFsfHag+Wd/2jQkibMWJ/0EfZ6W2swqVMUkgp88uqV/O9M0BEk/wY59B3lsy55yhGVmVnYVkwg++7Mn+rf35SQC8PKVZla5KiYRdB8a6CPY3XWof/vYliksam1yIjCzilUxiSDb7u5Dg/bPWtLKr598loNZHcpmZpWiMhNBV8+g/bOWtLL/YC8PbuwsU0RmZuVTmYkgp0ZwxqJZVMn9BGZWmSoyEezqGpwIpjfW8vx5050IzKwiORGkzlrSyuqNnezpHnrOzGwyq8hEsDtPIjh7SSu9fcFvnny2DBGZmZVPRSaCPd09Q44tP24G9TVV/MrzDplZhanIRJCvaaihtpoXLZzpfgIzqzgVmQj2HhhaI4Ckn+CxLXvYuqe7xBGZmZVPUROBpHMlPSZpraT35zn/bkmPSnpY0n9IOq6Y8WT7wYObuP+pZ9nc2UVvOuHc2Us8LbWZVZ6aYl1YUjXweeBVQAdwv6SVEfFoVrHVwIqI2C/pr4F/Ai4sVkzZ3nXLg/3b1VXimGkNzJ3eAMDffvtBug71cmzLFOaljyl11aUIy8ys5IqWCIDTgLURsQ5A0i3ABUB/IoiIn2eVvw+4pIjxDPLTd5/Dps4uNu3sYnNnV/92xge+97tB5Wc21fUnhWNbpjBvxkCSmDdjCjMaa5FUqvDNzI6aYiaCecDGrP0O4PQRyl8G/CjfCUmXA5cDtLe3H5XglsyeypLZU4cc//p96/nQ9x/h65edTn1tFZt2pkkiTRR/3LaXu5/Y1r+4TcaU2mqObWlg3oxG5rU09CeIY6cnX4+Z1kBNdUV2yZjZOFfMRFAwSZcAK4Bz8p2PiOuB6wFWrFhxVFaQOdjTR13N0DfmTD/Bk9v38pYzFvKihXnjoXP/oUEJor9W0dnFo5t3sX3vwUHPqRIcM62hvyaRr1bRWDcufh1mVmGK+c6zCViQtT8/PTaIpFcCfwecExEHihjPIF0He/MmguNmNTKvZQq/XLudt5yxMO9zJTGjqY4ZTXWcPG963jLdh3rZ1JkmiDRRdKTbv92wk9sffpqenFXRZjTW9vdLHNsyhfk5SWNWU52bn8zsqCtmIrgfWCrpeJIEcBHwF9kFJC0D/hU4NyK2FjGWIfYf6mE6tUOOS+LsJa386JGn6e0LqqsO7423obaaxW1TWdw2tPkJoLcv2LqnO0kQafNTJmk8tWMfv1q7fdACOgD1NVX9tYf+BJGVNI6Z3kCtm5/MbIyKlggiokfSFcCdQDVwQ0SskXQtsCoiVgLXAVOB76afdDdExPnFiinbld9czatPmpP0FbQ1M3/GFKrSN/0zl8zi26s28simXZyyoKUor19dJeZOn8Lc6VN4YZ5BsxHB7q4eOjr3s7mzm00797N5VzebdiY1iz/8YSvb9gyuQEkwp3lo89P8rO2p9W5+MrPBFDGxFm1fsWJFrFq1aszPW/j+H454vr6mikVtSQfyjMZabr53PReceiz/9IYXUF8zPoeOdh/q5eld3f01iY6sWsWmzi6e3tXFod7Bv9/pUwaan+bPmJJ0cLc0ph3dU2htqu9PiGY2eUj6bUSsyHeuoj8e/vMbT2HhrEbWbt2bPLbtZfWGnXSkw0h/8OBmbn/4adpnNrI4TRLZj3J/um6oreb41iaOb23Ke76vL9i29wAdOUNkk+ao/fx63Q725NxlXVdTxbHTG4Y0P2X2506fkrdvxcwmropOBO/97kNce8FJvDWnU7jrYC9vv/F+7l23g8vOPp6Nz+5n7da9/OLxrYM+YR8zraE/KSyePZUlabJonTo+OnWrqsScaQ3MmdbAC4+bkbfM7u5DQ+6lyIx+uuuxbWzN0/w0u7l+SILIboqa1jC078XMxq+KTgQA1/xgDbc9sIlv/NXp/Z/wp9RVc/lLF3Hvuh28dGkbZy9NhpQe6u1jQ5oU1m7dyx/TWsR3Vm0cdF/B9Cm1ad/D4BrEvJYp467ZZVpDLdPm1vK8udPynj/Q08szad/EoOGyu7p4ZNMufrJmCwd7B6/13NxQM2hYbG7SaJvq5iez8aTiEwHAQxs7OfnDd7Jk9lRevGgmyxbM4DnHNFNTJX65dnt/IqitruofCfSakwaeHxE8vat7UBPT2q17+envt/DtVQP31DXUVrGodWgT08JZTeO2uaW+pprjZjVx3Kzhm5+27zuQ1iq62dS5P00a3Wzq7OL+p55ld86037XVSUd5dqKYn7U9d3oDDbXjs1/GbDKq2M7il57QxppNu9ix7+Awzxjw7ledwLL2Fk5Z0DLmZo+d+w72J4bsx6bOgeksqqvEcTMbk+alrJrE4nHQD3E07Ok+NJAkOgdqF5mO7S17usn9M2xLm5/6E8T0zF3bSQKZNqVmXDS/mU0UI3UWV2wiALj57afx1ht+M+jYvJYpfPT8k9ix7wBX3zZ4viEJls6eyrIFM1jW3sLy42awpG3qYTVz7D/Yw7pt+wYlhye27mH9jv2DbjSbOz3ph8jurF46eyqzptaP+TXHq4M9fWzZ3T3kforNuwb6LA70DG5+mlpfk454Gtz8ND/dnt3ccNj3gJhNRk4EjD58NNc5J7TR0ljLDx7czIUrFnDKghY27tzPH57ezeqNnXTuTxa3aa6v4ZQFLSxvb2FZ+wxOXdDCjKa6MceXcai3j/U7BieItdv28set++g6NNAPMaOxdqCjOitJHDt9/PVDHKmIYMe+g4NqEoOSRmdX/+8jo6ZKzG1p6J/rKfteikzScPOTVRInAsaeCIYztb6G1ql17DvYO+SGrozGumpedeIcViycybIFLTz3mOYjnnCury/YvKtroKM6q7lpZ9ab4JTaahbPbhrSUX3crKZJfdfxvgM9g6bxyCSITO3imd3d5MzoQevUukFTjefO/9TiGWVtEnEi4MgTwWkLZzJ/xhSmTall+94DbNtzgG3p13xrII90nVefNIcT506jrbme2c0NR9zevWPvgUGd1JkRTZt3Day0VlMljpvVOLijuq2ZxbObKmKyu0O9SfPTppyaRPY9Ft2HBjc/NdZVDztB4LEtU5jTXO8ZZW3CcCJgbIng7CWt/HKYtYunNdRwavsMlre3sLx9Bqe2t1BXXTUoMWzbc4Ctew7w4MZO7n58W0GvmekcbZtaT1tz1iPdn91cT+vU+jEtkLPvQM+gmkMmWazfsb9/VTZI+kUW5xnuOvMImrgmmojg2X0H+zu1O7JHQXUm28/mDCzILGg0ZP6nrKThBY1svINXNf8AAA+ASURBVHAi4Og0DX3sz0/m0c27eGB9J49v3UPEQAfy8vYZyeO4Fha15u9A7j7Uy73rdvCTNVv4yZpnChqxlKu5voa25npacxJFduKY3VzPzKa6YT+tHuzJ6YfYNtDclP2peGZTHUvapg6MZurvh2ioyCaT/Qd70tpDd96b8J7Z3T0owULyM+zv1E6n8khmlU22Z3pGWSsRJwKOXh/B6r9/FTOa6tjdfYiHNnbywPpOHtiwk9UbdvaPl58+pZZTF7T0J4ZTF7TQPMyw02d2dbN6w05Wb+xk9YadPNyxq3+ETJVg/oxGFsycwvyWRuZMb2BP96H+WsdITVMSzGqqo3WYGkamltE2daBpqq8v2NTZlXZOD65FZHfGNtZVD+qgzmwfN6txUvdDjKant48tew4Mmu9p0N3aO7sGdfjDwIJGQ6Ydb/GCRnZ0ORFw9BIBwE1vP41zTmgbdKyvL1i3fW9/Ynhgw06e2Lq3v9Zwwuxmlh+XjCxa3j6DxW1NeT8JHurt4/dP72b1hs7+BLF+x34gaed/3txp/SOUlrW30D6zkQM9fUOapvLu7zkw5C5ggLrqqoFaRt7EUUeVxK6uQ3Ts7BrUWf10Tj/EwtahHdWL2iqjH2I0oy1otLmza9gFjY7N0/yUGQnVNAnuNbHicyLg6CYCSD71f+biZbxkSeuwwzV3dx/iwQ2ZGkPnkFrDsrSfYXn7DE5ZMH3YWsP2vQd4cEMnqzcm13loY2f/WgWzmupYlkkMC1p4wYKWYW9Ciwh2d/cMSRRb93QPShbb9x5gx76DQ27ygqFNU4211ezuPsSurkPs6uphd1eyvTdnMrt5LVOG3FG9pG3qEQ21nYxyFzTKrVU8s6t7yIJGLY21g2oS83Om9fCCRgZOBMDRTwQZi9qauPTMhfzX5fNHvQt4tFrDc+Y0pzWG5Ga1Ra35aw29fcHjW/awesNAs9Qft+0Dkk+QJ6TXSRLN8H0WI+np7ePZfQfZOkJNY3u6nzuD6VjUVCm5MS+ns3puhfZDjGakBY0yCWO4BY2OzRn1lEkac6Y1jNspTuzocSKgeIkg12tPPobnzZ1GY101TfU1NNZV01hXQ1NdNVNyjjXWVXOgpy/pa9iwkwfSWkOmzb+lsZZl/X0NMzhlhE/7u/Yf4sGOTh5YnzQnPZhV+8iMdFq2IEkwp85vYXrj0ZshtOtgL9v3JiOlhmui2j5C09RImuqqOWPxLF68aBZnLJ7Fc+Yc+T0Zk1m+BY0yo54y91hs3zv8gkaDJwgcWKtiuNqqTRxOBJQuEYxVdZVorK2msb6aproa6mureWZX16CbxPI956wlrbxkSSsnHjuNpvqsRFNXQ2N9NbVVVazbvm9Qs9TjW/b031S1uK2pv79iWXsLJ8xpLvqUDJk3qW17uweSRiZx7D7A75/Zw++f3j2ma85squOMRbM4fdFMFsxsLGjUVKXLLGiUu5Z2pr8i34JG0xpq0rmehk7r4QWNJgYnAsZvIiimuuoqWhprmVqfJIcIeGLrXg72DP+pvH1mI2ctmcWZi1tpn9mY1F6yEk1ddVVJmmyym6ae3L6P+9bt4N51O1iXNoEVKjNXUyGjpiwx0oJGme2RFjTKTOuRfQPeMdMbxu1Kf5WibIlA0rnAp0nWLP5KRPxjzvl64GbghcAO4MKIeGqkazoRTExTaquZ2VTHrKl1zGyqY2Zj8jWTqOprq6mWqKoSVUpqPVVKHtVVZG0PlOnpCzY+u59HN+/mN089O+Ykka25oYY50xo4Jl3IZ860eo6Z3sDs5oaBpNFc7/mJUpkFjbInB8xeKjXfgkZtU+uHTjueSRpe0KjoyrJUpaRq4PPAq4AO4H5JKyPi0axilwE7I2KJpIuATwAXFismK5+udDRM9vTb48me7h72dCdDYu3oi4Ct6R33qzd0ljucCeua153I288+/qhft5iNqKcBayNiXUQcBG4BLsgpcwFwU7p9K/AKuY5uZpZXsd4di3knyjxgY9Z+B3D6cGUiokfSLmAWMGiiH0mXA5cDtLe3FyteM8syZ1o9i9umcsz0BuZOT25qm9FYx/QptUxrqGVqQw31NVXUVldRV1NFfU0VVRJSpikP971MEBPilsSIuB64HpI+gsO5xlP/+KdHNSYzs8mimE1Dm4AFWfvz02N5y0iqAaaTdBqbmVmJFDMR3A8slXS8pDrgImBlTpmVwNvS7TcAP4uJNp7VzGyCK1rTUNrmfwVwJ8nw0RsiYo2ka4FVEbES+CrwNUlrgWdJkoWZmZVQUfsIIuIO4I6cY9dkbXcDbyxmDGZmNjLfg29mVuGcCMzMKpwTgZlZhXMiMDOrcBNu9lFJ24D1h/n0VnLuWh7HJkqsEyVOmDixOs6jb6LEWsw4j4uItnwnJlwiOBKSVg03+954M1FinShxwsSJ1XEefRMl1nLF6aYhM7MK50RgZlbhKi0RXF/uAMZgosQ6UeKEiROr4zz6JkqsZYmzovoIzMxsqEqrEZiZWQ4nAjOzCjcpE4GkcyU9JmmtpPfnOV8v6dvp+V9LWlj6KPtjGS3Wd0t6VNLDkv5D0nHjMc6scq+XFJLKMlSvkDglvSn9ma6R9M1Sx5gVx2i/+3ZJP5e0Ov39n1emOG+QtFXSI8Ocl6TPpN/Hw5KWlzrGNI7R4nxzGt/vJN0j6ZRSx5jGMWKcWeVeJKlH0huKHlRETKoHyZTXfwQWAXXAQ8CJOWX+BvhSun0R8O1xHOvLgcZ0+6/LEWshcablmoG7gfuAFeMxTmApsBqYke7PHse/++uBv063TwSeKlOsLwWWA48Mc/484EeAgBcDvx6ncZ6Z9Xt/7XiNM+vv42cksze/odgxTcYawWnA2ohYFxEHgVuAC3LKXADclG7fCrxC5VlcddRYI+LnEbE/3b2PZKW3UivkZwrwMeATQHcpg8tSSJzvAD4fETsBImJriWPMKCTWAKal29OBzSWMbyCIiLtJ1gsZzgXAzZG4D2iRNLc00Q0YLc6IuCfze6d8/0uF/DwBrgRuA0ry9zkZE8E8YGPWfkd6LG+ZiOgBdgGzShLdMHGk8sWa7TKST16lNmqcaXPAgoj4YSkDy1HIz/ME4ARJv5J0n6RzSxbdYIXE+hHgEkkdJJ8MryxNaGM21r/j8aBc/0ujkjQP+C/AF0v1mhNi8XoDSZcAK4Bzyh1LLklVwCeBS8scSiFqSJqHXkbyifBuSc+PiM6yRpXfxcCNEfEvks4gWc3v5IjoK3dgE5mkl5MkgrPLHcswPgVcHRF9pWqomIyJYBOwIGt/fnosX5kOSTUk1e4dpQkvbxwZ+WJF0iuBvwPOiYgDJYot22hxNgMnA3elf7jHACslnR8Rq0oWZWE/zw6StuFDwJOSHidJDPeXJsR+hcR6GXAuQETcK6mBZFKycjVnDaegv+PxQNILgK8Ar42IcvzPF2IFcEv6v9QKnCepJyK+X7RXLEdnSZE7YmqAdcDxDHTCnZRT5p0M7iz+zjiOdRlJp+LS8fwzzSl/F+XpLC7k53kucFO63UrSpDFrnMb6I+DSdPt5JH0EKtPfwEKG74T9UwZ3Fv+mHDEWEGc7sBY4s1zxFRJnTrkbKUFn8aSrEUREj6QrgDtJet5viIg1kq4FVkXESuCrJNXstSSdNheN41ivA6YC300/IWyIiPPHYZxlV2CcdwKvlvQo0Au8L8rwybDAWN8DfFnSVSQdx5dG+u5QSpK+RdKU1pr2V3wYqE2/jy+R9F+cR/Imux/4y1LHWGCc15D0BX4h/V/qiTLM9FlAnCXnKSbMzCrcZBw1ZGZmY+BEYGZW4ZwIzMwqnBOBmVmFcyIwMxvHCp2kLi37fyQ9mD4el1TQjZJOBDbhSfq7dCbRh9N/gNPT43WSPpXOivmEpB9Imp+eW5j7jyXpI5Lem27fKOnJ9HoPSXpFVrnTJN2dzhy6WtJXJDVKulTStqx/xAclnZgn3t6s6z4g6cxRvr9TyzXzqI0LN5LeWDiaiLgqIk6NiFOBzwLfK+R5k+4+Aqss6dQLrwOWR8QBSa0kN2gB/APJXc/PiYheSX8JfC+TKArwvoi4NZ2S4HpgqaQ5wHeBiyLi3jSGN6SvA8nssFeMct2u9B8VSa8BPs7IU4ecSnK36R0Fxm2TSETcrZyp8iUtBj4PtJHcu/GOiPhDzlMvJrlHYVSuEdhENxfYHunUGxGxPSI2S2okubHpqojoTc/9X+AA8CdjfI17GZhE7Z0kdybfmzkZEbdGxJbDjH8asBNA0s2S/jxzQtI3JF0AXAtcmNYiLpTUlDYX/CatkVyQlj8pPfZgWjtaepgx2fh3PXBlRLwQeC/wheyTStYtOZ5kKutRuUZgE91PgGvSOYN+SvKJ/BfAEpK7sHfnlF8FnEQybUehzgUy87yczMAU5vlcKCl7MrMzIqIrp8wUSQ8CDSSJLJOYvgpcBXxf0nSS+fPfBswgmbLjCgBJ/wD8LCLeLqkF+I2knwL/Hfh0RHxDUh3JHcs2yUiaSvK3kZltAKA+p9hFwK2ZD0GjcSKwCS0i9kp6IfASkkV8vq1kta8HRntqAcevS9905wNnFBjSWJuGzgBuTmcV/YWkL0hqA14P3JZORZH7/FcD52f6M0gSSjtJzeXv0n6Q70XEEwXGbBNLFdCZ+RsaxkUktdeCL2g2oUVEb0TcFREfBq4geRP9I9AuqTmn+AuBNSSzzc7IOTcT2J61/76IOAG4GrghPbYmvcbRiv1eksnv2tJDNwOXkDRr3TDM0wS8PtMpGBHtEfH7iPgmcD7QBdwhaaxNYDYBpLXcJyW9EfqXCu1fdlPSc0n+tu8d5hJDOBHYhCbpOTlt4acC6yNiH0kTziclVadl3wo0kjSr7AWezrxZSppJ0gT0yzwv8zmgKu3Y/RzwtuwOZ0n/Ne1EPpz4n0vShJOZ+O5G4G8BIuLR9NgeBjqjIZmo7kqlVQVJy9Kvi4B1EfEZ4AfACw4nJhtf0knq7gWeI6lD0mXAm4HLJD1E8uEke3W7i4BbxjJBoSedswktbRb6LNAC9JDMgHl5RGyXVA/8M8nMmH3AH4C/iYiN6XNPJBl5kakZXBcR30jP3QjcHhG3pvuvT5/7irQ555+A2el17yZp238TyWyx2XPx/01E3JMTcy/wu8wu8MHIWtlN0o+B72dmokyT1J0kM1R+HFhJsnjJmSQf5p6MiNelTWJvAQ4BzwB/ERGjLYlo5kRgNp6ko51+RzIcdle547HK4KYhs3FCyUp0vwc+6yRgpeQagZlZhXONwMyswjkRmJlVOCcCM7MK50RgZlbhnAjMzCrc/wcnFGujlh1TrAAAAABJRU5ErkJggg==\n"},"metadata":{"needs_background":"light"}}],"source":["# Make sbytes dbytes plot\n","x = UNSWNB15.select(\"sbytes\").rdd.flatMap(list).collect()\n","y = UNSWNB15.select(\"dbytes\").rdd.flatMap(list).collect()\n","# Title, Label axes, set margin, and show plot\n","plt.title(\"Sample Visualization\")\n","plt.xlabel('SOURCE Bytes')\n","plt.ylabel('Destination Bytes')\n","# Plot the data with Matplotlib defaults\n","plt.plot(x, y)\n","plt.show()"]},{"cell_type":"code","source":["# Make Service Dpkts plot\n","#rng = np.random.RandomState(0)\n","Service = UNSWNB15.select(\"service\").rdd.flatMap(list).collect()\n","Dpkts = UNSWNB15.select(\"Dpkts\").rdd.flatMap(list).collect()\n","Spkts = UNSWNB15.select(\"Spkts\").rdd.flatMap(list).collect()\n","# Title, Label axes, set margin, and show plot\n","plt.title(\"Dpkts against Spkts\")\n","plt.xlabel('Service')\n","plt.ylabel('Dpkts & dbytes')\n","\n","# Plot the data with Matplotlib defaults\n","import matplotlib as mpl \n","#plt.plot(x, y) \n","plt.scatter(Service[::10], Dpkts[::10])\n","plt.grid(color = 'hotpink', linewidth = 0.5)\n","plt.scatter(Service[::10], Spkts[::10])\n","plt.show()"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":295},"id":"7elXYuo5aPk0","executionInfo":{"status":"ok","timestamp":1648707311023,"user_tz":-60,"elapsed":60735,"user":{"displayName":"Abdulwahab Abdulliameed","userId":"10732377996348789858"}},"outputId":"e27658bb-19f4-4962-be6b-e78aa8000eb7"},"execution_count":null,"outputs":[{"output_type":"display_data","data":{"text/plain":["<Figure size 432x288 with 1 Axes>"],"image/png":"iVBORw0KGgoAAAANSUhEUgAAAYsAAAEWCAYAAACXGLsWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZxcVZ3//9c7nQ5EloTtF0knEBQIoxi2wOCXRTRIkEHMsAQUFVwGB5kBRgcnjH5ZIko0KsuMG5vg1wUCKMagRlYRkCVsYTOLAUkCQbZEAiHr5/fHPZVUd6q6lq6lu/r9fDz60XVP3Xs/51Z31afuueeeo4jAzMysOwOaXQEzM+v9nCzMzKwkJwszMyvJycLMzEpysjAzs5KcLMzMrCQnC2sJkkLSzs2uR7kknSjp982uR3ckHSJpUbPrYb2Dk4U1naRnJa2Q9LqkpZLulfSvkmry/9kbE0lE/DQiDuvpfkodm6RBkr4taZGk5em1vrgGcZ1I+hknC+stPhwRWwA7AlOA/wKubG6VWsLZwFhgP2AL4BDg4WZWyPomJwvrVSJiWURMB44HTpK0O4CkqyX9QNIt6QzkD5J2LLQPSQdKWpi+/d6Vih9L36yPl7StpBnpLOZVSX8sdhYj6ZK0r79LekjSQXnPDZZ0jaTXJD0t6Uv537YlTZL0l1TfpyT9c95zJ0u6O2850tnUvFSv70pSem7ndLzLJL0s6bpUvtGxFTiEfYFfRsTzkXk2In6cF/dZSWen+r0m6UeSNi3yWpye1tsR+C0wPMVdLmm4pP0kzUqv1YuSvlNoP9ZHRYR//NPUH+BZ4NAC5c8Bp6bHVwOvAwcDmwCXAHfnrRvAzsDhwEJgv67P5S1fCPwAaE8/BwEqUrePA9sAA4EvAkuATdNzU4A/AFsBI4DZwKK8bY8DhpN9KTseeAPYPj13coH6zwCGAjsALwGHp+d+Dnw57WdT4MBix1ag/l9Jr+Pngfd0Pc702j8BjAS2Bu4BLkjPHZI7HuAcsjOS7bo+l7evPwGfSI83B/Zv9v+Wf2r34zML682eJ/sAy7k5Iu6KiJVkH57vlTQy7/njgB8CH4qIB7rZ72pge2DHiFgdEX+M9AnXVUT8JCJeiYg1EfFtskQ1Oj09Efh6RLwWEYuAS7tse31k3+jXRcR1wDyy5qBipkTE0oh4DrgD2DOvvjsCwyPirYi4u+geNnYh8A3gRGAWsFjSSV3W+d+IWBgRrwJfAz6a95zSGcJhwPsj4qVuYq0Gdpa0bUQsj4j7Kqin9XJOFtabdQCv5i0vzD2IiOXpueF5z58JTIuIJ0rsdyowH/i9pAWSJhVbUdJ/piamZZKWAkOAbdPTw/Pr1OUxkj4p6dHUrLQU2D1v20KW5D1+k+zbOcCXAAEPSHpS0qdLHN96EbE2Ir4bEQeQnbV8DbhK0j8Uqfdf6fyaDgVOAS6MiGUlwn0G2BX4s6QHJR1Zbj2t93OysF5J0r5kySL/W/TIvOc3JzvreD7v+eOACZLO6G7fEfF6RHwxIt4BHAV8QdK4AnU4iOyDeiKwVUQMBZaRfXADvEDW/FSofjsClwP/BmyTtn0ib9uyRcSSiPiXiBgOfA74XjW9uyJiRUR8F3gNeFehepM1geW/pq8BRwI/knRA/u4K7H9eRHwU+P/IzmZukLRZpfW03snJwnoVSVumb6TXAj+JiMfznj4iXbweBHwVuC8i8r8VPw+MA86QdGpe+YvAO/JiHJkuGovsw38tsK5AdbYA1pBdPxgo6Rxgy7znpwFnS9pKUgdZYsjZjOwD9aUU81NkZxYVk3ScpFxSei3tN1ffTsdWYNsz04X+wZIGpiaoLYBH8lY7TdIISVuTNe9dl7+PiLiTrBnrF5JyzWgvAttIGpIX6+OStouIdcDSVFzodbU+yMnCeotfS3qdrEnky8B3gE91WednwLlkzU/7kF187iS1948DJkn6bCo+D7gmNQdNBHYBbgWWk12U/V5E3FGgTjOB3wFzyZpn3qJzk81kYBHwTNrfDcDKVI+ngG+n/b9IdnH5nvJeio3sC9wvaTkwHTgjIhYUObau3kz1WAK8DJwGHJO3PWSv6++BBcBfgAu67iQibgE+TfZ32jsi/kx24X1Bij2crHPBk6melwAnRMSKKo/ZehkVua5n1qtIupqs981Xml2XYtLZzAkR8b5m16Vckp4FPhsRtza7Lta7+czCrEqStpd0gKQBkkaTda39ZbPrZVYPA5tdAbM+bBBZV92dyNrorwW+19QamdWJm6HMzKwkN0OZmVlJLdkMte2228aoUaOq38Ebq2CzQTWrT0vHaFScVonRqDitEqNRcXwsADz00EMvR8R2BZ9s9ngj9fjZZ599okd+Padn2/enGI2K0yoxGhWnVWI0Ko6PJSIigFnhsaHMzKxaThZmZlaSk4WZmZXkZGFmZiU5WZiZWUlOFla92dPgot3hzinZ79nTml0jM6sTJwurzuxp8OvTYVkahHXZwmzZCcOsJTlZWHVumwyru4w+vXpFVm5mLcfJwqqzbFFl5WbWpzlZWHWGjKis3Mz6NCcLq864c6B9cOey9sFZuZm1nJYcSNAaYEyawfO2ydnkpENGZoliTKGZPc2sr3OysOqNmZj9zJgLR+7a7NqYWR25GcrMzEpysjAzs5KcLMzMrCQnCzMzK8nJwszMSnKyMDOzkpwszMysJCcLMzMrycnCzMxKcrIwM7OSnCzMzKwkJwszMyuprslC0n9IelLSE5J+LmlTSTtJul/SfEnXSRqU1t0kLc9Pz4/K28/ZqXyOpPF1q7DnlDYzK6huyUJSB3A6MDYidgfagBOAbwAXRcTOwGvAZ9ImnwFeS+UXpfWQ9K603buBw4HvSWqreYU9p7SZWVH1boYaCAyWNBB4G/AC8AHghvT8NcCE9PgjaZn0/DhJSuXXRsTKiHgGmA/sV/Oaek5pM7Oi6jafRUQslvQt4DlgBfB74CFgaUSsSastAjrS4w5gYdp2jaRlwDap/L68Xedvs56kU4BTAHYY1pHNsVCJxfuxPget+gdYfkz2eDmV76scc1+pz34bHaNRcVolRqPitEqMRsXxsZRUt2QhaSuys4KdgKXA9WTNSHUREZcBlwGMHTs2Kp6MZ94DG5qglh8Dm9+YPR4ysj4T+zRiwqBGTUrUKsfi16v3xWhUHB9LSfVshjoUeCYiXoqI1cAvgAOAoalZCmAEsDg9XgyMBEjPDwFeyS8vsE3teE5pM7Oi6pksngP2l/S2dO1hHPAUcAdwbFrnJOBX6fH0tEx6/vaIiFR+QuottROwC/BAzWs7ZiJ8+NLsTAKy3x++1HNKm5lR32sW90u6AXgYWAM8QtZMdDNwraQLUtmVaZMrgf8naT7wKlkPKCLiSUnTyBLNGuC0iFhbl0p7TunKzJ6WdQBYvF/WjDfuHCdXsxZVt2QBEBHnAud2KV5Agd5MEfEWcFyR/XwN+FrNK2jVy3U1Xr0C2G9DV2NwwjBrQb6D26rjrsZm/YqThVVn2aLKys2sT3OysOoMGVFZuZn1aU4WVh13NTbrV+p6gdtaWO4i9m2Ts7vch4x0byizFuZkYdVzV2OzfsPNUFY9D+lu1m84WVh1PKS7Wb/iZGHV8X0WZv2Kk4VVx/dZmPUrThZWHd9nYdavOFlYdXyfhVm/4q6zVh3fZ2HWrzhZWPV8n4VZv+FmKDMzK8nJwszMSnKyMDOzkpwsrHoe7sOs33CysOp4uA+zfsXJwqrj4T7M+hUnC6uOh/sw61ecLKw6Hu7DrF9xsrDqeLgPs37Fd3BbdTzch1m/4mRh1fNwH2b9hpuhzMysJCcLMzMrycnCzMxKcrIwM7OSnCzMzKwkJwszMyvJycLMzEpysjAzs5KcLMzMrCQnCzMzK6muyULSUEk3SPqzpKclvVfS1pJukTQv/d4qrStJl0qaL2m2pL3z9nNSWn+epJPqWWczM9tYRclC0laSxlSwySXA7yJiN2AP4GlgEnBbROwC3JaWAT4E7JJ+TgG+n2JuDZwL/COwH3BuLsFYk3laVbN+o2SykHSnpC3Th/bDwOWSvlPGdkOAg4ErASJiVUQsBT4CXJNWuwaYkB5/BPhxZO4DhkraHhgP3BIRr0bEa8AtwOEVHaXVnqdVNetXyhl1dkhE/F3SZ8k+zM+VNLuM7XYCXgJ+JGkP4CHgDGBYRLyQ1lkCDEuPO4CFedsvSmXFyjuRdArZGQk7DOvIRkKt1txXerZ9f4hx373w1hHZ41X/AMuPyR5Puxee27P28fr669XoOK0So1FxfCwllZMsBqZv+BOBL1e4772Bf4+I+yVdwoYmJwAiIiRFBfssKiIuAy4DGDt2bPRoyOxGDLnd12PMuho2T3+65cfA5jemJwRHXlz7eH399Wp0nFaJ0ag4PpaSyrlmMRmYCfwlIh6U9A5gXhnbLQIWRcT9afkGsuTxYko+pN9/S88vBkbmbT8ilRUrt2bytKpm/UrJZBER10fEmIg4NS0viIhjythuCbBQ0uhUNA54CpgO5Ho0nQT8Kj2eDnwy9YraH1iWmqtmAoeli+tbAYelMmsmT6tq1q+UbIaStCtZz6RhEbF76g11VERcUMb+/x34qaRBwALgU2QJapqkzwB/JWveAvgNcAQwH3gzrUtEvCrpq8CDab3JEfFquQdodeJpVc36lXKuWVwOnAX8ECAiZkv6GVAyWUTEo8DYAk+NK7BuAKcV2c9VwFVl1NXMzOqgnGsWb4uIB7qUralHZawPcddZs36lnGTxsqR3AgEg6Vjghe43sZZ322RYvaJz2eoVWbmZtZxymqFOI+uSupukxcAzwIl1rZX1fssWVVZuZn1aOckiIuJQSZsBAyLidUk71bti1ssNGbGhCapruZm1nHKaoW4EiIg3IuL1VHZD/arURDO+AOdvnY11dP7W2bIV5q6zZv1K0TMLSbsB7waGSDo676ktgU3rXbGGm/EFmHXlhuVYu2H5yJJDYfU/7jpr1q901ww1GjgSGAp8OK/8deBf6lmppnjoR8XLnSwKGzMx+2nUUAlm1jRFk0VE/Ar4laSDIuKPDaxTc8S6ysrNzPqRcq5ZXCXpeklHSFLda2Q9dtMjizlgyu1cfOs8DphyOzc94qG0zKxnykkWu5J1nf0EME/S19MQIK2lfbPKynupmx5ZzNm/eJzFS7N7IBYvXcHZv3jcCcPMeqScgQQjIm6JiI+SXas4CXhA0h8kvbfuNWyUD18MautcprasvA+ZOnMOK1av7VS2YvVaps6c06QamVkrKGcgwW2Aj5OdWbxINjjgdGBP4HqySY76vhbp3fP80hUVlZuZlaOcm/L+BPw/YEJE5N+eO0vSD+pTrSZpgd49w4cOXt8E1bXczKxa5VyzGB0RXwX+LmmL/Cci4hv1qZZV66zxoxnc3rk5bXB7G2eNH11kCzOz0so5s9hH0o+ALQBJWgp8OiIeqm/VrBoT9sqmJ586cw4sWUHH0MGcNX70+nIzs2qUkyyuAj6fu9dC0oHAj4Ax9ayYVW/CXh1ZcpgxlzP7aHOamfUu5TRDrc2/KS8i7sbzWZiZ9SvdjQ21d3r4B0k/BH5ONqfF8cCd9a+amZn1Ft01Q327y/K5eY+jDnUxM7Neqruxod7fyIqYmVnv1V0zVLeTOUSEh2I1M+snumuGyt1TMRrYl+yubciGK3+gnpUyM7PepbtmqPMBJN0F7J2bJU/SecDNDamdmZn1CuV0nR0GrMpbXpXKzMysnyjnprwfk40y+8u0PAG4um41MjOzXqdksoiIr0n6LXBQKvpURDxS32o1x02PLGbqzDkct2QF1z+xyMNkmJkl5ZxZEBEPAw/XuS5NlZs0KDcXRG7SIMAJw8z6vXKuWfQLnjSoCrOnwUW7w51Tst+zpzW7RmZWJ04WiScNqtDsafDr02HZwmx52cJs2QnDrCWVnSzy57KQtHN9qtM8xSYH8qRBRdw2GVZ3SaSrV2TlZtZyKjmzuFvSTZImAjPrVaFm8aRBFVq2qLJyM+vTuhvu423AqohYAxARe0g6lWz02RMaVL+G8aRBFRoyYkMTVNdyM2s53Z1Z3A5sm1uQ9M/AqcB44OT6Vqs5JuzVwT2TPsCZh+7CPZM+4ETRnXHnQHuXJrr2wVm5mbWc7pLF4IhYAiDpFOC/gXERcSu+g9vGTIQ9PgZKTXdqy5bHTGxuvcysLrq7z+IVSecCI4F/BnaOiNckbQ8MakjtrPeaPQ0e+xlE6m4ca7PlHfZ3wjBrQd2dWRwHrAXmAqcAv5d0FXAvMKUBdbPezL2hzPqVoskiIl6JiAsi4psRcSPwEeC3wOER8bNyA0hqk/SIpBlpeSdJ90uaL+k6SYNS+SZpeX56flTePs5O5XMkja/yWK2W3BvKrF8pu+tsRDwfEddHRKW3NJ8BPJ23/A3goojYGXgN+Ewq/wzwWiq/KK2HpHeR9b56N3A48D1Jnfu4WuMV6/Xk3lBmLamud3BLGgH8E3BFWhbwAeCGtMo1ZKPYQnbmck16fAMwLq3/EeDaiFgZEc8A84H96llvK4N7Q5n1K2UNJNgDFwNfYsOse9sAS3P3bgCLgFz/1A5gIUBErJG0LK3fAdyXt8/8bdZLPbZOAdhhWAfMmFt9ree+0rPt+0WMPeHt34dn/gCrOmDNp2Dk++C5d8NzdYjZ51+vBsdplRiNiuNjKalkspC0GbAiItZJ2hXYDfhtRKwusd2RwN8i4iFJh9Sktt2IiMuAywDGjh0bHLlr9TubMZcebd9vYuwK/HOLHEuDYjQqTqvEaFQcH0tJ5TRD3QVsKqkD+D3wCcqb/OgA4ChJzwLXkjU/XQIMlZRLUiOAxenxYrJuuqTnhwCv5JcX2KambnpkMQdMuZ2Lb53HAVNu56ZH6hLGzKzPKSdZKCLeBI4GvhcRx5FdbO5WRJwdESMiYhTZBerbI+JE4A7g2LTaScCv0uPpaZn0/O0REan8hNRbaidgF+CBso6uArn5LBanUWZz81k4YZiZlZksJL0XOBG4OZX1pDfSfwFfkDSf7JrElan8SmCbVP4FYBJARDwJTAOeAn4HnBYRazfaaw95Pgszs+LKucB9BnA28MuIeFLSO8jODsoWEXcCd6bHCyjQmyki3iK7EbDQ9l8DvlZJzEp5PosqzJ6W3YS3eD+Y90DWE8p3b5u1pHKSxbCIOCq3EBELJP2xjnVqiuFDB69vgupabgXkJj9avQLYb8PkR+CEYdaCymmGOrvMsj7N81lUyMN9mPUr3c1n8SHgCKBD0qV5T20JrCm8Vd/l+Swq5OE+zPqV7pqhngdmAUcBD+WVvw78Rz0r1SwT9urIksOMuZzZiD7XfZknPzLrV4omi4h4DHhM0lMR8WD+c+mGuxn1rpz1YuPOybtmkXi4D7OWVc41ix9K2j23IOmjwP+tX5WsTxgzET58KQxJ90sOGZkt++K2WUsqpzfUscANkj4GHAR8EjisrrWyvmHMxOynUUMlmFnTlEwWqavsCcBNwHPAYRHhmw/MzPqR7npDPQ5EXtHWZHdu3y+JiBhT78qZmVnv0N2ZxZENq4WZmfVq3fWG+mvusaS9gQPJzjTuiYiHG1A3MzPrJUr2hpJ0DtkMdtsA2wI/kvSVelfMzMx6j3J6Q50I7JEG+kPSFOBR4IJ6VszMzHqPcu6zeB7YNG95E+o0+ZCZmfVO5ZxZLAOelHQL2TWLDwIP5MaLiojT61g/MzPrBcpJFr9MPzl31qcq1ud4PguzfqOcm/KukbRdevxS/atkfYLnszDrV4pes1DmPEkvA3OAuZJeSr2jrL/zfBZm/Up3F7j/AzgA2Dcito6IrYB/BA6Q1JJDlLeKmx5ZzAFTbufiW+dxwJTbuemROvRH8HwWZv1Kd8niE8BHI+KZXEGaP/vjZIMJWi900yOLOfsXj6+fInbx0hWc/YvHa58wis1b4fkszFpSd8miPSJe7lqYrlu0169K1hNTZ85hxeq1ncpWrF6bzQBYS7sUGXi4WLmZ9WndJYtVVT7XZzWk+abOnl9aeEDgYuVVm31tZeVm1qd11xtqD0l/L1AuOt+k1xJyzTe5b+W55hugT83DPXzo4PVNUF3La2rVG5WVm1mfVvTMIiLaImLLAj9bRETLNUM1rPmmzt6/23YVlZuZlaOc4T76hYY139TZHX8ufCtMsfLqqcJyM+vLnCySYs00NW++qbOGJb2xn66s3Mz6NCeL5Kzxoxnc3tapbHB7G2eNH92kGlWnYUlvh/1BnV8v1JaVm1nLcbJIJuzVwYVHv4eO9KHaMXQwFx79nj51cRsamPRumwzR+RoPsdZ3cJu1qHIGEuw3JuzVkSWHGXM588hdm12dquSS29SZc2DJCjqGDuas8aNrn/R8B7dZv+Jk0YIakvSGjMgGDyxUbmYtx81QVp1x50B7l+sg7YOzcjNrOT6zsOrkhiG/bTIsB4aM9HwWZi3MycKqN2Zi9jNjLvTRazxmVh43Q5mZWUlOFmZmVlLdkoWkkZLukPSUpCclnZHKt5Z0i6R56fdWqVySLpU0X9JsSXvn7euktP48SSfVq86tohVGzzWz3qWeZxZrgC9GxLuA/YHTJL0LmATcFhG7ALelZYAPAbukn1OA70OWXIBzyWbp2w84N5dgbGMNm/zIzPqVuiWLiHghIh5Oj18HngY6gI8A16TVrgEmpMcfAX4cmfuAoZK2B8YDt0TEqxHxGnALcHi96t3XtcrouWbWuzSkN5SkUcBewP3AsIh4IT21BBiWHncA+Xd5LUplxcq7xjiF7IyEHYZlN6RVbe4rPdu+iTGOW7JhwMD918CZb6WFJSvqd0x9+PVqeIxGxWmVGI2K42Mpqe7JQtLmwI3AmRHxd2nDENYREZKiFnEi4jLgMoCxY8dGj7pyNqIraJ1iXP/EovVNUGe+BRenaao6hg6u393cffj1aniMRsVplRiNiuNjKamuvaEktZMlip9GxC9S8YupeYn0+2+pfDEwMm/zEamsWLkVcNb40Rw76F7uHnQ6Zwy8kbsHnc6xg+7tc6PnmlnvUs/eUAKuBJ6OiO/kPTUdyPVoOgn4VV75J1OvqP2BZam5aiZwmKSt0oXtw1KZFTCh7R6mtF/BiAEvI2DEgJeZ0n4FE9ruaXbVzKwPq+eZxQHAJ4APSHo0/RwBTAE+KGkecGhaBvgNsACYD1wOfB4gIl4Fvgo8mH4mpzIr5LbJDFz7VqeigWvf8tDhZtYjdbtmERF3U3yOzXEF1g/gtCL7ugq4qna1a12xbFHBF71YuZlZOXwHd4t5kW0rKrfW4hsyrV6cLFrMzDV7EF36l0Vk5dbafEOm1ZOTRYs5tO1R1KW9ScrKrbX5hkyrJyeLFrM9Lxcpf6XBNbFGe37piorKzSrhZNFi/qbtipTX4ZrF7Glw0e5w55Ts9+xptY9hZRs+dHBF5WaVcLJoMXftcCpvxqBOZW/GIO7a4dTaBpo9DX59+oZ5uJctzJadMJrmrPGjaW/r3AbZ3ibfkGk14WTRYi75215MWv1ZFq3blgAWrduWSas/yyV/26u2gW6bDKu7NG+sXuH7OZqt6+A5NRlMx8zTqnYyatLNQBpT6e55ADw75Z+aWaWKPb90BYs5kOmrDuTMNXDxqmMAUI3brX0/R+8zdeYcVq/rnB1WrwumzpzDhL02GnvTrCI+s0hyiaLc8t6qUe3Wvp+j9/EFbqsnJ4sWc9b40Qxub+tUNri9rebt1heuOq7gtZELVx1X0zhWPl/gtnpysshz1IC7O43WetSAu5tdpYpN2KuDC49+Dx3pA6Jj6GAuPPo9NW+GmLXlBwteG5m15QdrGsfK16gvCtY/+ZpFctSAu5nSfgVv0ypgw2itrAboW9ctJuzVkSWHGXPrNofF+3fbjp/ct/G1kY/vVrjrrtVf7gvB1JlzYMkKOoYO5qzxo329wmrCZxbJlwZOW58oct6mVXxpoLuCFjLjsRcKnonNeOyF0htb3UzYq4N7Jn2AMw/dhXsmfcCJwmrGySIZrsJ3Phcr780aMZjcwSvvKDhvxsEr76h5LDNrPieLZF2Rl6JYeW/VqMHkfCZm1r/0rU/COmpjXUXlvVWjBpNrpTMxMyvNySJZHIXvDyhW3ls1qq99q5yJmVl5/M5OvrlmIiujc7fDldHGN9dMbFKNqtOovvatciZmZuVxssgzkLXdLvcFZ40fzbGD7u3US+nYQffWvK/9q7F5ReVm1rc5WSRfb7+SLgN20qasvC+Z0HYPk3VZp15Kk3UZE9ruqWmcrhMslSo3s77NySLZjJUVlfdWi244u2AvpUU3nF3TOENZXqT8jZrGMbPewXdwt5jivZRqO1PeEm3L8AKz8i3RNgyvaaT6a4XRhnNuemQxU2fO4bglK7j+iUW+g9tqxsmixTwf2zKiQMJ4PrZhRA3jiLVEdG52ypb71nWeUZNuZsGgj2XHseYYztjkRiJg1KSf9bmEkbvHJtd1OnePDeCEYT3mZqgW8801EwuOBlvrXl1vj9c2uj4hZeV9SS5RSCBY/3jBoI81u2oVa9Q9NtY/OVm0mOnrDiw4Guz0dQc2u2q9Ui45lCrrCxYXuZemWLlZJdwM1YKmr9t4NFhrfW0Sa2PjeVTb+mLms17HZxYt6PyBVzF/k49zxsAbmb/Jxzl/4FXNrpI1QKFE0V25WSV8ZtFizh94FZ9su3V9M8pAreOTbbemZ/vWBVtw755KbDaojTdWbdzBYLNBbQXWNquMk0WLObHt9oJt8Ce23d6cCvWAe/dUplCi6K7c/GWkEm6GajGtNGaTe/dYPTVqOP9W4WRhvVajRtBtJa0wj3yj+MtIZdwM1WAPTv8hIx+eyrDlB/PirLtYuPdZ7HvU55pdrV5pcPsA3ly98RnR4HZ/xynkqAF38632yxikNUA2Lti32i/rk/PIN4K/jFTG77oGenD6D9n9oa/wdl5CwNt5id0f+goPTv9hs6tWuWK9MWvYS3PFmsJNZ8XK+7vz2n+8PlHkDNIazmv/cZNq1Ls1ajj/VuFk0UAjH57K4C6D/A3WKkY+PLVJNeqBYr0xa9hLs1iPT/cELWyrIoM7Fivv784aP5rB7Z17ig1ub6v5cP6tws1QDTQsXir4zZrAZfMAAA2hSURBVHtYvNT4yvQBbRL/pD/ypYHT6FjzPo4d9Ae+uWYiN8dBza5adS7cAVYug+XHwKwbYZMhcPZzza5VvzVhrw46Fs7ImoXXHMwJm6Zm4b0Ob3bVeiWfWTRSA5puWsmFuzzNd9q/32luju+0f58Ld3m62VWr3IU7EG8t61QUby3LEog1x+xp7PPwpE7Nwvs8PAlmT2t2zXqlPnNmIelw4BKgDbgiIqY0uUpWZ0f89RsMVOc2p4EKjvjrN4AvNqdSVYq3lhW8/yXeWtbnviusOWdINlHYG8fAgzeyNmDg5GUlt6vU2nOGMCAvzrqAthrGWXvTaRt1KR/Auqx8TG0H3vzLebvzjli4/lgWaCTvPO+Jmsaod+eZPnFmIakN+C7wIeBdwEclvau5tapCA9r5W0mrTEjVSnKJIn+U3jZl5bWUSxT5cQYoK6+VAWtXVVRerVyiyD+Wd8RC/nLe7jWL0YjOM30iWQD7AfMjYkFErAKuBT7S5DqZ9Tu5RJEvlzBqaUCROAP62mkYrE8U+XIJo1Ya0XmmrzRDdQD5r+wi4B/zV5B0CnAKwA7DOmDG3MoivJE3Ouuqf8guQuZUuq9Wj9GoOK0So1FxWiVGo+K00LEMW35wwRjDahijrySLkiLiMuAygLFjxwZH7lrZDh68ccOF5uXHwOY3ph0DR9Zo1NZWidGoOK0So0Fx4oEshrQhRkSawdCvV3NiNCjOi7Pu4u28tFGMF9mOt9coRl9phloMjMxbHpHKaib3pipV5hiNi9MqMRoVZ8DkZev3GXn7H1DLi8JFjmNtjV+vdUXirKthnJUxoGCMlVHbj8UFGlkwzgKNLLxBFRbufRYrusyQuSIGsXDvs2oWo68kiweBXSTtJGkQcAIwvZYBGvFGa5UYjYrTKjEaHUfnL0OHTELnL6v5/gdOXrY+YeSOox69odomL1ufMHJxat0batPJr61PGLkYK2MAm06u7dTA7zzvifUJIxen1r2h9j3qczyxzwUsYTsCWMJ2PLHPBbUdSigi+sQPcAQwF/gL8OXu1t1nn32iR349p2fb96cYjYrTKjEaFadVYjQqjo8lIiKAWVHkc7XPXLOIiN8Av2l2PczM+qO+0gxlZmZN5GRhZmYlOVmYmVlJThZmZlaSotYdynsBSS8Bf+3BLrYFXq5RdVo9RqPitEqMRsVplRiNiuNjyewYEdsVeqIlk0VPSZoVEWMdo/fEaZUYjYrTKjEaFcfHUpqboczMrCQnCzMzK8nJorDLHKPXxWmVGI2K0yoxGhXHx1KCr1mYmVlJPrMwM7OSnCzMzKwkJ4sGkHSepP+s8T5HSdpojGNJJ0sanrd8pqS31TJ2iTp9LG/5dElPS3qjp3OmSzpE0owS6+wp6YiexMnb19WSju3hPmp2/LUk6V8lPS7pUUl3N7tuufeHpDsl1b37ajl6+p6VdG8t61NmzPXvEUlHSZpUy/07WbSek4HhectnAg1JFsAo4GN5y58HPghcDzTiA2lPsqHse4tGH3+5fhYR74mIPYFvAt9pdoVaTUT8n65lkqoa5VuZij6rI2J6REypJl4xThZ1IunLkuZKuhsYncrulPQNSQ+k5w5K5e9OZY9Kmi1plzLDtEm6XNKTkn4v6RPAWOCnaV9nkCWOOyTdkWItl3RR2uY2SRvdrSlpM0k3S3pM0hOSjpf0rKQL035nSdpb0kxJf5H0r2nTKcBBaZ17gJ2B54CTgJ9LekHSO9PrcEla7wlJ+xV5DQ+X9GdJDwNH55XvJ+lPkh6RdK+k0WlSrMnA8Wm/xxdZr9CxTZH0VHrtv5VXhYPTdgsqPcuQ9APgHcAz6finpnqVffwF9jkqvR4/TWcsN0h6m6Rx6Rgfl3SVpE3S+s9K+mYqf0DSzgAR8fe83W4GRDd/8/MlPZz2sVva73mSrpH0R0l/lXR0XpzfSWrvLn56bqP3R3JcgfdHm6RvpXrNlvTvpfbfzWtYyd+/apKWp9+HpNdpOvBUsWMpsP0oSXMk/Rh4ArhS2fvuSUnn561X7D1ysqT/TY87nSXn1W17SXfl/R8e1O1BFZvowj89mqhpH+Bxsm/0WwLzgf8E7gS+HRsmc7o1Pf4f4MT0eBAwuIwYo4A1wJ5peRrw8RRjbN56zwLb5i1HXqxzgP8tsO9jgMvzloek/Zyali8CZgNbANsBL6byQ4AZedu9DCwBfk52xvEEWTK7M7d/4GDgiQJ12BRYCOxCNoPxtNy+02s6MD0+FLgxPT45/3gKrVfg2HYE5rChZ+DQ9PtqsjOCAWRnBfOr+D94lmzohauBY/PKSx5/N3/zAA5Iy1cBX0mv066p7MfAmXnxv5wef7LL3+Y0sonEcq9xsb/5v6flzwNXpMfnAXcD7cAewJvAh9JzvwQmdBefyt8fpwI35P0tty51fN28huX+/c8D/rMHnwHL894TbwA7dXcsRf7W64D9uxxzW3qdxtD9e+Rk0nuBjf//cnX7Yt7r1wZs0d0x+cyiPg4CfhkRb0b2LS5/CthfpN8Pkf1DAPwJ+G9J/0U2NsuKMuM8ExGPFthfd9YB16XHPwEOLLDO48AHlZ0FHRQRubksp+c9f39EvB4RLwErJQ0tEu8PwEpgFdmx5+L9HCAi7gK2LLD9bun45kX23/yTvOeGANcru2ZzEfDuIrELrdfp2Mjmcn+L7Jvb0WQffDk3RcS6iHgKGFYkRrVKHX8xCyPinvT4J8A4stdpbiq7hiwBdYqTfr83VxgR342IdwL/RZZwiv3NC/2/Avw2Ilan7dqA36Xyx7usVyh+pe+PQ4EfRsSaVPdXSx1fNyr5+9fKAxHxTHrc3bF09deIuC89npjOHh4h+z9+F92/R8rxIPApSecB74mI17tb2ckij6TT0inZo8q7SFxjK9PvtZDNVBgRPwOOAlYAv5H0gQr31Wl/FdroRpv0wbM32RvrAknndIm3rkvsdd3E7rr/KFaurFnrUUlXlKjzV4E7ImJ34MNk37DKWq/rsQH/DexH9m3vSDZ86EHnY1SJOlWq2OtS6XZLK1i/UIxryc4ESv3Nu/5/rQSIiHXA6vRhBRv/L5SK31WxeMVUtP8K//618kZPtpO0E9mZ17iIGAPcTPH/+ULWkD7rlV37GATrv6gcTJYwr5b0ye524mSRJ33b2jP9PN+DXd0FTJA0WNIWZB9URUl6B7AgIi4FfkV2ilmt18mah4otDwBy7ZcfI2tO6Fqf4cCbEfETYCrZm6ua2JCdhq8CtgYmALlvxcenWAcCyyJiWUSMT6/9Z4E/A6MkvTOt/9G8fQ4h+weH7HS7WPyN1itwbAcDQyKbtvc/yJpVaq3Q67LR8Ze5rx0k5b5BfwyYRfY65drrP0F2NtcpTvr9pxQz/5rYPwHzevA3L2Wj+FT4/gBuAT6ndIFY0tYl9l9Uk/7++bo7lmK2JEscyyQNAz6Uyrt7j+R7lqzpD7IvpblrSjuSNSFfDlxBib95n5mDuy+JiIclXQc8BvyN7HSvOxOBT0haTdbG//UehL8a+IGkFWSn5ZcBv5P0fES8n+yfbj9JX0l1O77APt5DdkF2HbCaDe2spcwG1kp6LNUD4GGyf9STgFeB11L5W5IeIfvH/XTXHUXEW5JOAW6W9CbwRzZ84H4TuCYdw815m90BTJL0KHBhkfW6HtsXgBmSNiU7e/hCGcdZqWuByyWdzoZE3e3xd2MOcJqkq4CngNOB+8ia2waS/a/9IG/9rSTNJvvGnvsw+TdJh5Id/2tkf5tq/+albBS/ivfHFcCuwOz0Hrkc+N9ujq87zfj75+vuWAqKiMfS/8qfya5R3JPKu3uP5Lsc+FV6X/6ODWc6hwBnpXosJ7vuU5SH++hnJC2PiM0bFOtksovt/9al/E6yi4ezGlGP3qba45c0iuwC5u5lrv8s2evfiHkaGh6/2cfX37gZyszMSvKZhZmZleQzCzMzK8nJwszMSnKyMDOzkpwszCqgbEyjJ9O4Po9K+sca7HNy6spq1mv5ArdZmdLNcN8BDomIlZK2BQaVcwOnpIG5IR7M+iKfWZiVb3vg5YjIDXXxckQ8L2kfSX+Q9FAasmR7WD/K8MWSZgFfVjZCa27Yhc0kLZTUrrxRQSXtq2yk28eUjaS6hbKRSqdKejCd0XyuWS+A9V9OFmbl+z0wUtnw2d+T9D5lw3H/D9monvuQjQT7tbxtBkXE2Ig4H3gUeF8qPxKYmQbjA0DZMOvXAWdExB5kg86tAD5DNiTIvsC+wL+k8YLMGsbDfZiVKSKWS9qHbNTU95N9sF8A7A7cIgmyEVhfyNvsui6PjycbluQE4HtdQowGXoiIB1O8vwNIOgwYow1zEgwhG5b6GcwaxMnCrAIRsZZsPoE7JT1ONi/EkxFRbHjs/BFHpwNfT4PH7QPcXmZYkc0rMbO6Wpv1nJuhzMqkbKa9/BFb9wSeBrbLjQSbrkEUnF8jIpaTDZp3CdkYT2u7rDIH2F7SvmlfW6TBAWcCp2rDDHS7StqslsdmVorPLMzKtznwP8omKlpDNsPbKWQj+14qaQjZe+pi4Mki+7iObAa+Q7o+ERGrJB2fYgwmu15xKNlIpaOAh5W1db1ENty7WcO466yZmZXkZigzMyvJycLMzEpysjAzs5KcLMzMrCQnCzMzK8nJwszMSnKyMDOzkv5/fLSoa3uanyQAAAAASUVORK5CYII=\n"},"metadata":{"needs_background":"light"}}]}],"metadata":{"colab":{"collapsed_sections":[],"name":"Task 3.1 -  Advanced Analytics using PySpark .ipynb","provenance":[],"authorship_tag":"ABX9TyNwNkUDTcrUDl55vna9NG1R"},"kernelspec":{"display_name":"Python 3","name":"python3"},"language_info":{"name":"python"}},"nbformat":4,"nbformat_minor":0}