Revert "Bug 1926041: Bump the fork to Hadoop 3.1.4"

Author: timflannagan

.gitignore

@@ -9,7 +9,6 @@
 *.suo
 *.vcxproj.user
 *.patch
-*.diff
 .idea
 .svn
 .classpath

BUILDING.txt

@@ -78,12 +78,12 @@ Optional packages:
   (OR https://github.com/01org/isa-l)
 * Bzip2
   $ sudo apt-get install bzip2 libbz2-dev
+* Jansson (C Library for JSON)
+  $ sudo apt-get install libjansson-dev
 * Linux FUSE
   $ sudo apt-get install fuse libfuse-dev
 * ZStandard compression
-  $ sudo apt-get install libzstd1-dev
-* PMDK library for storage class memory(SCM) as HDFS cache backend
-  Please refer to http://pmem.io/ and https://github.com/pmem/pmdk
+    $ sudo apt-get install zstd
 
 ----------------------------------------------------------------------------------
 Maven main modules:
@@ -248,32 +248,6 @@ Maven build goals:
    invoke, run 'mvn dependency-check:aggregate'. Note that this plugin
    requires maven 3.1.1 or greater.
 
- PMDK library build options:
-
-   The Persistent Memory Development Kit (PMDK), formerly known as NVML, is a growing
-   collection of libraries which have been developed for various use cases, tuned,
-   validated to production quality, and thoroughly documented. These libraries are built
-   on the Direct Access (DAX) feature available in both Linux and Windows, which allows
-   applications directly load/store access to persistent memory by memory-mapping files
-   on a persistent memory aware file system.
-
-   It is currently an optional component, meaning that Hadoop can be built without
-   this dependency. Please Note the library is used via dynamic module. For getting
-   more details please refer to the official sites:
-   http://pmem.io/ and https://github.com/pmem/pmdk.
-
-  * -Drequire.pmdk is used to build the project with PMDK libraries forcibly. With this
-    option provided, the build will fail if libpmem library is not found. If this option
-    is not given, the build will generate a version of Hadoop with libhadoop.so.
-    And storage class memory(SCM) backed HDFS cache is still supported without PMDK involved.
-    Because PMDK can bring better caching write/read performance, it is recommended to build
-    the project with this option if user plans to use SCM backed HDFS cache.
-  * -Dpmdk.lib is used to specify a nonstandard location for PMDK libraries if they are not
-    under /usr/lib or /usr/lib64.
-  * -Dbundle.pmdk is used to copy the specified libpmem libraries into the distribution tar
-    package. This option requires that -Dpmdk.lib is specified. With -Dbundle.pmdk provided,
-    the build will fail if -Dpmdk.lib is not specified.
-
 ----------------------------------------------------------------------------------
 Building components separately
 
@@ -364,50 +338,6 @@ export MAVEN_OPTS="-Xms256m -Xmx1536m"
 
 ----------------------------------------------------------------------------------
 
-Building on macOS (without Docker)
-
-----------------------------------------------------------------------------------
-Installing required dependencies for clean install of macOS 10.14:
-
-* Install Xcode Command Line Tools
-  $ xcode-select --install
-* Install Homebrew
-  $ /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
-* Install OpenJDK 8
-  $ brew tap AdoptOpenJDK/openjdk
-  $ brew cask install adoptopenjdk8
-* Install maven and tools
-  $ brew install maven autoconf automake cmake wget
-* Install native libraries, only openssl is required to compile native code,
-you may optionally install zlib, lz4, etc.
-  $ brew install openssl
-* Protocol Buffers 2.5.0 (required), since 2.5.0 is no longer in Homebrew,
-we need to compile it from source
-  $ wget https://github.com/protocolbuffers/protobuf/releases/download/v2.5.0/protobuf-2.5.0.tar.gz
-  $ tar zxvf protobuf-2.5.0.tar.gz
-  $ cd protobuf-2.5.0
-  $ ./configure
-  $ make
-  $ make check
-  $ make install
-  $ protoc --version
-
-Note that building Hadoop 3.1.1/3.1.2/3.2.0 native code from source is broken
-on macOS. For 3.1.1/3.1.2, you need to manually backport YARN-8622. For 3.2.0,
-you need to backport both YARN-8622 and YARN-9487 in order to build native code.
-
-----------------------------------------------------------------------------------
-Building command example:
-
-* Create binary distribution with native code but without documentation:
-  $ mvn package -Pdist,native -DskipTests -Dmaven.javadoc.skip \
-    -Dopenssl.prefix=/usr/local/opt/openssl
-
-Note that the command above manually specified the openssl library and include
-path. This is necessary at least for Homebrewed OpenSSL.
-
-----------------------------------------------------------------------------------
-
 Building on Windows
 
 ----------------------------------------------------------------------------------

Dockerfile

@@ -81,7 +81,7 @@ RUN chmod +x /usr/bin/tini
 
 ENV JAVA_HOME=/etc/alternatives/jre
 
-ENV HADOOP_VERSION 3.1.4
+ENV HADOOP_VERSION 3.1.1
 
 ENV HADOOP_HOME=/opt/hadoop
 ENV HADOOP_LOG_DIR=$HADOOP_HOME/logs
@@ -130,4 +130,4 @@ USER 1002
 LABEL io.k8s.display-name="OpenShift Hadoop" \
       io.k8s.description="This is an image used by operator-metering to to install and run HDFS." \
       io.openshift.tags="openshift" \
-      maintainer="<metering-team@redhat.com>"
+      maintainer="AOS Operator Metering <sd-operator-metering@redhat.com>"

Dockerfile.okd

@@ -78,7 +78,7 @@ RUN chmod +x /usr/bin/tini
 
 ENV JAVA_HOME=/etc/alternatives/jre
 
-ENV HADOOP_VERSION 3.1.4
+ENV HADOOP_VERSION 3.1.1
 
 ENV HADOOP_HOME=/opt/hadoop
 ENV HADOOP_LOG_DIR=$HADOOP_HOME/logs
@@ -126,4 +126,4 @@ USER 1002
 LABEL io.k8s.display-name="OpenShift Hadoop" \
       io.k8s.description="This is an image used by operator-metering to to install and run HDFS." \
       io.openshift.tags="openshift" \
-      maintainer="<metering-team@redhat.com>"
+      maintainer="AOS Operator Metering <sd-operator-metering@redhat.com>"

Dockerfile.rhel

@@ -32,7 +32,7 @@ RUN set -x; yum install --setopt=skip_missing_names_on_install=False -y \
 
 ENV JAVA_HOME=/etc/alternatives/jre
 
-ENV HADOOP_VERSION 3.1.4
+ENV HADOOP_VERSION 3.1.1
 
 ENV HADOOP_HOME=/opt/hadoop
 ENV HADOOP_LOG_DIR=$HADOOP_HOME/logs
@@ -80,4 +80,4 @@ USER 1002
 LABEL io.k8s.display-name="OpenShift Hadoop" \
       io.k8s.description="This is an image used by operator-metering to to install and run HDFS." \
       io.openshift.tags="openshift" \
-      maintainer="<metering-team@redhat.com>"
+      maintainer="AOS Operator Metering <sd-operator-metering@redhat.com>"

Jenkinsfile

@@ -553,7 +553,7 @@ For:
 hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/dataTables.bootstrap.js
 hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/dataTables.bootstrap.css
 hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/jquery.dataTables.min.js
-hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/dt-1.10.7/
+hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/dt-1.9.4/
 --------------------------------------------------------------------------------
 Copyright (C) 2008-2016, SpryMedia Ltd.
 
@@ -1767,7 +1767,7 @@ any resulting litigation.
 
 The binary distribution of this product bundles these dependencies under the
 following license:
-JSch 0.1.54
+JSch 0.1.51
 ParaNamer Core 2.3
 JLine 0.9.94
 leveldbjni-all 1.8

LICENSE.txt

@@ -8,10 +8,10 @@ following notices:
 * Copyright 2011 FuseSource Corp. http://fusesource.com
 
 The binary distribution of this product bundles binaries of
-AWS SDK for Java - Bundle 1.11.271,
-AWS Java SDK for AWS KMS 1.11.271,
-AWS Java SDK for Amazon S3 1.11.271,
-AWS Java SDK for AWS STS 1.11.271,
+AWS SDK for Java - Bundle 1.11.134,
+AWS Java SDK for AWS KMS 1.11.134,
+AWS Java SDK for Amazon S3 1.11.134,
+AWS Java SDK for AWS STS 1.11.134,
 JMES Path Query library 1.0,
 which has the following notices:
  * This software includes third party software subject to the following