Add Patch Adapter tool

patchAdapter/.gitignore

@@ -0,0 +1,6 @@
+*~
+target
+*.jar
+test/
+*originalclasses.out
+**/.DS_Store

patchAdapter/README.md

@@ -0,0 +1,59 @@
+# Patch Adapter
+  * a tool for adjusting software patches so that they can be used for hotfixing with a Java agent that uses the Instrumentation API
+  * this project depends on [Soot](https://github.com/soot-oss/soot)!
+
+
+## Motivation for a patch adapter:
+  * Not all changes in a patch are supported when performing hotfixing! Change types that are not supported will cause Java Agents to throw UnsupportedOperation Exceptions. If these are not handled properly it can result in JVM shutdown, and even if they are handled, it means that the patch simply cannot be used, until it is adjusted. This tool does that, automatically!
+  * currently follows [Instrumentation API](https://docs.oracle.com/en/java/javase/15/docs/api/java.instrument/java/lang/instrument/Instrumentation.html#redefineClasses(java.lang.instrument.ClassDefinition...)) constraints from `redefineClasses` method in JavaSE 15
+  * specific class file changes supported by `redefineClasses` described in [jvmti docs](https://docs.oracle.com/en/java/javase/15/docs/specs/jvmti.html#RedefineClasses)
+
+## How it works:
+  * the tool runs in two phases, where each invokes a new run of Soot
+    1) renames the set of original classes listed in x.originalclasses.out and places in to the directory specified by the `renamedOriginals` option
+    2) Soot loads both the renamed originals and the classes in the patch, then :
+	   * SemanticDiffer detects differences of interest between the versions
+	   * SemanticDiffer passes points of interest to PatchAdapter to transform the redefinition classes
+	   * a set of (possibly transformed) classes are emitted into the directory `adapterOutput`
+  * x.originalclasses.out must contain names of all classes to be analysed by the patch adapter, this includes any containing references to added/removed fields/methods, patch adapter does not currently support discovery of locations (in non-specified classes) that require adjustment
+  * patch class set must be located in directories, jar class loading is not currently supported
+
+## How to build:
+  * `mvn clean compile assembly:single` will build the application including dependencies
+
+
+## How to run:
+```
+java -cp $cp differ.SemanticDiffer -cp cpplaceholder -w -renameDestination tempDirectory -finalDestination adapterOutput -redefcp redefDir -runRename [true/false] -mainClass mainClass -fullDir [true/false] -originalclasslist x.originalclasses.out Example
+```
+  * where:
+     * $cp is the path to the SemanticDiffer package
+     * cpplaceholder contains the following (in the listed order):
+     	* originaldir: the directory of the original version of the class
+	* rt.jar: a path to the jvm's rt
+	* jce.jar: a path to the jvms crypto library
+	* the directory of the patch/redefined version of the relevant classes
+    * tempDirectory is the name that you would like to be used for the temporary directory that will contain a set of renamed classes that are output in the first invocation of Soot in the patch adapter
+    * adapterOutput is the name that you would like to be used for the directory that is used for the final output of the patch adapter
+    * redefDir is the directory of the patch/redefined version of the relevant classes
+    * runrename controls whether the first invocation of Soot is run, i.e., whether the renamed set of classes will be output to the tempDirectory. If this step has already been performed for some patch, and those files are still present, then runRename can safely be set to false
+    * mainClass is a (currently non) optional argument and can be used to set the "main class" that is used for the second invocation of Soot in the patch adapter. If the mainClass argument is omitted, then the first class in the originalclasses file will be used, however mainClass is currently used to set the package names when loading classes in the patch adapter specific run(s) of Soot. In the future a smarter approach would be to retain package names from x.originalclasses.out, or to have a loader discover classes in the patch directory, or to load from jars. 
+    * fullDir is an optional argument that specifies whether you would like Soot to use the redefcp option path as the directory to consider all classes from. If set to true, in Soot's callgraph (CG) generation, all classes in redefcp are considered application classes and each of their methods are modelled as entry points to the application. If set to false, only (all) methods in the mainClass class will be used as entry points for the CG, which may be unrealistic, depending on the patch structure.
+    * x.originalclasses.out is the file that contains the names of the classes that are relevant to the patch, and should be analysed by the patch adapter, both for diffing purposes and for fixing purposes
+    * Example is any class, as a placeholder to get Soot up and running in a way that it is familiar with, however Example is included in this project, so for simplicity sake, Example can be used. Currently compiled into `target/classes/`, so anything that runs needs to keep that in mind, cp-wise. 
+
+## Testing:
+   * to setup the patch classes required for testing, first run `./makePatchClasses.sh`
+   * then tests can be run with: `mvn test`
+   * testing requires additional heap, currently set in `pom.xml`, minimum required is 1gb
+   * to run only one test: `mvn -Dtest=ValidationTest#BTestRunFieldAddition test`
+   * to expand the testset:
+     1) create the **patch version** to use as a test class in `src/main/java/testexamples/<specific_testset_package_name>`
+     2) get the classfile for that, i.e., `mvn compile`
+     3) find the classfile for the new patch (`find . -name X.class`), then copy it and its source to `src/main/java/patch/testexamples/<specific_testset_package_name>'
+     4) rename the package for the source in `src/main/java/testexamplespatch/<specific_testset_package_name>/patch/testexamples/<specific_testset_package_name>` to `patch.testexamples.<specific_testset_package_name>` so that when this recompiles when rebuilding the project, it does not conflict with original version of same class. Additionally they are each in a separate test dir in case the process_dir option of Soot is used during testing (set by useFullDir).
+     5) now create the corresponding **original version** of that class in `src/main/java/testexamples/<specific_testset_package_name>`
+     6) add corresponding tests for this added setup (see below for important detail)
+     7) add to the `makePatchClasses.sh` script to assure that setup can occur correctly
+   * tests are setup to run in alphabetical order, a surefire way to ensure that they are run sequentially. If they are run in parallel, Soot will share many values, as it uses several singletons to model the application and this could cause issues if it is done unintentionally
+   * tests additionally must use the `TestSetup.testSetupRefresh` method to refresh all Soot settings, and pause to allow for that to commence, otherwise there will likely be overlapping data in Soot's model of the application, which can cause issues.

patchAdapter/makePatchClasses.sh

@@ -0,0 +1,66 @@
+######################
+#
+# Makes the set of patch classes
+# necesesary due to the fact that
+# maven will not allow 2 variants of
+# same class, which is necessary to have
+# a patch set and original set for testing
+#
+# If you expand the testset, add to this file!
+#
+# To cleanup the products of this, after testing, use:
+#     find resources/testexamplespatch/methodadditionpolymorphic/patch/testexamples/methodadditionpolymorphic/ -name "*.class" | xargs  rm
+#
+#
+# IF THIS SCRIPT FAILS PARTWAY THROUGH:
+# you should fix the issue then run the rest of the steps copy/paste style
+# if you restart the whole script you might erase the original copies of test setup classes
+# 
+######################
+
+mkdir TEMP
+echo "----------------------------------------"
+echo "Copying original versions of source files into TEMP"
+cp -r src/main/java/testexamples TEMP/
+echo "----------------------------------------"
+
+echo "----------------------------------------"
+echo "Copying patch sets into build location"
+cp resources/testexamplespatch/fieldaddition/patch/testexamples/fieldaddition/*.java src/main/java/testexamples/fieldaddition/
+cp resources/testexamplespatch/methodremoval/patch/testexamples/methodremoval/*.java src/main/java/testexamples/methodremoval/
+cp resources/testexamplespatch/nochange/patch/testexamples/nochange/*.java src/main/java/testexamples/nochange/
+cp resources/testexamplespatch/methodadditionmonomorphic/patch/testexamples/methodadditionmonomorphic/*.java src/main/java/testexamples/methodadditionmonomorphic
+cp resources/testexamplespatch/methodadditionpolymorphic/patch/testexamples/methodadditionpolymorphic/*.java src/main/java/testexamples/methodadditionpolymorphic
+echo "----------------------------------------"
+
+echo "----------------------------------------"
+echo "Compiling!"
+mvn compile
+echo "----------------------------------------"
+
+if [[ "$?" -ne 0 ]] ; then
+    echo 'Could not compile test setup'; exit $rc
+
+else
+
+    echo "----------------------------------------"
+    echo "Copying patch classfiles into expected directory for patch adapter testing"
+    cp target/classes/testexamples/fieldaddition/* resources/testexamplespatch/fieldaddition/patch/testexamples/fieldaddition/
+    cp target/classes/testexamples/methodremoval/* resources/testexamplespatch/methodremoval/patch/testexamples/methodremoval/
+    cp target/classes/testexamples/nochange/* resources/testexamplespatch/nochange/patch/testexamples/nochange/
+    cp target/classes/testexamples/methodadditionmonomorphic/* resources/testexamplespatch/methodadditionmonomorphic/patch/testexamples/methodadditionmonomorphic/
+    cp target/classes/testexamples/methodadditionpolymorphic/* resources/testexamplespatch/methodadditionpolymorphic/patch/testexamples/methodadditionpolymorphic/
+    echo "----------------------------------------"
+
+    echo "----------------------------------------"
+    echo "Replacing original versions of source files"
+    cp -r TEMP/testexamples/ src/main/java/testexamples/
+    echo "----------------------------------------"
+
+
+    echo "----------------------------------------"
+    echo "Cleaning up!"
+    rm -rf TEMP
+    echo "----------------------------------------"
+
+fi

patchAdapter/pom.xml

@@ -0,0 +1,105 @@
+<?xml version="1.0" encoding="utf-8"?>
+
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+  <modelVersion>4.0.0</modelVersion>
+
+  <groupId>differ</groupId>
+  <artifactId>ssDiffTool</artifactId>
+  <version>1.0-SNAPSHOT</version>
+
+  <name>ssDiffTool</name>
+
+  <properties>
+    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
+    <maven.compiler.source>1.8</maven.compiler.source>
+    <maven.compiler.target>1.8</maven.compiler.target>
+  </properties>
+
+  <dependencies>
+
+	<dependency>
+    <groupId>ca.mcgill.sable</groupId>
+    <artifactId>soot</artifactId>
+    <version>3.3.0</version>
+  </dependency>
+
+	<dependency>
+			<groupId>commons-cli</groupId>
+			<artifactId>commons-cli</artifactId>
+			<version>1.4</version>
+	</dependency>
+
+
+    <dependency>
+      <groupId>junit</groupId>
+      <artifactId>junit</artifactId>
+      <version>4.11</version>
+      <scope>test</scope>
+    </dependency>
+  </dependencies>
+
+  <build>
+    <pluginManagement><!-- lock down plugins versions to avoid using Maven defaults (may be moved to parent pom) -->
+      <plugins>
+        <!-- clean lifecycle, see https://maven.apache.org/ref/current/maven-core/lifecycles.html#clean_Lifecycle -->
+        <plugin>
+          <artifactId>maven-clean-plugin</artifactId>
+          <version>3.1.0</version>
+        </plugin>
+        <!-- default lifecycle, jar packaging: see https://maven.apache.org/ref/current/maven-core/default-bindings.html#Plugin_bindings_for_jar_packaging -->
+        <plugin>
+          <artifactId>maven-resources-plugin</artifactId>
+          <version>3.0.2</version>
+        </plugin>
+        <plugin>
+          <artifactId>maven-compiler-plugin</artifactId>
+          <version>3.8.0</version>
+        </plugin>
+        <plugin>
+          <artifactId>maven-surefire-plugin</artifactId>
+          <version>2.22.1</version>
+	  <configuration>
+	    <runOrder>alphabetical</runOrder>
+	    <argLine>-Xmx1g</argLine>
+		  </configuration>
+        </plugin>
+        <plugin>
+          <artifactId>maven-jar-plugin</artifactId>
+          <version>3.0.2</version>
+        </plugin>
+        <plugin>
+          <artifactId>maven-install-plugin</artifactId>
+          <version>2.5.2</version>
+        </plugin>
+        <plugin>
+          <artifactId>maven-deploy-plugin</artifactId>
+          <version>2.8.2</version>
+        </plugin>
+        <!-- site lifecycle, see https://maven.apache.org/ref/current/maven-core/lifecycles.html#site_Lifecycle -->
+        <plugin>
+          <artifactId>maven-site-plugin</artifactId>
+          <version>3.7.1</version>
+        </plugin>
+        <plugin>
+          <artifactId>maven-project-info-reports-plugin</artifactId>
+          <version>3.0.0</version>
+        </plugin>
+		 <plugin>
+      <artifactId>maven-assembly-plugin</artifactId>
+      <configuration>
+        <archive>
+          <manifest>
+            <mainClass>differ.SemanticDiffer</mainClass>
+          </manifest>
+        </archive>
+        <descriptorRefs>
+          <descriptorRef>jar-with-dependencies</descriptorRef>
+        </descriptorRefs>
+      </configuration>
+    </plugin>
+
+      </plugins>
+    </pluginManagement>
+  </build>
+</project>

patchAdapter/resources/testexamplespatch/fieldaddition/patch/testexamples/fieldaddition/AdditionFieldTest.java

@@ -0,0 +1,40 @@
+/*******************************************************************************
+ * Copyright (c) 2020, 2020 IBM Corp. and others
+ *
+ * This program and the accompanying materials are made available under
+ * the terms of the Eclipse Public License 2.0 which accompanies this
+ * distribution and is available at https://www.eclipse.org/legal/epl-2.0/
+ * or the Apache License, Version 2.0 which accompanies this distribution and
+ * is available at https://www.apache.org/licenses/LICENSE-2.0.
+ *
+ * This Source Code may also be made available under the following
+ * Secondary Licenses when the conditions for such availability set
+ * forth in the Eclipse Public License, v. 2.0 are satisfied: GNU
+ * General Public License, version 2 with the GNU Classpath
+ * Exception [1] and GNU General Public License, version 2 with the
+ * OpenJDK Assembly Exception [2].
+ *
+ * [1] https://www.gnu.org/software/classpath/license.html
+ * [2] http://openjdk.java.net/legal/assembly-exception.html
+ *
+ * SPDX-License-Identifier: EPL-2.0 OR Apache-2.0 OR GPL-2.0 WITH Classpath-exception-2.0 OR LicenseRef-GPL-2.0 WITH Assembly-exception
+ *******************************************************************************/
+
+package testexamples.fieldaddition;
+
+public class AdditionFieldTest {
+
+    public int newField;
+    public static int secondNewField = 1;
+
+    public int returnSame(int x){
+	int local = x;
+	newField = local;
+	return newField;
+    }
+
+    public int returnAValue(){
+	return secondNewField;
+    }
+
+}

patchAdapter/resources/testexamplespatch/methodadditionmonomorphic/patch/testexamples/methodadditionmonomorphic/AdditionMethodMonomorphicTest.java

@@ -0,0 +1,65 @@
+/*******************************************************************************
+ * Copyright (c) 2020, 2020 IBM Corp. and others
+ *
+ * This program and the accompanying materials are made available under
+ * the terms of the Eclipse Public License 2.0 which accompanies this
+ * distribution and is available at https://www.eclipse.org/legal/epl-2.0/
+ * or the Apache License, Version 2.0 which accompanies this distribution and
+ * is available at https://www.apache.org/licenses/LICENSE-2.0.
+ *
+ * This Source Code may also be made available under the following
+ * Secondary Licenses when the conditions for such availability set
+ * forth in the Eclipse Public License, v. 2.0 are satisfied: GNU
+ * General Public License, version 2 with the GNU Classpath
+ * Exception [1] and GNU General Public License, version 2 with the
+ * OpenJDK Assembly Exception [2].
+ *
+ * [1] https://www.gnu.org/software/classpath/license.html
+ * [2] http://openjdk.java.net/legal/assembly-exception.html
+ *
+ * SPDX-License-Identifier: EPL-2.0 OR Apache-2.0 OR GPL-2.0 WITH Classpath-exception-2.0 OR LicenseRef-GPL-2.0 WITH Assembly-exception
+ *******************************************************************************/
+
+package testexamples.methodadditionmonomorphic;
+
+/*
+ * Test scenarios for new methods:
+ * 
+ * (old == pre-existing)
+ *
+ * 1) new instance call in old method
+ * 2) new static call in old method
+ * 3) old instance method call in new method 
+ * 4) new instance method call in new method
+ * 5) old field access in new method
+ */
+
+public class AdditionMethodMonomorphicTest {
+
+    public int field;
+
+    public int returnSamePlusSeven(int x){
+	field = 7;
+	int temp = newMethod(x); //1
+	thirdNewMethod(); //2
+	return temp;
+    }
+
+    public void testPrinter(){
+	System.out.println("Hello World!");
+    }
+
+    public int newMethod(int y){
+	testPrinter(); //3
+	secondNewMethod(); //4
+	return field + y; //5
+    }
+
+    public void secondNewMethod(){
+	System.out.println("This is second new method!");
+    }
+
+    public static void thirdNewMethod(){
+	System.out.println("This is third new method!");
+    }
+}

patchAdapter/resources/testexamplespatch/methodadditionpolymorphic/patch/testexamples/methodadditionpolymorphic/AdditionMethodPolymorphicTestChildFive.java

@@ -0,0 +1,35 @@
+/*******************************************************************************
+ * Copyright (c) 2020, 2020 IBM Corp. and others
+ *
+ * This program and the accompanying materials are made available under
+ * the terms of the Eclipse Public License 2.0 which accompanies this
+ * distribution and is available at https://www.eclipse.org/legal/epl-2.0/
+ * or the Apache License, Version 2.0 which accompanies this distribution and
+ * is available at https://www.apache.org/licenses/LICENSE-2.0.
+ *
+ * This Source Code may also be made available under the following
+ * Secondary Licenses when the conditions for such availability set
+ * forth in the Eclipse Public License, v. 2.0 are satisfied: GNU
+ * General Public License, version 2 with the GNU Classpath
+ * Exception [1] and GNU General Public License, version 2 with the
+ * OpenJDK Assembly Exception [2].
+ *
+ * [1] https://www.gnu.org/software/classpath/license.html
+ * [2] http://openjdk.java.net/legal/assembly-exception.html
+ *
+ * SPDX-License-Identifier: EPL-2.0 OR Apache-2.0 OR GPL-2.0 WITH Classpath-exception-2.0 OR LicenseRef-GPL-2.0 WITH Assembly-exception
+ *******************************************************************************/
+
+package testexamples.methodadditionpolymorphic;
+
+/*
+ * grand-child class which relies on:
+ * 1) parent method in patch version
+ * 2) grandparent method in original version
+ *
+ */
+
+public class AdditionMethodPolymorphicTestChildFive extends AdditionMethodPolymorphicTestChildOne {
+
+
+}