Storm v1.0.5 breaks storm-mesos

[kiryllbondar]

Hello, i'm trying to build storm 1.0.5 mesos 1.4.0
STORM_RELEASE=1.0.5 MESOS_RELEASE=1.4.0 bin/build-release.sh
but getting error:

[WARNING] COMPILATION WARNING :
[INFO] -------------------------------------------------------------
[WARNING] bootstrap class path not set in conjunction with -source 1.7
[WARNING] /Users/build/Work/git/storm/storm/src/main/storm/mesos/util/PrettyProtobuf.java:[226,12] hasRole() in org.apache.mesos.Protos.Resource has been deprecated
[WARNING] /Users/build/Work/git/storm/storm/src/main/storm/mesos/util/PrettyProtobuf.java:[227,54] getRole() in org.apache.mesos.Protos.Resource has been deprecated
[WARNING] /Users/build/Work/git/storm/storm/src/main/storm/mesos/MesosNimbus.java:[255,32] NIMBUS_HOST in org.apache.storm.Config has been deprecated
[WARNING] /Users/build/Work/git/storm/storm/src/main/storm/mesos/MesosNimbus.java:[362,73] getRole() in org.apache.mesos.Protos.FrameworkInfo.Builder has been deprecated
[WARNING] /Users/build/Work/git/storm/storm/src/main/storm/mesos/MesosNimbus.java:[611,24] setRole(java.lang.String) in org.apache.mesos.Protos.Resource.Builder has been deprecated
[WARNING] /Users/build/Work/git/storm/storm/src/main/storm/mesos/MesosNimbus.java:[630,22] setRole(java.lang.String) in org.apache.mesos.Protos.Resource.Builder has been deprecated
[WARNING] /Users/build/Work/git/storm/storm/src/main/storm/mesos/MesosNimbus.java:[819,48] setRole(java.lang.String) in org.apache.mesos.Protos.FrameworkInfo.Builder has been deprecated
[WARNING] /Users/build/Work/git/storm/storm/src/main/storm/mesos/resources/AggregatedOffers.java:[68,43] getRole() in org.apache.mesos.Protos.Resource has been deprecated
[WARNING] /Users/build/Work/git/storm/storm/src/main/storm/mesos/ResourceRoleComparator.java:[27,36] getRole() in org.apache.mesos.Protos.Resource has been deprecated
[WARNING] /Users/build/Work/git/storm/storm/src/main/storm/mesos/ResourceRoleComparator.java:[27,13] getRole() in org.apache.mesos.Protos.Resource has been deprecated
[WARNING] /Users/build/Work/git/storm/storm/src/main/storm/mesos/ResourceRoleComparator.java:[29,20] getRole() in org.apache.mesos.Protos.Resource has been deprecated
[WARNING] /Users/build/Work/git/storm/storm/src/main/storm/mesos/util/MesosCommon.java:[75,102] NIMBUS_HOST in org.apache.storm.Config has been deprecated
[WARNING] /Users/build/Work/git/storm/storm/src/main/storm/mesos/MesosSupervisor.java: Some input files use unchecked or unsafe operations.
[WARNING] /Users/build/Work/git/storm/storm/src/main/storm/mesos/MesosSupervisor.java: Recompile with -Xlint:unchecked for details.
[INFO] 15 warnings
[INFO] -------------------------------------------------------------
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /Users/build/Work/git/storm/storm/src/main/storm/mesos/MesosSupervisor.java:[157,16] cannot access com.google.protobuf.GeneratedMessageV3
  class file for com.google.protobuf.GeneratedMessageV3 not found
[ERROR] /Users/build/Work/git/storm/storm/src/main/storm/mesos/MesosSupervisor.java:[163,9] cannot access com.google.protobuf.GeneratedMessageV3.Builder
  class file for com.google.protobuf.GeneratedMessageV3$Builder not found
[ERROR] /Users/build/Work/git/storm/storm/src/main/storm/mesos/MesosNimbus.java:[861,73] cannot find symbol
  symbol:   method getClass()
  location: variable credentialBuilder of type org.apache.mesos.Protos.Credential.Builder
[INFO] 3 errors
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] storm-parent ....................................... SUCCESS [ 43.225 s]
[INFO] storm-shim ......................................... SUCCESS [ 58.477 s]
[INFO] storm-shim-1x ...................................... SUCCESS [ 16.971 s]
[INFO] storm-mesos ........................................ FAILURE [ 18.742 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:17 min
[INFO] Finished at: 2017-10-17T23:13:48+03:00
[INFO] Final Memory: 37M/432M

[erikdw]

Thanks for the report, I can reproduce this issue, but not sure of any details. If you can determine the root cause that will be helpful otherwise this will not get any attention for weeks if not months.

[erikdw]

This is thematically related to #202 , #214 , #219 , and #222 . Notably, even if we fix #221 (this issue) you will likely still hit #222.

[kiryllbondar]

Thank you a lot. Hope we will have some fix soon

[erikdw]

@kiryllbondar : sorry to be obtuse, but are you saying "we" as in {you, your own team/company} will be providing a fix soon? "We" as in me and my team/company will not be able to work on this for a long time, maybe during Thanksgiving or Christmas time.

[kiryllbondar]

@erikdw : We, I mean all people who use this functionality :) Unfortunately I do not have the skills to deal with this problem on my own.

[erikdw]

@kiryllbondar : thanks for clarifying :)

[JessicaLHartog]

Looking into this, and leveraging the changes in #232 Storm 1.0.5 builds successfully with the Mesos versions in .travis.yml. This breakage in particular is related instead to Mesos 1.4.0.

Looking into this, the following is the result of successive compilations:

Storm Version	Mesos Version	Build Status
1.0.5	1.1.0	Successful
1.0.5	1.1.1	Successful
1.0.5	1.1.3	Successful
1.0.5	1.2.0	Successful
1.0.5	1.2.1	Successful
1.0.5	1.2.2	Successful
1.0.5	1.3.0	Successful
1.0.5	1.3.1	Successful
1.0.5	1.4.0	Unsuccessful
1.0.5	1.4.1	Unsuccessful

This looks to be related to changes in the protobufs for the Mesos version. There's a change in the protobuf version between 1.3.x and 1.4.x, namely that the version is increased to 3.3.0 per this commit

So I updated the protobuf version:

diff --git pom.xml pom.xml
index 2c3a74c..6980b09 100644
--- pom.xml
+++ pom.xml
@@ -120,7 +120,7 @@
       <dependency>
         <groupId>com.google.protobuf</groupId>
         <artifactId>protobuf-java</artifactId>
-        <version>2.6.1</version>
+        <version>3.3.0</version>
       </dependency>
       <dependency>
         <groupId>org.apache.storm</groupId>

And after this change these are the compilation results:

Storm Version	Mesos Version	Build Status
1.0.5	1.1.0	Successful
1.0.5	1.1.1	Successful
1.0.5	1.1.3	Successful
1.0.5	1.2.0	Successful
1.0.5	1.2.1	Successful
1.0.5	1.2.2	Successful
1.0.5	1.3.0	Successful
1.0.5	1.3.1	Successful
1.0.5	1.4.0	Successful
1.0.5	1.4.1	Successful

I'll submit a PR to update the protobuf version.

[JessicaLHartog]

I'll be closing this issue since this is solved once #233 is merged.