Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update samples to LAG (latest and greatest) #156

Closed
5 tasks done
onobc opened this issue Jan 14, 2022 · 10 comments · Fixed by #161, #163, #164, #166 or #167
Closed
5 tasks done

Update samples to LAG (latest and greatest) #156

onobc opened this issue Jan 14, 2022 · 10 comments · Fixed by #161, #163, #164, #166 or #167
Assignees

Comments

@onobc
Copy link
Collaborator

onobc commented Jan 14, 2022

Motivation

The samples are intended to give users a good starting place on how to do particular use cases w/ SCDF. We need to make sure what we are recommending is up-to-speed w/ latest libs and usage of the libs. We should also make sure the samples we recommend are still relevant.

Steps

Step Details

ITEM 1

  • I will need input from others on what we should/not support (@cppwfs @jvalkeal )

ITEM 2

  • TODO: list version of major libs to support here (assuming latest supported version of SCDF deps)

ITEM 3

  • Replace deprecations and any "glaring" older API uses; does not have to be exhaustive.

ITEM 4

ITEM 5

@onobc onobc changed the title Hygeine Bring samples up to LAG (latest and greatest) Jan 14, 2022
@onobc onobc changed the title Bring samples up to LAG (latest and greatest) Update samples to LAG (latest and greatest) Jan 14, 2022
@onobc onobc self-assigned this Jan 14, 2022
@onobc
Copy link
Collaborator Author

onobc commented Jan 16, 2022

INFO] Reactor Summary for ticktock 1.0.0-SNAPSHOT:
[INFO] 
[INFO] multi-output-time-rabbit ........................... SUCCESS [ 26.027 s]
[INFO] multi-input-log-rabbit ............................. SUCCESS [  4.110 s]
[INFO] multi-output-time-kafka ............................ FAILURE [  1.696 s]
[INFO] multi-input-log-kafka .............................. SKIPPED
[INFO] ticktock ........................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  34.873 s
[INFO] Finished at: 2022-01-16T16:00:32Z
[INFO] ------------------------------------------------------------------------
Error:  Failed to execute goal org.apache.maven.plugins:maven-javadoc-plugin:3.2.0:jar (javadoc) on project multi-output-time-kafka: MavenReportException: Error while generating Javadoc: 
Error:  Exit code: 1 - javadoc: warning - No source files for package org.springframework.cloud.stream.app.time.source.kafka
Error:  javadoc: error - No public or protected classes found to document.
Error:  
Error:  Command line was: /usr/lib/jvm/adoptopenjdk-8-hotspot-amd64/jre/../bin/javadoc @options @packages
Error:  
Error:  Refer to the generated Javadoc files in '/home/runner/work/spring-cloud-dataflow-samples/spring-cloud-dataflow-samples/multi-io-samples/ticktock/multi-output-time-kafka/target/apidocs' dir.
Error:  -> [Help 1]
Error:  
Error:  To see the full stack trace of the errors, re-run Maven with the -e switch.
Error:  Re-run Maven using the -X switch to enable full debug logging.
Error:  
Error:  For more information about the errors and possible solutions, please read the following articles:
Error:  [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
Error:  
Error:  After correcting the problems, you can resume the build with the command
Error:    mvn <args> -rf :multi-output-time-kafka
Error: Process completed with exit code 1.

@onobc
Copy link
Collaborator Author

onobc commented Jan 16, 2022

[INFO] Downloaded from maven-central: https://repo.maven.apache.org/maven2/org/springframework/spring-beans/5.2.8.RELEASE/spring-beans-5.2.8.RELEASE.jar (689 kB at 3.4 MB/s)
Warning:  Use of META-INF/dataflow-configuration-metadata-whitelist.properties is deprecated. Please use META-INF/dataflow-configuration-metadata.properties
[INFO] Attaching /home/runner/work/spring-cloud-dataflow-samples/spring-cloud-dataflow-samples/multi-io-samples/ticktock/multi-input-log-rabbit/target/multi-input-log-rabbit-1.0.0-SNAPSHOT-metadata.jar to current project

@onobc
Copy link
Collaborator Author

onobc commented Jan 16, 2022

[INFO] Downloaded: https://repo.maven.apache.org/maven2/org/springframework/session/spring-session-bom/Bean-SR3/spring-session-bom-Bean-SR3.pom (3 KB at 145.0 KB/sec)
Warning:  
Warning:  Some problems were encountered while building the effective model for org.springframework.cloud.dataflow.samples:http-ingest:jar:1.0.0.BUILD-SNAPSHOT
Warning:  'parent.relativePath' of POM org.springframework.cloud.dataflow.samples:http-ingest:1.0.0.BUILD-SNAPSHOT (/home/runner/work/spring-cloud-dataflow-samples/spring-cloud-dataflow-samples/kafka-samples/http-ingest/pom.xml) points at org.springframework.cloud.dataflow.samples:kafka-samples-parent instead of org.springframework.boot:spring-boot-starter-parent, please verify your project structure @ line 5, column 11
Warning:  
Warning:  Some problems were encountered while building the effective model for org.springframework.cloud.dataflow.samples:kstreams-join-user-clicks-and-region:jar:1.0.0.BUILD-SNAPSHOT
Warning:  'parent.relativePath' of POM org.springframework.cloud.dataflow.samples:kstreams-join-user-clicks-and-region:1.0.0.BUILD-SNAPSHOT (/home/runner/work/spring-cloud-dataflow-samples/spring-cloud-dataflow-samples/kafka-samples/kstreams-join-user-clicks-and-region/pom.xml) points at org.springframework.cloud.dataflow.samples:kafka-samples-parent instead of org.springframework.boot:spring-boot-starter-parent, please verify your project structure @ line 6, column 10
Warning:  
Warning:  Some problems were encountered while building the effective model for org.springframework.cloud.dataflow.samples:kstreams-log-user-clicks-per-region:jar:1.0.0.BUILD-SNAPSHOT
Warning:  'parent.relativePath' of POM org.springframework.cloud.dataflow.samples:kstreams-log-user-clicks-per-region:1.0.0.BUILD-SNAPSHOT (/home/runner/work/spring-cloud-dataflow-samples/spring-cloud-dataflow-samples/kafka-samples/kstreams-log-user-clicks-per-region/pom.xml) points at org.springframework.cloud.dataflow.samples:kafka-samples-parent instead of org.springframework.boot:spring-boot-starter-parent, please verify your project structure @ line 5, column 11
Warning:  
Warning:  Some problems were encountered while building the effective model for org.springframework.cloud.dataflow.samples:kstreams-word-count:jar:1.0.0.BUILD-SNAPSHOT
Warning:  'parent.relativePath' of POM org.springframework.cloud.dataflow.samples:kstreams-word-count:1.0.0.BUILD-SNAPSHOT (/home/runner/work/spring-cloud-dataflow-samples/spring-cloud-dataflow-samples/kafka-samples/kstreams-word-count/pom.xml) points at org.springframework.cloud.dataflow.samples:kafka-samples-parent instead of org.springframework.boot:spring-boot-starter-parent, please verify your project structure @ line 6, column 10
Warning:  
Warning:  It is highly recommended to fix these problems because they threaten the stability of your build.
Warning:  
Warning:  For this reason, future Maven versions might no longer support building such malformed projects.

@onobc
Copy link
Collaborator Author

onobc commented Jan 20, 2022

Another issue is the inconsistency w/ the version numbers of the samples. Some use snapshot starting at 0.0.1-snapshot, some w/ snapshot starting at 1.0.0-snapshot and some w/ plain versions such as 1.0.0. We should make this consistent.

@onobc
Copy link
Collaborator Author

onobc commented Jan 20, 2022

Another issue is the multiple ways we use to build docker containers:

  1. jib
  2. maven docker plugin
  3. spring-boot build image

@ilayaperumalg
Copy link
Contributor

ilayaperumalg commented Jan 20, 2022

Another issue is the inconsistency w/ the version numbers of the samples. Some use snapshot starting at 0.0.1-snapshot, some w/ snapshot starting at 1.0.0-snapshot and some w/ plain versions such as 1.0.0. We should make this consistent.

I agree. These samples were initially moved from different places and versioning was named from there. Given that these samples are already in the artifact repositories, we can have them consistent versioning starting with some higher version numbers, say 2.0.0-SNAPSHOT for the entire samples to avoid confusion to the already existing ones.

@ilayaperumalg
Copy link
Contributor

Another issue is the multiple ways we use to build docker containers:

yeah, we recently discussed about this for the main images for SCDF/Skipper/OOTB apps etc., we can be consistent here as well.

@cppwfs
Copy link
Contributor

cppwfs commented Jan 20, 2022

  • I updated the task samples to use spring-boot build image. Unless there is a reason to use the other image builders.
  • I like the use of 2.0.0-SNAPSHOT version for all samples except the task. We may want to set those to 2.0.0 because that is what we will use for our task samples in SCDF. Thoughts?

@onobc
Copy link
Collaborator Author

onobc commented Jan 20, 2022

I updated the task samples to use spring-boot build image. Unless there is a reason to use the other image builders.

I am in favor of SBBI as well.

I like the use of 2.0.0-SNAPSHOT version for all samples except the task. We may want to set those to 2.0.0 because that is what we will use for our task samples in SCDF. Thoughts?

Make sense to me.

@cppwfs
Copy link
Contributor

cppwfs commented Jan 21, 2022

Keep:

✅ transition-sample
✅ timestamp-task
✅ timestamp-batch
✅ scenario-task
✅ monitoring-samples
✅ kafka-samples
✅ security-ldap-uaa-example
✅ java-dsl
✅ function-samples
✅ dataflow-website
✅ dataflow-template-example
✅ file-ingest

Discuss

? spring-cloud-stream-function-bindings (refer them to the spring cloud stream apps?)
? dataflow-template-load-generator
? keystore
? multi-io-samples
? keystore
? dataflow-zuul
? custom-dataflow-builds

Discard:

❌ dataflow-migrate-schedules

onobc added a commit to onobc/spring-cloud-dataflow-samples that referenced this issue Jan 21, 2022
onobc added a commit to onobc/spring-cloud-dataflow-samples that referenced this issue Jan 21, 2022
onobc added a commit to onobc/spring-cloud-dataflow-samples that referenced this issue Jan 21, 2022
…on-metadata-whitelist.properties' to 'META-INF/(dataflow|spring)-configuration-metadata.properties'

Partially resolves spring-cloud#156
onobc added a commit to onobc/spring-cloud-dataflow-samples that referenced this issue Jan 22, 2022
onobc added a commit to onobc/spring-cloud-dataflow-samples that referenced this issue Jan 22, 2022
…on-metadata-whitelist.properties' to 'META-INF/(dataflow|spring)-configuration-metadata.properties'

Partially resolves spring-cloud#156
onobc added a commit to onobc/spring-cloud-dataflow-samples that referenced this issue Jan 22, 2022
…on-metadata-whitelist.properties' to 'META-INF/(dataflow|spring)-configuration-metadata.properties'

Partially resolves spring-cloud#156
onobc added a commit to onobc/spring-cloud-dataflow-samples that referenced this issue Jan 22, 2022
onobc added a commit to onobc/spring-cloud-dataflow-samples that referenced this issue Jan 22, 2022
onobc added a commit to onobc/spring-cloud-dataflow-samples that referenced this issue Jan 23, 2022
onobc added a commit to onobc/spring-cloud-dataflow-samples that referenced this issue Jan 23, 2022
onobc added a commit to onobc/spring-cloud-dataflow-samples that referenced this issue Jan 23, 2022
* Spring Boot 2.6.3
* Spring Cloud 2021.0.0
* SCS/F 3.2.1
* etc..

Partially resolves spring-cloud#156
@onobc onobc closed this as completed in 96c487a Jan 24, 2022
onobc added a commit to onobc/spring-cloud-dataflow-samples that referenced this issue Jan 24, 2022
…on-metadata-whitelist.properties' to 'META-INF/(dataflow|spring)-configuration-metadata.properties'

Partially resolves spring-cloud#156
onobc added a commit that referenced this issue Jan 24, 2022
…on-metadata-whitelist.properties' to 'META-INF/(dataflow|spring)-configuration-metadata.properties' (#161)

Partially resolves #156
onobc added a commit to onobc/spring-cloud-dataflow-samples that referenced this issue Jan 24, 2022
onobc added a commit to onobc/spring-cloud-dataflow-samples that referenced this issue Jan 24, 2022
onobc added a commit that referenced this issue Jan 24, 2022
* Set <relativePath/> on <parent> element to prevent warnings about pointing to the wrong parent.
* Fix package name - avoid warning about invalid javadoc.

Partially resolves #156
onobc added a commit to onobc/spring-cloud-dataflow-samples that referenced this issue Jan 24, 2022
* Spring Boot 2.6.3
* Spring Cloud 2021.0.0
* SCS/F 3.2.1
* etc..

Partially resolves spring-cloud#156
onobc added a commit that referenced this issue Jan 24, 2022
* Spring Boot 2.6.3
* Spring Cloud 2021.0.0
* SCS/F 3.2.1
* etc..

Partially resolves #156
onobc added a commit to onobc/spring-cloud-dataflow-samples that referenced this issue Jan 24, 2022
onobc added a commit that referenced this issue Jan 24, 2022
onobc added a commit to onobc/spring-cloud-dataflow-samples that referenced this issue Jan 24, 2022
onobc added a commit that referenced this issue Jan 24, 2022
onobc added a commit to onobc/spring-cloud-dataflow-samples that referenced this issue Jan 24, 2022
onobc added a commit that referenced this issue Jan 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment