Skip to content

Commit ee23a9c

Browse files
jklegarjklegar
authored
Migrate Java code from main Feast repo into feast-java (#1)
* move java files + copy license, .gitignore, .scalafmt.conf + modify main pom.xml, Makefile Signed-off-by: Jacob Klegar <[email protected]> * add core and serving dockerfiles Signed-off-by: Jacob Klegar <[email protected]> * add a github action Signed-off-by: Jacob Klegar <[email protected]> * add unit-test-java Signed-off-by: Jacob Klegar <[email protected]> * add java integration test action Signed-off-by: Jacob Klegar <[email protected]> * rebase changes from feast repo Signed-off-by: Jacob Klegar <[email protected]> * add datatypes/java to feast-java Signed-off-by: Jacob Klegar <[email protected]> * pom version should be 0.10 Signed-off-by: Jacob Klegar <[email protected]> * checkout submodules in github actions Signed-off-by: Jacob Klegar <[email protected]> * add build docker images action Signed-off-by: Jacob Klegar <[email protected]> * try fixing gcloud setup action Signed-off-by: Jacob Klegar <[email protected]> * simplify mvn cache download Signed-off-by: Jacob Klegar <[email protected]> * fix dockerfiles Signed-off-by: Jacob Klegar <[email protected]> * add release actions Signed-off-by: Jacob Klegar <[email protected]> * bump testcontainers patch Signed-off-by: Jacob Klegar <[email protected]> * clean up Makefile Signed-off-by: Jacob Klegar <[email protected]> * remove temporary actions test code Signed-off-by: Jacob Klegar <[email protected]> * add issue templates Signed-off-by: Jacob Klegar <[email protected]>
1 parent 6f0e7b1 commit ee23a9c

File tree

222 files changed

+21406
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

222 files changed

+21406
-0
lines changed

‎.github/ISSUE_TEMPLATE/bug_report.md

+22
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
---
2+
name: Bug report
3+
about: Create a report to help us improve
4+
title: ''
5+
labels: ''
6+
assignees: ''
7+
8+
---
9+
10+
## Expected Behavior
11+
12+
## Current Behavior
13+
14+
## Steps to reproduce
15+
16+
### Specifications
17+
18+
- Version:
19+
- Platform:
20+
- Subsystem:
21+
22+
## Possible Solution

‎.github/ISSUE_TEMPLATE/feature_request.md

+20
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
---
2+
name: Feature request
3+
about: Suggest an idea for this project
4+
title: ''
5+
labels: ''
6+
assignees: ''
7+
8+
---
9+
10+
**Is your feature request related to a problem? Please describe.**
11+
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
12+
13+
**Describe the solution you'd like**
14+
A clear and concise description of what you want to happen.
15+
16+
**Describe alternatives you've considered**
17+
A clear and concise description of any alternative solutions or features you've considered.
18+
19+
**Additional context**
20+
Add any other context or screenshots about the feature request here.

‎.github/workflows/complete.yml

+73
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
name: complete
2+
3+
on: [push, pull_request]
4+
5+
jobs:
6+
lint-java:
7+
container: gcr.io/kf-feast/feast-ci:latest
8+
runs-on: [ubuntu-latest]
9+
steps:
10+
- uses: actions/checkout@v2
11+
with:
12+
submodules: 'true'
13+
- name: Lint java
14+
run: make lint-java
15+
16+
unit-test-java:
17+
runs-on: ubuntu-latest
18+
needs: lint-java
19+
steps:
20+
- uses: actions/checkout@v2
21+
with:
22+
submodules: 'true'
23+
- name: Set up JDK 11
24+
uses: actions/setup-java@v1
25+
with:
26+
java-version: '11'
27+
java-package: jdk
28+
architecture: x64
29+
- uses: actions/cache@v2
30+
with:
31+
path: ~/.m2/repository
32+
key: ${{ runner.os }}-ut-maven-${{ hashFiles('**/pom.xml') }}
33+
restore-keys: |
34+
${{ runner.os }}-ut-maven-
35+
- name: Test java
36+
run: make test-java-with-coverage
37+
- uses: actions/upload-artifact@v2
38+
with:
39+
name: java-coverage-report
40+
path: ${{ github.workspace }}/docs/coverage/java/target/site/jacoco-aggregate/
41+
42+
integration-test:
43+
runs-on: ubuntu-latest
44+
needs: unit-test-java
45+
steps:
46+
- uses: actions/checkout@v2
47+
with:
48+
submodules: 'true'
49+
- name: Set up JDK 11
50+
uses: actions/setup-java@v1
51+
with:
52+
java-version: '11'
53+
java-package: jdk
54+
architecture: x64
55+
- uses: actions/setup-python@v2
56+
with:
57+
python-version: '3.6'
58+
architecture: 'x64'
59+
- uses: actions/cache@v2
60+
with:
61+
path: ~/.m2/repository
62+
key: ${{ runner.os }}-it-maven-${{ hashFiles('**/pom.xml') }}
63+
restore-keys: |
64+
${{ runner.os }}-it-maven-
65+
- name: Run integration tests
66+
run: make test-java-integration
67+
- name: Save report
68+
uses: actions/upload-artifact@v2
69+
if: failure()
70+
with:
71+
name: it-report
72+
path: spark/ingestion/target/test-reports/TestSuite.txt
73+
retention-days: 5

‎.github/workflows/master_only.yml

+45
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
name: master only
2+
3+
on:
4+
push:
5+
branches:
6+
- master
7+
tags:
8+
- 'v*.*.*'
9+
10+
jobs:
11+
build-docker-images:
12+
runs-on: ubuntu-latest
13+
strategy:
14+
matrix:
15+
component: [core, serving]
16+
env:
17+
MAVEN_CACHE: gs://feast-templocation-kf-feast/.m2.2020-08-19.tar
18+
steps:
19+
- uses: actions/checkout@v2
20+
with:
21+
submodules: 'true'
22+
- uses: GoogleCloudPlatform/github-actions/setup-gcloud@master
23+
with:
24+
version: '290.0.1'
25+
export_default_credentials: true
26+
project_id: ${{ secrets.GCP_PROJECT_ID }}
27+
service_account_key: ${{ secrets.GCP_SA_KEY }}
28+
- run: gcloud auth configure-docker --quiet
29+
- name: Get m2 cache
30+
run: |
31+
infra/scripts/download-maven-cache.sh \
32+
--archive-uri ${MAVEN_CACHE} \
33+
--output-dir .
34+
- name: Get version
35+
run: echo "RELEASE_VERSION=${GITHUB_REF#refs/*/}" >> $GITHUB_ENV
36+
- name: Build image
37+
run: make build-${{ matrix.component }}-docker REGISTRY=gcr.io/kf-feast VERSION=${GITHUB_SHA}
38+
- name: Push image
39+
run: make push-${{ matrix.component }}-docker REGISTRY=gcr.io/kf-feast VERSION=${GITHUB_SHA}
40+
- name: Push development Docker image
41+
run: |
42+
if [ ${GITHUB_REF#refs/*/} == "master" ]; then
43+
docker tag gcr.io/kf-feast/feast-${{ matrix.component }}:${GITHUB_SHA} gcr.io/kf-feast/feast-${{ matrix.component }}:develop
44+
docker push gcr.io/kf-feast/feast-${{ matrix.component }}:develop
45+
fi

‎.github/workflows/release.yml

+134
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,134 @@
1+
name: release
2+
3+
on:
4+
push:
5+
tags:
6+
- 'v*.*.*'
7+
8+
jobs:
9+
get-version:
10+
runs-on: ubuntu-latest
11+
outputs:
12+
release_version: ${{ steps.get_release_version.outputs.release_version }}
13+
version_without_prefix: ${{ steps.get_release_version_without_prefix.outputs.version_without_prefix }}
14+
highest_semver_tag: ${{ steps.get_highest_semver.outputs.highest_semver_tag }}
15+
steps:
16+
- uses: actions/checkout@v2
17+
with:
18+
submodules: 'true'
19+
- name: Get release version
20+
id: get_release_version
21+
run: echo ::set-output name=release_version::${GITHUB_REF#refs/*/}
22+
- name: Get release version without prefix
23+
id: get_release_version_without_prefix
24+
env:
25+
RELEASE_VERSION: ${{ steps.get_release_version.outputs.release_version }}
26+
run: |
27+
echo ::set-output name=version_without_prefix::${RELEASE_VERSION:1}
28+
- name: Get highest semver
29+
id: get_highest_semver
30+
env:
31+
RELEASE_VERSION: ${{ steps.get_release_version.outputs.release_version }}
32+
run: |
33+
source infra/scripts/setup-common-functions.sh
34+
SEMVER_REGEX='^v[0-9]+\.[0-9]+\.[0-9]+(-([0-9A-Za-z-]+(\.[0-9A-Za-z-]+)*))?$'
35+
if echo "${RELEASE_VERSION}" | grep -P "$SEMVER_REGEX" &>/dev/null ; then
36+
echo ::set-output name=highest_semver_tag::$(get_tag_release -m)
37+
fi
38+
- name: Check output
39+
env:
40+
RELEASE_VERSION: ${{ steps.get_release_version.outputs.release_version }}
41+
VERSION_WITHOUT_PREFIX: ${{ steps.get_release_version_without_prefix.outputs.version_without_prefix }}
42+
HIGHEST_SEMVER_TAG: ${{ steps.get_highest_semver.outputs.highest_semver_tag }}
43+
run: |
44+
echo $RELEASE_VERSION
45+
echo $VERSION_WITHOUT_PREFIX
46+
echo $HIGHEST_SEMVER_TAG
47+
48+
build-publish-docker-images:
49+
runs-on: [ubuntu-latest]
50+
needs: get-version
51+
strategy:
52+
matrix:
53+
component: [core, serving]
54+
env:
55+
MAVEN_CACHE: gs://feast-templocation-kf-feast/.m2.2020-08-19.tar
56+
steps:
57+
- uses: actions/checkout@v2
58+
with:
59+
submodules: 'true'
60+
- name: Set up QEMU
61+
uses: docker/setup-qemu-action@v1
62+
- name: Set up Docker Buildx
63+
uses: docker/setup-buildx-action@v1
64+
- name: Login to DockerHub
65+
uses: docker/login-action@v1
66+
with:
67+
username: ${{ secrets.DOCKERHUB_USERNAME }}
68+
password: ${{ secrets.DOCKERHUB_TOKEN }}
69+
- uses: GoogleCloudPlatform/github-actions/setup-gcloud@master
70+
with:
71+
version: '290.0.1'
72+
export_default_credentials: true
73+
project_id: ${{ secrets.GCP_PROJECT_ID }}
74+
service_account_key: ${{ secrets.GCP_SA_KEY }}
75+
- run: gcloud auth configure-docker --quiet
76+
- name: Get m2 cache
77+
run: |
78+
infra/scripts/download-maven-cache.sh \
79+
--archive-uri ${MAVEN_CACHE} \
80+
--output-dir .
81+
- name: Build and push versioned images
82+
env:
83+
RELEASE_VERSION: ${{ needs.get-version.outputs.release_version }}
84+
VERSION_WITHOUT_PREFIX: ${{ needs.get-version.outputs.version_without_prefix }}
85+
HIGHEST_SEMVER_TAG: ${{ needs.get-version.outputs.highest_semver_tag }}
86+
run: |
87+
docker build --build-arg VERSION=$RELEASE_VERSION \
88+
-t gcr.io/kf-feast/feast-${{ matrix.component }}:${GITHUB_SHA} \
89+
-t gcr.io/kf-feast/feast-${{ matrix.component }}:${VERSION_WITHOUT_PREFIX} \
90+
-t feastdev/feast-${{ matrix.component }}:${VERSION_WITHOUT_PREFIX} \
91+
-f infra/docker/${{ matrix.component }}/Dockerfile .
92+
docker push gcr.io/kf-feast/feast-${{ matrix.component }}:${VERSION_WITHOUT_PREFIX}
93+
docker push feastdev/feast-${{ matrix.component }}:${VERSION_WITHOUT_PREFIX}
94+
95+
echo "Only push to latest tag if tag is the highest semver version $HIGHEST_SEMVER_TAG"
96+
if [ "${VERSION_WITHOUT_PREFIX}" = "${HIGHEST_SEMVER_TAG:1}" ]
97+
then
98+
docker tag feastdev/feast-${{ matrix.component }}:${VERSION_WITHOUT_PREFIX} feastdev/feast-${{ matrix.component }}:latest
99+
docker tag gcr.io/kf-feast/feast-${{ matrix.component }}:${VERSION_WITHOUT_PREFIX} gcr.io/kf-feast/feast-${{ matrix.component }}:latest
100+
docker push feastdev/feast-${{ matrix.component }}:latest
101+
docker push gcr.io/kf-feast/feast-${{ matrix.component }}:latest
102+
fi
103+
104+
publish-java-sdk:
105+
runs-on: [ubuntu-latest]
106+
needs: get-version
107+
steps:
108+
- uses: actions/checkout@v2
109+
with:
110+
submodules: 'true'
111+
- name: Set up JDK 11
112+
uses: actions/setup-java@v1
113+
with:
114+
java-version: '11'
115+
java-package: jdk
116+
architecture: x64
117+
- uses: actions/setup-python@v2
118+
with:
119+
python-version: '3.6'
120+
architecture: 'x64'
121+
- uses: actions/cache@v2
122+
with:
123+
path: ~/.m2/repository
124+
key: ${{ runner.os }}-it-maven-${{ hashFiles('**/pom.xml') }}
125+
restore-keys: |
126+
${{ runner.os }}-it-maven-
127+
- name: Publish java sdk
128+
env:
129+
VERSION_WITHOUT_PREFIX: ${{ needs.get-version.outputs.version_without_prefix }}
130+
run: |
131+
echo -n ${{ secrets.GPG_PUBLIC_KEY }} > /etc/gpg/public-key
132+
echo -n ${{ secrets.GPG_PRIVATE_KEY }} > /etc/gpg/private-key
133+
echo -n ${{ secrets.MAVEN_SETTINGS }} > /root/.m2/settings.xml
134+
infra/scripts/publish-java-sdk.sh --revision ${VERSION_WITHOUT_PREFIX}

0 commit comments

Comments
 (0)