Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add custom labels in build request with conda and spack packges #422

Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
04108d2
labels added in spack dockerfile
munishchouhan Mar 12, 2024
4ae6caa
labels added in spack dockerfile
munishchouhan Mar 12, 2024
00c0517
fixed tests
munishchouhan Mar 12, 2024
8503a66
adding labels via kaniko
munishchouhan Mar 12, 2024
2b99d28
adding labels via kaniko
munishchouhan Mar 12, 2024
1d62b72
fixed test
munishchouhan Mar 13, 2024
d3a9e14
corrected label injection in kaniko
munishchouhan Mar 13, 2024
1318e68
tests fixed
munishchouhan Mar 13, 2024
f863e92
labels added to spack singularity file
munishchouhan Mar 13, 2024
a923649
labels added to spack singularity file
munishchouhan Mar 13, 2024
b7d8049
labels added in containerfile and removed from kaniko
munishchouhan Mar 13, 2024
e7b3e39
minor formatting
munishchouhan Mar 13, 2024
12926e1
labels moved to containerconfig
munishchouhan Mar 15, 2024
7559489
minor change
munishchouhan Mar 19, 2024
7566464
Merge branch 'master' into 396-support-custom-labels-when-building-co…
munishchouhan Mar 26, 2024
b0a73dd
changes as per review
munishchouhan Apr 10, 2024
d0f6e05
Merge remote-tracking branch 'origin/master' into 396-support-custom-…
munishchouhan Apr 10, 2024
a1c0dad
changes as per review
munishchouhan Apr 10, 2024
49b16e7
minor change
munishchouhan Apr 10, 2024
317b384
Merge branch 'master' into 396-support-custom-labels-when-building-co…
munishchouhan Apr 11, 2024
efdfc1a
changes made as per review
munishchouhan Apr 11, 2024
7cf792b
minor makeup
marcodelapierre Apr 12, 2024
14a7b94
consistent order for readability: labels after env
marcodelapierre Apr 12, 2024
c4dc428
added doc
munishchouhan Apr 16, 2024
5bcaa86
fixed typo
munishchouhan Apr 16, 2024
ae608d5
Merge remote-tracking branch 'origin/master' into 396-support-custom-…
munishchouhan Apr 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 27 additions & 24 deletions docs/api.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,8 @@ The endpoint returns the name of the container request made available by Wave.
entrypoint: string[],
cmd: string[],
env: string[],
workingDir: string
workingDir: string,
labels: string[],
layers: [
{
location: string,
Expand All @@ -47,29 +48,30 @@ The endpoint returns the name of the container request made available by Wave.
```
#### container token request attributes

| Attribute | Description |
| ----------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
| `containerImage` | Name of the container to be served, e.g., `docker.io/library/ubuntu:latest` (optional). If omitted, the `containerFile` must be provided. |
| `containerConfig.entrypoint` | The container entrypoint command e.g. `['/bin/bash']` |
| `containerConfig.cmd` | The launch command to be used by the Wave container, e.g., `['echo', 'Hello world']` (optional). |
| `containerConfig.env` | The environment variables to be defined in the Wave container e.g. `['FOO=one','BAR=two']` (optional). |
| `containerConfig.workingDir` | The work directory to be used in the Wave container e.g. `/some/work/dir` (optional). |
| Attribute | Description |
|-------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `containerImage` | Name of the container to be served, e.g., `docker.io/library/ubuntu:latest` (optional). If omitted, the `containerFile` must be provided. |
| `containerConfig.entrypoint` | The container entrypoint command e.g. `['/bin/bash']` |
| `containerConfig.cmd` | The launch command to be used by the Wave container, e.g., `['echo', 'Hello world']` (optional). |
| `containerConfig.env` | The environment variables to be defined in the Wave container e.g. `['FOO=one','BAR=two']` (optional). |
| `containerConfig.workingDir` | The work directory to be used in the Wave container e.g. `/some/work/dir` (optional). |
| `containerConfig.labels` | The lis of labels to be added in container image build by wave e.g. `['author=foo','org=bar']` (optional). |
| `containerConfig.layers.location` | Either a HTTP URL from where a container layer tar gzip file should be downloaded or tar gzip file base64 encoded and prefixed with `data:` string (optional). |
| `containerConfig.layers.tarDigest` | The SHA256checksum of the provided tar file e.g. `sha256:a7c724b02...`. |
| `containerConfig.layers.gzipDigest` | The SHA256 checksum of the provided layer tar gzip file e.g. `sha256:a7c724b02...`. |
| `containerConfig.layers.gzipSize` | The size in bytes of the the provided layer tar gzip file. |
| `containerFile` | Dockerfile used for building a new container encoded in base64 (optional). When provided, the attribute `containerImage` should be omitted. |
| `condaFile` | Conda environment file encoded as base64 string. |
| `spackFile` | Spack recipe file encoded as base64 string. |
| `containerPlatform` | Target container architecture of the built container e.g. `linux/amd64` (optional). Currently only supporting amd64 and arm64. |
| `buildRepository` | Container repository where container builds should be pushed e.g. `docker.io/user/my-image` (optional). |
| `cacheRepository` | Container repository used to cache build layers `docker.io/user/my-cache` (optional). |
| `timestamp` | Request submission timestap using ISO-8601. |
| `fingerprint` | Request unique fingerprint. |
| `freeze` | The container provisioned will be stored in the specified repository in a permanently. |
| `towerEndpoint` | Seqera Platform service endpoint from where container repositories credentials are retrieved (optional). Default `https://api.cloud.seqera.io`. |
| `towerAccessToken` | Access token of the user account granting the access to the Seqera Platform service specified via `towerEndpoint` (optional). |
| `towerWorkspaceId` | ID of the Seqera Platform workspace from where the container repositories credentials are retrieved (optional). When omitted the personal workspace is used. |
| `containerConfig.layers.tarDigest` | The SHA256checksum of the provided tar file e.g. `sha256:a7c724b02...`. |
| `containerConfig.layers.gzipDigest` | The SHA256 checksum of the provided layer tar gzip file e.g. `sha256:a7c724b02...`. |
| `containerConfig.layers.gzipSize` | The size in bytes of the the provided layer tar gzip file. |
| `containerFile` | Dockerfile used for building a new container encoded in base64 (optional). When provided, the attribute `containerImage` should be omitted. |
| `condaFile` | Conda environment file encoded as base64 string. |
| `spackFile` | Spack recipe file encoded as base64 string. |
| `containerPlatform` | Target container architecture of the built container e.g. `linux/amd64` (optional). Currently only supporting amd64 and arm64. |
| `buildRepository` | Container repository where container builds should be pushed e.g. `docker.io/user/my-image` (optional). |
| `cacheRepository` | Container repository used to cache build layers `docker.io/user/my-cache` (optional). |
| `timestamp` | Request submission timestamp using ISO-8601. |
| `fingerprint` | Request unique fingerprint. |
| `freeze` | The container provisioned will be stored in the specified repository in a permanently. |
| `towerEndpoint` | Seqera Platform service endpoint from where container repositories credentials are retrieved (optional). Default `https://api.cloud.seqera.io`. |
| `towerAccessToken` | Access token of the user account granting the access to the Seqera Platform service specified via `towerEndpoint` (optional). |
| `towerWorkspaceId` | ID of the Seqera Platform workspace from where the container repositories credentials are retrieved (optional). When omitted the personal workspace is used. |

### Response

Expand Down Expand Up @@ -112,7 +114,8 @@ The endpoint returns the name of the container request made available by Wave.
tarDigest: string
},
...
]
],
labels: string[]
},
condaFile: string,
spackFile: string,
Expand Down
10 changes: 10 additions & 0 deletions src/main/groovy/io/seqera/wave/core/ContainerAugmenter.groovy
Original file line number Diff line number Diff line change
Expand Up @@ -356,6 +356,16 @@ class ContainerAugmenter {
if( containerConfig.env ) {
config.Env = appendEnv(config.Env as List, containerConfig.env)
}
if ( containerConfig.labels ) {
def labelsMap = new HashMap<>()
for(String label: containerConfig.labels){
if(label.contains('=')) {
String[] singleLabelArray = label.split("=")
labelsMap.put(singleLabelArray[0], singleLabelArray[1])
}
}
config.Labels = labelsMap
}
if( entryChain ) {
config.Env = appendEnv( config.Env as List, [ "WAVE_ENTRY_CHAIN="+entryChain ] )
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,11 @@ class FreezeServiceImpl implements FreezeService {
result += '%environment\n'
result += " export ${containerConfig.env.join(' ')}\n"
}
//add Labels
if( containerConfig.labels ) {
result += '%labels\n'
result += " ${constructLabels(containerConfig.labels)}\n"
}
// add ENTRY
if( containerConfig.entrypoint ) {
result += '%runscript\n'
Expand Down Expand Up @@ -166,6 +171,10 @@ class FreezeServiceImpl implements FreezeService {
if( containerConfig.env ) {
result += "ENV ${containerConfig.env.join(' ')}\n"
}
// add Labels
if( containerConfig.labels ) {
result += "LABEL ${constructLabels(containerConfig.labels)}\n"
}
// add ENTRY
if( containerConfig.entrypoint ) {
result += "ENTRYPOINT ${containerConfig.entrypoint.collect(it->"\"$it\"")}\n"
Expand All @@ -177,4 +186,10 @@ class FreezeServiceImpl implements FreezeService {
return result
}

static String constructLabels(List<String> labels) {
return labels.collect {label ->
def labelParts = label.split('=')
"${labelParts[0]}=\"${labelParts[1]}\""
}.join(' ')
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -554,7 +554,7 @@ class ContainerAugmenterTest extends Specification {
def manifest = new JsonSlurper().parseText(MANIFEST) as Map

and:
def config = new ContainerConfig(workingDir: '/some/work/dir',entrypoint: ['my','entry'], env: ['THIS=THAT'])
def config = new ContainerConfig(workingDir: '/some/work/dir',entrypoint: ['my','entry'], env: ['THIS=THAT'], labels: ['key=value','key2=value 2'])
def scanner = new ContainerAugmenter().withContainerConfig(config)
def history = new ArrayList(manifest.history)

Expand All @@ -580,6 +580,7 @@ class ContainerAugmenterTest extends Specification {
v1Config.WorkingDir == '/some/work/dir'
v1Config.Entrypoint == ['my','entry']
v1Config.Env == ['PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'THIS=THAT']
v1Config.Labels == ['key2':'value 2', 'key':'value']

}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ class FreezeServiceImplTest extends Specification {
config = new ContainerConfig(
workingDir: '/some/work/dir',
env: ['FOO=one', 'BAR=two'],
labels: ['FOO=one', 'BAR=two 2'],
cmd:['/this','--that'],
entrypoint: ['/my','--entry'],
layers: layers)
Expand All @@ -75,6 +76,7 @@ class FreezeServiceImplTest extends Specification {
ADD layer-999fff.tar.gz /
WORKDIR /some/work/dir
ENV FOO=one BAR=two
LABEL FOO="one" BAR="two 2"
ENTRYPOINT ["/my", "--entry"]
CMD ["/this", "--that"]
'''.stripIndent()
Expand Down Expand Up @@ -117,7 +119,7 @@ class FreezeServiceImplTest extends Specification {
given:
def l1 = new ContainerLayer('/some/loc', 'digest1')
def l2 = new ContainerLayer('/other/loc', 'digest2')
def config = new ContainerConfig(env:['FOO=1', 'BAR=2'], entrypoint: ['bash', '--this', '--that'], layers: [l1, l2])
def config = new ContainerConfig(env:['FOO=1', 'BAR=2'], entrypoint: ['bash', '--this', '--that'], layers: [l1, l2], labels: ['FOO=1', 'BAR=value 2'])
def req = new SubmitContainerTokenRequest(containerImage: 'ubuntu:latest', freeze: true, format: 'sif', containerConfig: config)
when:
def result = freezeService.createBuildFile(req, Mock(PlatformId))
Expand All @@ -133,6 +135,8 @@ class FreezeServiceImplTest extends Specification {
export FOO=1 BAR=2
%runscript
bash --this --that
%labels
FOO="1" BAR="value 2"
'''.stripIndent(true)

}
Expand Down Expand Up @@ -277,4 +281,9 @@ class FreezeServiceImplTest extends Specification {
From: ubuntu:latest
'''.stripIndent()
}

def'should construct labels' () {
expect:
FreezeServiceImpl.constructLabels(['FOO=1', 'BAR=value 2']) == 'FOO="1" BAR="value 2"'
}
}
Loading