Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/s3 management #676

Merged
merged 33 commits into from
Nov 26, 2024
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
a79edee
feat: added accessKeys resource
rmocanu-ionos Sep 17, 2024
d84c3c2
fixed some plugin framework compatibilities
rmocanu-ionos Sep 18, 2024
456d7ba
feat: added data sources, some refactoring
rmocanu-ionos Sep 20, 2024
80e997c
fix: working data sources, use valueString and other fixes
rmocanu-ionos Sep 24, 2024
583b17b
fix: getting version and terraform version like in the old provider
rmocanu-ionos Sep 24, 2024
2a093e2
fix: rename bucket references
rmocanu-ionos Sep 24, 2024
61003e9
doc: added docs
rmocanu-ionos Sep 25, 2024
6376432
test: added tests wip, renamed files
rmocanu-ionos Sep 25, 2024
a5f0fea
test: fix tests
rmocanu-ionos Sep 26, 2024
466c487
fix: get specific client from bundle, added some checks to tests
rmocanu-ionos Sep 27, 2024
4b51cbf
fix: check for err and return if not nil to reduce complexity
rmocanu-ionos Sep 27, 2024
57dd3aa
doc: added comments
rmocanu-ionos Sep 27, 2024
554bf1e
fix: made s3 work with sdk bundle
rmocanu-ionos Sep 27, 2024
4724253
refactor: move cloudapi client initialization in method, sonar fixes
rmocanu-ionos Sep 27, 2024
e28f281
doc: added comments
rmocanu-ionos Sep 27, 2024
51b16c0
Merge branch 'master' into feat/s3-management
rmocanu-ionos Sep 27, 2024
19f2115
sonar fixes
rmocanu-ionos Sep 27, 2024
51920f0
added the sdk to run linter
rmocanu-ionos Sep 30, 2024
361e152
fix: linter issues
rmocanu-ionos Sep 30, 2024
ec42e76
fix: linter issues 2
rmocanu-ionos Sep 30, 2024
57473b1
fix: try fix imports
rmocanu-ionos Sep 30, 2024
f98abc7
fix: try fix imports 2
rmocanu-ionos Sep 30, 2024
ce3b8ac
chore: added sdk from repo, some renaming changes
rmocanu-ionos Nov 19, 2024
ef54732
Merge branch 'master' into feat/s3-management
rmocanu-ionos Nov 19, 2024
ba7653d
chore: rename to object-storage-management
rmocanu-ionos Nov 21, 2024
1997c64
chore: pr review changes and linter
rmocanu-ionos Nov 21, 2024
7a384ec
chore: linter error
rmocanu-ionos Nov 21, 2024
2e2bade
chore: undo linter changes
rmocanu-ionos Nov 21, 2024
41e4198
fix: make description computed
rmocanu-ionos Nov 25, 2024
0c46952
Merge branch 'master' into feat/s3-management
rmocanu-ionos Nov 25, 2024
045db72
feat: added option to get access key using accesskeyid or description
rmocanu-ionos Nov 25, 2024
3e7544b
doc: added changelog, summary, use sdk from github
rmocanu-ionos Nov 25, 2024
50958b2
fix: added check for when no identifier is set or calls fail, linter …
rmocanu-ionos Nov 25, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions docs/data-sources/s3_accesskey.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
---
subcategory: "S3 management"
layout: "ionoscloud"
page_title: "IonosCloud : s3_accesskey"
sidebar_current: "docs-datasource-s3_accesskey"
description: |-
Get information on a IonosCloud S3 Accesskeys
---

# ionoscloud\_s3\_accesskey

The **S3 Accesskey data source** can be used to search for and return an existing S3 Accesskeys.

## Example Usage

### By ID
```hcl
data "ionoscloud_s3_accesskey" "example" {
id = <accesskey_id>
}
```

## Argument Reference

* `id` - (Required) Id of an existing S3 accesskey that you want to search for.

## Attributes Reference

The following attributes are returned by the datasource:

- `id` - The ID (UUID) of the AccessKey.
- `description` - Description of the Access key.
- `accesskey` - Access key metadata is a string of 92 characters.
- `canonical_user_id` - The canonical user ID which is valid for user-owned buckets.
- `contract_user_id` - The contract user ID which is valid for contract-owned buckets
39 changes: 39 additions & 0 deletions docs/data-sources/s3_region.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
---
subcategory: "S3 management"
layout: "ionoscloud"
page_title: "IonosCloud : s3_region"
sidebar_current: "docs-datasource-s3_region"
description: |-
Get information on a IonosCloud S3 Region
---

# ionoscloud\_s3\_region
rmocanu-ionos marked this conversation as resolved.
Show resolved Hide resolved

The **S3 region data source** can be used to search for and return an existing S3 Regions.

## Example Usage

### By ID
```hcl
data "ionoscloud_s3_region" "example" {
id = <region_id>
rmocanu-ionos marked this conversation as resolved.
Show resolved Hide resolved
}
```

## Argument Reference

* `id` - (Required) Id of an existing S3 Region that you want to search for.

## Attributes Reference

The following attributes are returned by the datasource:

- `id` - The id of the region
- `version` - The version of the region properties
- `endpoint` - The endpoint URL for the region
- `website` - The website URL for the region
- `storage_classes` - The available classes in the region
- `location` - The data center location of the region as per [Get Location](/docs/cloud/v6/#tag/Locations/operation/locationsGet). *Can't be used as `LocationConstraint` on bucket creation.*
- `capability` - The capabilities of the region
* `iam` - Indicates if IAM policy based access is supported
* `s3select` - Indicates if S3 Select is supported
45 changes: 45 additions & 0 deletions docs/resources/s3_accesskey.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
---
subcategory: "S3 management"
layout: "ionoscloud"
page_title: "IonosCloud: s3_accesskey"
sidebar_current: "docs-resource-s3_accesskey"
description: |-
Creates and manages IonosCloud S3 Accesskeys.
---

# ionoscloud_s3_accesskey

Manages an **S3 Accesskey** on IonosCloud.

## Example Usage

```hcl
resource "ionoscloud_s3_accesskey" "example" {
description = "my description"
}
```

## Argument Reference

The following arguments are supported:

- `description` - (Optional)[string] Description of the Access key.
- `id` - (Computed) The ID (UUID) of the AccessKey.
- `accesskey` - (Computed) Access key metadata is a string of 92 characters.
- `secretkey` - (Computed) The secret key of the Access key.
- `canonical_user_id` - (Computed) The canonical user ID which is valid for user-owned buckets.
- `contract_user_id` - (Computed) The contract user ID which is valid for contract-owned buckets
- `timeouts` - (Optional) Timeouts for this resource.
- `create` - (Optional)[string] Time to wait for the bucket to be created. Default is `10m`.
- `delete` - (Optional)[string] Time to wait for the bucket to be deleted. Default is `10m`.
- `force_destroy` - (Optional)[bool] If true, the bucket and the contents of the bucket will be destroyed. Default is `false`.

## Import

An S3 accesskey resource can be imported using its `resource id`, e.g.

```shell
terraform import ionoscloud_s3_accesskey.demo {s3AccesskeyId}
```

This can be helpful when you want to import S3 Accesskeys which you have already created manually or using other means, outside of terraform.
87 changes: 85 additions & 2 deletions internal/framework/provider/provider.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,42 @@ package provider

import (
"context"
"fmt"
"net/http"
"os"
"runtime"

"github.com/hashicorp/terraform-plugin-framework/datasource"
"github.com/hashicorp/terraform-plugin-framework/path"
"github.com/hashicorp/terraform-plugin-framework/provider"
"github.com/hashicorp/terraform-plugin-framework/provider/schema"
"github.com/hashicorp/terraform-plugin-framework/resource"
"github.com/hashicorp/terraform-plugin-framework/types"
"github.com/hashicorp/terraform-plugin-sdk/v2/meta"

ionoscloud "github.com/ionos-cloud/sdk-go/v6"

"github.com/ionos-cloud/terraform-provider-ionoscloud/v6/internal/framework/services/s3"
s3service "github.com/ionos-cloud/terraform-provider-ionoscloud/v6/services/s3"
"github.com/ionos-cloud/terraform-provider-ionoscloud/v6/internal/framework/services/s3management"
"github.com/ionos-cloud/terraform-provider-ionoscloud/v6/services"
apiGatewayService "github.com/ionos-cloud/terraform-provider-ionoscloud/v6/services/apigateway"
autoscalingService "github.com/ionos-cloud/terraform-provider-ionoscloud/v6/services/autoscaling"
cdnService "github.com/ionos-cloud/terraform-provider-ionoscloud/v6/services/cdn"
"github.com/ionos-cloud/terraform-provider-ionoscloud/v6/services/cert"
crService "github.com/ionos-cloud/terraform-provider-ionoscloud/v6/services/containerregistry"
dataplatformService "github.com/ionos-cloud/terraform-provider-ionoscloud/v6/services/dataplatform"
dbaasService "github.com/ionos-cloud/terraform-provider-ionoscloud/v6/services/dbaas"
"github.com/ionos-cloud/terraform-provider-ionoscloud/v6/services/dbaas/inmemorydb"
"github.com/ionos-cloud/terraform-provider-ionoscloud/v6/services/dbaas/mariadb"
dnsService "github.com/ionos-cloud/terraform-provider-ionoscloud/v6/services/dns"
kafkaService "github.com/ionos-cloud/terraform-provider-ionoscloud/v6/services/kafka"
loggingService "github.com/ionos-cloud/terraform-provider-ionoscloud/v6/services/logging"
nfsService "github.com/ionos-cloud/terraform-provider-ionoscloud/v6/services/nfs"
s3Service "github.com/ionos-cloud/terraform-provider-ionoscloud/v6/services/s3"
s3managementService "github.com/ionos-cloud/terraform-provider-ionoscloud/v6/services/s3management"
"github.com/ionos-cloud/terraform-provider-ionoscloud/v6/services/vpn"
"github.com/ionos-cloud/terraform-provider-ionoscloud/v6/utils"
"github.com/ionos-cloud/terraform-provider-ionoscloud/v6/utils/constant"
)

// ClientOptions is the configuration for the provider.
Expand Down Expand Up @@ -120,6 +145,8 @@ func (p *IonosCloudProvider) Configure(ctx context.Context, req provider.Configu
accessKey := os.Getenv("IONOS_S3_ACCESS_KEY")
secretKey := os.Getenv("IONOS_S3_SECRET_KEY")
region := os.Getenv("IONOS_S3_REGION")
endpoint := os.Getenv("IONOS_API_URL")
terraformVersion := req.TerraformVersion

if !clientOpts.Token.IsNull() {
token = clientOpts.Token.ValueString()
Expand All @@ -145,6 +172,10 @@ func (p *IonosCloudProvider) Configure(ctx context.Context, req provider.Configu
region = clientOpts.S3Region.ValueString()
}

if !clientOpts.Endpoint.IsNull() {
endpoint = clientOpts.Endpoint.ValueString()
}

if token == "" && (username == "" || password == "") {
resp.Diagnostics.AddError("missing credentials", "either token or username and password must be set")
}
Expand All @@ -153,7 +184,47 @@ func (p *IonosCloudProvider) Configure(ctx context.Context, req provider.Configu
return
}

client := s3service.NewClient(accessKey, secretKey, region)
cleanedEndpoint := cleanURL(endpoint)
version := "DEV"

newConfig := ionoscloud.NewConfiguration(username, password, token, endpoint)
newConfig.UserAgent = fmt.Sprintf(
"terraform-provider/%s_ionos-cloud-sdk-go/%s_hashicorp-terraform/%s_terraform-plugin-sdk/%s_os/%s_arch/%s",
version, ionoscloud.Version, terraformVersion, meta.SDKVersionString(), runtime.GOOS, runtime.GOARCH, //nolint:staticcheck
)
if os.Getenv(constant.IonosDebug) != "" {
newConfig.Debug = true
}
newConfig.MaxRetries = constant.MaxRetries
newConfig.WaitTime = constant.MaxWaitTime
newConfig.HTTPClient = &http.Client{Transport: utils.CreateTransport()}
cloudapiClient := ionoscloud.NewAPIClient(newConfig)
digna-ionos marked this conversation as resolved.
Show resolved Hide resolved

version = ionoscloud.Version

client := &services.SdkBundle{
CDNClient: cdnService.NewCDNClient(username, password, token, endpoint, version, terraformVersion),
AutoscalingClient: autoscalingService.NewClient(username, password, token, cleanedEndpoint, version, terraformVersion),
CertManagerClient: cert.NewClient(username, password, token, cleanedEndpoint, version, terraformVersion),
CloudApiClient: cloudapiClient,
ContainerClient: crService.NewClient(username, password, token, cleanedEndpoint, version, terraformVersion),
DataplatformClient: dataplatformService.NewClient(username, password, token, cleanedEndpoint, version, terraformVersion),
DNSClient: dnsService.NewClient(username, password, token, cleanedEndpoint, version, terraformVersion),
LoggingClient: loggingService.NewClient(username, password, token, cleanedEndpoint, terraformVersion),
MariaDBClient: mariadb.NewMariaDBClient(username, password, token, cleanedEndpoint, version, terraformVersion),
MongoClient: dbaasService.NewMongoClient(username, password, token, cleanedEndpoint, version, terraformVersion),
NFSClient: nfsService.NewClient(username, password, token, cleanedEndpoint, version, terraformVersion),
PsqlClient: dbaasService.NewPsqlClient(username, password, token, cleanedEndpoint, version, terraformVersion),
KafkaClient: kafkaService.NewClient(username, password, token, cleanedEndpoint, version, terraformVersion),
APIGatewayClient: apiGatewayService.NewClient(
username, password, token, cleanedEndpoint, version, terraformVersion,
),
VPNClient: vpn.NewClient(username, password, token, cleanedEndpoint, terraformVersion),
InMemoryDBClient: inmemorydb.NewInMemoryDBClient(username, password, token, cleanedEndpoint, version, terraformVersion),
S3Client: s3Service.NewClient(accessKey, secretKey, region),
S3ManagementClient: s3managementService.NewClient(username, password, token, cleanedEndpoint, version, terraformVersion),
}

resp.DataSourceData = client
resp.ResourceData = client
}
Expand All @@ -172,6 +243,7 @@ func (p *IonosCloudProvider) Resources(_ context.Context) []func() resource.Reso
s3.NewBucketCorsConfigurationResource,
s3.NewBucketLifecycleConfigurationResource,
s3.NewBucketWebsiteConfigurationResource,
s3management.NewAccesskeyResource,
}
}

Expand All @@ -182,5 +254,16 @@ func (p *IonosCloudProvider) DataSources(_ context.Context) []func() datasource.
s3.NewObjectDataSource,
s3.NewBucketPolicyDataSource,
s3.NewObjectsDataSource,
s3management.NewRegionDataSource,
s3management.NewAccesskeyDataSource,
}
}

func cleanURL(url string) string {
cristiGuranIonos marked this conversation as resolved.
Show resolved Hide resolved
length := len(url)
if length > 1 && url[length-1] == '/' {
url = url[:length-1]
}

return url
}
108 changes: 108 additions & 0 deletions internal/framework/services/s3management/data_source_s3_accesskey.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
package s3management

import (
"context"
"fmt"

"github.com/ionos-cloud/terraform-provider-ionoscloud/v6/services"
"github.com/ionos-cloud/terraform-provider-ionoscloud/v6/services/s3management"
s3managementService "github.com/ionos-cloud/terraform-provider-ionoscloud/v6/services/s3management"

"github.com/hashicorp/terraform-plugin-framework/datasource"
"github.com/hashicorp/terraform-plugin-framework/datasource/schema"
)

var _ datasource.DataSourceWithConfigure = (*accessKeyDataSource)(nil)

// NewAccesskeyDataSource creates a new data source for the accesskey resource.
func NewAccesskeyDataSource() datasource.DataSource {
return &accessKeyDataSource{}
}

type accessKeyDataSource struct {
client *services.SdkBundle
}

// Metadata returns the metadata for the data source.
func (d *accessKeyDataSource) Metadata(ctx context.Context, req datasource.MetadataRequest, resp *datasource.MetadataResponse) {
resp.TypeName = req.ProviderTypeName + "_s3_accesskey"
}

// Configure configures the data source.
func (d *accessKeyDataSource) Configure(ctx context.Context, req datasource.ConfigureRequest, resp *datasource.ConfigureResponse) {
// Add a nil check when handling ProviderData because Terraform
// sets that data after it calls the ConfigureProvider RPC.
if req.ProviderData == nil {
return
}

client, ok := req.ProviderData.(*services.SdkBundle)
digna-ionos marked this conversation as resolved.
Show resolved Hide resolved
if !ok {
resp.Diagnostics.AddError(
"Unexpected Data Source Configure Type",
fmt.Sprintf("Expected *s3.Client, got: %T. Please report this issue to the provider developers.", req.ProviderData),
digna-ionos marked this conversation as resolved.
Show resolved Hide resolved
)

return
}

d.client = client
}

// Schema returns the schema for the data source.
func (d *accessKeyDataSource) Schema(ctx context.Context, req datasource.SchemaRequest, resp *datasource.SchemaResponse) {
resp.Schema = schema.Schema{
Attributes: map[string]schema.Attribute{
"id": schema.StringAttribute{
Required: true,
Description: "The ID (UUID) of the AccessKey.",
},
"description": schema.StringAttribute{
Description: "Description of the Access key.",
Optional: true,
},
"accesskey": schema.StringAttribute{
Description: "Access key metadata is a string of 92 characters.",
Computed: true,
},
"canonical_user_id": schema.StringAttribute{
Description: "The canonical user ID which is valid for user-owned buckets.",
Computed: true,
},
"contract_user_id": schema.StringAttribute{
Description: "The contract user ID which is valid for contract-owned buckets",
Computed: true,
},
},
}
}

// Read reads the data source.
func (d *accessKeyDataSource) Read(ctx context.Context, req datasource.ReadRequest, resp *datasource.ReadResponse) {
if d.client == nil {
resp.Diagnostics.AddError("s3 api client not configured", "The provider client is not configured")
return
}

var data *s3management.AccessKeyDataSourceModel
resp.Diagnostics.Append(req.Config.Get(ctx, &data)...)
if resp.Diagnostics.HasError() {
return
}

id := data.ID.String()

accessKey, apiResponse, err := d.client.S3ManagementClient.GetAccessKey(ctx, id)

if apiResponse.HttpNotFound() {
resp.Diagnostics.AddError("accesskey not found", "The accesskey was not found")
return
}
if err != nil {
resp.Diagnostics.AddError("an error occurred while fetching the accesskey with", err.Error())
return
}

s3managementService.SetAccessKeyPropertiesToDataSourcePlan(data, accessKey)
resp.Diagnostics.Append(resp.State.Set(ctx, &data)...)
}
Loading
Loading