feat: consideration for retry v2

README.md

@@ -163,13 +163,13 @@ Please ensure that the `MarkdownDescription` field is set in the schema for each
 To generate the documentation run either:
 
 ```sh
-$ make docs
+make docs
 ```
 
 or...
 
 ```sh
-$ go generate ./...
+go generate ./...
 ```
 
 ### Templates
@@ -210,6 +210,16 @@ provider_installation {
 }
 ```
 
+## Developer: Using the `skip_on` struct field tag
+
+The `skip_on` struct field tag is used to skip the external API call when only attributes that affect the internal state are modified, e.g. retry configuration. The `skip_on` struct field tag is a comma-separated list of operations that must be met in order to skip the field.
+
+The provider will compare the state with the plan, and check for changes. If the only fields to me modified are those with the `skip_on` struct field tag set to the supplied operation, e.g. `update`, the provider will skip the external API call.
+
+The following operations are supported:
+
+* `update` - Skip the external API call when the operation is an update.
+
 ## Credits
 
 We wish to thank HashiCorp for the use of some MPLv2-licensed code from their open source project [terraform-provider-azurerm](https://github.com/hashicorp/terraform-provider-azurerm).

docs/data-sources/resource.md

@@ -108,7 +108,7 @@ output "quarantine_policy" {
 	```
 
 To learn more about JMESPath, visit [JMESPath](https://jmespath.org/).
-- `retry` (Attributes) The retry block supports the following arguments: (see [below for nested schema](#nestedatt--retry))
+- `retry` (Attributes) The retry object supports the following attributes: (see [below for nested schema](#nestedatt--retry))
 - `timeouts` (Block, Optional) (see [below for nested schema](#nestedblock--timeouts))
 
 ### Read-Only
@@ -134,16 +134,16 @@ To learn more about JMESPath, visit [JMESPath](https://jmespath.org/).
 <a id="nestedatt--retry"></a>
 ### Nested Schema for `retry`
 
-Required:
-
-- `error_message_regex` (List of String) A list of regular expressions to match against error messages. If any of the regular expressions match, the error is considered retryable.
-
 Optional:
 
+- `error_message_regex` (List of String) A list of regular expressions to match against error messages. If any of the regular expressions match, the request will be retried.
 - `interval_seconds` (Number) The base number of seconds to wait between retries. Default is `10`.
 - `max_interval_seconds` (Number) The maximum number of seconds to wait between retries. Default is `180`.
 - `multiplier` (Number) The multiplier to apply to the interval between retries. Default is `1.5`.
 - `randomization_factor` (Number) The randomization factor to apply to the interval between retries. The formula for the randomized interval is: `RetryInterval * (random value in range [1 - RandomizationFactor, 1 + RandomizationFactor])`. Therefore set to zero `0.0` for no randomization. Default is `0.5`.
+- `response_is_nil` (Boolean) If set to `true`, the retry will be triggered when the response is `nil`. Default is `true`.
+- `status_forbidden` (Boolean) If set to `true`, the retry will be triggered when the status code is `403`. Default is `false`.
+- `status_not_found` (Boolean) If set to `true`, the retry will be triggered when the status code is `404`. Default is `false`.
 
 
 <a id="nestedblock--timeouts"></a>

docs/data-sources/resource_action.md

@@ -87,7 +87,7 @@ data "azapi_resource_action" "example" {
 	```
 
 To learn more about JMESPath, visit [JMESPath](https://jmespath.org/).
-- `retry` (Attributes) The retry block supports the following arguments: (see [below for nested schema](#nestedatt--retry))
+- `retry` (Attributes) The retry object supports the following attributes: (see [below for nested schema](#nestedatt--retry))
 - `sensitive_response_export_values` (Dynamic) The attribute can accept either a list or a map.
 
 - **List**: A list of paths that need to be exported from the response body. Setting it to `["*"]` will export the full response body. Here's an example. If it sets to `["properties.loginServer", "properties.policies.quarantinePolicy.status"]`, it will set the following HCL object to the computed property sensitive_output.
@@ -152,16 +152,16 @@ To learn more about JMESPath, visit [JMESPath](https://jmespath.org/).
 <a id="nestedatt--retry"></a>
 ### Nested Schema for `retry`
 
-Required:
-
-- `error_message_regex` (List of String) A list of regular expressions to match against error messages. If any of the regular expressions match, the error is considered retryable.
-
 Optional:
 
+- `error_message_regex` (List of String) A list of regular expressions to match against error messages. If any of the regular expressions match, the request will be retried.
 - `interval_seconds` (Number) The base number of seconds to wait between retries. Default is `10`.
 - `max_interval_seconds` (Number) The maximum number of seconds to wait between retries. Default is `180`.
 - `multiplier` (Number) The multiplier to apply to the interval between retries. Default is `1.5`.
 - `randomization_factor` (Number) The randomization factor to apply to the interval between retries. The formula for the randomized interval is: `RetryInterval * (random value in range [1 - RandomizationFactor, 1 + RandomizationFactor])`. Therefore set to zero `0.0` for no randomization. Default is `0.5`.
+- `response_is_nil` (Boolean) If set to `true`, the retry will be triggered when the response is `nil`. Default is `true`.
+- `status_forbidden` (Boolean) If set to `true`, the retry will be triggered when the status code is `403`. Default is `false`.
+- `status_not_found` (Boolean) If set to `true`, the retry will be triggered when the status code is `404`. Default is `false`.
 
 
 <a id="nestedblock--timeouts"></a>

docs/data-sources/resource_list.md

@@ -113,7 +113,7 @@ data "azapi_resource_list" "listSubnetsByVnet" {
 	```
 
 To learn more about JMESPath, visit [JMESPath](https://jmespath.org/).
-- `retry` (Attributes) The retry block supports the following arguments: (see [below for nested schema](#nestedatt--retry))
+- `retry` (Attributes) The retry object supports the following attributes: (see [below for nested schema](#nestedatt--retry))
 - `timeouts` (Block, Optional) (see [below for nested schema](#nestedblock--timeouts))
 
 ### Read-Only
@@ -136,16 +136,16 @@ To learn more about JMESPath, visit [JMESPath](https://jmespath.org/).
 <a id="nestedatt--retry"></a>
 ### Nested Schema for `retry`
 
-Required:
-
-- `error_message_regex` (List of String) A list of regular expressions to match against error messages. If any of the regular expressions match, the error is considered retryable.
-
 Optional:
 
+- `error_message_regex` (List of String) A list of regular expressions to match against error messages. If any of the regular expressions match, the request will be retried.
 - `interval_seconds` (Number) The base number of seconds to wait between retries. Default is `10`.
 - `max_interval_seconds` (Number) The maximum number of seconds to wait between retries. Default is `180`.
 - `multiplier` (Number) The multiplier to apply to the interval between retries. Default is `1.5`.
 - `randomization_factor` (Number) The randomization factor to apply to the interval between retries. The formula for the randomized interval is: `RetryInterval * (random value in range [1 - RandomizationFactor, 1 + RandomizationFactor])`. Therefore set to zero `0.0` for no randomization. Default is `0.5`.
+- `response_is_nil` (Boolean) If set to `true`, the retry will be triggered when the response is `nil`. Default is `true`.
+- `status_forbidden` (Boolean) If set to `true`, the retry will be triggered when the status code is `403`. Default is `false`.
+- `status_not_found` (Boolean) If set to `true`, the retry will be triggered when the status code is `404`. Default is `false`.
 
 
 <a id="nestedblock--timeouts"></a>

docs/ephemeral-resources/resource_action.md

@@ -98,7 +98,7 @@ ephemeral "azapi_resource_action" "listKeys" {
 	```
 
 To learn more about JMESPath, visit [JMESPath](https://jmespath.org/).
-- `retry` (Attributes) The retry block supports the following arguments: (see [below for nested schema](#nestedatt--retry))
+- `retry` (Attributes) The retry object supports the following attributes: (see [below for nested schema](#nestedatt--retry))
 - `timeouts` (Block, Optional) (see [below for nested schema](#nestedblock--timeouts))
 
 ### Read-Only
@@ -121,16 +121,16 @@ To learn more about JMESPath, visit [JMESPath](https://jmespath.org/).
 <a id="nestedatt--retry"></a>
 ### Nested Schema for `retry`
 
-Required:
-
-- `error_message_regex` (List of String) A list of regular expressions to match against error messages. If any of the regular expressions match, the error is considered retryable.
-
 Optional:
 
+- `error_message_regex` (List of String) A list of regular expressions to match against error messages. If any of the regular expressions match, the request will be retried.
 - `interval_seconds` (Number) The base number of seconds to wait between retries. Default is `10`.
 - `max_interval_seconds` (Number) The maximum number of seconds to wait between retries. Default is `180`.
 - `multiplier` (Number) The multiplier to apply to the interval between retries. Default is `1.5`.
 - `randomization_factor` (Number) The randomization factor to apply to the interval between retries. The formula for the randomized interval is: `RetryInterval * (random value in range [1 - RandomizationFactor, 1 + RandomizationFactor])`. Therefore set to zero `0.0` for no randomization. Default is `0.5`.
+- `response_is_nil` (Boolean) If set to `true`, the retry will be triggered when the response is `nil`. Default is `true`.
+- `status_forbidden` (Boolean) If set to `true`, the retry will be triggered when the status code is `403`. Default is `false`.
+- `status_not_found` (Boolean) If set to `true`, the retry will be triggered when the status code is `404`. Default is `false`.
 
 
 <a id="nestedblock--timeouts"></a>

docs/guides/retry-configuration.md

@@ -0,0 +1,60 @@
+---
+layout: "azapi"
+page_title: "Feature: Retry Configuration"
+description: |-
+  This guide will describe how to configure the retry behavior in the AzAPI provider. The retry configuration allows you to control how the provider handles failed requests and if it should retry them.
+---
+
+## Why retry?
+
+Sometimes, when managing cloud infrastructure, requests to the cloud provider may fail due to transient issues such as network problems, timeouts, eventual consistency, or rate limiting. In these cases, it can be beneficial to retry the request a few times before giving up. The AzAPI provider allows you to configure the retry behavior to suit your needs.
+
+## Retry configuration
+
+There are two types of retry configurations available in the AzAPI provider:
+
+1. Provider retry configuration
+2. Resource-specific retry configuration
+
+### Provider retry configuration
+
+The provider retry configuration is a global configuration that applies to all resources managed by the provider. You can configure the provider retry behavior by setting the following provider values:
+
+- `maximum_busy_retry_attempts`
+
+This value controls the number of times the provider will retry a failed request. The default value is `3`.
+A retry will be triggered if the request fails with HTTP 408, 429, 500, 502, 503, or 504.
+
+In the case that the response header contains a `Retry-After` value, the provider will wait for the specified duration before retrying the request.
+
+### Resource-specific retry configuration
+
+In addition to the provider retry configuration, you can also configure the retry behavior for individual resources. This allows you to fine-tune the retry behavior for specific resources.
+
+The resource-specific retry comes after the provider retry, that is to say that the provider retry will be attempted first, and if it fails or exceeds the maximum attempts, the resource-specific retry will be attempted.
+Note that the resource-specific retry does not honour the `Retry-After` header and is exponential backoff based.
+
+Resource specific retry is configured using the `retry` attribute.
+With `azapi_resource` and `azapi_data_plane_resource` resources, the provider performs a read after the resource has been created/updated to collect the read-only attributes of the resource. In this case there is a second retry attribute called `retry_read_after_create`, which controls the retry behavior of this operation.
+
+If you configure a retry configuration, the maximum elapsed time for the retry will be set to the resource's timeout value for that operation (create, update, read, delete).
+
+The schema of these retry attributes is as follows:
+
+- `error_message_regex` - A list of regular expressions to match against error messages. If any of the regular expressions match, the request will be retried.
+- `interval_seconds` - The initial number of seconds to wait before the 1st retry. The default value is `10`.
+- `max_interval_seconds` - The maximum number of times to retry the request. The default value is `180`.
+- `multiplier` - The multiplier to apply to the interval between retries. The default value is `1.5`.
+- `randomization_factor` - The randomization factor to apply to the interval between retries. The default value is `0.5`. The formula for the randomized interval is: `RetryInterval * (random value in range [1 - RandomizationFactor, 1 + RandomizationFactor])`. Set to zero `0.0` for no randomization.
+- `response_is_nil` - If the response is nil, should the request be retried. The default value is `true`.
+- `status_forbidden` - If the status code is 403, should the request be retried. The default value is `false`.
+- `status_not_found` - If the status code is 404, should the request be retried. The default value is `false`.
+
+## Default resource-specific retry configuration
+
+If you do not configure any retry values, the provider will use the following:
+
+For the initial create/read/update/delete operation we will retry on HTTP 429 status codes for a maximum time of 2 minutes.
+
+For the read-after-create operation we will retry on HTTP 404, 403 status codes as well as a nill response.
+We will do this up to the operation timeout configured in the `timeouts {}` block.

docs/index.md

@@ -72,6 +72,7 @@ provider "azapi" {
 - `enable_preflight` (Boolean) Enable Preflight Validation. The default is false. When set to true, the provider will use Preflight to do static validation before really deploying a new resource. When set to false, the provider will disable this validation.
 - `endpoint` (Attributes List) The Azure API Endpoint Configuration. (see [below for nested schema](#nestedatt--endpoint))
 - `environment` (String) The Cloud Environment which should be used. Possible values are `public`, `usgovernment` and `china`. Defaults to `public`. This can also be sourced from the `ARM_ENVIRONMENT` Environment Variable.
+- `maximum_busy_retry_attempts` (Number) The maximum number of retries to attempt if the Azure API returns an HTTP 408, 429, 500, 502, 503, or 504 response. The default is `3`. The resource-specific retry configuration may additionally be used to retry on other errors and conditions.
 - `oidc_azure_service_connection_id` (String) The Azure Pipelines Service Connection ID to use for authentication. This can also be sourced from the `ARM_OIDC_AZURE_SERVICE_CONNECTION_ID` environment variable.
 - `oidc_request_token` (String) The bearer token for the request to the OIDC provider. This can also be sourced from the `ARM_OIDC_REQUEST_TOKEN` or `ACTIONS_ID_TOKEN_REQUEST_TOKEN` Environment Variables.
 - `oidc_request_url` (String) The URL for the OIDC provider from which to request an ID token. This can also be sourced from the `ARM_OIDC_REQUEST_URL` or `ACTIONS_ID_TOKEN_REQUEST_URL` Environment Variables.

docs/resources/data_plane_resource.md

@@ -137,7 +137,8 @@ resource "azapi_data_plane_resource" "example" {
 	```
 
 To learn more about JMESPath, visit [JMESPath](https://jmespath.org/).
-- `retry` (Attributes) The retry block supports the following arguments: (see [below for nested schema](#nestedatt--retry))
+- `retry` (Attributes) The retry object supports the following attributes: (see [below for nested schema](#nestedatt--retry))
+- `retry_read_after_create` (Attributes) The retry object supports the following attributes: (see [below for nested schema](#nestedatt--retry_read_after_create))
 - `timeouts` (Block, Optional) (see [below for nested schema](#nestedblock--timeouts))
 - `update_headers` (Map of String) A mapping of headers to be sent with the update request.
 - `update_query_parameters` (Map of List of String) A mapping of query parameters to be sent with the update request.
@@ -162,16 +163,31 @@ To learn more about JMESPath, visit [JMESPath](https://jmespath.org/).
 <a id="nestedatt--retry"></a>
 ### Nested Schema for `retry`
 
-Required:
+Optional:
+
+- `error_message_regex` (List of String) A list of regular expressions to match against error messages. If any of the regular expressions match, the request will be retried.
+- `interval_seconds` (Number) The base number of seconds to wait between retries. Default is `10`.
+- `max_interval_seconds` (Number) The maximum number of seconds to wait between retries. Default is `180`.
+- `multiplier` (Number) The multiplier to apply to the interval between retries. Default is `1.5`.
+- `randomization_factor` (Number) The randomization factor to apply to the interval between retries. The formula for the randomized interval is: `RetryInterval * (random value in range [1 - RandomizationFactor, 1 + RandomizationFactor])`. Therefore set to zero `0.0` for no randomization. Default is `0.5`.
+- `response_is_nil` (Boolean) If set to `true`, the retry will be triggered when the response is `nil`. Default is `true`.
+- `status_forbidden` (Boolean) If set to `true`, the retry will be triggered when the status code is `403`. Default is `false`.
+- `status_not_found` (Boolean) If set to `true`, the retry will be triggered when the status code is `404`. Default is `false`.
+
 
-- `error_message_regex` (List of String) A list of regular expressions to match against error messages. If any of the regular expressions match, the error is considered retryable.
+<a id="nestedatt--retry_read_after_create"></a>
+### Nested Schema for `retry_read_after_create`
 
 Optional:
 
+- `error_message_regex` (List of String) A list of regular expressions to match against error messages. If any of the regular expressions match, the request will be retried.
 - `interval_seconds` (Number) The base number of seconds to wait between retries. Default is `10`.
 - `max_interval_seconds` (Number) The maximum number of seconds to wait between retries. Default is `180`.
 - `multiplier` (Number) The multiplier to apply to the interval between retries. Default is `1.5`.
 - `randomization_factor` (Number) The randomization factor to apply to the interval between retries. The formula for the randomized interval is: `RetryInterval * (random value in range [1 - RandomizationFactor, 1 + RandomizationFactor])`. Therefore set to zero `0.0` for no randomization. Default is `0.5`.
+- `response_is_nil` (Boolean) If set to `true`, the retry will be triggered when the response is `nil`. Default is `true`.
+- `status_forbidden` (Boolean) If set to `true`, the retry will be triggered when the status code is `403`. Default is `false`.
+- `status_not_found` (Boolean) If set to `true`, the retry will be triggered when the status code is `404`. Default is `false`.
 
 
 <a id="nestedblock--timeouts"></a>