Skip to content

confluence swallows any error that occurs during pagination #2394

@seanstory

Description

@seanstory

Part of #2393

Description

The confluence connector currently swallows errors during its paginated_api_call() function. See:

except Exception as exception:
self._logger.warning(
f"Skipping data for type {url_name} from {url}. Exception: {exception}."
)
break

It would be better for these to be re-raised if they can't be handled, and for callers to be able to determine how significant the error is. Otherwise, critical calls (like the first attempt to list all pages) could fail, leading to a mostly empty sync that thinks it's successful.

Acceptance Criteria

  • paginated_api_call catches some types of errors, and retries/handles the ones it can
  • if paginated_api_call cannot handle an error, it re-raises it
  • calling code of paginated_api_call handles errors as makes sense for the use case. We balance the need to "fail fast" with the need to not crash a large sync because of a small data issue.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions