Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(backup): paginate different types of content [WPB-10575] #3268

Conversation

vitorhugods
Copy link
Member

@vitorhugods vitorhugods commented Jan 31, 2025

StoryWPB-10575 Cross Platform Backup: Write common backup / restore library


PR Submission Checklist for internal contributors

  • The PR Title

    • conforms to the style of semantic commits messages¹ supported in Wire's Github Workflow²
    • contains a reference JIRA issue number like SQPIT-764
    • answers the question: If merged, this PR will: ... ³
  • The PR Description

    • is free of optional paragraphs and you have filled the relevant parts to the best of your ability

What's new in this PR?

Issues

We can make it easier to clients to consume the data in the same exact order every time.
Otherwise, clients can shoot themselves in the foot if the data being saved into the DB is not properly sorted, by failing to satisfy DB constraints, for example.

Solutions

Split the pager into smaller pagers. One for each:

  • Conversations
  • Users
  • Messages

These will make sure that the pages are always consumed in order.

Testing

Test Coverage

  • I have added automated test to this contribution

PR Post Merge Checklist for internal contributors

  • If any soft of configuration variable was introduced by this PR, it has been added to the relevant documents and the CI jobs have been updated.

References
  1. https://sparkbox.com/foundry/semantic_commit_messages
  2. https://github.com/wireapp/.github#usage
  3. E.g. feat(conversation-list): Sort conversations by most emojis in the title #SQPIT-764.

@echoes-hq echoes-hq bot added the echoes: product-roadmap/feature Work contributing to adding a new feature as part of the product roadmap. label Jan 31, 2025
@vitorhugods vitorhugods requested a review from saleniuk January 31, 2025 14:22
Copy link
Contributor

github-actions bot commented Jan 31, 2025

Test Results

3 506 tests   3 398 ✅  6m 41s ⏱️
  603 suites    108 💤
  603 files        0 ❌

Results for commit 2b411fd.

♻️ This comment has been updated with latest results.

@codecov-commenter
Copy link

codecov-commenter commented Jan 31, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Please upload report for BASE (epic/multiplatform-backup@d0818c7). Learn more about missing BASE report.

Additional details and impacted files
@@                     Coverage Diff                      @@
##             epic/multiplatform-backup    #3268   +/-   ##
============================================================
  Coverage                             ?   53.90%           
============================================================
  Files                                ?     1303           
  Lines                                ?    37404           
  Branches                             ?     3770           
============================================================
  Hits                                 ?    20162           
  Misses                               ?    15833           
  Partials                             ?     1409           

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d0818c7...2b411fd. Read the comment docs.

@datadog-wireapp
Copy link

datadog-wireapp bot commented Jan 31, 2025

Datadog Report

Branch report: feat/backup/paginate-different-types-of-content
Commit report: 63aadc3
Test service: kalium-jvm

✅ 0 Failed, 3398 Passed, 108 Skipped, 1m 4.84s Total Time

@vitorhugods vitorhugods force-pushed the epic/multiplatform-backup branch from 10253a9 to d0818c7 Compare January 31, 2025 15:21
Consolidated handling of paginated backup data into the new `BackupPage` concept to improve clarity and support pagination for users, conversations, and messages during import. Added tests to validate the refactored `BackupImportPager`.
@vitorhugods vitorhugods force-pushed the feat/backup/paginate-different-types-of-content branch from 4166be1 to 2b411fd Compare January 31, 2025 15:22
@vitorhugods vitorhugods merged commit 6f99a66 into epic/multiplatform-backup Feb 5, 2025
21 checks passed
@vitorhugods vitorhugods deleted the feat/backup/paginate-different-types-of-content branch February 5, 2025 14:42
github-merge-queue bot pushed a commit that referenced this pull request Feb 7, 2025
* chore: create backup module (#3117)

This commit introduces a new cross-platform backup module to the project with support for iOS, Web, and Android. Additionally, it enables JavaScript multiplatform support in existing `kalium/network-model` and `kalium/data` projects.

* chore: create backup protobuf models (#3116)

* feat: add initial crossplatform backup serialization [WPB-10575] (#3121)

* feat: support other message types [WPB-10575] (#3122)

Co-authored-by: Yamil Medina <[email protected]>

* refactor(backup): make visibility modifier mandatory (#3153)

* chore(backup): add encrypted stream [WPB-10575] (#3154)

* chore(backup): read and write the backup header (#3159)

* feat(backup): support export and import of backups on Web, iOS and Android [WPB-10575] (#3228)

* refactor(backup): enhance error handling and test coverage (#3248)

Refactored the backup import process to improve error handling by introducing more granular failure types and modularized the logic for decrypting and unzipping archives. Added comprehensive test coverage, including tests for edge cases in decryption, parsing, and unzipping.

* chore(backup): improve backup export error handling and result encapsulation (#3257)

Introduced `BackupExportResult` and `ExportResult` to encapsulate and represent export operation outcomes, including success and specific failure types (`IOError`, `ZipError`). Refactored relevant methods to use these types, added coroutine support annotations, and implemented error handling for zipping and I/O operations. Added unit tests to ensure correct error handling behavior.

* feat(backup): support file peeking [WPB-10575] (#3261)

Introduced `peekBackup` and related methods to retrieve backup metadata such as version, encryption status, and original creator. Also added unit tests to validate behavior for various cases, including unsupported versions and mismatched user IDs.

* feat(backup): paginate different types of content (#3268)

Consolidated handling of paginated backup data into the new `BackupPage` concept to improve clarity and support pagination for users, conversations, and messages during import. Added tests to validate the refactored `BackupImportPager`.

* docs(backup): add multiplatform sample implementations and documentation (#3269)

Introduced examples showcasing `MPBackupImporter` and `MPBackupExporter` usage for different platforms. Updated documentation, README, and build configurations to support multiplatform samples, including JS and non-JS targets.

---------

Co-authored-by: Yamil Medina <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
echoes: product-roadmap/feature Work contributing to adding a new feature as part of the product roadmap. 👕 size: L type: feature ✨
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants