Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Multiple Export Formats for lino pull Command #309

Open
youen opened this issue Aug 28, 2024 · 1 comment
Open

Support Multiple Export Formats for lino pull Command #309

youen opened this issue Aug 28, 2024 · 1 comment

Comments

@youen
Copy link
Collaborator

youen commented Aug 28, 2024

Currently, the lino pull command supports exporting data in JSONL format. While JSONL is a flexible and widely-used format, it may not always be suitable for all use cases. To address the needs of a broader range of users and applications, we propose enhancing the lino pull command to support multiple export formats.

New Formats to Support:

  1. SQL: Generate SQL scripts that can be executed or saved as files to initialize a test database. This is particularly useful for testing applications that do not support JSONL, such as Spring Boot applications.
  2. Avro/Parquet: Optimize CPU consumption and file compression by using binary serialization formats like Avro or Parquet.

Proposed Changes:

  • Add a --format flag to the lino pull command, allowing users to specify the export format.
  • Implement export options for SQL and Avro/Parquet formats.
  • Ensure compatibility with existing flags and options in the lino pull command.
  • Update the documentation to reflect the new export formats and usage examples.

Impact:

  • Users will have more flexibility in choosing the format that best suits their application's needs.
  • Improve performance and storage efficiency for users who require compressed binary formats.

Acceptance Criteria:

  • The lino pull command successfully exports data in the specified format (SQL, Avro, Parquet).
  • Documentation and usage examples are updated.
  • Comprehensive tests are added for each export format.
@youen
Copy link
Collaborator Author

youen commented Aug 28, 2024

ping @adrienaury

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant