Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Converters - allow passing meta in the run method #6554

Merged
merged 9 commits into from
Dec 15, 2023

Conversation

anakin87
Copy link
Member

Related Issues

Proposed Changes:

  • Make all Converters accept meta in the run method, so that users can provide their own metadata.
    The length of this list should match the number of sources.

How did you test it?

CI, new unit tests

Checklist

@anakin87 anakin87 requested review from a team as code owners December 14, 2023 15:31
@anakin87 anakin87 requested review from dfokina and masci and removed request for a team December 14, 2023 15:31
@github-actions github-actions bot added topic:tests 2.x Related to Haystack v2.0 type:documentation Improvements on the docs labels Dec 14, 2023
@anakin87 anakin87 marked this pull request as draft December 14, 2023 15:31
@anakin87 anakin87 changed the base branch from main to converters-standardize-sources December 14, 2023 15:32
@@ -56,8 +56,8 @@ def __init__(self, endpoint: str, api_key: Optional[str] = None, model_id: str =
self.endpoint = endpoint
self.model_id = model_id

@component.output_types(documents=List[Document], azure=List[Dict])
def run(self, sources: List[Union[str, Path, ByteStream]]):
@component.output_types(documents=List[Document], raw_azure_response=List[Dict])
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not related to this PR but the declaration did not match with the return value of the run method, so I fixed it.

@anakin87 anakin87 marked this pull request as ready for review December 14, 2023 15:34
Copy link
Contributor

@masci masci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 👍

@masci masci merged commit 0c08943 into converters-standardize-sources Dec 15, 2023
2 checks passed
@masci masci deleted the converter-allow-passing-meta branch December 15, 2023 13:48
anakin87 added a commit that referenced this pull request Dec 15, 2023
* standardize converters inputs: first draft

* fix precommit

* fix precommit 2

* fix precommit 3

* add default for optional param

* rm leftover

* install boilerpy in linting workflow

* add boilerpy3 to the core dependencies

* add reno

* remove boilerpy3 installation from test workflow

* fix pylint: import order and unused import

* fix import order

* add release note

* better Tika docstring

* rm boilerpy from linting

* leftover

* md link brackets

* feat: Converters - allow passing `meta` in the `run` method (#6554)

* first impl for html

* progressing on other components

* fix test

* add tests - run with meta

* release note

* reintroduce patches wrongly deleted

* add patch in test

* fix tika test

* Update haystack/components/converters/azure.py

Co-authored-by: Massimiliano Pippi <[email protected]>

---------

Co-authored-by: Massimiliano Pippi <[email protected]>

* Update releasenotes/notes/converters-standardize-inputs-ed2ba9c97b762974.yaml

Co-authored-by: Silvano Cerza <[email protected]>

* simplify test

---------

Co-authored-by: Massimiliano Pippi <[email protected]>
Co-authored-by: Julian Risch <[email protected]>
Co-authored-by: Daria Fokina <[email protected]>
Co-authored-by: Silvano Cerza <[email protected]>
@anakin87 anakin87 mentioned this pull request Dec 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2.x Related to Haystack v2.0 topic:tests type:documentation Improvements on the docs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants