Skip to content

Add support for Patch endpoints #291

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 40 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
e0bd94f
Adding patch endpoints to transactions extension to elasticsearch.
rhysrevans3 Aug 29, 2024
01c1563
Adding patch to opensearch backend.
rhysrevans3 Aug 29, 2024
d171c24
Pinning to pull request version for tests.
rhysrevans3 Aug 30, 2024
59661c5
Merge branch 'main' of github.com:stac-utils/stac-fastapi-elasticsear…
rhysrevans3 Aug 30, 2024
5400e42
Updating patch types.
rhysrevans3 Sep 16, 2024
fe8530a
Adding checks for existing properties.
rhysrevans3 Sep 17, 2024
7cf36eb
Updating utils.
rhysrevans3 Nov 19, 2024
5929b00
Adding model for path.
rhysrevans3 Mar 26, 2025
d94d8fe
Switch to use pydantic model for operation path.
rhysrevans3 Mar 26, 2025
4623a43
Merge branch 'main' of github.com:stac-utils/stac-fastapi-elasticsear…
rhysrevans3 Mar 26, 2025
093c593
Adding computed_field output type.
rhysrevans3 Mar 26, 2025
d4bd07a
pre-commit.
rhysrevans3 Mar 26, 2025
b6b0721
Add changes to opensearch.
rhysrevans3 Mar 26, 2025
90a287f
pre-commit.
rhysrevans3 Mar 26, 2025
f1c320a
Switch to model validator.
rhysrevans3 Mar 26, 2025
a2d6f48
Simplify conversion logic.
rhysrevans3 Mar 27, 2025
3a75b68
Opensearch update body not script.
rhysrevans3 Mar 27, 2025
ee90324
pre-commit.
rhysrevans3 Mar 27, 2025
173ef0d
Remove duplicate test name.
rhysrevans3 Mar 27, 2025
93350c7
PatchOperation not dict for tests.
rhysrevans3 Mar 27, 2025
bf6f96a
pre-commit.
rhysrevans3 Mar 27, 2025
29f6e2e
filter_expr not filter for aggregation request.
rhysrevans3 Mar 31, 2025
3d5b168
pre-commit.
rhysrevans3 Mar 31, 2025
ca64ba3
Filter fix.
rhysrevans3 Mar 31, 2025
b412576
remove PatchOperation in tests.
rhysrevans3 Mar 31, 2025
7368d8d
patch bugs.
rhysrevans3 Mar 31, 2025
2b359f6
Switch to http exception.
rhysrevans3 Mar 31, 2025
0274b64
test_item_search_temporal_window_get fix.
rhysrevans3 Mar 31, 2025
1c4fe43
Request not BadRequest for opensearch.
rhysrevans3 Mar 31, 2025
fb6fe82
Opensearch update body.
rhysrevans3 Mar 31, 2025
fa6afb3
Adding collection patch tests.
rhysrevans3 Apr 1, 2025
b1aa252
Array list fixes.
rhysrevans3 Apr 1, 2025
5c972d1
Consolidating add and copy commands.
rhysrevans3 Apr 1, 2025
b532058
remove debug prints.
rhysrevans3 Apr 1, 2025
3ffe9d5
Move extension replacement to ElasticPath.
rhysrevans3 Apr 2, 2025
0822417
Reset command set between different patches.
rhysrevans3 Apr 2, 2025
050254f
Correcting bad check command.
rhysrevans3 Apr 2, 2025
b5544dd
Update opensearch collection patch.
rhysrevans3 Apr 2, 2025
5887227
RequestError not BadRequestError for opensearch.
rhysrevans3 Apr 2, 2025
2b13818
Remove python 3.8 support.
rhysrevans3 Apr 2, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/cicd.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ jobs:

strategy:
matrix:
python-version: [ "3.8", "3.9", "3.10", "3.11", "3.12", "3.13"]
python-version: [ "3.9", "3.10", "3.11", "3.12", "3.13"]
backend: [ "elasticsearch7", "elasticsearch8", "opensearch"]

name: Python ${{ matrix.python-version }} testing with ${{ matrix.backend }}
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/deploy_mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,10 @@ jobs:
- name: Checkout main
uses: actions/checkout@v4

- name: Set up Python 3.8
- name: Set up Python 3.9
uses: actions/setup-python@v5
with:
python-version: 3.8
python-version: 3.9

- name: Install dependencies
run: |
Expand Down
3 changes: 3 additions & 0 deletions dockerfiles/Dockerfile.deploy.es
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,12 @@ FROM python:3.10-slim
RUN apt-get update && \
apt-get -y upgrade && \
apt-get -y install gcc && \
apt-get -y install build-essential git && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*



ENV CURL_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt

WORKDIR /app
Expand Down
2 changes: 1 addition & 1 deletion dockerfiles/Dockerfile.dev.es
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ FROM python:3.10-slim
# update apt pkgs, and install build-essential for ciso8601
RUN apt-get update && \
apt-get -y upgrade && \
apt-get install -y build-essential git && \
apt-get -y install build-essential git && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*

Expand Down
3 changes: 2 additions & 1 deletion dockerfiles/Dockerfile.dev.os
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,11 @@ FROM python:3.10-slim
# update apt pkgs, and install build-essential for ciso8601
RUN apt-get update && \
apt-get -y upgrade && \
apt-get install -y build-essential && \
apt-get -y install build-essential && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*

RUN apt-get -y install git
# update certs used by Requests
ENV CURL_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt

Expand Down
2 changes: 1 addition & 1 deletion dockerfiles/Dockerfile.docs
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM python:3.8-slim
FROM python:3.9-slim

# build-essential is required to build a wheel for ciso8601
RUN apt update && apt install -y build-essential
Expand Down
9 changes: 4 additions & 5 deletions stac_fastapi/core/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@
"attrs>=23.2.0",
"pydantic",
"stac_pydantic>=3",
"stac-fastapi.types==3.0.0",
"stac-fastapi.api==3.0.0",
"stac-fastapi.extensions==3.0.0",
"stac-fastapi.types@git+https://github.com/stac-utils/stac-fastapi.git@refs/pull/744/head#subdirectory=stac_fastapi/types",
"stac-fastapi.api@git+https://github.com/stac-utils/stac-fastapi.git@refs/pull/744/head#subdirectory=stac_fastapi/api",
"stac-fastapi.extensions@git+https://github.com/stac-utils/stac-fastapi.git@refs/pull/744/head#subdirectory=stac_fastapi/extensions",
"orjson",
"overrides",
"geojson-pydantic",
Expand All @@ -26,12 +26,11 @@
description="Core library for the Elasticsearch and Opensearch stac-fastapi backends.",
long_description=desc,
long_description_content_type="text/markdown",
python_requires=">=3.8",
python_requires=">=3.9",
classifiers=[
"Intended Audience :: Developers",
"Intended Audience :: Information Technology",
"Intended Audience :: Science/Research",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
Expand Down
48 changes: 47 additions & 1 deletion stac_fastapi/core/stac_fastapi/core/base_database_logic.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
"""Base database logic."""

import abc
from typing import Any, Dict, Iterable, Optional
from typing import Any, Dict, Iterable, List, Optional


class BaseDatabaseLogic(abc.ABC):
Expand Down Expand Up @@ -29,6 +29,30 @@ async def create_item(self, item: Dict, refresh: bool = False) -> None:
"""Create an item in the database."""
pass

@abc.abstractmethod
async def merge_patch_item(
self,
collection_id: str,
item_id: str,
item: Dict,
base_url: str,
refresh: bool = True,
) -> Dict:
"""Patch a item in the database follows RF7396."""
pass

@abc.abstractmethod
async def json_patch_item(
self,
collection_id: str,
item_id: str,
operations: List,
base_url: str,
refresh: bool = True,
) -> Dict:
"""Patch a item in the database follows RF6902."""
pass

@abc.abstractmethod
async def delete_item(
self, item_id: str, collection_id: str, refresh: bool = False
Expand All @@ -41,6 +65,28 @@ async def create_collection(self, collection: Dict, refresh: bool = False) -> No
"""Create a collection in the database."""
pass

@abc.abstractmethod
async def merge_patch_collection(
self,
collection_id: str,
collection: Dict,
base_url: str,
refresh: bool = True,
) -> Dict:
"""Patch a collection in the database follows RF7396."""
pass

@abc.abstractmethod
async def json_patch_collection(
self,
collection_id: str,
operations: List,
base_url: str,
refresh: bool = True,
) -> Dict:
"""Patch a collection in the database follows RF6902."""
pass

@abc.abstractmethod
async def find_collection(self, collection_id: str) -> Dict:
"""Find a collection in the database."""
Expand Down
131 changes: 120 additions & 11 deletions stac_fastapi/core/stac_fastapi/core/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -377,7 +377,7 @@ async def get_item(

@staticmethod
def _return_date(
interval: Optional[Union[DateTimeType, str]]
interval: Optional[Union[DateTimeType, str]],
) -> Dict[str, Optional[str]]:
"""
Convert a date interval.
Expand Down Expand Up @@ -459,7 +459,7 @@ async def get_search(
sortby: Optional[str] = None,
q: Optional[List[str]] = None,
intersects: Optional[str] = None,
filter: Optional[str] = None,
filter_expr: Optional[str] = None,
filter_lang: Optional[str] = None,
**kwargs,
) -> stac_types.ItemCollection:
Expand Down Expand Up @@ -493,11 +493,9 @@ async def get_search(
"token": token,
"query": orjson.loads(query) if query else query,
"q": q,
"datetime": datetime,
}

if datetime:
base_args["datetime"] = self._format_datetime_range(datetime)

if intersects:
base_args["intersects"] = orjson.loads(unquote_plus(intersects))

Expand All @@ -507,12 +505,12 @@ async def get_search(
for sort in sortby
]

if filter:
if filter_expr:
base_args["filter-lang"] = "cql2-json"
base_args["filter"] = orjson.loads(
unquote_plus(filter)
unquote_plus(filter_expr)
if filter_lang == "cql2-json"
else to_cql2(parse_cql2_text(filter))
else to_cql2(parse_cql2_text(filter_expr))
)

if fields:
Expand Down Expand Up @@ -594,10 +592,11 @@ async def post_search(
)

# only cql2_json is supported here
if hasattr(search_request, "filter"):
cql2_filter = getattr(search_request, "filter", None)
if search_request.filter_expr:
try:
search = self.database.apply_cql2_filter(search, cql2_filter)
search = self.database.apply_cql2_filter(
search, search_request.filter_expr
)
except Exception as e:
raise HTTPException(
status_code=400, detail=f"Error with cql2_json filter: {e}"
Expand Down Expand Up @@ -734,6 +733,62 @@ async def update_item(

return ItemSerializer.db_to_stac(item, base_url)

@overrides
async def merge_patch_item(
self, collection_id: str, item_id: str, item: stac_types.PartialItem, **kwargs
) -> Optional[stac_types.Item]:
"""Patch an item in the collection following RF7396..

Args:
collection_id (str): The ID of the collection the item belongs to.
item_id (str): The ID of the item to be updated.
item (stac_types.PartialItem): The partial item data.
kwargs: Other optional arguments, including the request object.

Returns:
stac_types.Item: The patched item object.

"""
base_url = str(kwargs["request"].base_url)

item = await self.database.merge_patch_item(
collection_id=collection_id,
item_id=item_id,
item=item,
base_url=base_url,
)
return ItemSerializer.db_to_stac(item, base_url=base_url)

@overrides
async def json_patch_item(
self,
collection_id: str,
item_id: str,
operations: List[stac_types.PatchOperation],
**kwargs,
) -> Optional[stac_types.Item]:
"""Patch an item in the collection following RF6902.

Args:
collection_id (str): The ID of the collection the item belongs to.
item_id (str): The ID of the item to be updated.
operations (List): List of operations to run on item.
kwargs: Other optional arguments, including the request object.

Returns:
stac_types.Item: The patched item object.

"""
base_url = str(kwargs["request"].base_url)

item = await self.database.json_patch_item(
collection_id=collection_id,
item_id=item_id,
base_url=base_url,
operations=operations,
)
return ItemSerializer.db_to_stac(item, base_url=base_url)

@overrides
async def delete_item(
self, item_id: str, collection_id: str, **kwargs
Expand Down Expand Up @@ -814,6 +869,60 @@ async def update_collection(
extensions=[type(ext).__name__ for ext in self.database.extensions],
)

@overrides
async def merge_patch_collection(
self, collection_id: str, collection: stac_types.PartialCollection, **kwargs
) -> Optional[stac_types.Collection]:
"""Patch a collection following RF7396..

Args:
collection_id (str): The ID of the collection to patch.
collection (stac_types.Collection): The partial collection data.
kwargs: Other optional arguments, including the request object.

Returns:
stac_types.Collection: The patched collection object.

"""
collection = await self.database.merge_patch_collection(
collection_id=collection_id,
base_url=str(kwargs["request"].base_url),
collection=collection,
)

return CollectionSerializer.db_to_stac(
collection,
kwargs["request"],
extensions=[type(ext).__name__ for ext in self.database.extensions],
)

@overrides
async def json_patch_collection(
self, collection_id: str, operations: List[stac_types.PatchOperation], **kwargs
) -> Optional[stac_types.Collection]:
"""Patch a collection following RF6902.

Args:
collection_id (str): The ID of the collection to patch.
operations (List): List of operations to run on collection.
kwargs: Other optional arguments, including the request object.

Returns:
stac_types.Collection: The patched collection object.

"""
collection = await self.database.json_patch_collection(
collection_id=collection_id,
operations=operations,
base_url=str(kwargs["request"].base_url),
)

return CollectionSerializer.db_to_stac(
collection,
kwargs["request"],
extensions=[type(ext).__name__ for ext in self.database.extensions],
)

@overrides
async def delete_collection(
self, collection_id: str, **kwargs
Expand Down
Loading