Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support _deleted selector in RxDB queries #18

Merged
merged 5 commits into from
Oct 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions dist/cjs/rx-query-helper.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion dist/cjs/rx-query-helper.js.map

Large diffs are not rendered by default.

25 changes: 21 additions & 4 deletions dist/cjs/rx-query.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion dist/cjs/rx-query.js.map

Large diffs are not rendered by default.

27 changes: 27 additions & 0 deletions dist/esm/rx-query-helper.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion dist/esm/rx-query-helper.js.map

Large diffs are not rendered by default.

27 changes: 22 additions & 5 deletions dist/esm/rx-query.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion dist/esm/rx-query.js.map

Large diffs are not rendered by default.

8 changes: 7 additions & 1 deletion dist/types/rx-query-helper.d.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import type { DeterministicSortComparator, FilledMangoQuery, MangoQuery, QueryMatcher, RxDocument, RxDocumentData, RxJsonSchema, RxQuery } from './types/index.d.ts';
import type { DeterministicSortComparator, FilledMangoQuery, MangoQuery, MangoQuerySelector, QueryMatcher, RxDocument, RxDocumentData, RxJsonSchema, RxQuery } from './types/index.d.ts';
/**
* Normalize the query to ensure we have all fields set
* and queries that represent the same query logic are detected as equal by the caching.
Expand All @@ -17,3 +17,9 @@ export declare function getSortComparator<RxDocType>(schema: RxJsonSchema<RxDocu
*/
export declare function getQueryMatcher<RxDocType>(_schema: RxJsonSchema<RxDocType> | RxJsonSchema<RxDocumentData<RxDocType>>, query: FilledMangoQuery<RxDocType>): QueryMatcher<RxDocumentData<RxDocType>>;
export declare function runQueryUpdateFunction<RxDocType, RxQueryResult>(rxQuery: RxQuery<RxDocType, RxQueryResult>, fn: (doc: RxDocument<RxDocType>) => Promise<RxDocument<RxDocType>>): Promise<RxQueryResult>;
/**
* Checks if a given selector includes deleted documents.
* @param selector The MangoQuerySelector to check
* @returns True if the selector includes deleted documents, false otherwise
*/
export declare function selectorIncludesDeleted<RxDocType>(selector: MangoQuerySelector<RxDocType> | undefined): boolean;
1 change: 1 addition & 0 deletions dist/types/rx-query.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ export declare class RxQueryBase<RxDocType, RxQueryResult, OrmMethods = {}, Reac
constructor(op: RxQueryOP, mangoQuery: Readonly<MangoQuery<RxDocType>>, collection: RxCollection<RxDocType>, other?: any);
get $(): BehaviorSubject<RxQueryResult>;
get $$(): Reactivity;
get includesDeleted(): boolean;
_latestChangeEvent: -1 | number;
_lastExecStart: number;
_lastExecEnd: number;
Expand Down
1 change: 1 addition & 0 deletions examples/vite-vanilla-ts/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
"test:e2e": "testcafe chrome -e test/ --hostname localhost --selector-timeout 8000 --retry-test-pages"
},
"devDependencies": {
"@swc/core": "1.6.0",
"async-test-util": "2.5.0",
"concurrently": "8.2.2",
"copyfiles": "2.4.1",
Expand Down
61 changes: 61 additions & 0 deletions src/rx-query-helper.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,10 @@ import type {
DeterministicSortComparator,
FilledMangoQuery,
MangoQuery,
MangoQueryOperators,
MangoQuerySelector,
MangoQuerySortDirection,
PropertyType,
QueryMatcher,
RxDocument,
RxDocumentData,
Expand Down Expand Up @@ -248,3 +251,61 @@ export async function runQueryUpdateFunction<RxDocType, RxQueryResult>(
return result as any;
}
}

/**
* Checks if a given selector includes deleted documents.
* @param selector The MangoQuerySelector to check
* @returns True if the selector includes deleted documents, false otherwise
*/
export function selectorIncludesDeleted<RxDocType>(
selector: MangoQuerySelector<RxDocType> | undefined
): boolean {
if (!selector) {
return false;
}

const isTrue = (value: unknown): boolean =>
value === true ||
(typeof value === 'object' &&
value !== null &&
'$eq' in value &&
(value as MangoQueryOperators<boolean>).$eq === true);


const isNotFalse = (value: unknown): boolean =>
value === true ||
(typeof value === 'object' &&
value !== null &&
'$ne' in value &&
(value as MangoQueryOperators<boolean>).$ne === false);

const hasDeletedTrue = (
condition: MangoQuerySelector<RxDocType>
): boolean =>
'_deleted' in condition &&
(isTrue(condition._deleted as PropertyType<RxDocType, '_deleted'>) ||
isNotFalse(
condition._deleted as PropertyType<RxDocType, '_deleted'>
));

if ('_deleted' in selector) {
return (
isTrue(selector._deleted as PropertyType<RxDocType, '_deleted'>) ||
isNotFalse(selector._deleted as PropertyType<RxDocType, '_deleted'>)
);
}

if ('$or' in selector && Array.isArray(selector.$or)) {
return selector.$or.some(hasDeletedTrue);
}

if ('$and' in selector && Array.isArray(selector.$and)) {
return selector.$and.some(hasDeletedTrue);
}

if ('$nor' in selector && Array.isArray(selector.$nor)) {
return !selector.$nor.every((condition) => !hasDeletedTrue(condition));
}

return false;
}
23 changes: 20 additions & 3 deletions src/rx-query.ts
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,8 @@ import {
getQueryMatcher,
getSortComparator,
normalizeMangoQuery,
runQueryUpdateFunction
runQueryUpdateFunction,
selectorIncludesDeleted

} from './rx-query-helper.ts';
import { RxQuerySingleResult } from './rx-query-single-result.ts';
Expand Down Expand Up @@ -199,6 +200,10 @@ export class RxQueryBase<
) as any;
}

get includesDeleted(): boolean {
return selectorIncludesDeleted(this.mangoQuery.selector);
}

// stores the changeEvent-number of the last handled change-event
public _latestChangeEvent: -1 | number = -1;

Expand Down Expand Up @@ -418,7 +423,14 @@ export class RxQueryBase<
)
};

(hookInput.mangoQuery.selector as any)._deleted = { $eq: false };
// Set _deleted to false if not explicitly set in selector
if (!this.includesDeleted) {
hookInput.mangoQuery.selector = {
...hookInput.mangoQuery.selector,
_deleted: { $eq: false },
};
}

if (hookInput.mangoQuery.index) {
hookInput.mangoQuery.index.unshift('_deleted');
}
Expand Down Expand Up @@ -816,7 +828,12 @@ async function __ensureEqual<RxDocType>(rxQuery: RxQueryBase<RxDocType, any>): P
}
}

if (rxQuery.op === 'count') {
if (rxQuery.includesDeleted) {
Copy link

@eliias eliias Oct 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't quite remember, but do we need to specifically handle count operations for any selector that contains a true-like value for _deleted and supports counting via an index (fast count). Or is _deleted always a slow count?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pseudo code

} else if (rxQuery.op === 'count' && rxQuery.includesDeleted) {
   rxQuery._execOverDatabase().then((newResultData) => {
    rxQuery._setResultData(newResultData ? newResultData.length : 0);
    return true;
  });
}

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good point that I have not tested yet. Could I verify this by using the field that Tristan mentioned here 'rxdbOnly.indexFields.triage_status_exists' and then check the correct count?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good point that I have not tested yet. Could I verify this by using the field that Tristan mentioned here 'rxdbOnly.indexFields.triage_status_exists' and then check the correct count?

Yeah… I think this could work, but it is also possible that the query planner “sees” the _deleted field in the selector, and just skips finding the best index, because it is an “unknown field” for the query planner. If you use rxdbOnly.indexFields.triage_status_exists and it actually picks up this index, you can potentially log the usage. https://github.com/readwiseio/rekindled/blob/ac42918be6d214fab78344a115405c9bc39ab3b7/reading-clients/shared/database/internals/storages/storage-sqlite/sqlite-storage-instance.ts#L360 for mobile, you could add a console.log and check which index is picked. It is possible though that no index, which results in RxDB to fall back to the default (this should be id).

return rxQuery._execOverDatabase().then((newResultData) => {
rxQuery._setResultData(newResultData);
return true;
});
} else if (rxQuery.op === 'count') {
// 'count' query
const previousCount = ensureNotFalsy(rxQuery._result).count;
let newCount = previousCount;
Expand Down
Loading