Skip to content

Issue with OP_MSG when cursor_id overflow 32bits #153

Open
@pierreMizrahi

Description

@pierreMizrahi

On a database where cursors overflow 32bits, responses that are big enough to trigger the "MoreToCome" mechanism fail with a 'fromJust' Error.

I think that there is a slight misinterpretation of the mongodb OP_MSG doc in the current implementation of the library :

id' = fromJust $ cast $ valueAt "id" doc' :: Int32

id', which is the gets casted as an Int32, whereas the specification says it is an Int64.

https://github.com/mongodb/specifications/blob/39add29e0a3b6721688845874383878b90811a12/source/change-streams/change-streams.md?plain=1#L319

The confusion might come from the fact that this is believed to be a ResponseId (Int32), on which the compatibility with the RequestId is checked, whereas it is actually a CursorId (Int64).

What happens is that this Int64 fails at being casted as an Int32, returning Nothing, and this fails unsafely because of the 'fromJust'.

I don't think it is necessary (nor possible) to check each batch of document. Note that the responseId is actually checked from the header (line 292).

Here is a very minimal patch that solves the issue for me: just remove the id check altogether:

diff --git a/Database/MongoDB/Internal/Protocol.hs b/Database/MongoDB/Internal/Protocol.hs
index dbf7630..08758ab 100644
--- a/Database/MongoDB/Internal/Protocol.hs
+++ b/Database/MongoDB/Internal/Protocol.hs
@@ -309,8 +309,7 @@ callOpMsg pipe request flagBit params = do
                         let (docs, docs') =
                               ( fromJust $ cast $ valueAt "nextBatch" doc :: [Document]
                               , fromJust $ cast $ valueAt "nextBatch" doc' :: [Document])
-                            id' = fromJust $ cast $ valueAt "id" doc' :: Int32
-                        (rt, check id' (rt, rep{ sections = docs' ++ docs })) -- todo: avoid (++)
+                        (rt, rep{sections= docs' ++ docs}) -- todo: avoid (++)
                         -- Since we use this to process moreToCome messages, we
                         -- know that there will be a nextBatch key in the document
                       _ ->  error "Impossible"

Please note that I haven't tried to make a simple reproducer outside of my production database, and I don't really expect it to be that easy.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions