FEATURE: Create a version of stream.Recv that returns bytes

[jchen42703]

Problem

Sometimes, when you call stream.Recv, you get json.Unmarshal errors, such as:

• invalid character ',' after top-level value
• unexpected end of JSON input
• looking for beginning of value
• invalid character ' ' in string escape code

This occurs because the stream.Recv implementation as of v2.12.4 is:

// Recv reads a message from the stream, returning io.EOF when
// all the messages have been read.
func (s Stream[T]) Recv() (T, error) {
	var value T
	bytes, err := s.reader.ReadFromStream()
	if err != nil {
		return value, err
	}
	if err := json.Unmarshal(bytes, &value); err != nil {
		return value, err
	}
	return value, nil
}

This occurs because the actual streamed value from reader.ReadFromStream is sometimes not properly JSON formatted, such as:

• \n
• truncated JSONs for long citations

Solution

Make fern generate an additional stream method like:

// Recv reads a message from the stream, returning io.EOF when
// all the messages have been read.
func (s Stream[T]) RecvRaw() ([]byte, error) {
	bytes, err := s.reader.ReadFromStream()
	if err != nil {
		return value, err
	}
	return bytes, nil
}

This way the user can call stream.RecvRaw() to handle the json.Unmarshal dynamically with some sort of JSON repairing.

Update: Created a fork that does exactly this:
https://github.com/Munchpass/cohere-go

Reproduce

The errors I encountered can be reproduced with any streaming workflow with tools with the SDK. It happens relatively rarely though.

[billytrend-cohere]

Thank you for your in depth research into this! I have shared with @amckinney who works on the code gen. Will see if we can fix upstream or if we should merge your fix. Appreciate you sharing this

[billytrend-cohere]

are you using the v1 chat or v2 chat? v2 chat uses SSE so may be more robust for you!

[jchen42703]

are you using the v1 chat or v2 chat? v2 chat uses SSE so may be more robust for you!

We're using the V2 chat with tools + citations enabled:

	stream, err := b.cohereClient.V2.ChatStream(ctx, &cohere.V2ChatStreamRequest{
		Model:           model,
		Messages:        messages,
		Tools:           tools,
		CitationOptions: citationOpts,
	})
	if err != nil {
		return nil, err
	}