Skip to content

[BLOG]Introducing Enhanced JSON Capabilities in Valkey #265

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

roshkhatri
Copy link
Member

Description

Adds a blog "Introducing Enhanced JSON Capabilities in Valkey" to the website

Check List

  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the BSD-3-Clause License.

Copy link
Member

@madolson madolson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's a good start, and will make a great blog. My only high level comment is it would be nice if the examples and the use cases lined up more. Right now they feel a little bit disconnected and I don't fully understand how to connect the two. I think either we interleave the examples with the use cases (Start with reference data, then do batch insertion, then do conditional updates).


It makes it that simple to query and filter through JSON documents.

### **Unlocking Real-World Potential with Valkey-JSON**
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
### **Unlocking Real-World Potential with Valkey-JSON**
### Unlocking Real-World Potential with Valkey-JSON

You shouldn't need to bold headers

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One other use case I have heard about was optimistic locking. You can do a conditional set, where a given row is a certain value and check to see if it returned or not.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you talk about optimistic locking, please rephrase the heading 'Unlocking' (unless you intend the pun, but I think it's confusing...)


This helps you find all users from the `state` of Washington - `"WA"`.

### Atomic Multi-Path Updates
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section is doing GET operations, it doesn't look like it's doing updates.

Comment on lines 8 to 12
On April 4, 2025 Valkey announced the GA release a powerful new upgrade—JSON support. With the introduction of native JSON data types, developers can now store, query, and manipulate structured data directly within Valkey without needing external serialization or complex parsing logic. This enhancement not only simplifies data handling for modern applications but also unlocks new capabilities for building real-time, data-driven experiences at scale. Whether you’re managing complex user profiles, caching API responses, or powering analytics dashboards, JSON support in Valkey streamlines data management while maintaining the speed and efficiency Valkey is known for.

JSON has become the universal format for data exchange, but effectively managing JSON data at scale remains a challenge in caching layers. The Valkey-JSON module addresses this by providing a dedicated JSON data type and a rich set of commands for inserting, updating, and retrieving JSON content using JSONPath queries. This approach reduces application complexity and accelerates performance, making it ideal for modern, data-driven applications.

Let’s first cover the key capabilities and performance of Valkey-JSON. We will then dive into how to work with Valkey-JSON and demonstrate how to get started with a real world example.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
On April 4, 2025 Valkey announced the GA release a powerful new upgrade—JSON support. With the introduction of native JSON data types, developers can now store, query, and manipulate structured data directly within Valkey without needing external serialization or complex parsing logic. This enhancement not only simplifies data handling for modern applications but also unlocks new capabilities for building real-time, data-driven experiences at scale. Whether you’re managing complex user profiles, caching API responses, or powering analytics dashboards, JSON support in Valkey streamlines data management while maintaining the speed and efficiency Valkey is known for.
JSON has become the universal format for data exchange, but effectively managing JSON data at scale remains a challenge in caching layers. The Valkey-JSON module addresses this by providing a dedicated JSON data type and a rich set of commands for inserting, updating, and retrieving JSON content using JSONPath queries. This approach reduces application complexity and accelerates performance, making it ideal for modern, data-driven applications.
Let’s first cover the key capabilities and performance of Valkey-JSON. We will then dive into how to work with Valkey-JSON and demonstrate how to get started with a real world example.
JSON is the universal format for semi-structured data, and modern developers expect to work with it natively across their stack, including in-memory stores like Valkey. Until now, using JSON in Valkey meant serializing entire objects into strings or flattening them into Hashes with limited nesting. These workarounds added complexity and made updates harder than they should be.
That changes with the general availability of native JSON support in Valkey. You can now store, query, and update JSON documents directly, without manual parsing or transformation. This brings a cleaner model to working with semi-structured data and makes your code easier to write and maintain.
In this post, I’ll walk through installing the Valkey JSON module and demonstrate how to use it to solve some common workloads.

I don't know, the current version did not spark joy in me. Here is an alternative suggestion that is hopefully uses a bit fewer buzzwords.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like Madelyn's wording a little better. Commenting on comments is a little clunky in GitHub, so I have some feedback on @madolson's version.

  • 'JSON is the universal format' I wouldn't use the definite article (see TOML, YAML, etc.), maybe something like 'JSON is a ubiquitous universal format'
  • I might be more harsh on serialized JSON. There are scenarios where you can lose updates if you're serializing in updates (without Valkey JSON).
  • You don't 'solve' workloads.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Edited this section


### **Per-User Event Counters for Ad or Notification Delivery**

Valkey JSON excels in high-throughput systems that require tracking per-user counters for ad impressions, push notifications, or message deliveries. For example, an ad platform may store a JSON document per user with nested metadata for each campaign — including impression counts, last delivery timestamps, and click history. Instead of serializing and deserializing large blobs, Valkey JSON enables in-place updates using `JSON.NUMINCRBY` or `JSON.SET` on specific paths (e.g., `$.ad_campaigns.ad_123.count`). This reduces network I/O and latency while ensuring atomicity. Microservices can also retrieve only the required subfields using JSONPath queries, like `$.ad_campaigns.ad_123.lastSeen`, allowing for efficient real-time decisioning. Compared to alternatives like managing multiple hash keys or plain strings, this approach is both cleaner and faster — making it well-suited for ad tech and notification delivery platforms operating at millions of ops/sec under tight latency constraints.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to be a bit more focused. There is a lot of "for X, Y, Z" and "may for example" in this paragraph. Just pick a use case and dive deep in to it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Valkey JSON excels in high-throughput systems ...

This seems sales-y and you say it excels at something but I don't see where the rest of the paragraph supports that. If you're going to say it's good at something, prove it or drop the quality verb.


Ensuring smooth integration across data stores and libraries is vital for minimizing migration friction and maximizing developer productivity. Valkey-JSON is fully compatible with **Valkey 8.0 and above**, making it accessible to users on the latest Valkey versions without requiring additional configuration.

It is also compliant with **RFC7159 and ECMA-404**, adhering to widely accepted JSON standards. This alignment enables consistent JSON handling across Valkey modules and supports integration with **Valkey-Search**, allowing full-text indexing and querying across JSON datasets for advanced analytics and data exploration. Additionally, Valkey-JSON integrates with **Valkey’s Access Control Lists (ACLs)**, providing fine-grained permissions for JSON commands alongside other data types like strings and hashes.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is Valkey search really dependent on the RFC spec? I assume it's more tightly coupled than that.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The RFC specs are syntax and semantic definitions of JSON, they don't actually have anything to do with search.

Comment on lines 168 to 173
Dive deeper into Valkey-JSON and see how easy it is to supercharge your JSON workloads with Valkey today!

For further exploration, refer to:

* [Valkey Official Documentation](https://valkey.io/topics/valkey-json/)
* [Valkey JSON GitHub Repository](https://github.com/valkey-io/valkey-json)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need a bit more concise of a call to action. Link to the documentation for learning more and link to the github if you have issues or feature requests.


### **Identity Graph and Profile Storage at Scale**

For companies operating large-scale identity platforms — such as those in fintech, healthtech, or fraud detection — managing complex user or entity profiles is a core requirement. These profiles often include deeply nested data like names, contact info, document verification, scores, and historical activity. Valkey JSON allows each profile to be stored as a single JSON document and updated atomically as new data arrives, without needing to rewrite the entire object. Queries like `$.email`, `$.history[-1]`, or `$.risk.score` can be executed efficiently with sub-millisecond latency. When paired with Valkey Search, indexed queries on nested fields become possible — enabling real-time lookups like “find all profiles with unverified addresses in California.” This architecture supports hundreds of thousands of concurrent reads and writes per second and can scale to multi-terabyte datasets using hybrid RAM + Flash configurations. For workloads that demand both schema flexibility and ultra-low latency, Valkey JSON offers a compelling alternative to rigid relational databases or slower document stores.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this mystical hybrid RAM + Flash configuration ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'When paired with Valkey Search, indexed queries on nested fields become possible' -> wait... Valkey Search is only VSS at the moment - this isn't possible.

In distributed systems like games, e-commerce platforms, or internal developer tools, multiple microservices often need fast access to consistent, structured reference data. This can include things like product attributes, game character metadata, tax codes, or ID mappings — which are naturally stored as JSON documents. Valkey JSON provides an ideal solution for centralizing this reference data in-memory. Teams can store large JSON blobs (hundreds of KB or more) using `JSON.SET`, and services can retrieve targeted subfields using path expressions like `$.items[?(@.rarity=="epic")]` or `$.idToName["1234"]`. Updates happen in bulk during patch releases or deployment cycles, but reads are constant and latency-sensitive. By keeping this metadata in Valkey, services avoid making remote API calls or parsing local files, achieving very low lookup latency even under load. This pattern greatly simplifies infrastructure, improves cache coherency, and is especially powerful in cloud-native environments where rapid bootstrapping and shared context matter.


### **Identity Graph and Profile Storage at Scale**
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really get this use case. I suppose I also don't really get how it's different from the previous use case.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In distributed systems like games, e-commerce platforms, or internal developer tools

I'm not sure I'd make the leap that these are all distributed systems.

Copy link
Member

@stockholmux stockholmux left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs some work.

Comment on lines 8 to 12
On April 4, 2025 Valkey announced the GA release a powerful new upgrade—JSON support. With the introduction of native JSON data types, developers can now store, query, and manipulate structured data directly within Valkey without needing external serialization or complex parsing logic. This enhancement not only simplifies data handling for modern applications but also unlocks new capabilities for building real-time, data-driven experiences at scale. Whether you’re managing complex user profiles, caching API responses, or powering analytics dashboards, JSON support in Valkey streamlines data management while maintaining the speed and efficiency Valkey is known for.

JSON has become the universal format for data exchange, but effectively managing JSON data at scale remains a challenge in caching layers. The Valkey-JSON module addresses this by providing a dedicated JSON data type and a rich set of commands for inserting, updating, and retrieving JSON content using JSONPath queries. This approach reduces application complexity and accelerates performance, making it ideal for modern, data-driven applications.

Let’s first cover the key capabilities and performance of Valkey-JSON. We will then dive into how to work with Valkey-JSON and demonstrate how to get started with a real world example.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like Madelyn's wording a little better. Commenting on comments is a little clunky in GitHub, so I have some feedback on @madolson's version.

  • 'JSON is the universal format' I wouldn't use the definite article (see TOML, YAML, etc.), maybe something like 'JSON is a ubiquitous universal format'
  • I might be more harsh on serialized JSON. There are scenarios where you can lose updates if you're serializing in updates (without Valkey JSON).
  • You don't 'solve' workloads.


Valkey’s JSON support goes beyond basic storage and retrieval. It introduces powerful querying and filtering capabilities that streamline complex data operations. Tasks that once required extensive development time and multiple network requests can now be executed through a single JSON command, significantly enhancing developer productivity and reducing operational costs.

Valkey-JSON supports six data types—**null, boolean, number, string, object, and array**—allowing developers to represent complex, nested data structures without the constraints of traditional string serialization. Unlike other composite types, JSON objects and arrays in Valkey can contain any combination of the six value types, enabling deeply nested and recursive data models to be stored natively.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Do you 'enhance' productivity? 'Increase' might be a better fit.
  • How Valkey JSON reduce operational costs?
  • Is it "Valkey-JSON" or "Valkey JSON"?

Check the server logs to confirm:

```
Module 'json' loaded successfully.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agree. MODULE LIST is also more universally available as not all users will have ready access to logs.

{\"name\":\"Charlie\",\"email\":\"[email protected]\",\"city\":\"Austin\",\"state\":\"TX\"}]"
```

That was easy! But what if we wanted to only retrieve the cities associated with our list of users rather than pulling all user information:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally, it's better to avoid calling anything 'easy' or 'hard'.

What's 'easy' to use by one person might be hard for another - just avoid this kind of statement is probably better.


## Integration with the Valkey Ecosystem

Ensuring smooth integration across data stores and libraries is vital for minimizing migration friction and maximizing developer productivity. Valkey-JSON is fully compatible with **Valkey 8.0 and above**, making it accessible to users on the latest Valkey versions without requiring additional configuration.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

making it accessible to users on the latest Valkey versions without requiring additional configuration.

What does this mean?


The introduction of the Valkey JSON module is a significant milestone for developers building modern, data-driven applications. By bringing native JSON support into Valkey’s high-performance in-memory engine, it bridges the gap between structured document modeling and real-time responsiveness. Whether you’re updating millions of ad counters per second, serving game metadata to distributed microservices, or powering identity search across terabytes of profiles, ValkeyJSON simplifies your architecture while boosting speed and scalability. With Valkey 8.0 and beyond, developers gain a powerful, intuitive, and production-grade toolset to manipulate JSON data at scale — all with the familiar simplicity and reliability of Valkey.

Dive deeper into Valkey-JSON and see how easy it is to supercharge your JSON workloads with Valkey today!
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoid easy.


The introduction of the Valkey JSON module is a significant milestone for developers building modern, data-driven applications. By bringing native JSON support into Valkey’s high-performance in-memory engine, it bridges the gap between structured document modeling and real-time responsiveness. Whether you’re updating millions of ad counters per second, serving game metadata to distributed microservices, or powering identity search across terabytes of profiles, ValkeyJSON simplifies your architecture while boosting speed and scalability. With Valkey 8.0 and beyond, developers gain a powerful, intuitive, and production-grade toolset to manipulate JSON data at scale — all with the familiar simplicity and reliability of Valkey.

Dive deeper into Valkey-JSON and see how easy it is to supercharge your JSON workloads with Valkey today!
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like a more concrete call to action


## Conclusion

The introduction of the Valkey JSON module is a significant milestone for developers building modern, data-driven applications. By bringing native JSON support into Valkey’s high-performance in-memory engine, it bridges the gap between structured document modeling and real-time responsiveness. Whether you’re updating millions of ad counters per second, serving game metadata to distributed microservices, or powering identity search across terabytes of profiles, ValkeyJSON simplifies your architecture while boosting speed and scalability. With Valkey 8.0 and beyond, developers gain a powerful, intuitive, and production-grade toolset to manipulate JSON data at scale — all with the familiar simplicity and reliability of Valkey.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels sales-y again. I don't think this paragraph adds much, I would drop it or replace it with something that tell a more before/after value.


### Installation and Setup

Valkey-JSON is a module, so you’ll need to first build locally:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're going to have a binary distribution right? Maybe not today, but we don't expect people to build it themselves forever.

If this is a temporary thing, I would say something like "At time of writing, you'll need to build the module locally"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I see that @madolson mentioned that it's available from valkey-extensions - maybe just go with that)


Ensuring smooth integration across data stores and libraries is vital for minimizing migration friction and maximizing developer productivity. Valkey-JSON is fully compatible with **Valkey 8.0 and above**, making it accessible to users on the latest Valkey versions without requiring additional configuration.

It is also compliant with **RFC7159 and ECMA-404**, adhering to widely accepted JSON standards. This alignment enables consistent JSON handling across Valkey modules and supports integration with **Valkey-Search**, allowing full-text indexing and querying across JSON datasets for advanced analytics and data exploration. Additionally, Valkey-JSON integrates with **Valkey’s Access Control Lists (ACLs)**, providing fine-grained permissions for JSON commands alongside other data types like strings and hashes.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Valkey-search doesn't support full-text indexing yet.

Signed-off-by: Roshan Khatri <[email protected]>
Signed-off-by: Roshan Khatri <[email protected]>
Signed-off-by: Roshan Khatri <[email protected]>
Signed-off-by: Roshan Khatri <[email protected]>
Signed-off-by: Roshan Khatri <[email protected]>
Copy link
Member

@stockholmux stockholmux left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Progress. I would reduce your usage of adjectives - too many things are said to be great/amazing/compelling/sophisticated/significant/etc. without support. I think if you reduce that throughout the document it will overall sound less 'sales-y'


Valkey’s JSON support goes beyond basic storage and retrieval. It introduces powerful querying and filtering capabilities that streamline complex data operations. Tasks that once required extensive development time and multiple network requests can now be executed through a single JSON command, significantly increasing developer productivity and reducing operational costs.

Valkey JSON supports six data types—**null, boolean, number, string, object, and array**—allowing developers to represent complex, nested data structures without the constraints of traditional string serialization. Unlike other composite types, JSON objects and arrays in Valkey can contain any combination of the six value types, enabling deeply nested and recursive data models to be stored natively.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you give an example of how Valkey JSON can store something recursively?

### Installation and Setup

Valkey JSON is a module and at time of writing, you'll need to build the module locally.
However, it is also available in the [valkey-extensions](https://hub.docker.com/r/valkey/valkey-extensions), a docker image with all the valkey modules built-in.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: Might change ... valkey-extensions


Let's connect to the valkey-server with `valkey-cli` client for the following commands.

We will not verify the module is loaded:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems weird.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

haha
yes


## Advanced JSON Queries

Valkey JSON supports sophisticated [JSONPath](https://goessner.net/articles/JsonPath/) expressions
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On this and the following lines you 'sophisticated' and 'powerful' - I think you tone down the use of adjectives and it will sound more authentic (here and generally)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you only need to link JSONPath once, btw

{\"name\":\"Bob\",\"email\":\"[email protected]\",\"city\":\"Bellevue\",\"state\":\"WA\"}]"
```

This helps you find all users from the `state` of Washington - `"WA"`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems repetitive, maybe explain the syntax that filters instead of restating the heading.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

may be something like this:

Here, JSONPath syntax is used to filter data from a JSON document.
Let's break it down on how that works:
- `$ `represents the root of the JSON document
- `[...]` indicates we're working with an array
- `?()` is a filter expression that applies a condition
- `@` refers to the current object being processed in the array
- `.state` accesses the "state" property of the current object
- `=="WA"` checks if that property equals the string "WA"

"[\"Alice\",\"Bob\"]"
```

This would return an array of names matching the condition
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

repetitive

"[\"[email protected]\",\"[email protected]\"]"
```

This makes querying and filtering JSON documents straightforward.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not needed and 'straightforward' is subjective.


### Key Use Cases for with Valkey JSON

Valkey JSON is both powerful and straightforward to implement, making it an ideal solution for a variety of scenarios. With native JSON support, developers can manage structured, nested data without complex serialization or parsing logic, enabling rapid, targeted updates and high-performance queries. Here are three impactful use cases that showcase how Valkey JSON can drive value:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'Valkey JSON is both powerful and straightforward to implement, making it an ideal solution for a variety of scenarios.' <- not needed. just get rid of it.

Signed-off-by: Roshan Khatri <[email protected]>
Signed-off-by: Roshan Khatri <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants