[BLOG]Introducing Enhanced JSON Capabilities in Valkey #265
Conversation
Signed-off-by: Roshan Khatri <[email protected]>
There was a problem hiding this comment.
I think it's a good start, and will make a great blog. My only high level comment is it would be nice if the examples and the use cases lined up more. Right now they feel a little bit disconnected and I don't fully understand how to connect the two. I think either we interleave the examples with the use cases (Start with reference data, then do batch insertion, then do conditional updates).
|
|
||
| It makes it that simple to query and filter through JSON documents. | ||
|
|
||
| ### **Unlocking Real-World Potential with Valkey-JSON** |
There was a problem hiding this comment.
| ### **Unlocking Real-World Potential with Valkey-JSON** | |
| ### Unlocking Real-World Potential with Valkey-JSON |
You shouldn't need to bold headers
There was a problem hiding this comment.
One other use case I have heard about was optimistic locking. You can do a conditional set, where a given row is a certain value and check to see if it returned or not.
There was a problem hiding this comment.
If you talk about optimistic locking, please rephrase the heading 'Unlocking' (unless you intend the pun, but I think it's confusing...)
|
|
||
| This helps you find all users from the `state` of Washington - `"WA"`. | ||
|
|
||
| ### Atomic Multi-Path Updates |
There was a problem hiding this comment.
This section is doing GET operations, it doesn't look like it's doing updates.
| On April 4, 2025 Valkey announced the GA release a powerful new upgrade—JSON support. With the introduction of native JSON data types, developers can now store, query, and manipulate structured data directly within Valkey without needing external serialization or complex parsing logic. This enhancement not only simplifies data handling for modern applications but also unlocks new capabilities for building real-time, data-driven experiences at scale. Whether you’re managing complex user profiles, caching API responses, or powering analytics dashboards, JSON support in Valkey streamlines data management while maintaining the speed and efficiency Valkey is known for. | ||
|
|
||
| JSON has become the universal format for data exchange, but effectively managing JSON data at scale remains a challenge in caching layers. The Valkey-JSON module addresses this by providing a dedicated JSON data type and a rich set of commands for inserting, updating, and retrieving JSON content using JSONPath queries. This approach reduces application complexity and accelerates performance, making it ideal for modern, data-driven applications. | ||
|
|
||
| Let’s first cover the key capabilities and performance of Valkey-JSON. We will then dive into how to work with Valkey-JSON and demonstrate how to get started with a real world example. |
There was a problem hiding this comment.
| On April 4, 2025 Valkey announced the GA release a powerful new upgrade—JSON support. With the introduction of native JSON data types, developers can now store, query, and manipulate structured data directly within Valkey without needing external serialization or complex parsing logic. This enhancement not only simplifies data handling for modern applications but also unlocks new capabilities for building real-time, data-driven experiences at scale. Whether you’re managing complex user profiles, caching API responses, or powering analytics dashboards, JSON support in Valkey streamlines data management while maintaining the speed and efficiency Valkey is known for. | |
| JSON has become the universal format for data exchange, but effectively managing JSON data at scale remains a challenge in caching layers. The Valkey-JSON module addresses this by providing a dedicated JSON data type and a rich set of commands for inserting, updating, and retrieving JSON content using JSONPath queries. This approach reduces application complexity and accelerates performance, making it ideal for modern, data-driven applications. | |
| Let’s first cover the key capabilities and performance of Valkey-JSON. We will then dive into how to work with Valkey-JSON and demonstrate how to get started with a real world example. | |
| JSON is the universal format for semi-structured data, and modern developers expect to work with it natively across their stack, including in-memory stores like Valkey. Until now, using JSON in Valkey meant serializing entire objects into strings or flattening them into Hashes with limited nesting. These workarounds added complexity and made updates harder than they should be. | |
| That changes with the general availability of native JSON support in Valkey. You can now store, query, and update JSON documents directly, without manual parsing or transformation. This brings a cleaner model to working with semi-structured data and makes your code easier to write and maintain. | |
| In this post, I’ll walk through installing the Valkey JSON module and demonstrate how to use it to solve some common workloads. |
I don't know, the current version did not spark joy in me. Here is an alternative suggestion that is hopefully uses a bit fewer buzzwords.
There was a problem hiding this comment.
I like Madelyn's wording a little better. Commenting on comments is a little clunky in GitHub, so I have some feedback on @madolson's version.
- 'JSON is the universal format' I wouldn't use the definite article (see TOML, YAML, etc.), maybe something like 'JSON is a ubiquitous universal format'
- I might be more harsh on serialized JSON. There are scenarios where you can lose updates if you're serializing in updates (without Valkey JSON).
- You don't 'solve' workloads.
|
|
||
| ### **Per-User Event Counters for Ad or Notification Delivery** | ||
|
|
||
| Valkey JSON excels in high-throughput systems that require tracking per-user counters for ad impressions, push notifications, or message deliveries. For example, an ad platform may store a JSON document per user with nested metadata for each campaign — including impression counts, last delivery timestamps, and click history. Instead of serializing and deserializing large blobs, Valkey JSON enables in-place updates using `JSON.NUMINCRBY` or `JSON.SET` on specific paths (e.g., `$.ad_campaigns.ad_123.count`). This reduces network I/O and latency while ensuring atomicity. Microservices can also retrieve only the required subfields using JSONPath queries, like `$.ad_campaigns.ad_123.lastSeen`, allowing for efficient real-time decisioning. Compared to alternatives like managing multiple hash keys or plain strings, this approach is both cleaner and faster — making it well-suited for ad tech and notification delivery platforms operating at millions of ops/sec under tight latency constraints. |
There was a problem hiding this comment.
This needs to be a bit more focused. There is a lot of "for X, Y, Z" and "may for example" in this paragraph. Just pick a use case and dive deep in to it.
There was a problem hiding this comment.
Valkey JSON excels in high-throughput systems ...
This seems sales-y and you say it excels at something but I don't see where the rest of the paragraph supports that. If you're going to say it's good at something, prove it or drop the quality verb.
|
|
||
| Ensuring smooth integration across data stores and libraries is vital for minimizing migration friction and maximizing developer productivity. Valkey-JSON is fully compatible with **Valkey 8.0 and above**, making it accessible to users on the latest Valkey versions without requiring additional configuration. | ||
|
|
||
| It is also compliant with **RFC7159 and ECMA-404**, adhering to widely accepted JSON standards. This alignment enables consistent JSON handling across Valkey modules and supports integration with **Valkey-Search**, allowing full-text indexing and querying across JSON datasets for advanced analytics and data exploration. Additionally, Valkey-JSON integrates with **Valkey’s Access Control Lists (ACLs)**, providing fine-grained permissions for JSON commands alongside other data types like strings and hashes. |
There was a problem hiding this comment.
Is Valkey search really dependent on the RFC spec? I assume it's more tightly coupled than that.
There was a problem hiding this comment.
The RFC specs are syntax and semantic definitions of JSON, they don't actually have anything to do with search.
| Dive deeper into Valkey-JSON and see how easy it is to supercharge your JSON workloads with Valkey today! | ||
|
|
||
| For further exploration, refer to: | ||
|
|
||
| * [Valkey Official Documentation](https://valkey.io/topics/valkey-json/) | ||
| * [Valkey JSON GitHub Repository](https://github.com/valkey-io/valkey-json) |
There was a problem hiding this comment.
I think we need a bit more concise of a call to action. Link to the documentation for learning more and link to the github if you have issues or feature requests.
|
|
||
| ### **Identity Graph and Profile Storage at Scale** | ||
|
|
||
| For companies operating large-scale identity platforms — such as those in fintech, healthtech, or fraud detection — managing complex user or entity profiles is a core requirement. These profiles often include deeply nested data like names, contact info, document verification, scores, and historical activity. Valkey JSON allows each profile to be stored as a single JSON document and updated atomically as new data arrives, without needing to rewrite the entire object. Queries like `$.email`, `$.history[-1]`, or `$.risk.score` can be executed efficiently with sub-millisecond latency. When paired with Valkey Search, indexed queries on nested fields become possible — enabling real-time lookups like “find all profiles with unverified addresses in California.” This architecture supports hundreds of thousands of concurrent reads and writes per second and can scale to multi-terabyte datasets using hybrid RAM + Flash configurations. For workloads that demand both schema flexibility and ultra-low latency, Valkey JSON offers a compelling alternative to rigid relational databases or slower document stores. |
There was a problem hiding this comment.
What is this mystical hybrid RAM + Flash configuration ?
There was a problem hiding this comment.
'When paired with Valkey Search, indexed queries on nested fields become possible' -> wait... Valkey Search is only VSS at the moment - this isn't possible.
| In distributed systems like games, e-commerce platforms, or internal developer tools, multiple microservices often need fast access to consistent, structured reference data. This can include things like product attributes, game character metadata, tax codes, or ID mappings — which are naturally stored as JSON documents. Valkey JSON provides an ideal solution for centralizing this reference data in-memory. Teams can store large JSON blobs (hundreds of KB or more) using `JSON.SET`, and services can retrieve targeted subfields using path expressions like `$.items[?(@.rarity=="epic")]` or `$.idToName["1234"]`. Updates happen in bulk during patch releases or deployment cycles, but reads are constant and latency-sensitive. By keeping this metadata in Valkey, services avoid making remote API calls or parsing local files, achieving very low lookup latency even under load. This pattern greatly simplifies infrastructure, improves cache coherency, and is especially powerful in cloud-native environments where rapid bootstrapping and shared context matter. | ||
|
|
||
|
|
||
| ### **Identity Graph and Profile Storage at Scale** |
There was a problem hiding this comment.
I don't really get this use case. I suppose I also don't really get how it's different from the previous use case.
There was a problem hiding this comment.
In distributed systems like games, e-commerce platforms, or internal developer tools
I'm not sure I'd make the leap that these are all distributed systems.
| On April 4, 2025 Valkey announced the GA release a powerful new upgrade—JSON support. With the introduction of native JSON data types, developers can now store, query, and manipulate structured data directly within Valkey without needing external serialization or complex parsing logic. This enhancement not only simplifies data handling for modern applications but also unlocks new capabilities for building real-time, data-driven experiences at scale. Whether you’re managing complex user profiles, caching API responses, or powering analytics dashboards, JSON support in Valkey streamlines data management while maintaining the speed and efficiency Valkey is known for. | ||
|
|
||
| JSON has become the universal format for data exchange, but effectively managing JSON data at scale remains a challenge in caching layers. The Valkey-JSON module addresses this by providing a dedicated JSON data type and a rich set of commands for inserting, updating, and retrieving JSON content using JSONPath queries. This approach reduces application complexity and accelerates performance, making it ideal for modern, data-driven applications. | ||
|
|
||
| Let’s first cover the key capabilities and performance of Valkey-JSON. We will then dive into how to work with Valkey-JSON and demonstrate how to get started with a real world example. |
There was a problem hiding this comment.
I like Madelyn's wording a little better. Commenting on comments is a little clunky in GitHub, so I have some feedback on @madolson's version.
- 'JSON is the universal format' I wouldn't use the definite article (see TOML, YAML, etc.), maybe something like 'JSON is a ubiquitous universal format'
- I might be more harsh on serialized JSON. There are scenarios where you can lose updates if you're serializing in updates (without Valkey JSON).
- You don't 'solve' workloads.
|
|
||
| Valkey’s JSON support goes beyond basic storage and retrieval. It introduces powerful querying and filtering capabilities that streamline complex data operations. Tasks that once required extensive development time and multiple network requests can now be executed through a single JSON command, significantly enhancing developer productivity and reducing operational costs. | ||
|
|
||
| Valkey-JSON supports six data types—**null, boolean, number, string, object, and array**—allowing developers to represent complex, nested data structures without the constraints of traditional string serialization. Unlike other composite types, JSON objects and arrays in Valkey can contain any combination of the six value types, enabling deeply nested and recursive data models to be stored natively. |
There was a problem hiding this comment.
- Do you 'enhance' productivity? 'Increase' might be a better fit.
- How Valkey JSON reduce operational costs?
- Is it "Valkey-JSON" or "Valkey JSON"?
| Check the server logs to confirm: | ||
|
|
||
| ``` | ||
| Module 'json' loaded successfully. |
There was a problem hiding this comment.
agree. MODULE LIST is also more universally available as not all users will have ready access to logs.
content/blog/2025-05-27-introducing-enhanced-json-capabilities-in-valkey.md
Show resolved
Hide resolved
| {\"name\":\"Charlie\",\"email\":\"[email protected]\",\"city\":\"Austin\",\"state\":\"TX\"}]" | ||
| ``` | ||
|
|
||
| That was easy! But what if we wanted to only retrieve the cities associated with our list of users rather than pulling all user information: |
There was a problem hiding this comment.
Generally, it's better to avoid calling anything 'easy' or 'hard'.
What's 'easy' to use by one person might be hard for another - just avoid this kind of statement is probably better.
|
|
||
| ## Integration with the Valkey Ecosystem | ||
|
|
||
| Ensuring smooth integration across data stores and libraries is vital for minimizing migration friction and maximizing developer productivity. Valkey-JSON is fully compatible with **Valkey 8.0 and above**, making it accessible to users on the latest Valkey versions without requiring additional configuration. |
There was a problem hiding this comment.
making it accessible to users on the latest Valkey versions without requiring additional configuration.
What does this mean?
|
|
||
| The introduction of the Valkey JSON module is a significant milestone for developers building modern, data-driven applications. By bringing native JSON support into Valkey’s high-performance in-memory engine, it bridges the gap between structured document modeling and real-time responsiveness. Whether you’re updating millions of ad counters per second, serving game metadata to distributed microservices, or powering identity search across terabytes of profiles, ValkeyJSON simplifies your architecture while boosting speed and scalability. With Valkey 8.0 and beyond, developers gain a powerful, intuitive, and production-grade toolset to manipulate JSON data at scale — all with the familiar simplicity and reliability of Valkey. | ||
|
|
||
| Dive deeper into Valkey-JSON and see how easy it is to supercharge your JSON workloads with Valkey today! |
|
|
||
| The introduction of the Valkey JSON module is a significant milestone for developers building modern, data-driven applications. By bringing native JSON support into Valkey’s high-performance in-memory engine, it bridges the gap between structured document modeling and real-time responsiveness. Whether you’re updating millions of ad counters per second, serving game metadata to distributed microservices, or powering identity search across terabytes of profiles, ValkeyJSON simplifies your architecture while boosting speed and scalability. With Valkey 8.0 and beyond, developers gain a powerful, intuitive, and production-grade toolset to manipulate JSON data at scale — all with the familiar simplicity and reliability of Valkey. | ||
|
|
||
| Dive deeper into Valkey-JSON and see how easy it is to supercharge your JSON workloads with Valkey today! |
There was a problem hiding this comment.
I would like a more concrete call to action
|
|
||
| ## Conclusion | ||
|
|
||
| The introduction of the Valkey JSON module is a significant milestone for developers building modern, data-driven applications. By bringing native JSON support into Valkey’s high-performance in-memory engine, it bridges the gap between structured document modeling and real-time responsiveness. Whether you’re updating millions of ad counters per second, serving game metadata to distributed microservices, or powering identity search across terabytes of profiles, ValkeyJSON simplifies your architecture while boosting speed and scalability. With Valkey 8.0 and beyond, developers gain a powerful, intuitive, and production-grade toolset to manipulate JSON data at scale — all with the familiar simplicity and reliability of Valkey. |
There was a problem hiding this comment.
This feels sales-y again. I don't think this paragraph adds much, I would drop it or replace it with something that tell a more before/after value.
|
|
||
| ### Installation and Setup | ||
|
|
||
| Valkey-JSON is a module, so you’ll need to first build locally: |
There was a problem hiding this comment.
We're going to have a binary distribution right? Maybe not today, but we don't expect people to build it themselves forever.
If this is a temporary thing, I would say something like "At time of writing, you'll need to build the module locally"
There was a problem hiding this comment.
(I see that @madolson mentioned that it's available from valkey-extensions - maybe just go with that)
|
|
||
| Ensuring smooth integration across data stores and libraries is vital for minimizing migration friction and maximizing developer productivity. Valkey-JSON is fully compatible with **Valkey 8.0 and above**, making it accessible to users on the latest Valkey versions without requiring additional configuration. | ||
|
|
||
| It is also compliant with **RFC7159 and ECMA-404**, adhering to widely accepted JSON standards. This alignment enables consistent JSON handling across Valkey modules and supports integration with **Valkey-Search**, allowing full-text indexing and querying across JSON datasets for advanced analytics and data exploration. Additionally, Valkey-JSON integrates with **Valkey’s Access Control Lists (ACLs)**, providing fine-grained permissions for JSON commands alongside other data types like strings and hashes. |
There was a problem hiding this comment.
Valkey-search doesn't support full-text indexing yet.
Signed-off-by: Roshan Khatri <[email protected]>
Signed-off-by: Roshan Khatri <[email protected]>
Signed-off-by: Roshan Khatri <[email protected]>
Signed-off-by: Roshan Khatri <[email protected]>
Signed-off-by: Roshan Khatri <[email protected]>
|
|
||
| Valkey’s JSON support goes beyond basic storage and retrieval. It introduces powerful querying and filtering capabilities that streamline complex data operations. Tasks that once required extensive development time and multiple network requests can now be executed through a single JSON command, significantly increasing developer productivity and reducing operational costs. | ||
|
|
||
| Valkey JSON supports six data types—**null, boolean, number, string, object, and array**—allowing developers to represent complex, nested data structures without the constraints of traditional string serialization. Unlike other composite types, JSON objects and arrays in Valkey can contain any combination of the six value types, enabling deeply nested and recursive data models to be stored natively. |
There was a problem hiding this comment.
can you give an example of how Valkey JSON can store something recursively?
| ### Installation and Setup | ||
|
|
||
| Valkey JSON is a module and at time of writing, you'll need to build the module locally. | ||
| However, it is also available in the [valkey-extensions](https://hub.docker.com/r/valkey/valkey-extensions), a docker image with all the valkey modules built-in. |
There was a problem hiding this comment.
Note: Might change ... valkey-extensions
|
|
||
| Let's connect to the valkey-server with `valkey-cli` client for the following commands. | ||
|
|
||
| We will not verify the module is loaded: |
|
|
||
| ## Advanced JSON Queries | ||
|
|
||
| Valkey JSON supports sophisticated [JSONPath](https://goessner.net/articles/JsonPath/) expressions |
There was a problem hiding this comment.
On this and the following lines you 'sophisticated' and 'powerful' - I think you tone down the use of adjectives and it will sound more authentic (here and generally)
There was a problem hiding this comment.
you only need to link JSONPath once, btw
| {\"name\":\"Bob\",\"email\":\"[email protected]\",\"city\":\"Bellevue\",\"state\":\"WA\"}]" | ||
| ``` | ||
|
|
||
| This helps you find all users from the `state` of Washington - `"WA"`. |
There was a problem hiding this comment.
This seems repetitive, maybe explain the syntax that filters instead of restating the heading.
There was a problem hiding this comment.
may be something like this:
Here, JSONPath syntax is used to filter data from a JSON document.
Let's break it down on how that works:
- `$ `represents the root of the JSON document
- `[...]` indicates we're working with an array
- `?()` is a filter expression that applies a condition
- `@` refers to the current object being processed in the array
- `.state` accesses the "state" property of the current object
- `=="WA"` checks if that property equals the string "WA"
| "[\"Alice\",\"Bob\"]" | ||
| ``` | ||
|
|
||
| This would return an array of names matching the condition |
| "[\"[email protected]\",\"[email protected]\"]" | ||
| ``` | ||
|
|
||
| This makes querying and filtering JSON documents straightforward. |
There was a problem hiding this comment.
Not needed and 'straightforward' is subjective.
|
|
||
| ### Key Use Cases for with Valkey JSON | ||
|
|
||
| Valkey JSON is both powerful and straightforward to implement, making it an ideal solution for a variety of scenarios. With native JSON support, developers can manage structured, nested data without complex serialization or parsing logic, enabling rapid, targeted updates and high-performance queries. Here are three impactful use cases that showcase how Valkey JSON can drive value: |
There was a problem hiding this comment.
'Valkey JSON is both powerful and straightforward to implement, making it an ideal solution for a variety of scenarios.' <- not needed. just get rid of it.
Signed-off-by: Roshan Khatri <[email protected]>
Signed-off-by: Roshan Khatri <[email protected]>
Description
Adds a blog "Introducing Enhanced JSON Capabilities in Valkey" to the website
Check List
--signoffBy submitting this pull request, I confirm that my contribution is made under the terms of the BSD-3-Clause License.