
With the help of AI and collaborative robots, 6 River Systems has become an industry leader in fulfillment operations. Companies in the United States, Canada, and Europe use their comprehensive solutions to optimize their shipping workflows and fulfill millions of units every week. Since they were founded in 2015, 6 River Systems has grown rapidly and experienced a handful of common hurdles that come with scaling a business. During this journey, they found that JSON Schema was able to help them through some of these challenges.


## Challenges

Like many organizations that experience rapid growth, 6 River Systems has faced a series of organizational, technical, and evolutionary challenges as they've scaled. When 6 River Systems' number of warehouse clients increased by a factor of 10, the company itself had to grow as well. The operations, engineering, and analytics teams at 6 River Systems expanded significantly, and as a result, they realized they needed a better way to stay aligned and communicate expectations, contracts, and understandings—not only with one another, but also with the fulfillment system.

6 River Systems uses a large amount of robotics and hardware, and many of those technologies don't always work well together. They struggled to integrate best-in-class data processing, storage, and analytics tools into their existing systems. For example, the robots in their facility use the Robot Operating System (ROS) framework, which stores messages, commands, and system status information. While tools like ROSbag make it easier to collect and analyze robot data, they aren't compatible with modern data stack tools like Elasticsearch and BigQuery. Additionally, the most widely used version of ROS depends on Python 2.7, which is deprecated.

One inevitable byproduct of rapid growth is technical debt. 6 River Systems found early success, enough for Shopify to acquire them in 2019. Some of the solutions they implemented were short-sighted, and they knew that their next phase of growth would require them to focus on projects and practices that were sustainable in the long term.

<figure class="mt-10">
  <img src="/img/posts/2023/6-river-systems-case-study/chuck.webp"/>
  <figcaption class="mt-2 mb-10 text-sm text-center text-gray-500">Automation robot "Chuck"</figcaption>
</figure>


## Solution

By expanding their use of JSON Schema, 6 River Systems was able to solve a number of organizational and technical problems. In fact, JSON Schema has become the nucleus for automation across the organization, which has helped them achieve more consistency and higher-quality results.

To address their communication challenges, 6 River Systems' analysts started to directly collaborate with product engineers to draft new schemas. Using JSON Schema, the analysts can write a production-ready specification that the engineers understand immediately.

6 River Systems has found JSON Schema to be extremely helpful for generating typed SDKs. When an analyst and an engineer finish iterating on a schema, they use the schema to generate TypeScript interfaces for use by robots and throughout the application. The strongly typed nature at the data generation point makes development much smoother and eliminates bugs before they reach production, which makes all the difference.

6 River Systems also uses their schemas to automatically update destination databases, identify and isolate non-conforming data, and provide insight into the data that flows through the pipelines.

Here's a look at some of the technical solutions they've implemented:

- Every time a new schema is released, a series of database reconciliation steps run to make sure everything is in sync. Database tables are created or migrated automatically, and schemas are pushed to registries.
- Data processing systems use JSON Schemas to continuously validate data. Any data that doesn't adhere to the schema specification is redirected to an "invalid events" table (also known as "dead-lettering"), and an alert is triggered. They track the volume of invalid events and notify stakeholders as needed.
- They use JSON Schemas to build event-level namespaces that are incorporated into all of their metrics and monitoring systems. They can identify the exact instrumentation and version of what's running through the pipes, and any issues are immediately visible on a graph.
- Tools for data discovery, such as data catalogs, are seeded with JSON Schemas, so the tools no longer have to manually scrape metadata.
- They're starting to use JSON Schemas to help automate the generation of sources in [dbt](https://www.getdbt.com/), a popular warehouse analytics platform. As a result, an analytics engineer or data analyst no longer has to help write a new JSON Schema and then write the same thing all over again in YAML.


## Impact

As a result of using JSON Schema, 6 River Systems has seen benefits across the organization. They've found that JSON Schema saves time across all layers of their tech stack. Some would say that structuring data at the point of generation slows things down, and there's some truth to that sentiment. But 6 River Systems has found that isn't the case when the team has TypeScript interfaces to point out incorrect data types and missing attributes in an IDE.

JSON Schema is a tool that both a data analyst and a product engineer can understand. This enables two people who work with very different languages, such as TypeScript and SQL, to communicate directly with one another—a benefit that "cannot be understated," says Jacob Thomas, Data Platform team lead at 6 River Systems.

JSON Schema has enabled 6 River Systems to implement automation across the stack. They've implemented TypeScript interfaces for instrumentation SDKs, used it to generate tables for data warehouses, and put it to use in streaming environments to generate Kafka topics and dbt sources. In a large fulfillment center, it's very difficult to track down data problems after code has been released, so using JSON Schema to validate code before it goes out saves time and brings peace of mind.

As a company scales, having individuals chase things down through complicated infrastructure is not sustainable. Using JSON Schema as the shared language across the organization has improved the quality and visibility of data at all levels, which enables sustainable practices.

<figure class="mt-10">
  <img src="/img/posts/2023/6-river-systems-case-study/manage.webp"/>
  <figcaption class="mt-2 mb-10 text-sm text-center text-gray-500">Operator Command Center - "The Bridge"</figcaption>
</figure>

### Key Impact Results

The biggest advantage 6 River Systems has seen from using JSON Schema is the time they have saved. For such a small team managing such a large amount of work, every minute counts! They want to automate everything that can be automated.

<p className="text-2xl my-10">"We have processed hundreds of millions of events—each and every one of them is 'known good'. It's a model that works, and it works very well."</p>

6 River Systems discovered that by using JSON Schema, they could:

- Deploy known-good instrumentation to thousands of robots in the field.
- Enable effective collaboration between product engineers who think in TypeScript and JavaScript, data engineers who think in Python, and analysts who think in SQL.
- Validate every payload that reaches their data collection infrastructure.
- Proactively seed data discovery mechanisms and catalogs, as opposed to having the tools manually scrape metadata.

As a result of using JSON Schema, they've been able to implement tools like [Quicktype](https://quicktype.io/) and [TypeBox](https://github.com/sinclairzx81/typebox) on the producer side and JSON Schema-based validators on the client side. With some logic to connect the two, their data warehouses stay up-to-date.


## 6 River Systems' Journey

6 River Systems was founded in Waltham, Massachusetts, in 2015 and became a subsidiary of Shopify in 2019. Like many companies that grow quickly, 6 River Systems discovered that they needed to make some changes in order to sustain their early success. Let's take a look at how 6 River Systems went through the process of identifying the challenges they were facing and the different solutions they considered.


### Identifying Pain Points

When 6 River Systems was starting out, they dealt with a lot of semi-structured data sent without context. There was no way of knowing what the data meant, how to structure it, or what was acceptable and what wasn't. They formed a team to analyze this data, but as the volume of data increased, scaling the team became unsustainable. On top of that, instrumentation would often change and sometimes be removed entirely, making it even harder to contextualize the data and causing customer-facing dashboards to break.

All this was happening as new customers continued to come on board. Costs were increasing across the organization, and 6 River Systems knew that they needed to expand as fast as possible using sustainable, long-term solutions—all without blocking current development.

<p className="text-2xl my-10">"We identified very quickly that we needed a solution for expressing expectations, contracts, and understanding—an IDL for both systems and humans within various teams."</p>


### Finding a Solution

In their search for solutions to maintain explicit contracts with their data, here are some of the options they considered:

- **Protobuf**: While Protobuf works very well, it's less common than JSON. Analysts tend not to know it, while JSON is something they work with every day. Because 6 River Systems' engineering and analytics teams work so closely together, they needed a solution that was equally understandable to both.
- **Apache Avro**: There's a lot to like about Avro, but it's heavily tied to the Kafka ecosystem, which 6 River Systems doesn't use.
- **JSON Schema**: Rather than completely rewriting the stack with new technology, using JSON Schema was an incremental solution that was easy to understand. JSON was already widely used by front-end engineers, it integrates well with the Node.js and TypeScript ecosystem, and it can be "bolted onto" existing code with little to no changes.


### Validating the Solution

Using JSON and JSON Schema turned out to be the best solution at all levels of the stack. Instead of starting from scratch, they could use JSON Schema to enhance upstream processes while automating downstream ones. They were able to make each schema more strict than before and enforce more constraints around the data.

<p className="text-2xl my-10">"The 'aha!' moment was when teams no longer needed to talk to us to push new instrumentation. The product team and analysts could talk to each other, and good data would show up in BigQuery."</p>

While 6 River Systems had previously used JSON Schema in a number of areas across the company, they now use it to power UI components, workflow and configuration management systems, and incoming and outgoing integrations to warehouse management systems. By implementing JSON Schema consistently across the organization, they are confident in their ability to keep growing and scaling while always maintaining the high-quality solutions they are known for.

Thank you to [Jacob Thomas](https://www.linkedin.com/in/jake-thomas/) and 6 River Systems for enabling us to share this case study with you.

Images used with permission.


Melinda Gutermuth

Ben Hutton

How 6 River Systems saves time and boosts collaboration with JSON Schema


## The JSON Schema documentation project

The main goal of JSON Schema's [documentation project](https://github.com/orgs/json-schema-org/discussions/687) is to create a scalable documentation architecture that is clear, consistent, and comprehensive. To achieve this through Google Season of Docs (GSoD), we focused on:

- **[Restructuring the specification docs](https://github.com/json-schema-org/website/pull/823)**: The specification docs need a structure that helps all [personas](https://github.com/json-schema-org/community/issues/336#personas) understand how JSON Schema has evolved across different releases and the features introduced in each version.
- **[Revamping the information architecture](https://github.com/json-schema-org/website/issues/790)**: The documentation needs a new information architecture that better aligns with the [documentation developer journey](https://www.figma.com/board/TRjQUw33K93y8RlJMSRkJs/JSON-dev-journey?node-id=0-1&t=fF2pJ8IwpJvbFzDU-1) and organizes content based on the [diataxis framework](https://diataxis.fr/). The new information architecture's goal is to improve the onboarding experience and content findability.

## The GSoD project accomplishments

Since May, the team ([Valeria Hernandez](https://www.linkedin.com/in/valeriahhdez/), [Blessing Ene Anyebe](https://www.linkedin.com/in/anyebe-blessing-ene-kwennb/), [Dhairya Amrish Majmudar](https://www.linkedin.com/in/dhairya-majmudar/), and [Benjamin Granados](https://www.linkedin.com/in/benjagranados/)) has been working hard to achieve these goals. On the part of documentation, we have completed the following: 

- Conducted a thorough content audit
- Mapped out the [developer experience journey](<(https://www.figma.com/board/TRjQUw33K93y8RlJMSRkJs/JSON-dev-journey?node-id=0-1&t=gYkdGtHk2sKrCQM0-0)>)
- Implemented the [keywords page](https://json-schema.org/understanding-json-schema/keywords)
- Restructured the [specification docs](https://github.com/json-schema-org/website/pull/823)

The team has also introduced changes to the website, like a table of contents and a [Next-Previous](https://github.com/json-schema-org/website/pull/807) component that aim to improve content scannability and navigation. 

## The road ahead

So, what's pending? We're currently working on the [JSON Hyper-Schema](https://github.com/json-schema-org/website/issues/896) and [Migration pages](https://github.com/json-schema-org/website/issues/897), which are part of the specification docs. 

We're also documenting [guidelines for docs contributions and text formatting](https://github.com/json-schema-org/website/pull/902), documenting missing keywords (see issues [951](https://github.com/json-schema-org/website/issues/951), [952](https://github.com/json-schema-org/website/issues/952), and [961](https://github.com/json-schema-org/website/issues/961)), and writing step-by-step guides (see issues [180](https://github.com/json-schema-org/website/issues/180), [950](https://github.com/json-schema-org/website/issues/950), and [954](https://github.com/json-schema-org/website/issues/954)). 

## Conclusion

Our documentation efforts are expanding daily, and your contributions can improve them. There are several ways you can help, including:

- Breaking down large documents to improve the overall structure of the documentation website
- Creating missing documentation for the Troubleshooting and Scaling stages of the developer journey
- Contributing to the Migration page

Our docs meeting is held bi-weekly every Tuesday at 3:00 PM (UTC). You're welcome to join the #gsod24 channel in the JSON Schema [Slack workspace](https://json-schema.org/slack) and send us a message with your email address so we can add you to the meeting.

We appreciate your interest in the JSON Schema documentation project and look forward to your contributions and their benefits to the wider developer community.

**The JSON Schema's Google Season of Docs Team.**

Photo by <a href="https://unsplash.com/@miinyuii?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Duy Pham</a> on <a href="https://unsplash.com/photos/people-holding-shoulders-sitting-on-wall-Cecb0_8Hx-o?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Unsplash</a>
  

Blessing Ene Anyebe

Valeria Hernandez

Dhairya Amrish Majmudar

Benjamin Granados

Update from the Google Season of Docs 2024 Project


Hi, everybody!

I'm Greg Dennis.  I'm the author of [`json-everything`](https://github.com/gregsdennis/json-everything), which aims to be an all-encompassing tool suite for JSON needs in .Net, including JSON Path, JSON Patch, and (of course) JSON Schema, amongst others.  I'm also responsible for driving and authoring the output requirements in the JSON Schema specification.

Piggybacking on [Julian's news](hello-world-hello-postman), I'm also joining Postman to work on JSON Schema full time as a Specification & Tooling Architect!  This means that I will be working on updating the specification as well as helping define and implement validation and other JSON-Schema-adjacent tooling.  I also expect to be highly involved in the community by supporting other tooling implementors.

## Being Involved

JSON Schema has always been a passion project for me.  I didn't really see it at work\*, and I never had a problem for which JSON Schema was the solution.  Nevertheless, I recognized its potential that it _could_ be the solution other people needed, and I wanted to contribute by giving them an easy adoption path.

\* *I have primarily worked in the .Net world, where there are a lot of other data validation tools, most of which are built into the framework and centered around deserialization.  However, with the popularity of JSON Schema growing (especially around code generation), we could find a proper place for it in .Net soon.*

I first became involved with JSON Schema in 2015 when I was looking for JSON-related functionality to implement in my library, Manatee.Json.  I liked the ability to validate JSON data, and I especially liked that the validation requirements themselves could be represented in JSON.

At that time, Draft 4 had been available for a couple years, and the project was just starting to be picked up again after having been nearly abandoned.  Although I didn't know it then, I kind of just stumbled upon the project at the right time.  I started reading through some of the open issues, commenting on a few, and I joined the Slack workspace.

It soon became apparent that I wasn't going away, and I was invited to contribute as a core member of the team.  I spent my spare time discussing changes and new features, while trying to balance how helpful or easy to use a particular feature would be to JSON Schema users (primarily schema authors) against how hard that feature would be to implement.

Since then, I have contributed to countless discussions, commented on alterations to the specification, and even made a few authorship contributions of my own, including the aforementioned section on output formatting.

I've learned a lot from my involvement with JSON Schema.

Most significantly, having a focus on interoperability has opened my eyes to the needs of programming languages and frameworks that I haven't used.  Working primarily in C-based languages, I had never considered that different programming languages could actually _do_ different things.  Previous to my experience here, I had thought different languages all provided the same set of functionality, the only difference being that some made certain tasks easier.

I've also learned that JSON Schema is used for a lot more than just validation: hyperschema, code generation, and form generation are primary examples.  The diversity of application really makes JSON Schema a great concept.

## Being an Implementor

I've had two libraries (so far) that implement JSON Schema.

[Manatee.Json](https://www.nuget.org/packages/Manatee.Json) was the first, and it was half a learning platform for me to develop my coding skills and half intended to actually be useful to people.  It wasn't fast by any stretch of the imagination (I later found out that it was just my home-brew parsing that was slow), but it was accurate to the specification, and it had done moderately well in the nuget.org download stats.  At the time of this writing, it continues to be downloaded 300 times per day and has reached 1 million downloads despite it being deprecated for two years.

The second library, [JsonSchema.Net](https://www.nuget.org/packages/JsonSchema.Net), released in 2020, aims to be a more practical production-use library.  I created it as a JSON Schema companion to .Net's new built-in `System.Text.Json` parser/serializer, which was released in 2019.  In the two years that it's been available, it has amassed almost 450K downloads at about 670 per day.

JsonSchema.Net also has several "expansion pack" libraries that provide additional functionality, such as schema generation from .Net code, data generation from a schema, and two vocabularies that I wrote to address a couple open GitHub issues on the specification repository: external data access and identifying item uniqueness within arrays by specified keys.

I always keep an eye out for other extended uses for JSON Schema (and JSON in general) to add to the suite, and now I have more time to do it!

## Being an Employee

Last year, when Ben announced he was joining Postman to work on JSON Schema full time, I had reservations about the direction of the project.  Was Postman looking to own the project?  What would that mean for its future or for our ability (as volunteer contributors) to enact changes?

Ben's response was incredible and perfect:  he championed JSON Schema joining the OpenJS Foundation, cementing its independence.  And that he did it while being a Postman employee was reassurance that Postman was in it for the betterment of the open source community.

Then business carried on as usual.  We were all working hard toward the release of an editorial, non-functional patch release for 2020-12 (now released!), and Ben found that he needed some more dedicated help.  So in Slack DMs, he asked me if I would be interested in joining up.  Even though I was just getting settled in at a new company, I couldn't let this opportunity pass me by.  Getting paid to work on a passion project is the dream!

And now, I'm extremely excited to announce that I'm joining Postman Open Technologies as a Specification & Tooling Architect.  I will have three focuses: work on the JSON Schema specification, support for its implementors, and development support of adjacent uses.

To start with the specification, the output format will continue to be a primary focus for me.  Earlier this year, I opened a [discussion](https://github.com/orgs/json-schema-org/discussions/63) on potential improvements.  I think that we're in a good place, and I need to start creating PRs that move the current language in the specification to the new design.

I'm not sure yet what implementor support will look like, but based on our [implementations page](https://json-schema.org/tools), it seems that many of them haven't updated to supporting 2020-12 yet.  Outreach is probably the best place to start here.  We need to find out the status of each of these projects.  Are they still maintained?  Do they _intend_ to stay up-to-date with the specification?  Do they need help?  Answering these questions will be the first step.

And finally adjacent tooling.  I'm really excited about this part because it will highlight for me how JSON Schema is being used in the real world, thereby _validating_ (ha!) its usefulness.

I'm really excited about the opportunity that Postman has given me to work full time on a project that I love and its adjacency, and I foresee a bright future for this project and the community that continues to support and build it.

Thank you to everyone I've met on this journey, used my libraries, asked questions, contributed code, or just offered opinions!  I love being a part of this community!

Cover photo by me 😁


Greg Dennis

And Then There Were Three


Last month, we had the privilege of partnering with AsyncAPI and OpenAPI with the first API Specs and Standards booth at the [API Days Paris Conference](https://www.apidays.global/paris/). This resulted in a fantastic experience where attendees could easily connect with experts in one booth. We had the opportunity to learn from the experience of OpenAPI and AsyncAPI conference tracks.

This is our first time having a booth at a conference, but we are sure it won't be the last. We returned energized for the great conversations, inspired by the great work of the other standards, excited with the great content, and impressed by the great work done by the API Days Team.

## About API Days Paris
Apidays is the leading business and technology conference series in APIs, and the programmable economy and Paris edition is the biggest event of the series.

<div className='flex flex-wrap justify-center items-center gap-4 w-full'>
  <iframe
    className='w-full md:w-1/2 lg:w-1/3 xl:w-2/4' 
    height="300"
    src="https://apiglobal.b-cdn.net/wp-content/uploads/2020/04/APIDAYS-PARIS-22-TRAILER-2.mp4"
    frameborder="0"
    allowfullscreen
  ></iframe>
</div>

### The Conference Theme
In today's intricate digital landscape, APIs have emerged as powerful conduits for innovation, bridging the gaps and unlocking new realms of possibility. This year, API Days Paris revolves around four strong pillars: "Software, Sovereignty, Society, and Sustainability."

* **Software**: Software & APIs are the pulsing heart of the digital revolution, from the frameworks that power global platforms to the microservices optimizing niche solutions, Serveless, Kubernetes, Kafka, and many other technologies.
* **Sovereignty**: How should sovereignty be defined in the age of the cloud, decentralization, and ubiquitous connectivity, all while empowering individuals and organizations to chart their distinct digital paths?
* **Society**: Let's understand the societal implications of rapid technological advances to ensure that we leave no member of society behind, fostering an equitable and harmonious future.
* **Sustainability**: Let's explore the sustainable development, deployment, and management of software solutions to grasp the importance of eco-friendly digital infrastructures, ethical AI, and the long-term impact of our digital choices.

## Connecting with JSON Schema and API practitioners
It has been great having the chance to connect with JSON Schema practitioners in person. We received lots of feedback, we discussed about challenges and use cases and we created great connections with amazing people.

<div className='flex flex-wrap justify-center items-center gap-4 w-full'>
    <img className='max-w-400 px-20' src='/img/posts/2023/apidays-paris-2023-recap/booth-apidays.jpeg'/>
</div>

## Meaningful conversations with organizations
It has been great having the chance to connect with JSON Schema practitioners in person. We received lots of feedback, discussed challenges and use cases, and created great connections with amazing people.

At JSON Schema, we are looking to connect with stakeholders from Organizations invested in JSON Schema and its ecosystem, and this event was perfect for it. We want Organizations to be more directly part of our community and be involved in driving our roadmap.
We have an ambitious mission, but we need help. We are looking for sponsors!. We recently updated our [sponsorship packages](https://json-schema.org/overview/sponsors), increasing the benefits for every sponsorship package, but this is not only about getting funds; we'd love to explore other types of collaborations like "people-resource".

If you are reading this, your Organization is using JSON Schema, and you like to get involved, [let's talk](mailto:info@json-schema.org)!

## Learning from the AsyncAPI and the OpenAPI conference tracks
At JSON Schema, we have been discussing organizing a JSON Schema conference, but doing it alone will require a lot of resources and some years to get its own status. Instead, we returned inspired by great OpenAPI and AsyncAPI tracks, which we think is our path forward! We'd love to host our own track in the future!

## We hope to see you next year!
We'd like to thank AsyncAPI, OpenAPI, and most importantly, the API Days Team for their generosity in having us in the Conference, and we hope to repeat this in the future!

In the meantime, you can connect with us in our [Slack workspace](https://json-schema.org/slack), the best place to ask questions, learn, get help, or discuss all things JSON Schema.

API Days Paris 2023 - Recap

## Introduction
[Apidays Paris](https://www.apidays.global/paris/), the flagship event of the apidays family, returned this year with a focus on "The Future API Stack for Mass Innovation." This annual gathering of API enthusiasts, developers, and industry leaders brought together experts to discuss the latest trends and best practices in API design, API development, and API management.

This year's agenda highlighted the increasing role of AI in API development, exploring how AI can be used to automate tasks, improve API design, and enhance API performance.
Continuing our successful partnership with [AsyncAPI](https://www.asyncapi.com/en) and [OpenAPI](https://www.openapis.org/), this year's event marked a significant milestone with the inaugural JSON Schema conference track. This dedicated track, sponsored by [Octue](https://www.octue.com/), delved deep into the world of JSON Schema, exploring its role in API design, validation, and documentation.

## The first JSON Schema conference

The inaugural [JSON Schema conference track](https://conference.json-schema.org/) at apidays Paris 2024 was a resounding success. As part of the larger conference theme "The Future API Stack for Mass Innovation," the track showcased JSON Schema's crucial role in modern API development and data contract design. 

<figure className='flex flex-wrap justify-center items-center gap-4 w-full'>
    <img className='max-w-400 px-20' src='/img/posts/2024/apidays-paris-2024-recap/json-schema-team.jpg' alt="Five people standing at a booth labeled 'Standards'. They are smiling and looking at the camera. From left to right, their names are Benjamin Granados, Thomas Clark, Andreas Eberhart, Juan Cruz Viotti, and Ben Hutton."/>
    <figcaption class="text-center text-gray-600 text-sm">Community members at the standards booth shared with AsyncAPI, OpenAPI, and GraphQL. From left to right: Benjamin Granados (community manager), Thomas Clark (Octue CEO), Andreas Eberhart (JSON Schema ambassador), Juan Cruz Viotti (TSC member), and Ben Hutton (TSC member).</figcaption>
</figure>
<br>

[Ten comprehensive presentations](https://conference.json-schema.org/sessions/) spanned the spectrum of JSON Schema applications, from foundational concepts to cutting-edge implementations. The list of talks included:

- JSON Schema for data design and contract, client and code generation by Tom Collins (DVLA, UK Gov)
- IEC standards: Toward digital standards with JSON Schema by Thomas Clark (Octue CEO, conference sponsor)
- Introducing Strands - a JSON Schema repository by Thomas Clark
- JSON Schema: powering FINOS architecture as code by James Gough (Morgan Stanley)
- Generic and extensible web of things manager using JSON Schema and AI by Andreas Eberhart (Dashjoin, JSON Schema ambassador)
- High-performance JSON Schema validation in .NET by Matthew Adams (Endjin, TSC member)
- JSON schema and relational databases: bridging the gap by Loic Lefèvre (Oracle)
- Applying software engineering practices to JSON Schemas by Juan Cruz Viotti (Sourcemeta CEO, TSC member)
- The state of JSON schema by Benjamin Granados (Celonis, TSC member)
- Unlocking dynamic frontend development through JSON Schema by Neha Singla (Apple)

Three major themes recurred throughout the track: data design and standardization, tooling infrastructure, and modern application integration. Particularly notable was the emphasis on JSON Schema's role in AI integration and automated API development, aligning perfectly with the broader conference focus on AI's increasing importance in the API landscape.

Octue's sponsorship significantly bolstered the track's success, demonstrating their commitment to the JSON Schema community. This support enabled a diverse range of speakers to share their expertise and experiences.

Audience engagement exceeded expectations. Attendees from various industries and organizations - such as fintech, renewable energy, IT, and government - participated in robust Q&A sessions after each presentation. The discussions revealed both the growing adoption of JSON Schema across different sectors and the community's hunger for more knowledge about its practical applications.

<figure className='flex flex-wrap justify-center items-center gap-4 w-full'>
    <img className='max-w-400 px-20' src='/img/posts/2024/apidays-paris-2024-recap/loic-lefevre-talk.jpg' alt="Loic Lefevre presenting on stage at the APIDays Paris 2024 conference. The audience is seated in rows of chairs, facing the stage."/>
    <figcaption class="text-center text-gray-600 text-sm">A room full of people listening to Loïc Lefèvre’s engaging presentation on utilizing JSON schemas to manage relational databases.</figcaption>
</figure>
<br>

Looking ahead, this first dedicated track has laid a strong foundation for future JSON Schema events. The feedback highlighted the need to further showcase JSON Schema's versatility across different use cases, and plans are underway to expand and enhance next year's track. This successful launch not only validates JSON Schema's importance in modern API development but also signals the beginning of a new chapter in the community's growth and evolution.

## Collaboration with organizations

The success of the JSON Schema track at apidays Paris 2024 highlighted the vital role of community collaboration in our growth. It also demonstrated how organizations collaborating with JSON Schema gain more than just visibility - they become part of a broader movement toward standardized, high-quality API development. 

Here's how your organization can benefit:
- **Standardization & interoperability**: Ensure seamless data exchange and API integration.
- **Improved data quality**: Minimize errors with robust data validation.
- **Enhanced API design**: Streamline development with clear and concise schemas.
- **Increased productivity**: Automate tasks and boost efficiency with JSON Schema tools.
  
Here’s how you can become part of the initiative:
- [Be an ambassador](https://github.com/json-schema-org/community/tree/main/programs/ambassadors): Share your JSON Schema expertise and champion its adoption.
- [Become a sponsor](https://json-schema.org/overview/sponsors): Gain visibility and support the standard's growth.
- [Contribute to the project](https://github.com/json-schema-org): Help us shape the future of JSON Schema development.
- [Join our Slack workspace](https://json-schema.org/slack): Connect with the vibrant JSON Schema community.


Together, we can build a more robust and interoperable API ecosystem. 

A final thank you to Octue for sponsoring the conference, apidays Paris for hosting it, the organizers for their dedication and hard work, and the attendees for your valuable input and insightful questions. On behalf of JSON Schema, we look forward to seeing you again next year!

Acknowledgments: The author would like to thank Onyedikachi Hope, Ben Hutton, and Benjamin Granados for their valuable feedback, which enhanced the quality of this article. 


JSON Schema conference: A milestone event at Apidays Paris 2024

It all starts with applicability - JSON Schema Fundamentals part 1


*This is the first in what will hopefully be an intermittent series on Bowtie.
I speak here without speaking on behalf of the JSON Schema organization in any official capacity -- Bowtie isn't a tool we The JSON Schema Team are "blessing" in some way today, though I have personal hopes it becomes a sort of official tool, and have developed it intending it to be owned by "the community" in whatever sense that means.
For the moment, I speak only on behalf of myself as its author :)*

The JSON Schema

```json
{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "type": "string",
  "maxLength": 3
}
```

validates strings of length at most 3.
This behavior is what the specification says *should* happen.
Does every *implementation* of the specification, across the wide spectrum of programming languages, properly *follow* the specification?
What about for other more complicated schema / instance pairs?
This is the key question Bowtie tries to address -- how can we compare behavioral differences between JSON Schema implementations and between the specification, in the hopes that we make it easier to fix implementation issues and clarify any unclear portions of the specification?

## What It Is

[Bowtie](https://github.com/bowtie-json-schema/bowtie/) is what I'm calling a "meta-validator", by which I mean a program which can execute *other* JSON Schema validation implementations and collect results from them.

There's prior art in doing this sort of thing.
The [JSONPath Comparison project](https://cburgmer.github.io/json-path-comparison/) does this extremely well for [JSONPath](https://github.com/ietf-wg-jsonpath/draft-ietf-jsonpath-base), and [Nicolas Seriot's "Parsing JSON is a Minefield"](https://seriot.ch/projects/parsing_json.html) is a fantastic example for JSON itself.
Bowtie attempts to bring these ideas to JSON Schema.

From the existing [list of JSON Schema implementations](https://json-schema.org/tools#validators), Bowtie already [supports](https://github.com/orgs/bowtie-json-schema/packages) *12* implementations across *9* programming languages, allowing anyone to run any of these implementations and see what they have to say about schemas and instances.

It ships with a [command line program](https://docs.bowtie.report/en/stable/cli/) but perhaps more excitingly, ongoing automated runs of this CLI have been set up, such that Bowtie emits a [report across all of its supported implementations](https://bowtie.report).

To produce this report, Bowtie runs the [official JSON Schema Test Suite](https://github.com/json-schema-org/JSON-Schema-Test-Suite), which is our existing set of tests meant to exercise compliance with the JSON Schema specifications.
Many implementations already [use the suite within their own continuous integration](https://github.com/json-schema-org/JSON-Schema-Test-Suite#who-uses-the-test-suite), but this is the first time both users of JSON Schema as well as implementers can see results of running the suite in a single place, across many implementations.
The test suite already has great coverage of our specifications (specifically the validation portions of them).
The suite certainly isn't perfect, but Bowtie's results therefore cover the validation specification really well.

## Why It Might Be Interesting

The most noticeable thing Bowtie enables is an easy way to compare how close an implementation follows the specification.
This gives transparency to users of implementations, and also to the community about areas which might be hard to implement, or are commonly misimplemented.
Our hope is that this leads to improvement and energy to help fix issues, and overall to a stronger community!

Even beyond the test suite, Bowtie is capable of providing a uniform interface to the implementations it supports, meaning you can quickly access results from each one without needing to learn the language-specific API each implementation offers.
If you have a schema and instance and want to run it across many implementations, or a single implementation which you're not already familiar with or don't have set up, Bowtie can help.

## How Do I Run It?

If you're just interested in its output (i.e. reports on runs of the test suite), you can currently find them at the following links, corresponding to each version of the specification:

* [draft2020-12](https://bowtie.report/#/dialects/draft2020-12)
* [draft2019-09](https://bowtie.report/#/dialects/draft2019-09)
* [draft7](https://bowtie.report/#/dialects/draft7)
* [draft6](https://bowtie.report/#/dialects/draft6)
* [draft4](https://bowtie.report/#/dialects/draft4)
* [draft3](https://bowtie.report/#/dialects/draft3)

The medium-term goal is to combine these onto one unified report (at which point the links may change, though hopefully we'll put a redirect in place).

If you want to go beyond the test suite reports, you can run Bowtie locally as well, on whatever input you'd like.
Bowtie is written in Python and published [on PyPI](https://pypi.org/project/bowtie-json-schema/).
If you have no existing preferred setup for installing Python applications, [install `pipx` using the platform-specific instructions for your OS](https://pipx.pypa.io/stable/), then run:

```sh
$ pipx install bowtie-json-schema
```

which should give you a working Bowtie, which you can check via:

```sh
$ bowtie --help
```

The usage instructions are in [Bowtie's documentation](https://bowtie-json-schema.readthedocs.io/), but more documentation is definitely needed, so do ping myself (Julian) if there are things you can't figure out, it will motivate documenting them.

## How Does It Work?

I won't go into full details of Bowtie's implementation in this post, but at a high level, Bowtie is a command-line program written in Python which orchestrates spinning up and down OCI containers ("Docker containers" in a loose sense).
The containers it runs are "test harnesses" -- little bridge programs which take an implementation of JSON Schema, written in any programming language, and then allow incoming requests from Bowtie to call into the library under test.
Adding support for an implementation essentially means implementing a test harness in this fashion (which is generally simple to do), and once the harness is present, the implementation will light up in Bowtie's reports, and any user of Bowtie will be able to execute the implementation via Bowtie's uniform CLI.

## How to Help

There are two areas you (a user of JSON Schema, author of an implementation, or community member) can help with:

### Improving Implementations

The first and most obvious is by helping to take any issues uncovered by Bowtie back to implementations (after confirming them), and then helping implementations fix them.

Find your favorite JSON Schema implementations and the tests it fails.
Are there existing open issues on the implementation's bug tracker about the failures?
If they aren't previously known, an issue ticket (with minimal reproducer) is likely going to be welcome by the implementer, so you may want to file one.
If they *are* known, you can file a pull request in Bowtie which contains information about the downstream issue, which Bowtie can make use of in reports (by showing that the test is explicitly skipped).
Here's an [example pull request](https://github.com/bowtie-json-schema/bowtie/pull/73) showing how to do so.

Look out for [Bowtie-generated badges](https://github.com/bowtie-json-schema/bowtie/issues/55) in the near future which you can use to show off your spec-compliance (nothing could be cooler, eh?).

### Improving Bowtie

The second is by helping to improve Bowtie itself, which I already have a long list of ideas for, which can only grow as more people start to use it.

To repeat part of the above, I myself have implemented support for [many implementations](https://github.com/bowtie-json-schema/bowtie/tree/main/implementations).
If you maintain or use one, look at the harness I wrote -- it's possible I misused your implementation (though so far thankfully hasn't happened).

I'll also mention that I've been experimenting with doing work on Bowtie on [livestreams here](https://www.twitch.tv/julianberman) -- without too much self-promotion, feel free to stop by and say hello, or even to flip through previous streams where I fight with various Bowtie-adjacent things, it may give you some pointers on how to work on it.

Below is some specific guidance around areas in need of help:

*This is current as of the publication of this post, but my hopes are that it swiftly becomes out of date. If it seems to be, feel free to reach out!*

#### Give Me Something Easy to Get My Feet Wet

There's a list of [good first issues](https://github.com/bowtie-json-schema/bowtie/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) on Bowtie's issue tracker which I've attempted to curate.
They cover various portions of Bowtie's codebase (the Python bits, the frontend bits, and its surrounding infrastructure).
The label doesn't indicate that the issue is necessarily "small" to fix, though many are.
But it does indicate the relevant functionality or fix is "straightforward" or likely well-scoped.
If you don't see activity on one, feel free to leave a comment if you begin to work on it.

#### I Want to Learn About a New Implementation or Programming Language

This is possibly the most "fun" thing to work on, or at least I quite enjoyed doing it.
Pick an implementation Bowtie doesn't already support from the [list of implementations](https://json-schema.org/tools#validators) and follow the [tutorial for writing a harness](https://bowtie-json-schema.readthedocs.io/en/latest/implementers/) which will guide you through what Bowtie expects from a harness.
If there are gaps in the tutorial, do raise them or ping me!
But this can be a nice way to play with a simple program in a language you haven't used before, as well as a way to learn a JSON Schema implementation which may have made different choices than one you're already familiar with.

#### The Report is Ugly, I'm a Frontend Developer or UI Designer Who Can Help

This is probably the most beneficial area you can help in.
I myself am not a frontend developer, in case that wasn't obvious.
What I put together (with Bootstrap) is essentially the bare minimum of what's needed to show off results Bowtie emits.
If you have more experience or headspace to think about how to effectively report test results, or compare implementations, please help!
All of Bowtie's "frontend code" today lives in one place, [a Jinja2 template which gets formatted into a static single page site](https://github.com/bowtie-json-schema/bowtie/blob/b23644eb86cf9bcbdf1d98149cf3d976339f7cdf/bowtie/templates/report.html.j2).
Beyond generally "make it prettier, more responsive, or more usable", here are a few specific issues to look at:

* [Add client-side filtering and sorting](https://github.com/bowtie-json-schema/bowtie/issues/37)
* [Show combined errors+failures counts](https://github.com/bowtie-json-schema/bowtie/issues/53), or more generally improve the summary from a "table of counts" into a more graphically pleasing representation of what happened in a run
* [Display schemas & instances more prettily](https://github.com/bowtie-json-schema/bowtie/issues/28), or more generally, design a good widget or component for representing test cases, their schemas, instances, and (sub)tests
* [Show average compliance numbers](https://github.com/bowtie-json-schema/bowtie/issues/54), or more generally, what summary statistics *across* implementations should we show, and how should we show them?

#### I Develop Other Tooling in the JSON Schema Ecosystem

Great!
I have hopes Bowtie may hook in well with other upstream or downstream tooling.
As a concrete example, given that Bowtie provides uniform interfaces to downstream implementations, an "obvious" complimentary tool might be to fuzz-test across *all* implementations, looking for cases they disagree, or blow up, or more generally produce behavior non-compliant with the specification which isn't already covered by an explicit test in the suite.
Doing this likely simply means hooking such a tool up to Bowtie and letting it rip.
See [this issue](https://github.com/bowtie-json-schema/bowtie/issues/23) though it doesn't contain much beyond the above in the way of detail.

Other tools may also have nice interactive properties when combined with Bowtie, so if you have other ideas, reach out, or try it!

#### I Want to Contribute Tests

This is definitely also helpful.
If they're tests of the specification itself, check whether they're already present in the official suite, and if not, do submit them there.
If they're *not* tests that the official suite covers today, e.g. because they cover pathological cases like causing implementations to "blow up" rather than produce a result, they are now "fair game" to add (somewhere, TBD where), because Bowtie can "gently" run implementations and catch these sorts of errors.
Bowtie also allows for a wider range of `$ref`-related testing, because its protocol specifically instruments harnesses with a sort of "registry" of schemas that the implementation is expected to be able to reference.
[This issue](https://github.com/bowtie-json-schema/bowtie/issues/61) is the relevant one, but ideas are welcome here on kinds of tests we can now add.

#### I Found a Bug, or Have an Idea for Bowtie Itself

If it's with an implementation, you likely should take it to the implementation's issue tracker.
Please be polite, as there's still a small but non-zero chance that issues are caused by Bowtie itself.
And even if there isn't, please do be kind to maintainers, many (including myself!) are already aware of issues Bowtie is flagging and may really want to fix them but have things which make doing so difficult.
Help out implementations if you can!

If it's with Bowtie itself, or is a new idea related to Bowtie, definitely [file an issue](https://github.com/bowtie-json-schema/bowtie/issues), [start a discussion](https://github.com/bowtie-json-schema/bowtie/discussions), or start a thread on the JSON Schema Slack.

#### I Want to Do Something Random and Cool

There are some funky ideas in this bin, even already.
I would love to figure out whether we can make use of Bowtie to *benchmark* implementations uniformly, rather than just test them.
Being able to [jump into a language-specific REPL](https://github.com/bowtie-json-schema/bowtie/issues/38) is also a kind of out-there idea but one which might be fun to play with.
There likely are many others!

#### Setting Up Bowtie Was Hard But I Figured It Out / I Want to Help With CI, "Infrastructure" or Triaging

These are all under-appreciated areas you can help with, so please do reach out if you're interested in anything here.
Even something like making Bowtie easier to install for those who don't use Python is a nice help!
(Right now, builds produce a `shiv`, but not a cross-platform one.)

## Conclusion

Thanks for taking the time to hear a bit about Bowtie.
Special thanks also must be given to [Postman](https://www.postman.com/) who employs me full-time to be able to do work like this on behalf of the community.
Without Postman, this work would never have happened!
I hope there's a lot more to come.
Please do share feedback, it's very welcome, and if you do want to get involved, that'd be very much appreciated!

Cover photo generated via Stable Diffusion.


Julian Berman

Introducing: Bowtie


*I’ve been known to say “If you haven’t rewritten your OpenAPI bundling implementation recently, then you don’t support OpenAPI 3.1”. This observation may be true, but perhaps some more detail would be helpful? When implementing support for OAS 3.1 and JSON Schema draft 2020-12 in `oas-kit`, reading the sections of the JSON Schema spec on bundling compound documents, I still wasn’t totally clear on what was expected of compliant tooling. Thankfully, Ben Hutton is here to set the record straight with a worked example.* - Mike Ralphson, OAI TSC

## Bundling has renewed importance

OpenAPI has long since put the spotlight on JSON Schema, and the release of OpenAPI 3.1 has huge implications for the future of both projects. I'm truly excited.

Developers of platforms and libraries that use OpenAPI haven't had such a shake up before, and my feeling is it may take more than a few releases to correctly implement all the new shiny features full JSON Schema has to offer.

While the number of changes from JSON Schema draft-04 to draft 2020-12 are vast and the subject of more blog posts than are likely interesting, one of the key "features" of draft 2020-12 is a defined bundling process. (draft-04 is the version of JSON Schema that OAS used prior to version 3.1.0; or rather, a subset/superset of it.)

Indeed, bundling, if anything, is going to be more important to get right than ever. OAS 3.1 ushering in full JSON Schema support dramatically increases the likelihood that developers with existing JSON Schema documents will use them __by reference__ in new and updated OpenAPI definitions. Ultimate source of truth matters, and it's often the JSON Schemas.

Many tools don't support referencing external resources.
Bundling is a convenient way to package up schema resources spread across multiple files in a single file for use elsewhere, such as an OpenAPI document.

## Existing solutions? New solutions!

There are several libraries which offer bundling solutions, however they all have caveats, and I haven't seen any to date which are fully JSON Schema aware. The most popular of these libraries is called `json-schema-ref-parser`, however it [reports](https://github.com/APIDevTools/json-schema-ref-parser/issues/97#issuecomment-415963663) that it was not intended to be JSON Schema aware, and is only intended to cover the JSON Reference specification (Which has been bundled back into the JSON Schema specification now).

We are hoping to provide you with a canonical implementation (Right, Mike?!) and enough information to get started building your own in your language of choice. (Although, it's always best to read the full specification when developing implementations.)

## Bundling fundamentals

Firstly, let's visit some key definitions in JSON Schema draft 2020-12.
The `$id` keyword is used to identify a "schema resource". In the example below, the `$id` is `https://jsonschema.dev/schemas/mixins/integer` for the resource.

```json
{
  "$id": "https://jsonschema.dev/schemas/mixins/integer",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "description": "Must be an integer",
  "type": "integer"
}
```

A "Compound Schema Document" is a JSON document which has multiple embedded JSON Schema Resources. Below is a simplified example of one we’ll unpack a bit later.

```json
{
  "$id": "https://jsonschema.dev/schemas/examples/non-negative-integer-bundle",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "description": "Must be a non-negative integer",
  "$comment": "A JSON Schema Compound Document. Aka a bundled schema.",
  "$defs": {
    "https://jsonschema.dev/schemas/mixins/integer": {
      "$schema": "https://json-schema.org/draft/2020-12/schema",
      "$id": "https://jsonschema.dev/schemas/mixins/integer",
      "description": "Must be an integer",
      "type": "integer"
    },
    "https://jsonschema.dev/schemas/mixins/non-negative": {
      "$schema": "https://json-schema.org/draft/2020-12/schema",
      "$id": "https://jsonschema.dev/schemas/mixins/non-negative",
      "description": "Not allowed to be negative",
      "minimum": 0
    },
    "nonNegativeInteger": {
      "allOf": [
        {
          "$ref": "/schemas/mixins/integer"
        },
        {
          "$ref": "/schemas/mixins/non-negative"
        }
      ]
    }
  },
  "$ref": "#/$defs/nonNegativeInteger"
}
```
> Note that schema bundling and the use of multiple definitions are not necessary to represent non-negative integers.
> This example is purely for illustrative purposes and the schema below is perfectly suitable for representing non-negative integers without the use of bundling.
> ```json
> {"type": "integer", "minimum": 0}
> ```

Last, let's look at the carefully crafted definition of "bundling" according to the JSON Schema specification:

"The bundling process for creating a Compound Schema Document is
   defined as taking references (such as "$ref") to an external Schema
   Resource and embedding the referenced Schema Resources within the
   referring document.  Bundling SHOULD be done in such a way that all
   URIs (used for referencing) in the base document and any referenced/
   embedded documents do not require altering."

With these definitions in mind, now we can look at the defined bundling process for JSON Schema resources! We will only cover the ideal situation in this article. The goal here is to have no external Schema Resources.

Note, this article does NOT cover "total dereferencing", which is removing all uses of `$ref` from a schema. This is not advised, and is not always even possible, such as when there are self references.

## Bundling Simple External Resources

In our first example, we have an ideal situation for bundling. Each schema has an `$id` and `$schema` defined, making the bundling process simple.
We'll cover various other situations and edge cases in further examples, but having each resource define its own identity and dialect is always preferable.
Our primary schema resource references two other schema resources using the in-place applicator `$ref` with the value being a relative URI. The relative URI is resolved against the base URI, which in this instance is found in the primary schema resource's `$id` value. By combining "integer" and "non-negative" schemas, we create a "non-negative integer" schema.

```json
{
  "$id": "https://jsonschema.dev/schemas/mixins/integer",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "description": "Must be an integer",
  "type": "integer"
}
```

```json
{
  "$id": "https://jsonschema.dev/schemas/mixins/non-negative",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "description": "Not allowed to be negative",
  "minimum": 0
}
```

```json
{
  "$id": "https://jsonschema.dev/schemas/examples/non-negative-integer",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "description": "Must be a non-negative integer",
  "$comment": "A JSON Schema that uses multiple external references",
  "$defs": {
    "nonNegativeInteger": {
      "allOf": [
        {
          "$ref": "/schemas/mixins/integer"
        },
        {
          "$ref": "/schemas/mixins/non-negative"
        }
      ]
    }
  },
  "$ref": "#/$defs/nonNegativeInteger"
}
```

Should "non-negative-integer" schema be used as the primary schema in an implementation, the other schemas would need to be available to the implementation. At this point, exactly how that implementation loads in the schemas doesn't matter, as they have fully qualified URIs as their identity defined in `$id`. Any implementation that loads in schemas should build an internal local index of schema URIs defined in `$id` to schema resources.

Remember, any schema which provides a value for `$id` is considered a Schema Resource.

Let's resolve (dereference) one of the references in our primary schema. `"$ref": "/schemas/mixins/integer"` resolves to a fully qualified URI of `https://jsonschema.dev/schemas/mixins/integer` by following the rules for first determining the base URI and then resolving the relative URI against that base URI. The implementation should then check its internal index of schema identifiers and schema resources, finding a match, and using the appropriate previously loaded schema resource.

The bundling process is done. The previously externally referenced schemas are copied into `$defs` in our primary schema, as is. The keys for the `$defs` object are the identifying URIs, but they can be anything, as those values won't be referenced (They could be UUIDs if you like). Looking at our final bundled schema… I mean "Compound Schema Document", we now have multiple Schema Resources embedded in a single Schema document.

```json
{
  "$id": "https://jsonschema.dev/schemas/examples/non-negative-integer-bundle",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "description": "Must be a non-negative integer",
  "$comment": "A JSON Schema Compound Document. Aka a bundled schema.",
  "$defs": {
    "https://jsonschema.dev/schemas/mixins/integer": {
      "$schema": "https://json-schema.org/draft/2020-12/schema",
      "$id": "https://jsonschema.dev/schemas/mixins/integer",
      "description": "Must be an integer",
      "type": "integer"
    },
    "https://jsonschema.dev/schemas/mixins/non-negative": {
      "$schema": "https://json-schema.org/draft/2020-12/schema",
      "$id": "https://jsonschema.dev/schemas/mixins/non-negative",
      "description": "Not allowed to be negative",
      "minimum": 0
    },
    "nonNegativeInteger": {
      "allOf": [
        {
          "$ref": "/schemas/mixins/integer"
        },
        {
          "$ref": "/schemas/mixins/non-negative"
        }
      ]
    }
  },
  "$ref": "#/$defs/nonNegativeInteger"
}
```

When the bundled schema is initially loaded and evaluated, the implementation should create its own internal index of schema identifiers and schema resources, just as before. The relative URIs used to reference those schema resources need not change.

The simplest way to see this bundled schema working as expected is to paste it into https://json-schema.hyperjump.io and then try different values for the instance. I'm hopeful to bring several updates to https://jsonschema.dev over the next few months, but times are busy as we continue to elevate JSON Schema as an organisation.

It’s worth remembering that the example in this article shows the ideal situation, when best practices have been followed. The JSON Schema specification does define additional processes for non-ideal situations and edge cases (such as when `$id` or `$schema` are not set), however, some solutions may be indirectly related to Compound JSON Schema Documents. For example, establishing the base URI follows the steps laid out in RFC3986, which JSON Schema does not redefine.

## OpenAPI Specification Example

Let’s look at an example of how this might work with an OpenAPI definition.

```yaml
openapi: 3.1.0
info:
  title: API
  version: 1.0.0
components:
  schemas:
    non-negative-integer:
      $ref: 'https://jsonschema.dev/schemas/examples/non-negative-integer'
```

We start with our input OpenAPI 3.1.0 specification document. For brevity, we’re only showing the components section with a single component, but let’s assume some other part of the document uses the component schema “non-negative-integer”.

“non-negative-integer” has a single reference to a JSON Schema resource. The reference URI is an absolute URI, including domain and path, meaning there’s no need to do any “resolve the relative URI against the base URI” dance.

All the schemas required to resolve and bundle the reference are provided to the bundling tooling. After the schemas are loaded into the implementation, their originating physical location no longer matters.

```yaml
openapi: 3.1.0
info:
  title: API
  version: 1.0.0
components:
  schemas:
    # This name has not changed, or been replaced, as it already existed and is likely to be referenced elsewhere
    non-negative-integer:
      # This Reference URI hasn't changed
      $ref: 'https://jsonschema.dev/schemas/examples/non-negative-integer'
    # The path name already existed. This key doesn't really matter. It could be anything. It's just for human readers. It could be an MD5!
    non-negative-integer-2:
      $schema: 'https://json-schema.org/draft/2020-12/schema'
      $id: 'https://jsonschema.dev/schemas/examples/non-negative-integer'
      description: Must be a non-negative integer
      $comment: A JSON Schema that uses multiple external references
      $defs:
        nonNegativeInteger:
          allOf:
          # These references remain unchanged because they rely on the base URI of this schema resource
          - $ref: /schemas/mixins/integer
          - $ref: /schemas/mixins/non-negative
      $ref: '#/$defs/nonNegativeInteger'
    integer:
      $schema: 'https://json-schema.org/draft/2020-12/schema'
      $id: 'https://jsonschema.dev/schemas/mixins/integer'
      description: Must be an integer
      type: integer
    non-negative:
      $schema: 'https://json-schema.org/draft/2020-12/schema'
      $id: 'https://jsonschema.dev/schemas/mixins/non-negative'
      description: Not allowed to be negative
      minimum: 0
```

The schemas are inserted into the `components/schemas` location of the OAS document. The keys used in the `schemas` object have no importance for reference resolution, although you will want to avoid potential duplications. References need not change, and a processor of the resulting bundled or Compound Document, should look for the use of embedded Schema Resources within the OAS document, keeping track of the `$id` values.


## But what about...

The astute among you might have noticed that Compound Documents may not be correctly validated using a meta-schema for the dialect defined at the document root. One of our principal contributors distilled a great explanation which he has agreed to let us share with you.

“*If an embedded schema has a different `$schema` than the parent schema, then a Compound Schema Document can't be validated against a meta-schema without deconstructing it into separate schema resources and applying the appropriate meta-schema to each. That doesn't mean the Compound Schema Document is not usable without deconstruction, it just means that implementations need to be aware that the `$schema` can change during evaluation and handle such changes appropriately.*” - Jason Desrosiers.

If you’d like a more in-depth look at edge case situations, please do let us know.

You can reach out to us [@jsonschema](https://www.x.com/jsonschema) or our [Slack server](/slack).

*I hope you’ll agree, Ben has clarified the process for us all here, and we can use this example to fully meet JSON Schema’s bundling expectations when writing tools which bundle multiple resources into compound OpenAPI documents. Thanks, Ben!* - Mike

<a href="https://www.freepik.com/vanitjan1">Business photo created by vanitjan - www.freepik.com</a>

This article was first published on the JSON Schema Blog and is canonically located at https://json-schema.org/blog/posts/bundling-json-schema-compound-documents


Mike Ralphson

JSON Schema bundling finally formalised


If the JSON Schema project had a Code of Conduct in the early days, I suspect we would be years ahead of where we are today. It may seem unlikely, and you may feel like you rarely see a code of conduct in action, but having a Code of Conduct lays some foundational expectations from contributors. You could argue that you shouldn't NEED a Code of Conduct, that people should use common sense and be nice... but it's not so clear cut. Let me explain.

## My culture is not your culture

Star Trek fans will no doubt be well aware of the "Prime Directive". For the rest of us, the Prime Directive is a top priority general order to not interfere with other cultures and civilizations. The theory goes, humans are more likely to have a disastrous effect regardless of the intent because cultures can be very different.

While the failure of an open-source project is unlikely to cause the collapse of another civilization in our reality, the general principle still applies; You cannot presume to know the culture of others. What is normal or acceptable in one culture is not in another.

How do these concepts translate from fiction to reality? While there are many extreme examples, one TV advert always stuck with me as a good example of cultural differences causing undesirable results.

The advert depicts an English man sitting down for a meal with 10 or so Chinese businessmen. The English man orders an eel, possibly by mistake as he looks worried, but eats the whole thing. The voiceover says, "The English believe it's a slur on your host's food if you don't clear your plate, whereas the Chinese feel you're questioning my generosity if you do." The restaurant brings out another eel, this time much larger.

This advert was one of a series published for HSBC, highlighting the importance of "local knowledge". While it makes a nice soundbite, it is a good demonstration of different cultural norms and expectations.

## Communication is mostly non-verbal

Avoiding cultural clashes is even harder when using mostly text to communicate. Experts estimate that only 7% of how we communicate is by using the meaning of the words we use. That leaves 93% attributes to tone and body language. Wow.

When you communicate through text with people you usually communicate with in person, it's usually not difficult to fill in for the lack of tone. But, when you've never met or seen the person you're interacting with, you have no reference or basis to make assumptions. What sometimes happens is we fill in those missing parts to draw our own conclusions, which may not match the intent.

So how should an open-source project on GitHub avoid cultural clashes and misunderstandings?

Emojis can help increase the likelihood of understanding the correct intent, as they can convey tonal and emotional implications. Increasing the level of understanding and intent certainly helps avoid misunderstandings, but this isn't always going to help with cultural differences.

## Our culture and colloquial language

It's easy to forget how much culture affects how we interact every day. "Shaka, when the walls fell." You might recognize this as a quote from StarTrek. Imagine you land on a planet and you have a universal translator, but the words just don't make any sense. In this fictional situation, the alien language only uses colloquialisms; phrases that have meaning beyond the words used.

We find colloquialisms in one language which are nonsensical to most others. Many of you will find yourself part of various internet cultures, and these too will have their own colloquialisms. With images being a way we can communicate online, we've built visual colloquial language, which we call memes.

What I'm implying here is that we can, and do, create our own cultures, and that's OK. We as JSON Schema chose to adopt the Contributor Covenant for [several reasons](https://github.com/json-schema-org/community/blob/main/docs/adr/2021-08-26-code-of-conduct.md), and this sets our cultural tone. It defines how we should interact, and what our expectations should be when we interact. In addition to the Contributor Covenant, we continue to keep the IETF Guidelines for Conduct (BCP 54), creating our own Code of Conduct.

## Why a Code of Conduct?

We've explored that cultural differences can make it hard to understand intent. But, even if the intent is clear, sometimes specific conduct can be unwelcome.

With open-source, people come and go. The people who were super helpful today may be busy tomorrow... or de-prioritise a specific project for years. Aside from making people feel welcome to be nice, it makes good sense in terms of developing and maintaining the project long term. People are more likely to stay if they continue to feel welcomed and valued.

Some get involved because it's their job, some because it's fun and rewarding. Some are lucky enough to be able to combine those two aspects. But whatever your reason, we want open source to be a positive experience.

While I often suggest people avoid making assumptions, when your communication is limited, you have to make some assumptions or ask for clarification. However, a Code of Conduct allows you to frame your interactions with expectations. "If a comment reads bad, but I assume their intent was good, can I understand the likely meaning differently?"

Beyond setting the expectations of good interactions, there is conduct we (agreeing with the Contributor Covenant) defined as not acceptable such as public or private harassment. While it's hard to imagine a culture where that would be acceptable, there are some cultural norms and expectations that are totally unacceptable in many cultures.

The Contributor Covenant makes clear the expectations in the form of a pledge and provides examples of meeting those expectations, but it doesn't give any specific hard rules for working (how to do "business"). CBP 54 does give some clear expectations, such as how we should work: "We dispute ideas by using reasoned argument rather than through intimidation or personal attack."

We feel the combination we presented and agreed upon in full view and comment from the community aligns with our expectations, and we hope you agree. If that changes or your opinions differ, we will gladly hear them, and try to understand. This is part of what consensus building looks like, some of which is set out in CBP 54.

Our choice for the Code of Conduct was partly based on the document's reputation and existing usage. The Contributors Covenant has been adopted by many projects and organizations and has undergone several revisions. It has been developed and updated by many with much experience and knowledge of code of conducts, which is something we can't say for anyone in our team.

## What if we just... did nothing?

Without getting too philosophical, humans seem to often find conflict even when full communication is possible. It's almost inevitable that with text-only communication, there are going to be some misunderstandings, and that drives conflict.

JSON Schema as a project previously stalled because the core team (as much as I can tell) had [differing expectations](https://github.com/json-schema/json-schema/issues/52). Discussions got heated. Tempers were raised. There was no default laid expectation to frame potential misunderstandings. There was no framework for resolving or remediating conflicts.

Right now, there is a fair amount of fragmentation of JSON Schema tooling and version support. We are actively trying to help ease the issue, but my feeling is this only came about where the specification development stopped. If it had continued, and those projects and tools had been supported, we might have found ourselves in an easier situation.

We've lost knowledge. We're trying to capture knowledge and decision reasoning moving forward, but there are some things we just simply don't know, and likely now never will. Key people of the previous team that worked on JSON Schema seem to have stepped away from any public open-source work. It may be a coincidence, but based on very public arguments, I suspect not.

When we frame our interactions with expectations, we give the benefit of the doubt. With an increasingly wide user base, we have to expect contributors may not have English as their first language, making graceful interpretation increasingly important.

One strength of JSON Schema is the ability to use any compliant implementation, and swap out one for another with minimal effort if needed. Do others need to use the schema in another programming language? No problem. That's great for validation, but what about generating types or classes? What about UI generation and database generation? What about generating documentation?

Having fixed many problems and inconsistencies, developed many new features, and re-established active development of JSON Schema, the legitimacy and trust of the project are growing. We are now interacting with and guiding several initiatives to define standardized extensions to JSON Schema. Standardized to ensure interoperability.

As our community grows, so must our approach with which we engage with our community, and our expectations. Establishing a Code of Conduct is just one of several initiatives that seek to elevate JSON Schema from being an unsung silent hero, into the legitimate, established, and well used critical standard we know it can be. We must learn from our history if we don't want to repeat its mistakes.

## How do I get involved?

We recently updated our homepage to include a few links to our active community and regular activities, but let's recap here anyway.

The majority of our discussions happen on our [open slack server](https://json-schema.org/slack). Our general channel is for all discussions, including those looking for help. We have other channels for specification development and implementers, the official test suite, even for monitoring GitHub and StackOverflow activity. It's our central hub. But, it is ephemeral, and we lose access to older messages (a limitation of the free account).

The more long-living and searchable location for activity and discussion is now our [GitHub Discussions](https://github.com/json-schema-org/community/discussions). Currently, these are limited to our community repository, but we plan to open these up across other repositories later.

Currently, there are two sets of regular calls we hold.

First is our [JSON Schema Office Hours](https://github.com/json-schema-org/community/discussions/34); an hour every week at the same time where anyone can come and ask questions or just discuss anything JSON Schema related. It has seen low traffic, but it sends an important signal to our community: "We're here to help you as much as we can." Every Tuesday at 15:00 UTC.

Second is our [Open Community Working Meetings](https://github.com/json-schema-org/community/discussions/35). For these we alternate between times; First Friday of the month at 20:00 UTC, third Friday of the month at 14:00 UTC. This enables the calls to be open to a broader audience. These meetings have a semi-formal agenda and serve as a call to action to gather feedback and thoughts on key decisions and to progress work that needs to be done. We always make sure we capture the resulting action items and follow up in the next session.

Both sets of calls are open for anyone to join. Both are recorded, but only the Open Community Working Meeting is shared publicly. The Office Hours session is not shared, hopefully allowing people to speak more freely.

Besides Slack, GitHub Discussions, and regular meetings, we do make use of X. You can find me running the [@jsonschema account](https://www.x.com/jsonschema). Any mention of "JSON Schema" feeds into a channel on Slack, so we see most of the discussion, and can lend a hand or point people in a direction as appropriate.

Hopefully, this has been a helpful look at why JSON Schema specifically needs a Code of Conduct. Maybe you're considering if your project needs a Code of Conduct. If you have any questions, thoughts, or comments as a result of this article, please feel free to use any of the above methods to make contact.

Image by <a href="https://pixabay.com/users/broesis-5213623/?utm_source=link-attribution&amp;utm_medium=referral&amp;utm_campaign=image&amp;utm_content=2366955">Maike und Björn Bröskamp</a> from <a href="https://pixabay.com/?utm_source=link-attribution&amp;utm_medium=referral&amp;utm_campaign=image&amp;utm_content=2366955">Pixabay</a>


Why JSON Schema needs a Code of Conduct


## Challenge

Cookpad Mart is one of the new businesses that Cookpad Inc. is focusing on. It is an e-commerce platform that mainly deals with fresh food, and retailers in the city and local producers participate in Cookpad Mart as sellers. Dedicated refrigerators are installed as pick-up locations for users in various places such as convenience stores, drug stores, train stations, and condominiums. Users can place orders from the app and receive fresh food from the refrigerator.

Cookpad Mart was developing a management screen for sellers that provided functions for sellers to register products and perform daily shipping operations.

"It is necessary to enter different types of data depending on the type and state of the product, but which information should be manually entered from the form in which all items are listed? It is very difficult to judge each time," says Kenshi Shiode, Engineer, Cookpad Mart Team. There was a need for a mechanism to sort out appropriate forms according to the type and condition of the product.

In addition, it is not possible to completely prevent the registration of invalid data simply by controlling the distribution of forms on the front-end.
Although in-house operation members perform product screening before the product starts selling, it is desirable that the product is registered after being validated on the back-end side in order to avoid the burden of product screening and mistakes.

## Solution

The team considered using custom code to generate complicated forms, but were concerned about consistent validation in the browser and on the server side. In looking to create a common Schema, JSON Schema was chosen as the mechanism. "We decided to introduce JSON Schema as a mechanism that can separate and validate complicated forms," says Kenshi.

## Impact

JSON Schema has improved the accuracy and experience when registering products, and has greatly reduced the operational burden of product screening.

Using JSON Schema has reduced the burden of changing requirements. "Even if new requirements increase in the future, it can be solved by simply updating the JSON Schema definition, and validation of the form distribution control logic on the front-end side and validation on the back-end side can be easily performed," says Kenshi.

## Key Impact Results

<Bigquote>
  From 10% of submissions being deficient, to zero.
</Bigquote>

"Previously, about 10% of newly registered products had incomplete input of quality assurance items, but with the introduction of JSON Schema, there are no deficiencies in the quality assurance items," says Kenshi.

## About Cookpad Inc.

<p className="text-2xl my-10">Cookpad is a global recipe sharing platform, boasting an average of 100 million people every month across the world, spanning 76 countries, supporting 34 languages, and boasting more than 5.5 million shared recipes. It is the largest recipe sharing service in Japan.</p>

<div className="container flex flex-col sm:flex-row sm:space-x-4 space-y-4 sm:space-y-0 place-content-between">
  <figure className="group m-auto">
    <img className="flex-1" src="/img/posts/2021/cookpad-case-study/fridge.webp" />
    <figcaption className="absolute text-2xl -mt-9 px-4 bg-gray-600 bg-opacity-50">
        <span className="text-white">Cookpad Mart Fridges</span>
    </figcaption>
  </figure>
  <figure className="group m-auto">
    <img className="flex-1" src="/img/posts/2021/cookpad-case-study/office.webp" />
    <figcaption className="absolute text-2xl -mt-9 px-4 bg-gray-600 bg-opacity-50">
        <span className="text-white">Cookpad Offices</span>
    </figcaption>
  </figure>
</div>

As a tech company, they work to "make everyday cooking fun". They believe that cooking is the key to a happier and healthier life for people, communities and the planet. Providing easier access to fresh quality ingredients enables better cooking, which is where Cookpad Mart has an impact.

"Delicious things while they are delicious" is one of the primary thoughts of Cookpad Mart, looking to connect customers more directly with producers. Fresh ingredients, recommendations, and recipes, make for an appealing proposition. "The crispy and elastic texture and the dripping milky texture are the real pleasures of fresh mozzarella," reads one example product.

High quality ingredients that come from small producers tend to be difficult to find in the general market, making the most sensible approach to be mostly self-service listing. While Cookpad Mart operates in a limited area for now, the ability to scale and reduce manual review and approval is important, in addition to enabling customers to get the freshest ingredients.

## Benefits in development

"We thought that we should use the same logic for backend and frontend applications. At first we thought about creating our own custom JSON Structure, but we had to write many `if` statements, and I do not want to imagine!"

<Bigquote>
"JSON Schema helps us to implement backend validation and frontend form generation easily. It was quite an excellent experience!" explains Kenshi, estimating that time spent building validation has been cut in half as a result of using JSON Schema.
</Bigquote>

One mantra developers often hear is DRY: 'don't repeat yourself'. "We could focus ONLY on JSON Schema definitions and everything has been automatically generated and applied," says Kenshi, "That was an excellent moment." Reducing duplication of logic reduces the likelihood of a mismatch in expectations between the client and server side.

While AI is often compared to many `if` statements, experienced application developers usually seek to reduce complexity and therefore risk. Reducing complexity by removing many conditional blocks has made code reviews easier, and even had a positive impact on continuing maintenance tasks. "Engineers would like to reduce "if" statements as much as possible I think," quips Kenshi.

Many JSON Schemas we see are kept quite simple, but there's a lot of power on offer for those willing to read the documentation. "I was really astonished with the vocabulary of JSON Schema," Kenshi continues, "There are not only simple expressions like type and length, but also complex expressions like `oneOf` and even `If-Then-Else`."

Conditional validation is new when you consider the long life of JSON Schema. Once you unlock the understanding of the multiple approaches to applying subschemas, composition can be satisfying and powerful. "When I found the rich vocabulary prepared by JSON Schema, I felt confident to use JSON Schema. Very nice!" We happen to agree, Kenshi!

Thank you to Kenshi Shiode, Engineer, and Cookpad, for allowing and enabling us to share this case study with you.

Note:
Some parts of this case study are originally located in a blog posting by Cookpad, located at https://techlife.cookpad.com/entry/mart-json-schema. Used with permission.


JSON Schema deduplicated complex logic and validation at Cookpad


## 挑戦

クックパッドマートとは、クックパッド株式会社が力を入れて取り組んでいる新規事業のひとつです。生鮮食品を中心として扱っているECプラットフォームで、街の食材専門店や地域の生産者が、販売者としてクックパッドマートに参加しています。コンビニエンスストア・ドラッグストア・駅・マンションなどの様々な場所に、ユーザーの受け取り場所として専用の冷蔵庫が設置されています。ユーザーはアプリから注文を行い、冷蔵庫から生鮮食品を受け取ることができます。

クックパッドマートでは、販売者が商品の登録や日々の出荷作業などを行うための機能を提供する販売者向け管理画面を開発しています。

「商品の種類や状態によって異なる種類のデータを入力する必要がありますが、全項目が羅列されるフォームから適切な項目の入力を都度行うことはとても難しい状態でした。」とクックパッドマートの開発に携わっているエンジニアの塩出さんは語ります。商品の種類や状態に応じて、適切なフォームが出し分けされる仕組みが求められていました。

また、フロントエンド側でフォームの出し分け制御がされるだけでは、不正なデータの登録を完全に防ぐことはできません。社内の運用メンバーが商品の販売開始前に審査を行っていましたが、商品審査の負担やミスを避けるために、バックエンド側でバリデーションされた上で商品が登録されることが望ましい状態でした。

## 解決策

開発チームでは、独自のコードを使用して複雑なフォームを生成することも検討しましたが、フロントエンドとバックエンドでのバリデーションの一貫性について懸念していました。そこで、共通の定義を作成するために、JSON Schemaが仕組みとして選択されました。「複雑なフォームをバリデーションできる仕組みとしてJSON Schemaを導入することにしました。」と塩出さんは言います。

## 結果

JSON Schemaにより、商品登録時の正確性や体験が改善され、商品審査にかかる運用の負担が大幅に軽減されました。

また、JSON Schemaを利用することで、要件の変更に対して柔軟に対応することができるようになりました。「今後新しい要件が増えても、JSON Schemaの定義を更新するだけで解決し、フロントエンド側のフォームの出し分け制御ロジック・バックエンド側のバリデーションを容易に追加・更新することが可能な状態になりました。」と塩出さんは言います。


## 主な成果

<Bigquote>
品質保証に関する入力不備が10％から0％に
</Bigquote>


主な成果として「以前は、新規登録商品の約10％で品質保証の項目の入力不備がありましたが、JSON Schemaの導入により、品質保証の項目の入力不備はほぼゼロになりました。」と塩出さんは言います。

## クックパッド株式会社について

<p className="text-2xl my-10">クックパッドは、クックパッド株式会社がグローバルに展開している料理レシピ投稿・検索サービスです。世界の76か国で毎月平均1億人のユーザー数を誇り、34の言語をサポートし、日本では550万件以上のレシピが共有されている国内最大のレシピサービスです。</p>

<div className="container flex flex-col sm:flex-row sm:space-x-4 space-y-4 sm:space-y-0 place-content-between mb-6">
  <figure className="group m-auto">
    <img className="flex-1 " src="/img/posts/2021/cookpad-case-study/fridge.webp" />
    <figcaption className="absolute text-l -mt-6 px-4 bg-gray-600 bg-opacity-50">
        <span className="text-white">クックパッドマートの冷蔵庫</span>
    </figcaption>
  </figure>
  <figure className="group m-auto">
    <img className="flex-1" src="/img/posts/2021/cookpad-case-study/office.webp" />
    <figcaption className="absolute text-l -mt-6 px-4 bg-gray-600 bg-opacity-50">
        <span className="text-white">クックパッド株式会社のオフィス</span>
    </figcaption>
  </figure>
</div>

テクノロジー企業であるクックパッド株式会社は、「毎日の料理を楽しみにする」というミッション実現に向けて様々な事業に取り組んでいます。料理は、人・地域社会・地球での生活に幸せと健康をもたらす大事なものであると考えています。クックパッドマートでは、新鮮で高品質の食材を手軽に入手できるようにすることで、料理をより良い体験にすることができると考えています。

「おいしいものを新鮮なうちに」は、クックパッドマートの指針のひとつであり、ユーザーと生産者をより直接的に結び付けることを目指しています。新鮮な食材、おすすめを紹介する特集やレシピなど、ユーザーへ魅力的な提案をしています。代表的な商品のひとつには「キシっとした弾力ある食感とジュっと滴るミルク感は、フレッシュなモッツァレラならではの醍醐味です。」と記載されています。

クックパッドマートは現在限られた地域でのみサービス展開されていますが、ユーザーが新鮮な食材を入手できるようにする仕組みだけでなく、事業規模の拡大に向けて、商品のひとつひとつの審査・承認にかかる運用コストの削減も重要です。


## 開発でのメリット

「バックエンドとフロントエンドのアプリケーションに同じロジックを使用する必要がありました。最初は独自のJSONフォーマットの作成を考えましたが、多数の`if`文を自分たちで作成する必要が出てくるため、避けたいと思っていました。」

<Bigquote>
"「JSON Schemaのおかげで、バックエンド側でのバリデーションとフロントエンド側のフォーム生成をとても簡単に実装することができました。とても素晴らしい体験でした！」と塩出さん。JSON Schemaを使用した結果、バリデーションおよびフォームの実装にかかる時間が半分以下で済んだ、と見積もっています
</Bigquote>

開発者はDRY「Don't repeat yourself」という言葉をよく耳にします。「JSON Schemaの定義の構築のみに集中できました。その後は、JSON Schemaをサポートするライブラリによって、ほぼ全てが自動生成されました。最高の瞬間でしたね！」と塩出さんは語ります。ロジックの重複を減らすことで、フロントエンドとバックエンドの間で実装の不一致が起こりにくくなります。

AIは多数の`if`文と比較されることがよくありますが、経験豊富なアプリケーション開発者は複雑さを軽減し、リスクを軽減しようとすることが多いです。多くの条件付きブロックを削除して複雑さを軽減することでコードレビューが簡単になり、継続的なメンテナンスタスクにもプラスの影響を与えました。「多くのエンジニアは`if`文を必要最小限に減らしたいのではないでしょうか。」と塩出さんは言います。

私たちが目にする多くのJSON Schemaはとてもシンプルなものですが、ドキュメントを読む人に多大なる力を与えます。「JSON Schemaの構文の種類には本当に驚きました。型や文字列長などの単純な制約だけでなく、`oneOf`や`If-Then-Else`などの複雑なものもありました。」と塩出さんは続けます。

JSON Schemaはかなり前からありますが、条件付きのバリデーションはまだ新しいものです。あるデータ構造の制約を定義するために、複数のアプローチが存在することを理解できるようになると、JSON Schemaはより強力なものになります。「JSON Schemaの豊富な表現力を見つけたとき、JSON Schemaを自信を持って使うことができました。すごく良かったです！」塩出さん、本当にその通りです！

JSON Schemaの活用事例の紹介に協力してくださった、エンジニアの塩出さんおよびクックパッド株式会社に深く感謝いたします。

注：
本事例紹介の一部は、https://techlife.cookpad.com/entry/mart-json-schema にあるクックパッド株式会社による開発者ブログから許可を得て引用しています。


クックパッド株式会社におけるJSON Schemaの活用事例


[Last time](the-last-breaking-change), I wrote about how we had to remove support for unknown keywords.  I ended that post with a note saying that we were still searching for a way that we can support non-functional custom keywords, that is, simple annotations.

In this post, I'd like to review the solution and how we came to it, as well as a brief overview of some of the other solutions we dreamed up and were offered.  Buckle up, 'cause it's gonna be a mild ride!

## What's the solution?

The `tl;dr` of it is that going forward, JSON Schema will treat any keyword that starts with `x-` as an annotation.

This solution was chosen for several reasons.

1. Ad-hoc keywords necessarily cannot have any functionality tied to them.  That is, they're only ever annotations; their values are just returned to the user or application without any processing by the schema.
2. Having ad-hoc annotations follow a convention makes them very easy to identify.
3. A prefix is a good convention to follow.
4. Reserving this prefix for ad-hoc annotations means that there can be no collision with keywords defined by vocabularies, now or in the future (preserving our compatibility promise).
5. Developers are generally familiar with the `x-` prefix as it's already commonly used in other arenas, such as HTTP headers, to denote custom data intended for recipients that understand it.

The one hesitancy we had for using `x-` was that its origins denoted experimental behavior.  However, it seems that in practice, this prefix used quite liberally for any custom data.  Since we're making custom data our expressed purpose, it seems like a good fit.

## Why a prefix and not some other solution?

After releasing the previous blog post and publicizing it as far across the internet as we could, we received feedback that custom keywords were used quite extensively and not supporting them would break users more severely than we wanted.  While we had already planned on still supporting annotative custom keywords, we didn't know what that would look like.  So, I started a [discussion](https://github.com/json-schema-org/community/discussions/57) with some of the loose ideas that the team already had.

There's a lot in that discussion and quite a few ideas, but I'll just go through some highlights here.  If you're interested in the details, feel free to read the discussion.

### Alternative #1 - Defining the prefix in a new keyword

This option actually builds on the one we selected by defining a new Core keyword (one that starts with `$`) that would contain the prefix used for that schema.  It's a fascinating concept that would allow schema authors to use the prefix they wanted.

It was pointed out, however, that in order for a schema to be validated by the meta-schema, the meta-schema would need to be able to read this new keyword to get the prefix so that it could ignore the keywords that started with it.  This requires a whole lot of new mechanisms that we don't currently have in JSON Schema, so it's not very practical at this time.

We also noted that we couldn't figure out what the scope of this keyword would be.  Would it be only the schema resource (indicated by `$id`) where the keyword was used?  Would it be the whole document?  What if we `$ref` to another schema resource or document that doesn't define a prefix?  There's a balance somewhere between inferring intent and requiring too much repetition.

### Alternative #2 - Listing the custom keywords to ignore in a new keyword

This option defines a new Core keyword such as `$ignored` that would hold an array of the names of keywords to ignore.  This would allow schema authors to explicitly define the keywords they wanted to use.

Like alternative #1, this has the problem that JSON Schema doesn't currently have the mechanisms to perform the kind of meta-schema validation that would be required, as well as the same scoping issues.  It's also possible that a schema author could ignore a keyword that would later be added to the spec or some vocabulary, meaning that it _shouldn't_ be ignored, resulting in surprisingly wrong validations and violates our compatibility requirements.

### Alternative #3 - Inlined vocabularies that define keywords

This option allows for a vocabulary to be defined and described _within_ a meta-schema's `$vocabulary` keyword.  This is like alternative #2, except that the ad-hoc keywords are defined by a vocabulary so the normal JSON Schema process that enforces "no unknown keywords" wouldn't pick them up; they would be known.

We decided against this because it requires a lot of further development of vocabularies, a concept which is still under development anyway.  This solution would also skew that development toward solving this problem, which may not be the right direction for the vocabulary concept.

### Alternative #4 - A new keyword to contain all custom annotations

This option creates a new Core keyword such as `$extra` that simply houses all of the custom keywords that a schema author may want to use.

Undesirably, this creates a layer of separation between the annotations and the data they're trying to annotate.  But more importantly, a keyword can only create a single annotation (as it's defined now), and so a keyword like `$extra` would lump all of the annotations into a single large object rather than being more targeted like individual keywords would be.

### Alternative #5 - Support unknown keywords behind an option

Finally, this option just requires implementations to provide a configuration option to allow unknown keywords, defaulting to "disallow."  While allowing unknown keywords breaks our compatibility promises, a user explicitly setting this option is effectively acknowledging that risk.

This felt like [a step backwards](https://vlipsy.com/vlip/the-emperors-new-groove-this-is-a-step-backwards-0YryOW3W).  It just didn't feel like it aligned with the spirit of supportability we want for this project.

## How did we arrive at `x-` as the prefix of choice?

In the previous post, I linked to the discussion and invited everyone to support their preferred ideas or propose new ones.  We saw a lot more interaction than we have in the past, and that was great!

Additionally, we posted about the discussion on social media and we even found several posts (e.g. on reddit) that others made linking back to the blog post or the discussion (or both) and discussing on that platform.

Once we had determined that the prefix option looked like the favorite, Benjamin Granados, our Community Manager, created a survey that listed options for what that prefix could be.  It included the proposals we had in the discussion so far, `x-` and a handful of easily typeable symbols, as well as a "make your own" option.

We had 53 respondents, which may not seem like much, but it's more than we've ever had.  The results clearly showed that `x-` (with 17 votes) was the preferred prefix.  Tying for second were the "at" symbol `@` and the asterisk `*` with 12 votes each.

## Too long; Read anyway

Moving forward, prefix your custom annotation keywords with `x-`.

Another nice thing about this solution is that you don't have to wait for the next version of JSON Schema to come out.  You can start updating your schemas today.  `x-` keywords are compatible with all versions of JSON Schema that are currently published; they'll still just be collected as annotations.  And when the next version comes out, you'll already have migrated!

_Cover photo by [Mick Haupt](https://unsplash.com/@rocinante_11) on [Unsplash](https://unsplash.com/photos/dtTu8Ec_uAU)_


Custom Annotations Will Continue


One of the most common questions we get is how to represent a concept from a strongly typed programming language in JSON Schema.  Class hierarchies, polymorphisms, generics, etc.  These ideas define strongly typed languages, and influence our data models.

The topic in this post can apply to any programming paradigm where you have a defined data model and support for something like generic types.  This will likely be the first in a disjoint series on the relation between data modelling in programming languages and JSON Schema.

Today, I want to cover the idea of generics or templates or some other label you may have heard.  To start, let's cover what I mean by "generic types."  This isn't intended as a lesson on them; I just want to ensure that we're all on the same page.

## Generic Types

By "generic types" I mean the feature in many programming languages to create a type that requires the additional information of one or more secondary types to be complete.  In object-oriented programming, generics can also apply to services as well as data models, but since we use JSON Schema to describe data models, we can be fairly certain that the generic types we care about for this use case are wrappers and containers.

In .Net (C#), these are denoted by angle brackets at the end of the type name, so `List<T>` or `Dictionary<TKey, TValue>` where `T`, `TKey`, and `TValue` represent the secondary types.  (These examples are both container types, but you can also do this with envelope types like [Cloud Events](https://cloudevents.io/) (e.g. `CloudEvent<T>`).)

In C++, these types are called "templates" and are denoted by the keyword `template` with the additional type information afterward, also in angle brackets:

```cpp
template <class T>
class List { ... }

template <class TKey, class TValue>
class Dictionary { ... }
```

Typescript also has this concept and mostly follows the C# syntax.

These types are then completed by defining the required secondary type(s).  This is typically done by replacing the type placeholder (e.g. `T`) with the secondary type, so `List<T>` or `Dictionary<string, int>` for the C# examples.  One of the interesting consequences is that a generic type cannot be instantiated on its own: it needs the secondary type so that the compiler or script engine (or whatever runs the code) can know various things about the type, like memory footprint.

So, the question, then, is how to represent a partially defined type in JSON Schema.

## Representing Generics with `$dynamicRef` and `$dynamicAnchor`

The dynamic keywords, collectively `$dynamic*`, enable references that generally can't be resolved until evaluation time, unlike `$ref`, which can be resolved statically with just the schema.  Typically this is most evident when the schema also defines conditionals (`if`/`then`/`else`) which can change the resolution based on the JSON instance being evaluated.

However to support generics, we want to use this dynamic behavior a bit differently.  The strategy we'll use is a two-step process.

1. For the generic type itself, we're going to write a schema with a reference that initially resolves to a subschema that always fails validation.
2. For each derivation, we write a subschema that
    - references the generic type schema from #1
    - defines a new subschema for the same reference that describes the secondary type

To see this in action, let's write schemas for the `List<T>` from above.  Then we'll write two more that will use it to help us define `List<string>` and `List<int>`.

### The Generic Schema: `List<T>`

We start simply with a list of things.

```json
{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://json-schema.example/list-of-t",
  "type": "array"
}
```

Now we define the items.  This is where `$dynamic*` does some work for us.

```json
{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://json-schema.blog/list-of-t",
  "$defs": {
    "content": {
      "$dynamicAnchor": "T",
      "not": true
    }
  },
  "type": "array",
  "items": { "$dynamicRef": "#T" }
}
```

_**NOTE** I used `T` here to match up with the C# `List<T>` to better illustrate what's happening.  You can name it whatever you want._

If we validate an instance against only this schema, `"$dynamicRef": "#T"` resolves to the subschema with `"$dynamicAnchor": "T"` that we included at `/$defs/content`.  In this case, `$dynamicRef` and `$dynamicAnchor` work just like `$ref` and `$anchor`.

The `"not": true` means that all instances will fail validation.  Typically, to ensure all instances fail validation, we'd use the `false` schema, but in this case, we need to include a dynamic anchor, so a simple `false` doesn't work.  I think `"not": true` is probably the cleanest alternative, but you can also use something like `"allOf": [ false ]` if that makes more sense to you.

_**NOTE** An empty array will still pass validation for this schema, but any array that has items will fail._

You can also use multiple dynamic anchors to support types like `Dictionary<TKey, TValue>` that need multiple secondary types.

```json
{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://json-schema.blog/list-of-t",
  "$defs": {
    "key": {
      "$dynamicAnchor": "TKey",
      "not": true
    },
    "value": {
      "$dynamicAnchor": "TValue",
      "not": true
    }
  },
  "type": "array",
  "items": {
    "type": "object",
    "properties": {
      "key": { "$dynamicRef": "#TKey" },
      "value": { "$dynamicRef": "#TValue" }
    }
  }
}
```

And that's it for the generic type.  The magic comes when we define the content.

### Defining Contents

As mentioned before, we need a schema that references `list-of-t` and also provides a definition for `T`.  Let's write one for `List<string>`.

```json
{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://json-schema.blog/list-of-string",
  "$defs": {
    "string-items": {
      "$dynamicAnchor": "T",
      "type": "string"
    }
  },
  "$ref": "list-of-t"
}
```

Here's how this works:

1. When evaluation begins, it creates a "dynamic scope" which starts with the root schema (`list-of-string`) and remains throughout evaluation.
2. This root schema defines a `"$dynamicAnchor": "T"`.
3. Evaluation then `$ref`s into the generic schema, `list-of-t`.  This is a new _lexical_ scope, but the _dynamic_ scope remains unchanged.
4. The generic schema also declares `"$dynamicAnchor": "T"`, but that dynamic anchor is already defined, so the new declaration is ignored.
5. When evaluation hits `"$dynamicRef": "#T"`, it uses the first one from the beginning of the dynamic scope.

If we wanted `int` items instead of `string`s, we just create a new schema where the subschema with `$dynamicAnchor` defines an integer.

```json
{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://json-schema.blog/list-of-int",
  "$defs": {
    "int-items": {
      "$dynamicAnchor": "T",
      "type": "integer"
    }
  },
  "$ref": "list-of-t"
}
```

## Conclusion

By using `$dynamicRef` and `$dynamicAnchor`, you don't need to write full schemas for classes which share structure but vary in a content type.  Instead, you can write a partial, reusable schema for the structure, which makes the fully-defined schemas significantly smaller and easier to maintain.

_**ASIDE** With Draft 2020-12, including `$dynamicAnchor` in the generic schema is required for `$dynamicRef` to work.  In future versions, this requirement will be removed as it isn't strictly needed: any resolution attempt will simply fail.  (This requirement is a holdover from its Draft 2019-09 predecessor `$recursive*`.)  However for the particular application of modelling generic types, I'd still keep it in because it serves as an analog for not being able to instantiate a generic type, like `List<T>`.  The net outcome is the same (validation failure), but I think including it more explicitly describes the intent._


_Cover photo by [Nick Fewings](https://unsplash.com/@jannerboy62) on [Unsplash](https://unsplash.com/photos/-sAtFFj28W4) with some edits by me._ 😁


Using Dynamic References to Support Generic Types


I have a problem: when I read GitHub issues, they occasionally resonate with me, and I obsess about them until they're resolved.  That may not sound like a problem to some, but when that resolution causes JSON Schema implementation developers to ask fundamental design questions for three years... yeah, that's a problem.

And that's precisely what happened coming out of draft 2019-09.  For this version of the specification we released the very first official output format.  It was actually multiple formats that were supposed to cater to multiple needs.

- While most people wanted to know what the errors were, some just wanted a pass/fail result, so we created the `flag` format.
- Among those who wanted some more detail about what actually failed, some preferred a flat list, while others figured a hierarchy that matched the schema would work better.  So we created `basic` for the list people.
- Finally of those that wanted a hierarchy, some wanted a condensed version (which became `detailed`), and others wanted the fully-realized hierarchy (`verbose`).

_**ASIDE** Some people wanted a hierarchy that mimicked the instance data, but we couldn't figure out how to make that work in a realistic way, so we just kinda swept it under the rug and moved on._

## Adding requirements to the spec

At the time, I hadn't contributed anything major to the spec, but I'd been pretty involved in making direction-type decisions, so I thought I'd take a crack at authorship.  That's not to say that I hadn't contributed _any_ text to the spec; just not anything as significant as an entire section.

So I spent a couple weeks writing up the new requirements for output based on the lengthy github issues that results from weeks (months?) of discussion.

Man, I thought I had everything!  I defined the properties, the overall structure, validation examples, and I wrote it all in that wonderful spec-ese that we all love.

I even implemented it in my library, Manatee.Json, before the spec was released just to make sure that it worked.

But I missed something: annotations.  I mean, I considered them, and provided requirements for them.  But I didn't provide an example of the results from a passing instance that generated annotations.  I guess _technically_ I did, but it was buried, nested way down inside the `verbose` example, which happened to be so big that I decided it needed to be in its own file, separate from the specification document.  (Yeah, like anyone was gonna read _that_.)

The highlight of the following years would be the numerous questions I would receive from fellow implementors regarding confusion about the output, mainly around how annotations should be represented.  And my general response to these questions wasn't great either:  "They're just like errors."  I thought it was a trivial exercise.

Fortunately, we listed the output as a whole as a "SHOULD" requirement, so implementations weren't _required_ to do it.  The idea behind this was that we were in the early stages of defining it and we didn't want to put too much of a burden on implementations knowing that we were likely going to tweak it in future releases.

## Tasting my own medicine

It wasn't until I decided to deprecate Manatee.Json to build JsonSchema.Net that I realized why everyone was asking questions.  Having to reimplement the output opened my eyes.

Wow.  I left out a lot!

Knowing what I originally intended helped me quite a bit, but I can't imagine what it must have been like trying to implement what I wrote while not also having written it.

So, I started taking notes.

## Time for an update

Draft 2020-12 has been published for over a year, and I've decided something needs to be done about the output.  I created this mess, and I feel it's my responsibility to clean it up.  (Now it's actually my job to clean it up! 😁)  I organized all of my notes and dumped out a [massive opening discussion comment](https://github.com/orgs/json-schema-org/discussions/63) on improvements that I think could be made to the formats.

The first thing that everyone agreed on was isolating purpose for and renaming some of the output unit properties.  These properties served a distinct purpose, but [naming things is hard](https://martinfowler.com/bliki/TwoHardThings.html), so of course the names for these could be better.  After some back-n-forth, proposed alternatives, and refinements, this got a quick-n-easy PR that's already been merged, so that's one thing done.

- `keywordLocation` ➡️ `evaluationPath`
- `absoluteSchemaLocation` (mostly optional) ➡️ `schemaLocation` (required)
- `errors`/`annotations` ➡️ `details`

You can read the discussion for the rest of the proposed changes, but I want to focus on one in particular.  At some point in the discussion, I had an epiphany:

> Why is the output designed to capture errors and annotations from individual keywords instead of from subschemas when it's the subschemas that ultimately collect errors and annotations and provide the final result?

## Status quo

To understand what I mean by this, let's take a look at the existing output.  We'll start with a simple example, and for brevity we'll only cover the `basic`, or list, form.

```json
// props { "isSchema": true}
{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "example-schema",
  "type": "object",
  "title": "foo object schema",
  "properties": {
    "foo": {
      "title": "foo's title",
      "description": "foo's description",
      "type": "string",
      "pattern": "^foo ",
      "minLength": 10,
    }
  },
  "required": [ "foo" ],
  "additionalProperties": false
}

// instance (passing)
{
  "foo": "foo isn't a real word"
}
```

As you can see, this schema defines that a JSON value must be an object with a single string-valued property, `foo`, and the instance meets these requirements.  Additionally, the schema defines several annotations.

The 2019-09 / 2020-12 specs would require the following output for this evaluation:

```json
{
  "valid": true,
  "keywordLocation": "",
  "instanceLocation": "",
  "annotations": [
    {
      "valid": true,
      "keywordLocation": "/title",
      "instanceLocation": "",
      "annotation": "foo object schema"
    },
    {
      "valid": true,
      "keywordLocation": "/properties",
      "instanceLocation": "",
      "annotation": [
        "foo"
      ]
    },
    {
      "valid": true,
      "keywordLocation": "/properties/foo/title",
      "instanceLocation": "/foo",
      "annotation": "foo's title"
    },
    {
      "valid": true,
      "keywordLocation": "/properties/foo/description",
      "instanceLocation": "/foo",
      "annotation": "foo's description"
    }
  ]
}
```

So what's bad about this?

- Annotations are being rendered as full nodes.  This results in a lot of unnecessary or duplicate information.  This is actually more apparent in the hierarchical formats where everything is grouped by location making the repeated location properties redundant.
- All nodes carry the `valid` property, which makes it difficult to tell what is the result of an annotation vs the result of a validation.
- The top-level node has a plural `annotations` property with an array of nodes, whereas the inner nodes each have a singular `annotation` property with the annotation value.  This is just confusing.

This is just a simple example.  You can see how this can become considerably larger as the size and complexity of the schema grows.

## There's gotta be a better way

There is: reporting output by subschema rather than keyword.

In the above example, this means that we'd get two nodes:  one for the root schema, and one for the `foo` property subschema.  (Also note the aforementioned property name changes.)

```json
{
  "valid": true,
  "evaluationPath": "",
  "instanceLocation": "",
  "details": [
    {
      "valid": true,
      "evaluationPath": "/properties/foo",
      "instanceLocation": "/foo"
    }
  ]
}
```

That _does_ look a lot simpler.  But what about annotations?  Well, we can group them in a new property.  And, since we know that any keyword is only going to produce a single annotation value, we can utilize an object to report those annotations by using the keyword as the property names.

```json
{
  "valid": true,
  "evaluationPath": "",
  "instanceLocation": "",
  "annotations": {
    "title": "foo object schema",
    "properties": [
      "foo"
    ]
  },
  "details": [
    {
      "valid": true,
      "evaluationPath": "/properties/foo",
      "instanceLocation": "/foo",
      "annotations": {
        "title": "foo's title",
        "description": "foo's description"
      }
    }
  ]
}
```

Alternatively for the `basic` format, which is supposed to be a list, the result for the root schema could be moved inside the root output node as shown below.  This is the proposed idea, anyway.  Let us know in a comment on the [PR](https://github.com/json-schema-org/json-schema-spec/pull/1249) which way you prefer.  I'll be using this format for the rest of the post because it's what is currently proposed.

```json
{
  "valid": true,
  "details": [
    {
      "valid": true,
      "evaluationPath": "",
      "instanceLocation": "",
      "annotations": {
        "title": "foo object schema",
        "properties": [
          "foo"
        ]
      }
    },
    {
      "valid": true,
      "evaluationPath": "/properties/foo",
      "instanceLocation": "/foo",
      "annotations": {
        "title": "foo's title",
        "description": "foo's description"
      }
    }
  ]
}
```

The last thing is that the absolute URI of the subschema is now required, so let's add that in.

_**NOTE** All of these examples (both old and new) are generated by my implementation, which uses a default base URI of `https://json-everything/base`.  I've implemented this new output on an experimental branch, and you can view the impact of those changes on my library suite [here](https://github.com/gregsdennis/json-everything/pull/308)._

```json
{
  "valid": true,
  "details": [
    {
      "valid": true,
      "evaluationPath": "",
      "schemaLocation": "https://json-everything/example-schema#",
      "instanceLocation": "",
      "annotations": {
        "title": "foo object schema",
        "properties": [
          "foo"
        ]
      }
    },
    {
      "valid": true,
      "evaluationPath": "/properties/foo",
      "schemaLocation": "https://json-everything/example-schema#/properties/foo",
      "instanceLocation": "/foo",
      "annotations": {
        "title": "foo's title",
        "description": "foo's description"
      }
    }
  ]
}
```

And that's it!  All of the information we had before in a much more concise package.  Moreover, all of the related annotations are grouped together, which increases readability.

## How this affects errors

I wanted to start off with annotations because that's what I missed in the previous iteration.  Now, let's take a look at how a couple of failing instances would be reported.  There is an interesting nuance that isn't immediately apparent, and I had to do some double- and triple-checking to ensure that it was right.

Our first failing instance

```json
{
  "baz": 42
}
```

This will fail because

- `foo` is required but missing
- `baz` isn't allowed

The current error output has the same problems as the current annotations output:

```json
{
  "valid": false,
  "keywordLocation": "#",
  "instanceLocation": "#",
  "errors": [
    {
      "valid": false,
      "keywordLocation": "#/required",
      "instanceLocation": "#",
      "error": "Required properties [\"foo\"] were not present"
    },
    {
      "valid": false,
      "keywordLocation": "#/additionalProperties",
      "instanceLocation": "#/baz",
      "error": "All values fail against the false schema"
    }
  ]
}
```

Note how even though all of the errors actually result from the root schema, they are reported from child locations.  This just seems wrong.

Let's look at the new output:

```json
{
  "valid": false,
  "details": [
    {
      "valid": false,
      "evaluationPath": "",
      "schemaLocation": "https://json-everything/example-schema#",
      "instanceLocation": "",
      "errors": {
        "required": "Required properties [\"foo\"] were not present"
      }
    },
    {
      "valid": false,
      "evaluationPath": "/additionalProperties",
      "schemaLocation": "https://json-everything/example-schema#/additionalProperties",
      "instanceLocation": "/baz",
      "errors": {
        "": "All values fail against the false schema"
      }
    }
  ]
}
```

Again, we see the errors exist as a single `errors` property, which is reported at the subschema level.

Also, that nuance I mentioned appears: that `false` under `additionalProperties` is reported as a separate subschema (because it technically _is_ a subschema), and the error is reported as an empty-string keyword.  Looking at the evaluation path, though, it still appears that we're reporting at the keyword level.  That's the nuance:  we're actually reporting at the subschema level; it's just that the subschema happens to be located at a keyword.  Let's take a look at another failing instance to see this better.

```json
{
  "foo": "baz"
}
```

```json
{
  "valid": false,
  "details": [
    {
      "valid": false,
      "evaluationPath": "/properties/foo",
      "schemaLocation": "https://json-everything/example-schema#/properties/foo",
      "instanceLocation": "/foo",
      "errors": {
        "pattern": "The string value was not a match for the indicated regular expression",
        "minLength": "Value is not longer than or equal to 10 characters"
      }
    }
  ]
}
```

Here, you can see that the evaluation path shows that we are at the subschema located at `/properties/foo`.  Compare this to the previous example, where we were evaluating the subschema `false` at the location `/additionalProperties`, and you can see the similarity.

## Wrap-up

So that's one way that I would like to update the output and the reasoning behind it.  If you have any thoughts on this, please do let us know in a comment either in the [discussion](https://github.com/orgs/json-schema-org/discussions/63) or on the [PR](https://github.com/json-schema-org/json-schema-spec/pull/1249).

Again, if you'd like to see what the impact of making this change in my implementation was, please have a look at [this PR](https://github.com/gregsdennis/json-everything/pull/308).  All of these changes were driven by updating the output, but I think most of them are specific to my architecture, and some of them could be made to the library even without implementing the new output.  The short summary, though, is the net -343 lines of code!

_Cover photo by [Daria Nepriakhina](https://unsplash.com/@epicantus) on [Unsplash](https://unsplash.com/photos/zoCDWPuiRuA)_ 😁


Fixing JSON Schema Output


About this time last year, I hosted a discussion at the API Specification
Conference about the future of JSON Schema. The most popular topic of discussion
was, when JSON Schema is going to be "done". Of course we've heard that question
a lot. The question stems from the "draft" label we put on your releases. The
term "draft" has led to quite a bit of confusion for our community over the
years, so let's take a moment to understand where it comes from.

## Why is it a "draft"?

JSON Schema has been loosely following the IETF standards track RFC process.
That means our releases are in the form of an Internet Draft (I-D). That's why
we refer to them as drafts. However, because of JSON Schema's wide use in
production systems, it's not really possible to treat the spec like a typical
I-D. Therefore, our use of the term "draft" is a bit of legacy artifact from
when the IETF process made more sense for JSON Schema.

This is has been a problem because when people hear "draft", they hear
"unfinished" or "not production ready". That's not the way we treat our
releases. Every release is expected and encouraged to be used in production.
It's no different than OpenAPI releasing a new version. No one asks when OpenAPI
is going to be "done". JSON Schema is just perceived differently because we call
our releases "drafts".

## The real problem

But this isn't just a branding problem. When people ask when JSON Schema is
going to be out of "draft", what they really mean is, when is JSON Schema going
to be "stable". They want to be able to write a schema and be sure that it will
continue to work the same way no matter how JSON Schema evolves in the future.
They want to be able to update their dependencies and not have to update their
schemas that were working just fine.

This effects library maintainers as well. Needing to support multiple versions
of JSON Schema with no backward or forward compatibility guarantees gets
cumbersome and has led many maintainers to
[drop support](https://github.com/gregsdennis/json-everything/issues/310) for
older releases. When this happens, it can leave users with a choice to have to
update all of their existing schemas that weren't broken or pin to a
no-longer-supported version of the JSON Schema library they are using.

## Our solution

Those are the problems we're aiming to solve in the next release. Instead of
continuing to release a new immutable and incompatible version of JSON Schema
with each release, our next release will be a long-lived version that is stable,
but evolving. In this case, "stable" means that there will be strict backward
and forward compatibility requirements that must be followed for any change. It
will be much like JavaScript in that as it evolves you can always be sure that
your existing schemas will continue to work with any JSON Schema library you're
using, but you take a risk using newer features because not all libraries will
have implemented those features yet.

That vision of a stable yet continuously evolving spec doesn't fit well with the
IETF process. There are paths we considered, but nothing was proposed that we
thought would allow us to continue to evolve the standard and get out of "draft"
any time soon. Therefore, the first step to achieving our vision is to [decouple
our main spec development from the IETF process](https://github.com/json-schema-org/json-schema-spec/pull/1277).
This split allows us to pursue a new model for the main spec development that is
more conducive to our vision.

Whether or not you're a fan of the direction the JavaScript language has
evolved, it seems clear that they've come up with an
[effective process](https://2ality.com/2015/11/tc39-process.html) for allowing
for continuous evolution without sacrificing interoperability and longevity.
That's why we've chosen to base our new process on the process that is used to
evolve the JavaScript language. In our next release, most of the keywords and
features you use today will be declared stable and they will never change in a
backward incompatible way again. Features and keywords we aren't comfortable
making stable yet will become part of a new staged release process that we are
working on defining now. The goal of the staged release process is to ensure
that the feature gets sufficient implementation, testing, and real world vetting
for us to be confident in declaring it stable. The process should not only make
us more confident, but also allow us to achieve that confidence much more
quickly.

## Standards considerations

Starting with the next release, the JSON Schema specification will be
self-published on our website.

One of the concerns about self-publishing is about other standards being able to
reference the JSON Schema specification. We have received
[feedback](https://github.com/json-schema-org/json-schema-spec/pull/1277#issuecomment-1228734352)
from people involved in standards development that our approach would be
acceptable for them to reference our specification in their standards based on
our membership with the OpenJS Foundation. We don't know if _all_ standards
bodies will come to the same conclusion, but this feedback gives us confidence
that it isn't going to be a significant problem.

Although the main specification will be self published, we are continuing to
work through the IETF process where it makes sense. For example, we are in the
process of registering our media types such as `application/schema+json` through
the HTTPAPIs WG. We are also looking into standardizing reusable components such
as Relative JSON Pointer through IETF.

## The bottom line

Details about the new process will be shared in a separate post when they are
finalized, but here are some outcomes users can expect.

* If you use only stable features, you are guaranteed interoperability between
  JSON Schema libraries and you will never need to update your schemas just for
  the sake of keeping up with new releases.
* You can safely use new features before they are stable, as long as the
  libraries consuming your schemas support the feature.
* Compatibility/Interoperability guarantees will only apply to the next release
  and beyond. You will need to update your schemas to the stable version, but
  you won't have to continue to update them as JSON Schema evolves.
* Custom dialects and vocabularies will continue to be central concepts for
  customizing and extending JSON Schema.
* Implementers won't need to maintain distinct code to support past stable
  releases. A library that supports the 2025 release will automatically support
  the 2023 and 2024 releases. The past stable releases will no longer need to be
  maintained as distinct versions. However, implementations that continue to
  support the "draft" releases would still need to maintain those as distinct
  versions from the current stable release.


Jason Desrosiers

Towards a stable JSON Schema


> Originally published at [simonplend.com](https://simonplend.com/get-started-with-validation-in-node-js/).

The first step in putting together request validation for your Node.js application is finding a way to create flexible validation rules. When you try to pick a validation library to use, it's often more difficult than you expect. They're all different from each other, and it's not clear what benefits one has over another.

Perhaps you've tried to build your own custom validation before, but it starts to feel messy and difficult to maintain. You want to put some reliable validation in place and move on to building exciting new features in your application. Why is adding in validation *such* a headache?!

In this article, we're going to learn how the JSON Schema specification can help us create flexible validation rules. We'll write a schema which describes the format we expect data to be in, then we'll write some JavaScript that uses the Ajv validator library to validate our data against it.

Let's get started!

## A powerful validation duo

The [JSON Schema specification](/specification) defines a JSON-based format for describing the structure of JSON data. This includes validation keywords, such as `type`, `required` and `properties`. These keywords allow us to create a definition of the format we expect data to be in. This is a "schema". It can be as simple as:

```json
// props { "isSchema": true }
{ "type": "string" }
```

To validate data against a schema, we need to use a validator library which implements the JSON Schema specification. The [Ajv (Another JSON Schema Validator)](https://ajv.js.org/) library is the most popular JSON Schema validator for client and server side JavaScript, downloaded over [50 million times every week](https://www.npmtrends.com/ajv) from npm.

Let's get hands-on and see what validating data with JSON Schema and Ajv looks like.

> For a more detailed introduction to JSON Schema, check out the blog post [JSON Schema in 5 minutes](/blog/posts/json-schema-in-5-minutes).

## Creating a schema and validating data

First, we need to run the command `npm install ajv` to install the Ajv library.

Then we can jump straight into defining a JSON schema. It describes the structure and types we expect:

```javascript
const iceCreamSchema = {
  type: "object",
  properties: {
    flavour: { type: "string" },
    price: { type: "number" },
    stock: { type: "number" },
  },
  required: ["flavour", "price", "stock"],
};
```

Now we'll define the data which we want to validate:

```javascript
const iceCreamData = {
  flavour: "Pistachio",
  price: 1.99,
  stock: null,
};
```

Then we'll import the Ajv library and create a new Ajv instance:

```javascript
import Ajv from "ajv";

const ajv = new Ajv();
```

And use it to validate the data against our schema:

```javascript
const isDataValid = ajv.validate(iceCreamSchema, iceCreamData);
```

Lastly, we'll add some code to handle the validation results:

```javascript
if (isDataValid) {
  console.log("The ice cream data is valid! 🍨");
} else {
  console.error("The ice cream data is invalid:", ajv.errors);
}
```

When we put this code together and run it, we get the following output:

```
The ice cream data is invalid: [
  {
    instancePath: '/stock',
    schemaPath: '#/properties/stock/type',
    keyword: 'type',
    params: { type: 'number' },
    message: 'must be number'
  }
]
```

Pretty slick, huh?

Alright, we've created a schema which describes the structure and types we expect, validated data against it, and handled any validation errors. So far so good!

> A note from the editor: In the latest major release of AJV, a "strict mode" was introduced, enabled by default, which sometimes throws an exception when a valid JSON Schema is used. If you run into problems with strict mode related errors, check out [the documentation](https://ajv.js.org/strict-mode.html) for details on how to disable it, or even specific rules. We feel this may be better as linter functionality, however it does help prevent accidental errors in schema authorship. - Ben Hutton

As one final step, we'll look at how we can potentially improve the way we create our JSON schema.

## Schema generation

JSON schemas are an expressive way of defining validation rules, but writing schemas "by hand" can sometimes get a bit much. There's a handy library called [fluent-json-schema](https://npm.im/fluent-json-schema) which can help us generate our JSON schemas. Let's give it a try.

Note that while using a tool to define validaiton rules to be converted into a JSON Schema gives you control, other methods of generating JSON Schema, for example from your instance data, often leaves you with a partially complete or sometimes incorrect JSON Schema. Other approaches might be better called "scaffolding" rather than "generating", as you'll have the basics, but still have work to do before it's complete and useable.

First we need to install the library by running the command `npm install fluent-json-schema`. Then we can import it and use it to generate our schema:

```
import S from "fluent-json-schema";

const iceCreamSchema = S.object()
  .prop("flavour", S.string().required())
  .prop("price", S.number().required())
  .prop("stock", S.number().required())
  // This method call returns the generated schema as an object.
  .valueOf();
```

If we `console.log` the `iceCreamSchema` object, we can see the JSON schema we've generated:

```json
{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "object",
  "properties": {
    "flavour": { "type": "string" },
    "price": { "type": "number" },
    "stock": { "type": "number" }
  },
  "required": ["flavour", "price", "stock"]
}
```

You'll notice this generated schema is almost identical to the `iceCreamSchema` we previously wrote "by hand". We can replace our handcrafted schema with this generated schema and the validation will behave in the same way as it did before.

If we're writing our applications in TypeScript, the [TypeBox](https://www.npmjs.com/package/@sinclair/typebox) library is a great alternative to `fluent-json-schema`.

Schema generation is down to your personal preference: some folks like to write raw JSON schemas "by hand", whereas others prefer to use a library which helps generate them. Try out both and pick whichever approach works best for you.

## Try it yourself

- **[Browse the code on GitHub](https://github.com/simonplend/creating-flexible-validation-rules)**
- **[Run the code in your browser with StackBlitz](https://stackblitz.com/github/simonplend/creating-flexible-validation-rules)** (works in Chrome, Edge or Brave browsers)
- **[Take on the validation coding challenge](https://github.com/simonplend/creating-flexible-validation-rules#ready-for-a-challenge)**! 🏆

## Wrapping up

We can use Ajv as a standalone library or we can integrate it with the framework we're using to build our Node.js application. Some Node.js frameworks even provide JSON Schema based validation with Ajv [built in](https://fastify.dev/docs/latest/Reference/Validation-and-Serialization/).

When we combine JSON Schema and Ajv, we have a flexible solution for implementing validation in our Node.js applications:

- **Learn once, use everywhere.** The JSON Schema specification is cross-platform, and there are [validation libraries](/tools) available for every popular programming language. We're not locked into a library, framework or language. Once we've learnt the fundamentals of JSON Schema, we can use it everywhere.
- **Portability.** Because JSON Schema is cross-platform, even if we decide to rewrite our applications in another framework or language, we can take our schemas with us.
- **Speed**. Under the hood, Ajv compiles our JSON schemas into JavaScript code. This improves the performance of repeated validation of data against a schema. For example, schemas can be compiled by Ajv when our Node.js application starts. HTTP request data which the application receives can then be validated against the pre-compiled schemas.
- **Active and supportive community**. There's an active community of folks on Slack who are happy to help out (the [JSON Schema website](https://json-schema.org/) has a link you can use to join).

### Further reading

If you'd like to learn more about JSON Schema, you might find these links helpful:

- **[How to handle request validation in your Express API](https://simonplend.com/how-to-handle-request-validation-in-your-express-api/)**. If you're using the Express Node.js framework, I wrote this guide to help you integrate the Ajv library into your applications.
- **[Fastify framework validation documentation](https://fastify.dev/docs/latest/Reference/Validation-and-Serialization/)**. The Fastify Node.js framework integrates the Ajv library, making it straightforward to add validation with JSON Schema.
- **[Understanding JSON Schema](https://json-schema.org/understanding-json-schema/)**. A free online book which will teach you the fundamentals of JSON Schema. Also available in PDF format for offline reading.
- **[JSON Schema Cheat Sheet](https://simonplend.com/wp-content/uploads/2020/12/JSON-Schema-Cheat-Sheet-v1.1.pdf)**. I created this cheat sheet PDF as a handy reference for JSON Schema validation keywords. Use it to give you a helping hand any time you're writing schemas.

> Photo by [Manik Rathee](https://unsplash.com/@manikrathee) on [Unsplash](https://unsplash.com/photos/a8YV2C3yBMk).


Simon Plenderleith

Get started with JSON Schema in Node.js


## Challenge

GitHub stands as a ubiquitous and invaluable tool for software developers and those in related fields, enabling efficient code management, seamless collaboration, automation, project management and more.
More than 100 million users rely on the platform, many of them making use of it daily for building and sharing open source software.

There is a staggering amount of functionality packed into GitHub, assisting across all parts of the development lifecycle, and supporting this ever-growing list of functionality is an undoubtedly powerful and complex underlying platform developed within GitHub, as well as extensive documentation covering features new and old.

Documentation at GitHub is housed at [docs.github.com](https://docs.github.com), itself a massive undertaking, covering pages upon pages of functionality within the platform.
Historically, this documentation was split into two statically generated sites (help.github.com and developer.github.com), but in 2020 these static sites were combined in a new dynamic application hosted at docs.github.com.
The full story of the rebirth of GitHub's documentation is told best on the GitHub blog [here](https://github.blog/2020-10-14-how-we-open-sourced-docs-github-com/) and [here](https://github.blog/2020-07-02-how-we-launched-docs-github-com/), but an important outcome of this change was a move towards data-driven documentation, and in particular a big investment in the use of JSON both to drive content as well as for intercommunication within the application's platform itself.

More specifically, some page content is assembled fully automatically from JSON data files and some JSON is manually authored by content writers.
Within the platform, the application's entire queryable in-memory context is retrievable as JSON.

"Not being able to validate our JSON data against a JSON schema would result in bugs in production like missing data on automated documentation pages or unavailable pages", explain Rachael Sewell and Robert Sese, software engineers who work on GitHub's Docs Engineering team.

Without proper schemas for their JSON data, there was no way to verify whether code changes introduced new bugs, or to ensure that data consumed from external sources was in the format needed for automated documentation pages.

<figure className="mt-10">
  <img src="/img/posts/2023/github-case-study/working.webp"/>
  <figcaption className="mt-2 mb-10 text-sm text-center text-gray-500">Hubbers (GitHub staff) working to empower developers.</figcaption>
</figure>

## Solution

The team began to introduce JSON Schemas to validate all of the JSON data files, context data and API request bodies consumed or produced as part of the application.
Each time a change to the data model is made, the schemas are updated accordingly, and in any circumstance the schemas are used to validate that the application functions correctly with each change.

This occurs in three main ways:

  * While the application runs in production, each API call has its request body validated via a corresponding JSON Schema before passing the event off to a data warehouse
  * When retrieving external data in automation pipelines, whenever data is transformed from a source format into JSON, the generated data is validated against a JSON Schema to ensure it has been correctly transformed.
    If validation succeeds, the generated data is checked into a git repository to be used in production.
    Otherwise, the failure is raised for investigation.
  * When running continuous integration each time a change is made to the application, various additional schemas ensure:

    * that YAML frontmatter properties are correctly included in Markdown files which are used to generate pages within the application
    * that YAML or JSON data files which contain page content and which have been hand authored by content writers are correctly formed
    * that the context object created at runtime which contains the entirety of the site content along with a site tree for the application is itself correctly formed

<p className="text-2xl my-10">"Choosing JSON schema to allow JSON schema validation was a natural and obvious choice that our team made. It has been a fundamental part of our application since we moved from having a static site to a dynamic application about 3 years ago." - Rachael Sewell & Robert Sese, Docs Engineers at GitHub</p>

## Impact

Introducing JSON Schema into the platform produced meaningful impact in productivity, in discoverability, as well as in reliability.

<p className="text-2xl my-10">"JSON schema makes it so much easier to see the shape of a data and its property types. I can quickly open the file on disk and understand what the data structure looks like. This saves the whole team time when extending a feature that relies on data backed by a schema." - Rachael Sewell & Robert Sese, Docs Engineers at GitHub</p>

The GitHub team moves quickly, with the documentation team releasing to production 20 times per day or more, and relying heavily on continuous integration checking each and every commit to ensure changes work as intended.
Failures in continuous integration alert the team before the change is shipped out to production, with JSON Schema validation an integral piece of ensuring all of the various pieces of JSON data needed for the application are properly formed.

<figure className="mt-10">
  <img src="/img/posts/2023/github-case-study/building.webp"/>
  <figcaption className="mt-2 mb-10 text-sm text-center text-gray-500">GitHub Offices.</figcaption>
</figure>

## GitHub - The Company

<p className="text-2xl my-10">GitHub is the world's largest developer platform, helping developers and organizations around the world to build, scale and deliver secure software.</p>

It is based in San Francisco, CA, though its hiring and work culture is remote-first.

Particular to the subject at hand, within GitHub is an internal organization called Education, Community, and Open Source Software, which houses the Docs team responsible for the documentation application.
This Docs team consists of content writers, content designers, docs product managers, and docs designers and docs engineers.

Thank you to [Rachael Sewell](https://github.com/rachmari) and [Robert Sese](https://github.com/rsese), Software Engineers at GitHub's Docs Engineering team who maintain the wonderful [docs.github.com](https://docs.github.com), as well as to the entire team at GitHub, for sharing their experiences and allowing us to further share them with you.


How JSON Schema Was an Obvious Choice at GitHub


Last year, JSON Schema proudly participated in Google Summer of Code (GSoC) 2024, a global program designed to bring fresh talent into open-source communities. Through this initiative, we tackled some of our most exciting projects yet, creating opportunities for contributors to make meaningful impacts while gaining valuable experience.

We’re thrilled to share the highlights of this journey and the incredible work our contributors and mentors achieved together.

## A Unique Start: Setting the Stage for Success
For JSON Schema, GSoC 2024 wasn’t just about welcoming contributors but about setting a high standard. Before selection, applicants underwent a qualification test, allowing us to assess technical skills and alignment with project goals. This ensured each participant had the tools and mindset to thrive.

Here’s what stood out in our journey:

* 8 projects completed across diverse aspects of JSON Schema.
* A record number of applicants, with a community brimming with enthusiasm.
* Returning mentors and contributors, bringing experience and new perspectives.

## Transformative Projects and Talent Contributors

### 1. Bowtie-Trend: Long-Term Reporting With Bowtie

* Contributor: [Adwait Godbole](https://github.com/adwait-godbole)
* Mentors: [Julian Berman](https://github.com/Julian), [Agnivesh Chaubey](https://github.com/AgniveshChaubey)
* [Idea on GitHub](https://github.com/json-schema-org/community/issues/607)

The project aims to enhance [Bowtie](https://github.com/bowtie-json-schema/bowtie), a meta-validator for the JSON Schema specification that coordinates the execution of validator implementations and reports their results. While Bowtie already generates compliance reports accessible [here](https://bowtie.report), this project introduces a mechanism to track compliance metrics over time. The goal is to enable graphing and querying of how test results evolve as implementations fix bugs or new tests are added. Key outcomes include a new bowtie trend command to aggregate results into trend reports and a dedicated webpage displaying graphs of failed tests over time, providing deeper insights into long-term compliance trends.

### 2. Bowtie-Perf: A Performance Tester for JSON Schema Implementations

* Contributor: [Dhruv Singh](https://github.com/sudo-jarvis)
* Mentors: [Julian Berman](https://github.com/Julian), [Agnivesh Chaubey](https://github.com/AgniveshChaubey)
* [Idea on Github](https://github.com/json-schema-org/community/issues/605)

The project enhances [Bowtie](https://github.com/bowtie-json-schema/bowtie), a meta-validator of the JSON Schema specification. Bowtie provides universal access to JSON Schema implementations, enabling users to validate instances and identify bugs or gaps in functionality. Currently, Bowtie supports validating instances against schemas and comparing implementations based on correctness. This project expands its capabilities by developing a performance tester integrated within Bowtie. This addition provides a critical dimension for performance optimization and enables more comprehensive comparisons across JSON Schema implementations.

### 3. Setting up the CI/CD Pipeline for the JSON Schema Website

* Contributor: [Alok Gupta](https://github.com/aialok)
* Mentor: [Benjamin Granados](https://github.com/benjagm)
* [Idea on Github](https://github.com/json-schema-org/community/issues/603)

The Improved CI/CD Workflow for the [website](https://github.com/json-schema-org/website) project with GitHub Actions project focuses on enhancing the existing CI/CD pipelines by adding essential features like linting, formatting, unit testing, UI testing, broken link checks, and build processes. The project involves streamlining workflows for pull requests, pushes, and issues, with special emphasis on first-time contributors. Key deliverables include creating and optimizing workflows such as PR workflows, push workflows, issue workflows, stale issue and PR reminders, unauthorized file detection, and CodeQL for security analysis. Additionally, the project aims to provide clear documentation on workflow roles and guidelines for updates, ensuring smoother community operation integration and more efficient code management.

### 4. Building a New Version of the JSON Schema Tooling Page

* Contributor: [DV](http://github.com/darhkvoyd)
* Mentor: [Benjamin Granados](https://github.com/benjagm)
* [Idea on Github](https://github.com/json-schema-org/community/issues/602)

The JSON Schema [Tooling Page](https://json-schema.org/tools) plays a vital role in helping developers explore the ecosystem of JSON Schema tools and implementations. However, its current design presents usability challenges that hinder adoption. This project aims to rebuild the tooling page using a data-driven approach while adhering to the updated UI/UX standards of the JSON Schema website. By improving accessibility, simplifying navigation, and enhancing clarity, the revamped tooling page will make JSON Schema implementations more discoverable and user-friendly, ultimately reducing friction and encouraging broader adoption within the developer community.

### 5. JSON Schema Language Server Contributions

* Contributor: [Diya Solanki](https://github.com/diyaayay)
* Mentor: [Jason Desrosiers](https://github.com/jdesrosiers)
* [Idea on Github](https://github.com/json-schema-org/community/issues/601)

The project focuses on enhancing the [Language Server Protocol (LSP)](https://github.com/hyperjump-io/json-schema-language-tools) for JSON Schema, enabling improved support for various editors and IDEs. Unlike the existing VSCode integration, which lacks support for recent JSON Schema versions and struggles with performance for large schemas, this project aims to expand functionality. Key improvements include inline diagnostics for invalid schemas, semantic highlighting for JSON Schema and deprecated keywords, support for multiple drafts (04/06/07/2019-09/2020-12), and enhanced configuration management. The project also focuses on adding code completion for $schema, making JSON Schema development more efficient and accessible across platforms.

### 6. Define Upgrade/Downgrade Language-Agnostic Declarative Transformation Rules for all JSON Schema Dialects

* Contributor: [Suprith KG](https://github.com/suprith-hub)
* Mentor: [Juan Cruz Viotti](https://github.com/jviotti)
* [Idea on Github](https://github.com/json-schema-org/community/issues/599)

The project addresses the challenges of maintaining software compatibility across different schema versions. It focuses on creating declarative, language-agnostic rules to streamline the process of [upgrading or downgrading JSON Schemas](https://github.com/json-schema-org/upgrade-downgrade-rules). This is crucial for users working with high-level languages like Python, managing large documents, or navigating diverse network communication setups such as HTTP, WebSocket, microservices, or IoT. By simplifying schema transformations and introducing standardized procedures, this project aims to save time, ensure reliability, and improve JSON Schema’s versatility across various implementations and use cases.

### 7. Source Generation Analyzer Powered by Corvus.JsonSchema (.NET)

* Contributor: [Pranay Joshi](https://github.com/pranayjoshi)
* Mentors: [Matthew Adams (Endjin)](https://github.com/mwadams), [Greg Dennis](https://github.com/gregsdennis)
* [Idea on Github](https://github.com/json-schema-org/community/issues/614)

The project focuses on creating a source generator that integrates seamlessly with [Corvus.JsonSchema](https://github.com/corvus-dotnet/corvus.jsonschema). This tool will automatically generate code from JSON Schema files within a .NET project at compile time, streamlining the development process. The ultimate goal is to foster collaboration and consistency across software development teams using diverse technology stacks by establishing JSON Schema as a unified source of truth for data modeling. This ensures that generated code adheres to a standardized data structure, improving efficiency and alignment within organizations.

### 8. A Tour of JSON Schema

* Contributor: [Zeel Rajodiya](https://github.com/JeelRajodiya)
* Mentors: [Bence Eros](https://github.com/erosb), [Benjamin Granados](https://github.com/benjagm)
* [Idea on Github](https://github.com/json-schema-org/community/issues/645)

The [Tour of JSON Schema](https://github.com/json-schema-org/tour) project is an interactive learning platform designed to help beginners quickly grasp JSON Schema concepts and best practices. Through step-by-step lessons and hands-on exercises, users can learn by doing, using a simple interface and built-in code editor. The platform provides practical examples, real-time validation of JSON Schema, and immediate feedback, making the learning process engaging and effective. By following structured lessons, users will gain the confidence to create and work with JSON Schemas independently.

## Mentorship: A Two-Way Learning Process

The success of GSoC 2024 for JSON Schema wouldn’t have been possible without our dedicated mentors. They not only guided contributors but also gained fresh perspectives from working closely with emerging talent.

## Watch the Journey

Want to hear more about the contributors’ stories? Watch our [YouTube presentation](https://www.youtube.com/watch?v=2V9k1Hqhgnw) where contributors reflect on their challenges, successes, and learning moments.

## Looking Forward

GSoC 2024 has left an indelible mark on JSON Schema. From automation to tooling, each project has strengthened our ecosystem and laid a foundation for continued innovation.
We are excited about the future and invite developers from around the world to join us in shaping it.

## Ready to contribute?
Explore our [GitHub repository](https://github.com/json-schema-org), get involved in discussions, and be a part of the JSON Schema community!

Background photo by <a href="https://unsplash.com/@jessbaileydesigns?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Jess Bailey</a> 
on <a href="https://unsplash.com/photos/colored-pencil-lined-up-on-top-of-white-surface-l3N9Q27zULw?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Unsplash</a>


Onyedikachi Hope Amaechi-Okorie

Celebrating JSON Schema’s Google Summer of Code 2024 Journey


Since Ben Hutton joined Postman to work full-time on JSON Schema [the past year](https://blog.postman.com/ben-hutton-joins-postman-to-lead-json-schema-strategy/), other top contributors like Jason Desrosiers, Julian Berman, and Greg Dennis have joined, and now all are full-time supporting JSON Schema. So today, I am excited to announce that I am joining as Developer Advocate with a focus on serving the Community, nurturing relationships with other organizations, launching programs to support and grow the Community, and increasing awareness through content, public speaking, and discussions in X, Linkedin, Stackoverflow or Reddit.

## Who is this guy?

My whole career, APIs have been at the core of everything. In the first part of my career, I focused on the engineering aspect of building APIs, playing all the possible team roles. Later, I focused on API Design and Governance, leading API Programs, and most recently, my focus has been Developer Experience and Developer Relations because this human aspect is the most critical for a technology or product to be successful.

Having said that, I see myself as a mix of 1/3 Software Engineering 1/3 API Strategy 1/3 Developer Relations, but I am constantly learning and testing new things, and today I am investing in Product Strategy/Growth. With this diverse background, I hope to be a good complement to the current JSON Schema Team.

## But who is this guy for real?

I am a father of two who loves to spend time with my family and friends. I love mountaineering and team sports, and I think nothing is impossible if you truly believe it. I love this sentence by Nelson Mandela "It always seems impossible until it's done".

## Like to know more about me?

If you like to know more about me, check out [this intro post](https://www.twilio.com/blog/introducing-twilio-developer-evangelist-benjamin-granados) I wrote when I joined Twilio.

## Let's do this!

2023 will be a fantastic year for JSON Schema, and I am excited to be part of this journey with all of you! I can't wait to start connecting with you in the upcoming [working meetings](https://github.com/json-schema-org/community/discussions/35), in [Slack](https://json-schema.org/slack), and [Github](https://github.com/orgs/json-schema-org/discussions).

>* Email: benjamin.granados@postman.com
>* Twitter: [@benjagm](https://www.x.com/benjagm)
>* Linkedin: [benjagranados](https://www.linkedin.com/in/benjagranados/)
>* Github: [benjagm](https://github.com/benjagm)

Cover photo by [Jai Cano](https://www.jaicano.com/redes) on [Instagram](https://www.instagram.com/jaicano).


Hello 2023, Hello JSON Schema Community


Hi! I'm Julian Berman.

If you're a user of JSON Schema, you may know me as the author of [a popular Python implementation of JSON Schema](https://github.com/python-jsonschema/jsonschema). If you're an implementer of the JSON Schema specification, you may instead know me as the primary maintainer of [the cross-language JSON Schema test suite](https://github.com/json-schema-org/JSON-Schema-Test-Suite).

I'd love to share some good news with the JSON Schema community.

In short: I've joined [Postman's](https://www.postman.com/) Open Technology team to work on JSON Schema full-time. In doing so I'm joining a growing existing group of folks from the ecosystem, including amongst them [Ben Hutton](https://www.x.com/relequestual), JSON Schema's specification lead. This brings the number of people working on JSON Schema full-time up to 2 (and counting!).

What does this mean? Primarily, on my part, more mindspace dedicated to JSON Schema, and hopefully thereby some long-needed improvements to the test suite, to my implementation, to new implementations, and beyond. We hope to grow the community even further, and to connect more strongly with it.

Before talking more about that, I'd love to take you through my journey to get here. It's been a fun one.

## At the Start

My connection to JSON Schema started in 2011, more than 10 years ago. I was working on a now long-forgotten web project which needed a validation layer, and stumbled across JSON Schema as something I might use to fill the need. [As is not uncommon in the programming world](https://codewithoutrules.com/2016/09/09/side-projects/), the side project became the main project, and the web application is now long-forgotten. I don't even think I remember what it was meant to do.

My first public commit of the Python `jsonschema` project was in [December of that year](https://github.com/python-jsonschema/jsonschema/commit/2c25dc77051fbdb10b7e5552f317edb72bb6bcc4).

At the time, I decided to write a Python implementation not because one didn't already exist – on the contrary, at the time in 2011, there already were one or two out there. My implementation was instead almost born out of coincidence. I was growing as a developer at the time, and starting to form opinions about what I thought good code looked like (occasionally overly strong ones). When I found some specification inconsistencies in the pre-existing implementations, it gave me an itch to write something I could "own" and learn from as a maintainer.

It wasn't long before my implementation was used by other projects, and those other projects were reaching out with issues, requests, thanks or feedback.
Hearing others appreciated my work was definitely good motivation to continue to invest effort into it.

## The Test Suite

Not long after writing my little Python library, I started what would become the [official, language-agnostic test suite for implementers of JSON Schema](https://github.com/json-schema-org/JSON-Schema-Test-Suite). For awhile I churned away on the suite, hoping it would make it easier for others to do what I'd done – to write a spec-conformant implementation of JSON Schema.

At the time, the implementations I found were each replicating their own incomplete, internal test suites. It seemed eminently beneficial to centralize the effort, giving someone authoring a library a way of running a "known" set of tests which would tell them whether their libraries were working as expected and specified.

The test suite was originally not "official" by any means. I worked on it from my own account with no real connections to the work happening on the specification itself. Graciously however, a member of the specification team at the time reached out and proactively asked me to bring it within the JSON Schema organization, while continuing to maintain it. While I wasn't surprised in the interest in such a thing, I certainly was proud of having produced something useful.

I've never been the one to drive new functionality in JSON Schema. My role instead has been more of an interpreter, taking what has been written and using the test suite to shine a light on corner cases. From early on, I was firm that the test suite wasn't a vehicle for enacting change to the specification, it was simply a representation of what the specifications already prescribed. Doing so was critical for its success (success both of the suite and the specification itself). Specifications are never perfect, and still today the JSON Schema specification continues to be improved and iterated on; nonetheless the test suite has given implementers a shared understanding of what the specification says (allowing discussions for how to improve it to carry on elsewhere).

## A Bit More Reflection & Thanks

I owe a decent amount of debt to JSON Schema as a whole. It's been with me for a very long time in my programming career thus far, and I've learned a lot through needing to maintain bits and pieces of its orbit. Maintaining a widely used library is not easy at all at times, and JSON Schema gave me a way to exercise these muscles. At times, it has moved up or down on my (long) list of interests, but never completely off it. For over 10 years I've had it somewhere in my conscious.

It's led me to serendipitous encounters sitting halfway around the world with developers who unknown to both of us had just dealt with an outage I'd caused to their internal systems by doing a release (n.b. pin your dependencies folks!). Luckily we both could crack a smile at the coincidence by that point.

I've enjoyed waking up to unexpected emails from respectable open source projects' maintainers telling me they use it, or thanking me for it when we bump into each other. I've come across important looking documents with government seals on them giving guidance on how to use the library I wrote initially as an afterthought, and couldn't help but be amused.

## What's Next

And so, proudly, it's time for JSON Schema to play a bigger part in my day-to-day. I join Postman with an expected bit of tentativeness – joining a for-profit company, one of course worries about any undue influences on direction of an open source project. I can say happily that Postman have been nothing but gracious supporters so far – of JSON Schema, of open source more broadly, and certainly of myself. The tentativeness I start with I expect to disappear quite quickly and be replaced with gratitude. There are many many knowledgeable people to both learn from and interact with here, something I think will prove to be invaluable.

As for work on JSON Schema, there are many things I've wanted to do these past few years, particularly with the test suite, that there was just never enough time to dedicate to. Hardening the suite has even more potential to help expand JSON Schema's usage than we've tapped so far. There are things about maintainership which embarrass me somewhat, as any project occasionally does to its maintainers. The glacial pull request queue is definitely up there on that list; if you're a contributor, you have my deep apologies if a pull request of yours has sat stagnant. It will be one of the things I definitely aim to fix quickly. There are of course many things I am quite proud of as well, and hope to continue doing.

Through the years I've occasionally spoken about JSON Schema at conferences or meetups, or occasionally poked my head beyond the borders of my implementation or the test suite. I intend to do more of both, and be better at it all.

And more broadly than myself, we have even bigger dreams. Stay tuned for more details there, both here on the blog as well as on the issue trackers and discussion boards throughout the organization.

Please help! Your feedback, as a user, as an implementer of the specification, or as an observer, are all likely invaluable to making sure JSON Schema moves forward. Please do reach out – to Ben, to myself, to other wonderful folks who form JSON Schema's contributors, and/or via the [JSON Schema Slack](https://json-schema.org/slack). I know we'd all love to hear feedback, things you'd like to see, constructive complaints, or general thoughts on how we can grow the community.

In particular, if you're an open source maintainer who depends on my library, or on the test suite, please introduce yourself if we don't already know each other! I'd love to hear about what either can do better for you.

Thanks for the ride so far!

Cover photo by <a href="https://unsplash.com/@jimkalligas?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Jim Kalligas</a> on <a href="https://unsplash.com/?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a>.


Hello World, Hello Postman

Yanick Champoux

Astonishing Serializations & Schemas of Hyperborea


I've received a lot of questions (and purported bugs) and had quite a few discussions over the past few years regarding JSON Schema output, and by far the most common is, "Why does my passing validation contain errors?"

Let's dig in.

_The last time we [talked about output](fixing-json-schema-output), it was to announce changes from the 2019-09/2020-12 version.  I'm going to use the new formats because it's easier to read and more compact._

## No Problem

Before we get into where the output may be confusing, let's have a review of a happy path, where either

- all of the child nodes are valid, so the overall validation is valid, or
- one or more of the child nodes is invalid, so the overall validation is invalid.

These cases are pretty easy to understand, so it serves as a good place to start.

```json
{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://json-schema.org/blog/interpreting-output/example1",
  "type": "object",
  "properties": {
    "foo": { "type": "boolean" },
    "bar": { "type": "integer" }
  },
  "required": [ "foo" ]
}
```

This is a pretty basic schema, where this is a passing instance:

```json
{ "foo": true, "bar": 1 }
```

with the output:

```json
{
  "valid": true,
  "evaluationPath": "",
  "schemaLocation": "https://json-schema.org/blog/interpreting-output/example1#",
  "instanceLocation": "",
  "annotations": {
    "properties": [
      "foo",
      "bar"
    ]
  },
  "details": [
    {
      "valid": true,
      "evaluationPath": "/properties/foo",
      "schemaLocation": "https://json-schema.org/blog/interpreting-output/example1#/properties/foo",
      "instanceLocation": "/foo"
    },
    {
      "valid": true,
      "evaluationPath": "/properties/bar",
      "schemaLocation": "https://json-schema.org/blog/interpreting-output/example1#/properties/bar",
      "instanceLocation": "/bar"
    }
  ]
}
```

All of the subschema output nodes in `/details` are valid, and the root is valid, and everyone's happy.

Similarly, this is a failing instance (because `bar` is a string):

```json
{ "foo": true, "bar": "value" }
```

with the output:

```json
{
  "valid": false,
  "evaluationPath": "",
  "schemaLocation": "https://json-schema.org/blog/interpreting-output/example1#",
  "instanceLocation": "",
  "details": [
    {
      "valid": true,
      "evaluationPath": "/properties/foo",
      "schemaLocation": "https://json-schema.org/blog/interpreting-output/example1#/properties/foo",
      "instanceLocation": "/foo"
    },
    {
      "valid": false,
      "evaluationPath": "/properties/bar",
      "schemaLocation": "https://json-schema.org/blog/interpreting-output/example1#/properties/bar",
      "instanceLocation": "/bar",
      "errors": {
        "type": "Value is \"string\" but should be \"integer\""
      }
    }
  ]
}
```

The subschema output at `/details/1` is invalid, and the root is invalid, and while we may be a bit less happy because it failed, we at least understand why.

So is that always the case?  Can a subschema that passes validation have failed subschemas?  Absolutely!

## More Complexity

There are limitless ways that we can create a schema and an instance that pass it while outputting a failed node.  Pretty much all of them have to do with keywords that present multiple options (`anyOf` or `oneOf`) or conditionals (`if`, `then`, and `else`).  These cases, specifically, have subschemas that are _designed_ to fail while still producing a successful validation outcome.

For this post, I'm going to focus on the conditional schema below, but the same ideas pertain to schemas that contain "multiple option" keywords.

```json
{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://json-schema.org/blog/interpreting-output/example2",
  "type": "object",
  "properties": {
    "foo": { "type": "boolean" }
  },
  "required": ["foo"],
  "if": {
    "properties": {
      "foo": { "const": true }
    }
  },
  "then": { "required": ["bar"] },
  "else": { "required": ["baz"] }
}
```

This schema says that if `foo` is true, we also need a `bar` property, otherwise we need a `baz` property.  Thus, both of the following are valid:

```json
{ "foo": true, "bar": 1 }
```

```json
{ "foo": false, "baz": 1 }
```

When we look at the validation output for the first instance, we get output that resembles the happy path from the previous section: all of the output nodes have `valid: true`, and everything makes sense.

However, looking at the validation output for the second instance (below), we notice that the output node for the `/if` subschema has `valid: false`.  But the overall validation passed.

```json
{
  "valid": true,
  "evaluationPath": "",
  "schemaLocation": "https://json-schema.org/blog/interpreting-output/example2#",
  "instanceLocation": "",
  "annotations": {
    "properties": [
      "foo"
    ]
  },
  "details": [
    {
      "valid": true,
      "evaluationPath": "/properties/foo",
      "schemaLocation": "https://json-schema.org/blog/interpreting-output/example2#/properties/foo",
      "instanceLocation": "/foo"
    },
    {
      "valid": false,
      "evaluationPath": "/if",
      "schemaLocation": "https://json-schema.org/blog/interpreting-output/example2#/if",
      "instanceLocation": "",
      "details": [
        {
          "valid": false,
          "evaluationPath": "/if/properties/foo",
          "schemaLocation": "https://json-schema.org/blog/interpreting-output/example2#/if/properties/foo",
          "instanceLocation": "/foo",
          "errors": {
            "const": "Expected \"true\""
          }
        }
      ]
    },
    {
      "valid": true,
      "evaluationPath": "/else",
      "schemaLocation": "https://json-schema.org/blog/interpreting-output/example2#/else",
      "instanceLocation": ""
    }
  ]
}
```

How can this be?

## Output Includes Why

Often more important than the simple result that an instance passed validation is _why_ it passed validation, especially if it's not the expected outcome.  In order to support this, it's necessary to include all relevant output nodes.

If we exclude the failed output nodes from the result,

```json
{
  "valid": true,
  "evaluationPath": "",
  "schemaLocation": "https://json-schema.org/blog/interpreting-output/example2#",
  "instanceLocation": "",
  "annotations": {
    "properties": [
      "foo"
    ]
  },
  "details": [
    {
      "valid": true,
      "evaluationPath": "/properties/foo",
      "schemaLocation": "https://json-schema.org/blog/interpreting-output/example2#/properties/foo",
      "instanceLocation": "/foo"
    },
    {
      "valid": true,
      "evaluationPath": "/else",
      "schemaLocation": "https://json-schema.org/blog/interpreting-output/example2#/else",
      "instanceLocation": ""
    }
  ]
}
```

we see that the `/else` subschema was evaluated, from which we can infer that the `/if` subschema MUST have failed.  However, we have no information as to _why_ it failed because that subschema's output was omitted.  But looking back at the full output, it's clear that the `/if` subschema failed because it expected `foo` to be true.

For this reason, the output must retain the nodes for all evaluated subschemas.

It's also important to note that the [specification](https://json-schema.org/draft/2020-12/json-schema-core#name-if) states that the `if` keyword doesn't directly affect the overall validation result.

## A Note About Format

Before we finish up, there is one other aspect of reading output that can be important: format.  All of the above examples use the _Hierarchical_ format (formerly _Verbose_).  However, depending on your needs and preferences, you may want to use the _List_ format (formerly _Basic_).

Here's the output from the simple schema in _List_ format:

```json
{
  "valid": false,
  "details": [
    {
      "valid": false,
      "evaluationPath": "",
      "schemaLocation": "https://json-schema.org/blog/interpreting-output/example1#",
      "instanceLocation": ""
    },
    {
      "valid": true,
      "evaluationPath": "/properties/foo",
      "schemaLocation": "https://json-schema.org/blog/interpreting-output/example1#/properties/foo",
      "instanceLocation": "/foo"
    },
    {
      "valid": false,
      "evaluationPath": "/properties/bar",
      "schemaLocation": "https://json-schema.org/blog/interpreting-output/example1#/properties/bar",
      "instanceLocation": "/bar",
      "errors": {
        "type": "Value is \"string\" but should be \"integer\""
      }
    }
  ]
}
```

This is easy to read and process because all of the output nodes are on a single level.  To find errors, you just need to scan the nodes in `/details` for any that contain errors.

Here's the output from the conditional schema in _List_ format:

```json
{
  "valid": true,
  "details": [
    {
      "valid": true,
      "evaluationPath": "",
      "schemaLocation": "https://json-schema.org/blog/interpreting-output/example2#",
      "instanceLocation": "",
      "annotations": {
        "properties": [
          "foo"
        ]
      }
    },
    {
      "valid": true,
      "evaluationPath": "/properties/foo",
      "schemaLocation": "https://json-schema.org/blog/interpreting-output/example2#/properties/foo",
      "instanceLocation": "/foo"
    },
    {
      "valid": false,
      "evaluationPath": "/if",
      "schemaLocation": "https://json-schema.org/blog/interpreting-output/example2#/if",
      "instanceLocation": ""
    },
    {
      "valid": true,
      "evaluationPath": "/else",
      "schemaLocation": "https://json-schema.org/blog/interpreting-output/example2#/else",
      "instanceLocation": ""
    },
    {
      "valid": false,
      "evaluationPath": "/if/properties/foo",
      "schemaLocation": "https://json-schema.org/blog/interpreting-output/example2#/if/properties/foo",
      "instanceLocation": "/foo",
      "errors": {
        "const": "Expected \"true\""
      }
    }
  ]
}
```

Here, it becomes obvious that we can't just scan for errors because we have to consider where those errors are coming from.  The error in the last output node only pertains to the `/if` subschema, which (as mentioned before) doesn't affect the validation result.

## Wrap-up

JSON Schema output gives you all of the information that you need in order to know what the validation result is and how an evaluator came to that result.  Knowing how to read it, though, takes understanding of why all the pieces are there.

If you have any questions, feel free to ask on our Slack workspace (link in the footer) or [open a discussion](https://github.com/orgs/json-schema-org/discussions).

_All output was generated using my online evaluator <https://json-everything.net/json-schema>._

_Cover image by [Tim Gouw](https://unsplash.com/@punttim) on [Unsplash](https://unsplash.com/photos/man-wearing-white-top-using-macbook-1K9T5YiZ2WU)_


Interpreting JSON Schema Output


In the last couple of weeks, you may have seen that Postman has been investing
heavily in JSON Schema's future by hiring some of its top contributors giving
them the opportunity to work full time on improving the JSON Schema
specification and tooling ecosystem. I'm happy to announce that I'm the next
addition to that team.

JSON Schema has always been a small group of volunteers working in their free
time with no financial support for their work. Thanks to Postman, we now have a
team of four that has the financial support and creative freedom to focus on the
projects and goals we never had time for in the past. I'm excited to see what
this team can achieve in the next couple years.

When Postman hired Ben Hutton last year to work full time on JSON Schema, there
was some concern within our community that Postman might try to own or influence
the evolution of JSON Schema in someway the way we often see with certain big
companies having undue influence on web standards today. Fortunately, this
hasn't been the case with Postman and none of us who have joined in the past
month would have joined if it were. Postman is just making it possible for us to
put more time and effort into doing what we've been doing for years.

## Background

I discovered JSON Schema around 2013 when draft-04 was new. I was designing APIs
and I was doing a deep dive on REST and hypermedia. I discovered JSON
Hyper-Schema and it's unique approach to hypermedia allowed me to introduce
hypermedia to APIs in an unobtrusive way and express relationships that weren't
possible with other approaches. But, the biggest win was that using a JSON
Hyper-Schema implementation called Jsonary along with some basic hand rolled
tooling, I could design APIs in a interactive way. I could try out workflows by
clicking links and filling out forms before we invested in building anything
out. It was API first design before it had a name.

Around 2015-ish, I started contributing to specification discussions and
supporting the community through StackOverflow where I'm the all time top
answerer of JSON Schema questions. For a while I was too busy with other
obligations necessary for paying bills to be as involved as much as I wanted to
be. But, a couple years ago I managed to transition to doing consulting part
time with the goal of doing work to pay the bills when necessary and spending as
much time as I could afford on things that bring me joy like working on JSON
Schema.

During that time I decided to work on a JSON Hyper-Schema implementation. I was
hoping to make use of an existing validation library and build a JSON
Hyper-Schema client on top of it, but nothing existed that was capable of that.
I ended up writing my own validation implementation which became one of the
first 2019-09 implementations. The goal of this implementation was to be a set
of tools to create JSON Schema related libraries. The validation implementation
was just a proof of concept of something you can create with these tools.
Eventually, I'll get back to actually implementing that JSON Hyper-Schema
client.

## Goals

Now that I can focus full time on JSON Schema, I have many goals. The following
are my top three objectives at the moment, but who knows how the winds will
change and priorities will shift over time.

My first priority is to get the `application/schema+json` media type registered
with IANA. It's been used for years in many applications and it's about time it
was officially recognized and defined. This is a unique challenge because of how
much JSON Schema has evolved over the years and will evolve in the years to
come. Whatever we end up with needs to be inclusive of past and future JSON
Schema.

Another goal is to complete the updates to the Understanding JSON Schema
documentation. Because the JSON Schema community was unfunded and limited to
contributing in our free time, things like documentation usually didn't get the
attention it deserved. Last year I began an effort to update the documentation
to cover the most recent two (!) drafts that had gone undocumented. I got most
of the way through it, but there are a few things like dynamic references
and the vocabulary system that I didn't get to before I had to take a consulting
gig and make some money.

As I said before, I'm a huge fan of JSON Hyper-Schema. Due to not having
sufficient resources and lack of adoption, we've paused work on the JSON
Hyper-Schema specification. Now that I'm able to work on JSON Schema full time,
I want to bring JSON Hyper-Schema (or something like it) back. This is a big job
that above all needs tooling developed so people can see how powerful it can be.

## Personal

Since this is something of an introduction post, I guess I should include some
personal trivia as well. I live in San Luis Obispo, California. When I'm not at
a keyboard you'll probably find me hiking and/or camping deep in the mountains,
rock climbing, or coaching gymnastics. The cover image for this post is me doing
a handstand at the summit of Mt Baldy in Southern California.


Joining Postman


Data consistency and interoperability are critical for success in today's interconnected reality. While JSON is undoubtedly the most popular format for exchanging data, JSON Schema is essential for ensuring data consistency, validity, and interoperability at scale. Despite the widespread adoption of JSON Schema, there has yet to be a comprehensive list of adopters where individuals and organizations can seek inspiration and real-world use cases. This is why we are thrilled to announce the launch of the Adopters list.

## What is the Adopters list?

The [Adopters list](https://github.com/json-schema-org/community/blob/main/ADOPTERS.md) is a directory that features organizations that have successfully implemented JSON Schema. It's your go-to place to discover how industry leaders use JSON Schema.

When you explore the Adopters list, you'll uncover how organizations use JSON Schema in Production. It's a valuable resource for inspiration and learning. Whether you are seeking insights to enhance your projects or to remain informed about the latest trends in the JSON Schema Ecosystem, the Adopters list provides a window into the best practices within the community.

## We need you!

Adding your organization to the JSON Schema Adopters list demonstrates your commitment to supporting the JSON Schema Ecosystem and encouraging others to do the same. Additionally, adding your organization to this list provides valuable feedback to the JSON Schema team, helping them to improve the project's adoption and impact.

By joining the list, you play an active role in the growth and sustainability of the JSON Schema Ecosystem.
It's a small contribution to the project with a big impact to encourage and grow JSON Schema adoption.

**This is the Open Source way:** Learn from others and give back to the community! ❤️

## Ready to join?

You can add your Company using this [Adopter form](https://forms.gle/HNZs4vt49GWPPGQq5) or following [these simple steps](https://github.com/json-schema-org/community/blob/main/ADOPTERS.md#how-to-add-your-company).

<div className='flex flex-wrap justify-center items-center gap-4 w-full'>
    <a href='https://forms.gle/vyFskw1RshJ55LY46'>
      <img className='max-w-400 px-20' src='/img/posts/2023/json-schema-adopters/join-button.png'/>
    </a>
</div>

## Join the JSON Schema Community

If your organization uses JSON Schema, please consider [contributing](https://github.com/json-schema-org/.github/blob/main/profile/README.md) in other ways. You can report issues, suggest improvements, improve our documentation, [become a sponsor](https://opencollective.com/json-schema), or even make code contributions.

**Join us and shape the future of JSON Schema!**

Photo by [Patrick Tomasso](https://unsplash.com/@impatrickt?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash) on [Unsplash](https://unsplash.com/photos/lighted-vintage-light-bulbs-1NTFSnV-KLs?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash)


Introducing the JSON Schema Adopters


## Ambassadors program?

You may have noticed our new Ambassadors Program! If not, don't worry—we're here to explain what it is and how you can get involved.

I'm personally very excited to see this program come to life. Recognizing our community champions in a meaningful and sustainable way has always been a goal. The Ambassadors Program is our first step in publicly supporting our most experienced and helpful members.

We have many community members who frequently help others, provide feedback, answer questions, and debug schemas and code. They also assist newcomers in avoiding basic misconceptions. JSON Schema wouldn't be where it is today without this support.

## The time is right - we are growing

For the past few years, a small team has worked full time on JSON Schema, focusing on ongoing projects. Now, we have the capacity to launch new initiatives, including this Ambassadors Program. We want to better recognize and support those who are giving back to our community.

We are better placed today than we ever have been to support and empower our existing ambassadors, and build a program which recognizes and supports them to further their efforts. It's always encouraging to see others as excited about JSON Schema as you are.

<Regularquote>
  _"The new website, and assorted outreach programmes have started to connect the huge JSON Schema user base, and we've really started to see the benefit in terms of the number of people actively participating." - **Matthew Adams** - Founder at [Endjin](https://endjin.com) - Candidate Ambassador._
</Regularquote>

## How we benefit, from today

When thinking about our programs, two aspects we should consider are how they will impact the project overall, and the community. These are not the only aspects, but ones I want to touch on in this context.

### As a Project

Having ambassadors not only recognizes individual contributions but also benefits JSON Schema. It will increase visibility and encourage positive community engagement.

Despite JSON Schema's presence, many still see it as part of another standard. By promoting JSON Schema through various channels, we enhance its recognition and usage. This increased visibility can attract more sponsorships, funding further programs like the Ambassadors Program and fostering a better ecosystem. It's a positive cycle; once we start, momentum will help us grow.

As the project thrives, so too does our community. The Ambassadors Program brings several benefits specifically for our community.

<Regularquote>
  _"The community is very friendly, welcoming, and extremely smart. I already learnt a LOT by participating in it. I want the JSON Schema ecosystem to grow, as it will likely power my business as a result. The Ambassadors Program is a great opportunity to recognise key contributors and encourage them to continue being involved." - **Juan Cruz Viotti** - Founder and CTO at [Intelligence.AI](https://intelligence.ai) - Candidate Ambassador_
</Regularquote>

### As the Community

The project and the community both benefit from the Ambassadors Program. Ambassadors will contribute valuable content, enhancing learning and engagement. To keep their ambassador status each year, ambassadors will be expected to make a number of contributions that benefit the community, such as articles, videos, podcasts, or even talking or running a workshop at a conference.

Given the prominence of the ambassadors, we will, collectively, check any and all published work they produce, making sure it's accurate. If you work with code, you've probably seen your fair share of poorly documented projects, and you know the impact that can have. High-quality, up-to-date content makes it easier for both newcomers and veterans to learn and stay current.

Further, we don't require or even expect that all the content ambassadors produce will go onto the project's website. No. There's actually improved reach by ambassadors using their existing channels to reach their community, hopefully expanding the JSON Schema community in addition to educating other spaces.

## A portrait of an Ambassador

We've long planned the Ambassadors Program and have identified initial candidates. (You can nominate yourself or others!) Here are the qualities we seek in ambassadors.

Firstly, an ambassador has to be with the community. This means in the slack server, on Github, and occasionally on calls or other places representing or being a recognised expert or source of knowledge for JSON Schema.

Critically, we expect ambassadors to be aligned with the JSON Schema project's mission and values, be respectful of our code of conduct, and continue to be active in the community.

## Recognising our Ambassadors from the community

We easily identified several community members to invite into the Ambassadors Program. The TSC and the Community Team spend a LOT of time in our community spaces, so even if we don't always reply or comment, we do notice those who are active and helpful.

When I first started the JSON Schema Slack server, the numbers were very few. Gradually, it grew. The core team answered most of the questions. I recall the feeling when I came back in the morning to find other people had answered questions from the community while I'd been asleep. I've previously [recognised](https://words.benhutton.me/2023-05-23-two-words-big-win) that saying "thank you" can be a big win, but I also know that we can do several magnitudes better.

We already have several members of our community who have become ambassadors for JSON Schema, just without any sort of formal recognition. Recognising, supporting, and further enabling these individuals, and new ambassadors, is going to have a positive impact, and I can't wait to see what it unlocks for JSON Schema in the future.

There's a swag kit for all accepted into the Ambassadors Program, and I can't wait to meet the individuals excited and proud to be wearing the exclusive swag at events and conferences in the future. We see you, ambassadors!

## How to apply today

OK, OK. If you've read all the above, maybe you haven't yet made it to the [Ambassador Program documentation](https://json-schema.org/ambassadors) file in our community repo. That would be your next stop if you're thinking about applying to become an ambassador.

We will be looking for existing contributions from individuals who have already been in our community for a while. If you're brand new to JSON Schema, becoming an ambassador is probably not something immediately in your reach, and that's OK. If you have some experience with JSON Schema, go write about it, apply to talk at events and conferences, or make videos. Be part of our community, answer questions. Then your application is much more likely to be accepted.

As I mentioned, we have a number of people in mind already, and I'd love to introduce a few of them to you today. If you're in our community spaces, you've probably seen them around.

"As a small team ourselves at Endjin, we live by the mantra 'helping small teams achieve big things'. I believe we can have a disproportionate positive impact by working on JSON Schema as it is so widely adopted, across so many different disciplines and industry verticals." - Matthew Adams

"JSON is the world's most popular data format, and it's impossible to not work with it in some way or another. Once you want to embrace JSON, JSON Schema is the most powerful and expressive schema language for it, elegantly enabling a wide variety of use cases." - Juan Cruz Viotti

## You (yes, you!) can help support and expand Ambassadors and other programs

If this program sounds great and you want to support it, you can! Being part of our community is a great first step. Our Slack server may sometimes get busy, but you could mute it and only get notified of announcements, if you wanted. We actively use and monitor GitHub Discussions, so if Slack isn't your thing or you have one too many already, that's going to be a space for you. We're also more active on LinkedIn over this past year, and it seems to be increasing in popularity with professionals; We love to hear how JSON Schema is used in industry.

Sharing content from the JSON Schema project and community promotes growth and enhances your professional credibility. With ambassadors contributing more content, they may even seek ideas from you!

We are sending out swag at our expense, and we have a little money saved in our Open Collective budgeted and set aside for doing so. However, we would love to be able to support ambassadors to cover the expenses involved in getting to the larger conferences, including our own in the future. If your organization supports open source projects, please consider financial donations, especially if you use JSON Schema in production. We also support GitHub Sponsors.

If that's easy, great, but we do appreciate it's not always attainable. What might be more possible is to get your organization added to our Adopters list. You can read all about that in our community repository.

I'm thrilled to see our Ambassadors Program get off the ground, and I'm positive it will have a really impactful effect on the project, the community, and the ambassadors that take part. Join us, get involved, and let's see what we can achieve together!

Photo by <a href="https://unsplash.com/@goian?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Ian Schneider</a> on <a href="https://unsplash.com/photos/two-person-standing-on-gray-tile-paving-TamMbr4okv4?utm_content=creditCopyText&utm_medium=referral&utm_source=unsplash">Unsplash</a>


JSON Schema Ambassadors Are Here!


If you have never heard of JSON Schema till now, then you are in the right place to learn more.
If you are confident in using JSON Schema, this article still holds some value for you, and you'll likely still learn something new.

By the end of this article, you will:

- Understand what JSON Schema is
- Know what JSON Schema can do
- Understand some key terminology for talking about JSON Schema
- Know some of the basics of how JSON Schema works
- Outshine even those with years of JSON Schema experience (hopefully)

Although this is a technical introduction, we won't cover writing any code today. But we will link you to a list of implementations in almost any programming language.

## JSON What?
Primarily, JSON Schema is used to validate data.
The sorts of data JSON Schema can validate is data encoded in JSON.

JSON stands for JavaScript Object Notation and is a subset of Javascript.
JSON is both human and machine-readable, making it a popular format choice for data interchange.

JSON Schema is both the name given to the project and the artefact (A JSON Schema) that defines the required validation. The validation rules are expressed by defining constraints against the JSON data.

The JSON data you are validating is called a "JSON instance".

## JSON Schema
JSON Schema the project is located [on GitHub](https://github.com/json-schema-org) under the organization name "json-schema-org". It is comprised of several git repositories used to track changes and enable collaboration on the specification and supporting resources.

The JSON Schema specification is made up of technical specification documents currently published canonically through the Internet Engineering Task Force (IETF) as a series of "personal drafts". A "personal draft" is the category given to documents published on the IETF which are from an IETF working group.

A JSON Schema document is written in JSON itself. Being written in JSON means that it can be read by almost every programming language, making it an interoperable validation solution. It can also be written using other formats which translate to JSON, or using code that can serialize to JSON.

## A JSON Schema (document)
We mentioned JSON Schema is itself JSON. It used to be that JSON had to be an Object or an Array at the root level, but today, it can also be any of the allowed values.

A JSON Schema may be an Object or a Boolean. The Boolean form results in the same value as the validation assertion result. The Object form allows for keywords and values to express constraints for validation.

JSON Schema is constraints based, meaning that anything not defined (or constrained) is allowed and considered valid.

It may seem odd to allow Boolean values for a Schema, but we will explore why that's useful later. For now, let's look at our very first example Schemas.

## Introduction to constraints

```json
// props { "caption": "An empty Object"}
{}
```

```json
// props { "caption": "A \"true\" Boolean value"}
true
```

```json
// props { "caption": "A \"false\" Boolean value"}
false
```

```json
// props { "caption": "Not, empty Object"}
{ "not": { } }
```

Let's take each Schema in turn and explain the validation result.

The empty Object Schema expresses no constraints. Any instance data will always be considered valid.
The "true" Boolean Schema will always assert that the instance data is valid.

As you may have guessed, the inverse is true for the remaining two Schemas. The "false" Boolean Schema will always assert false, and the "Not, empty Object" Schema will have the same result.

The `not` keyword takes a JSON Schema as its value. These "schema values" are called subschemas. The subschema is applied to the instance data, and the resulting assertion is inverted. As we now know, the empty Object schema results in a "true" assertion, so the inverse is "false".

## The Object Schema
The keys in an Object Schema are the JSON Schema keywords. These keywords have four classes: Identifiers, Applicators, Assertions, and Annotations. Some keywords don't fit into these classes, as they have unique functions.

The `$schema` keyword has a unique function. The value of this keyword is a URI that identifies the JSON Schema Dialect to use when processing the schema.

You can think of a JSON Schema Dialect as a set of keywords with a defined meaning. In most cases, the different dialects you will deal with are merely different versions (or "drafts") of JSON Schema.

It is best practice to define the JSON Schema Dialect in use using the `$schema` keyword. In doing so, implementations should throw an error if they find a version of JSON Schema that they do not support.

If your Schema does not define the Dialect of JSON Schema it is using, implementations may pick any dialect. This may result in false positives when used with an implementation that does not support that version.

In this introduction, we will use the 2020-12 version of JSON Schema, however, most of what you will learn applies to almost all of the versions used in production today.

The 2020-12 JSON Schema Dialect URI is `https://json-schema.org/draft/2020-12/schema`.

## Assertions
Remembering that JSON Schema provides constraints based validation, JSON Schema defines keywords that, when applied to the instance, provide an assertion result (pass or fail).

### The "const" keyword

The simplest assertion keyword is `const`. The value can be any valid JSON. To pass validation data must be the same as the value of `const`. Here's an example:

```json
// props { "isSchema": true}
{ "const": 1234 }
```

```json
// props { "valid": true, "caption": "valid - It is exactly the same" }
1234
```

```json
// props { "valid": false, "caption": "invalid - It is not the same" }
"foobar"
```

### The "type" keyword

Another assertion keyword is `type`. Its value is an array of strings that are the allowed types. The types are the six primitive types defined by JSON, plus "integer". Let's look at an example.

```json
// props { "isSchema": true}
{ "type": ["object", "boolean", "null"] }
```

```json
// props { "valid": true, "caption": "valid - An Object is allowed." }
{ "ok": "yes" }
```
```json
// props { "valid": true, "caption": "valid - A Boolean is allowed." }
true
```
```json
// props { "valid": true, "caption": "valid - Null is allowed." }
null
```

```json
// props { "valid": false, "caption": "invalid - A Number is not allowed." }
123
```
```json
// props { "valid": false, "caption": "invalid - A string is not allowed." }
"foobar"
```

These two assertion keywords are applicable to ANY type of data in JSON, but some assertion keywords are only applicable to specific types. If a keyword is NOT applicable to the instance data, it is ignored and performs no validation.

### The "required" keyword

`required` is an assertion keyword that is only applicable for Objects. Let's explore the implications of this with some examples.

`{ "required": ["name"] }` - Our JSON Schema
`{ "name": "Bob" }` - valid - "name" is in the Object.
`{ "fullName": "Bob" }` - invalid - "name" is missing from the Object.
`"Bob"` - valid - The `required` keyword only applies to an Object.
`true` - valid - Same as above.

This may be surprising, but it allows for the composition of constraints that only logically otherwise make sense given a specific type of data.

You should combine using `type` with other keywords in the same Schema to avoid type expectation mismatch.

## Applicators

Validation begins by applying the root Schema to the complete instance document. Applicator keywords apply subschemas to the instance location.

### The "properties" keyword
The most commonly used applicator keyword is `properties`, which has an object value, where the values are subschemas. Let's see it in action.

```json
// props { "caption": "Our JSON Schema", "isSchema": true }
{
  "properties": {
    "name": {
      "type": ["string"]
    }
  }
}
```

```json
// props { "caption": "instance has name, which is a string", "valid": true }
{
  "name": "Alice"
}
```

```json
// props { "caption": "instance object has no `name` property", "valid": true }
{
  "fullName": "Alice"
}
```

```json
// props { "caption": "instance is not an object, therefore `properties` isn't applicable", "valid": true }
[ "name", 123 ]
```


The subschemas are only applied to the instance location where the key matches a key in the instance locations object.

If you want to make sure the instance data is an Object, use the `type` keyword.

If you want to make sure that the Object has the "name" property, use the `required` keyword.

## Recap
The most common reason a JSON Schema appears to not be working is the lack of the fully required constraints.

JSON Schema is "constraints based", and keywords may only be applicable to the correct data types.
Using multiple keywords to tighten constraints allows you to make sure type-specific keyword mismatch is handled by a different keyword.

Identifying subschemas, and knowing that Schemas can always be Boolean values allows you to confirm assumptions about correct applicability. Subschemas can always be tested on their own if required.

## Next steps / Further Reading
I hope you've found this very brief introduction to the fundamentals of JSON Schema beneficial in how you read, reason, and develop JSON Schemas.

My recommendation for writing JSON Schemas today is to use VSCode. When a file is in JSON language mode, and a `$schema` value is added to a root object, VSCode provides auto-complete and IntelliSense  (Although this is limited to draft-07).

Here are some places you may find helpful moving forward:

- [Up to date getting started guide](/learn/getting-started-step-by-step)
- [Weekly Office Hours (Tuesday, 15:00 UTC)](https://github.com/json-schema-org/community/discussions/34)
- [Open Community Slack Server](/slack)
- [Community GitHub Discussions](https://github.com/json-schema-org/community/discussions)
- [List of implementations](/tools)
- [Human-friendly documentation for understanding JSON Schema](/understanding-json-schema)
- [JSON Schema live playground (ajv) for draft-07 with shareable links](https://jsonschema.dev)
- [JSON Schema live playground (hyperjump validator) for all drafts (from draft-04)](https://json-schema.hyperjump.io)
- [Links to RFC specification documents](/specification)

Photo by <a href="https://unsplash.com/@saadx?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Saad Salim</a> on <a href="https://unsplash.com/s/photos/construction?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a>


JSON Schema in 5 minutes


Joining Postman to work full time on JSON Schema has been a great move for me personally, and I feel for the JSON Schema project. Working full time on JSON Schema has been something I never expected, but I'm grateful and honoured to be able to do.

It's been almost a year, and one of the first meetings I organized was to chat with the OpenJS Foundation to learn more and discuss the potential for JSON Schema to join the foundation. That's why it gives me great pleasure to announce that JSON Schema has joined the OpenJS Foundation!

I'll give a few other small updates, but joining the OpenJS Foundation is our headline.

# Joining the OpenJS Foundation

The application to join could have been put in a year ago, but that wouldn't have been in line with the way we operate, and we wanted to do it right. JSON Schema works as a loose collective of individuals looking to form general consensus, and act accordingly.

There had long been a question over where JSON Schema would call home in terms of belonging to an organization. Would we formally join the IETF or the W3C, or maybe even some other open source / open standards organization? While myself and others had done some investigating and had a few calls, we were still unsure. We had made a few connections and could pick up threads again if required.

However, with the prospect of working on JSON Schema full time, I decided we should re-engage with the OpenJS Foundation. I wanted to provide confidence to the core team and community that JSON Schema would remain neutrally owned, and have an assurance of continuation should myself or others vanish from the space.

You can read my full proposal and report in a JSON Schema Community GitHub Discussion: https://github.com/json-schema-org/community/discussions/28

I primed the application, which was then reviewed and revised by the team and community. You can read the full application in our Community GitHub Repository: https://github.com/json-schema-org/community/blob/main/docs/openjsf/application.md


# OpenJS Foundation announcement

Here's the majority of the OpenJS Foundation announcement regarding JSON Schema joining the foundation. You can read the full announcement here: https://openjsf.org/blog/2022/01/31/json-schema-joins-openjs-foundation/

__JSON Schema is the newest technical project hosted under the OpenJS Foundation!__

JSON Schema is a vocabulary that allows you to annotate and validate JSON documents. It defines how a JSON should be structured, making it easy to ensure that a JSON is formatted correctly, and it is useful for automated testing and validating. In addition, JSON Schema provides clear human- and machine-readable documentation.

"We are thrilled to welcome JSON Schema into the OpenJS Foundation. Building a community requires dedicated people and great technology, which JSON Schema already has. It also requires a reliable structure for open governance and legal support that allows worldwide communities to grow. As the vendor-neutral home to almost 40 open source projects, JSON Schema already fits in well with our ecosystem of projects," said Robin Ginn, OpenJS Foundation executive director. "We look forward to providing resources and support to JSON Schema to help their community to grow."

"JSON Schema's supportive community has in part enabled us to get this far. This has been critical to its success. JSON Schema is primarily a validation tool, plus it's gaining additional uses such as generating forms, generating databases, or generating other UIs. We want to make sure the community and technology can continue to grow, possibly in unforeseen directions," said Ben Hutton, JSON Schema specification lead at Postman. "By joining the OpenJS Foundation, we gain the community structure and support - with a strong focus on open governance - to continue to build and enlarge the community. We remain committed to being an interoperability focused standard, and want to provide assurance that JSON Schema will remain open and owned by the community that needs it."

"The OpenJS Foundation continues to grow, and JSON Schema is a great addition. It is a key foundational technology, and by joining the OpenJS Foundation, it now has a strong home for further growth," said Todd Moore, OpenJS Foundation Board Chairperson and Chief Developer Advocate IBM. "We are looking forward to working with and supporting JSON Schema."

"The OpenJS Foundation is continuing to support key technologies that JavaScript communities rely upon. JSON Schema is an important addition," said Joe Sepi, Open Source Program Director at IBM, and chairperson of the OpenJS Foundation Cross Project Council." JSON Schema is a great example of how interconnected JavaScript technologies can be. Providing a structure for sharing data is critical."

"The Cross Project Council carefully considers the projects that join our neutral home at the OpenJS Foundation. We are pleased to have JSON Schema onboard so we can support the project's growth, and the maintainers can contribute their expertise to the broader JavaScript ecosystem through OpenJS," said Eemeli Aro, Staff Software Engineer at Mozilla, and OpenJS Cross Project Council (CPC) member.

JSON Schema will be designated "At-Large," which includes many different types of projects but is most often used for stable projects with minimal needs. They are now officially in the incubation process where projects complete their on-boarding to join the foundation.


# JSON Schema is hiring! (at Postman)

On joining Postman, one thing we discussed is the potential to bring on more people to work on JSON Schema full time. While JSON Schema currently does not provide or maintain any software as such, that could change.

In the immediate future, we're looking to hire for Software Engineer and for Technical Community Manager / Developer Relations roles.

Feel free to reach out to me directly via [X](https://www.x.com/jsonschema) or [Slack](https://json-schema.org/slack) should you have any questions.


# Regular calls

JSON Schema now holds two regular calls. We hold weekly Office Hours and twice monthly Open Community Working Meetings.

Office Hours are 15:00 UTC every Tuesday. The objective is to provide a space for people to ask questions or just chat about JSON Schema in real time with video.

Open Community Working meetings are every 1st and 3rd Friday of the month. We have split the time across multiple zones, but we will be consolidating the time to 12:00 PT for both calls moving forward due to low attendance.

We still want more people to join us on these calls. If you want to have a voice in the future of JSON Schema, please join!

Links to both regular calls can be found on our home page: [https://json-schema.org](https://json-schema.org)


# Community, Slack, and GitHub Discussions

We started using GitHub Discussions as a place for our community outside of Slack. The free tier of Slack limits history, and the valuable answers and discussions are lost. While they can be deep-linked, people still have to be registered with the JSON Schema slack to read the content, and it's not search engine indexed.

After joining the OpenJS Foundation, I reached out to Slack again to see if they would bless us with a free Pro account. This time, they agreed! Enjoy access to the full history of our Slack server!

As we expand the ways in which we interact with our community, it feels fitting to enable the community to discuss how they would like to see things develop, and have an open space to see and read proposals. Our Community repo is that space: [https://github.com/json-schema-org/community](https://github.com/json-schema-org/community)


# JSON Schema Blog

JSON Schema now has a blog! [https://json-schema.org/blog](https://json-schema.org/blog)

We wanted a space to provide best practice examples, thought pieces, case studies, and updates. You may even be reading this article on the blog itself!

Currently there are only a handful of articles, but several more case studies are in the works.

The blog has an RSS feed, so if you're still using an RSS reader you can catch every new update.

There were long time plans to rebuild the website as a whole, but our need for a blog escalated after some discussion around referencing on X. Our first article was a joint post between myself and Mike Ralphson on the new JSON Schema bundling process.

We've published one case study in two languages, English and Japanese.

If you're interested in writing a guest post or collaborating on a case study, please reach out.


# JSON Schema on YouTube

In addition to our written case studies, we wanted to provide some easy watching chat style case studies. This series is called JSON Schema in Production.

We needed a YouTube channel to host the videos, which you can [now subscribe to](https://www.youtube.com/channel/UCrl3fjmHSp6FhJUKcEJhisA) - We need 100 subscribers to claim a custom URL, so any sharing is appreciated.


# Open Objectives

With updates to GitHub Projects, I figured I'd see if they could be useful for tracking JSON Schema work in a more public way. Ultimately, I've ended up using task lists in issues, which seems to be working just fine for now. (Projects are still in beta.)

Postman uses the Objective, Key Results (ORK) approach to defining work. I created OKRs for JSON Schema.

Here is a [Project view for 2022 Q1 OKRs grouped by objective](https://github.com/orgs/json-schema-org/projects/4/views/5).

Here's the [Issue which lists all the Key Results as individual tasks](https://github.com/json-schema-org/community/issues/72).


# In Summary

If you've read this far, you must be interested in what we're doing at JSON Schema and our plans for the future.

The most effective way to keep up to date is to join our Slack server. We're also using X, which is our primary way to reach out to our community when we have questions.

There are lots more things we want to bring you, including essential tooling to enable a modern JSON Schema ecosystem.



Photo by [Abraham Barrera](https://unsplash.com/@abebarrera?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText) on [Unsplash](https://unsplash.com/?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText).


JSON Schema joins the OpenJS Foundation and 2022 updates


[Manfred](https://www.getmanfred.com/) es una plataforma de gestión de talento tecnológico con sede en España que permite a sus usuarios gestionar sus carreras profesionales manteniendo un control absoluto sobre sus datos. Manfred ha sido creada por desarrolladores para desarrolladores, liderando un cambio de paradigma en la industria de contratación técnica. Este case study muestra cómo Manfred utilizó JSON Schema para potenciar la transparencia, empoderar a los desarrolladores y fomentar la interoperabilidad dentro del ecosistema de reclutamiento tecnológico.

## Reto

Manfred inició su andadura con la ambiciosa misión de revolucionar el panorama de reclutamiento y para ello decidió apostar por la transparencia total y una estrategia Open Data. Las plataformas de reclutamiento tradicionales se han caracterizado por ofrecer un acceso limitado a los datos personales, a menudo proporcionando archivos poco accionables como PDFs. Manfred apostó por ofrecer una propuesta de valor única ofreciendo una solución que empoderara a los desarrolladores para poseer sus datos en el formato de intercambio universal: JSON.

<Regularquote>
  _"Mientras que otras plataformas de recruiting se basan en la captura de tus datos para luego entregarlos en formatos poco interoperables como PDF, nuestra propuesta fue completamente diferente. Tus datos son tuyos y en JSON, el estándar de facto." - **David Bonilla** - Founder_
</Regularquote>

<div className='flex flex-wrap justify-center items-center gap-4 w-full'>
    <img className='w-full md:w-full lg:w-3/5 xl:w-3/5 2xl:w-3/5 px-20' src='/img/posts/2024/manfred-case-study/period.webp'/>
</div>

## Solution

Reconociendo la necesidad de un enfoque developer-friendly, Manfred apostó por JSON Schema como la piedra angular de su plataforma al crear el [MAC](https://github.com/getmanfred/mac), un formato estándar y open source para definir y compartir currículums. Al ofrecer a los desarrolladores sus datos en formato JSON ofreciendo además el JSON Schema (el MAC) como el mapa para usarlo, Manfred transformó la forma en que los profesionales interactuaban con su propia información. Este enfoque innovador no sólo permitió a Manfred diferenciarse de sus competidores, sino que también propició un nuevo nivel de transparencia en los procesos de contratación al ofrecer un enfoque basado en datos estructurados y accionables con enormes capacidades de interoperabilidad.

<Regularquote>
  _"Tras probar diferentes formatos, descubrimos que usando JSON junto a JSON Schema tendríamos la flexibilidad de poder hacer prácticamente cualquier cosa." - **David Bonilla** - Founder_
</Regularquote>

<div className='flex flex-wrap justify-center items-center gap-4 w-full'>
    <img className='w-full md:w-full lg:w-3/5 xl:w-3/5 2xl:w-3/5 px-20 pt-10' src='/img/posts/2024/manfred-case-study/MAC_Structure.webp'/>
</div>

## Impacto

La adopción de JSON Schema tuvo un impacto profundo en la estrategia de Manfred, contribuyendo a la transparencia, flexibilidad y escalabilidad de la plataforma. A diferencia de las plataformas convencionales que proporcionan datos estáticos y poco accionables, Manfred empoderó a los desarrolladores para utilizar sus propios datos de manera efectiva. La adopción de JSON Schema afectó muy positivamente en distintas áreas:

* **Transparencia y Accesibilidad**: El compromiso de Manfred con los datos abiertos sumado al uso de JSON Schema permitió a los desarrolladores acceder, comprender y navegar sus datos profesionales de una manera fácil e intuitiva. Esta transparencia se convirtió en un factor clave de la confianza entre Manfred y la comunidad.
* **Comunidad**: El uso de JSON Schema por parte de Manfred fue percibido positivamente por la comunidad de desarrolladores, fortaleciendo su posición como una plataforma que pone a los desarrolladores en el centro.
* **Ventaja competitiva**: La propuesta de valor única de proporcionar datos accionables en formato JSON diferenció a Manfred en una industria muy competitiva. Los desarrolladores ahora podían aprovechar el poder de sus propios datos, tomando mejores decisiones sobre sus carreras.
* **Flexibilidad y Escalabilidad**: JSON Schema proporcionó la flexibilidad necesaria para adaptarse a las necesidades de una industria cambiante y facilitando un crecimiento más rápido de la base de usuarios.

<div className='flex flex-wrap justify-center items-center gap-4 w-full'>
    <img className='w-full md:w-full lg:w-3/5 xl:w-3/5 2xl:w-3/5 px-20' src='/img/posts/2024/manfred-case-study/MAC_Export.webp'/>
</div>

## Resultados clave

La adopción de JSON Schema permitió que partners y 3rd parties se integraran con la plataforma de manera más fácil y rápida. La validación e intercambio de información entre Manfred y sus partners se convirtió en un proceso trivial.

<Regularquote>
  _“Usar JSON Schema demostró ser una decisión clave para que la plataforma fuera lo suficientemente flexible y escalable como para soportar al crecimiento de la Empresa.” - **David Bonilla** - Founder_
</Regularquote>

## Sobre Manfred

En sus propias palabras:

<p>“Manfred es una empresa de reclutamiento. Bueno, para ser justos, es una empresa que nació para cambiar el reclutamiento, ayudando a la comunidad con procesos más estructurados y transparentes y un enfoque basado en las personas.</p>

<p>Ayudamos a las personas a alcanzar sus metas profesionales conectando individuos y empresas que comparten objetivos, valores e intereses. Buscamos, entrevistamos, asesoramos y presentamos perfiles tecnológicos a empresas. Pero nuestra misión va más allá: buscamos ayudarlos a lo largo de su trayectoria profesional. Estamos allí, independientemente de la etapa en la que te encuentres, ya sea que estés buscando activamente un trabajo o no, ya sea que necesites consejos o no estés seguro de cómo reorientar tu carrera.</p>

<p>Nuestra prioridad siempre han sido las personas. Nuestro objetivo es devolver a todos los desarrolladores lo que no nos han dado. Formamos parte de una comunidad abierta y colaborativa, pero tiene un problema: la adquisición y gestión de talento.</p>

<p>No movemos cajas, no movemos recursos. <del>Movemos</del>Ayudamos a las personas.”</p>

<div className='flex flex-wrap justify-center items-center gap-4 w-full'>
    <img className='w-full md:w-full lg:w-3/5 xl:w-3/5 2xl:w-3/5 px-20 pt-10 pb-20' src='/img/posts/2024/manfred-case-study/manfred_team.webp' alt='image'/>
</div>


Yeray Darias

Transformando la industria de la contratación técnica con JSON Schema

[Manfred](https://www.getmanfred.com/) is a Spanish based technical recruiting platform allowing developers to manage their careers while keeping absolute control over their data. Manfred has been built by developers for developers, leading a paradigm shift in the technical recruiting industry. This case study showcases how Manfred harnessed the power of JSON Schema to redefine transparency, empower developers, and foster interoperability within the tech recruitment ecosystem.

## Challenge
Manfred started their journey with the ambitious mission of revolutionizing the recruitment landscape by prioritizing total transparency and open data. Traditional recruiting platforms offered limited accessibility to personal data, often providing static, non-actionable PDFs. To establish a unique value proposition, Manfred sought a solution that would empower developers to own their data in a universally understood format—JSON.

<Regularquote>
  _"While other recruiting platforms rely on capturing your data and then giving it back using inefficient formats like PDF, our approach was completely different. Your data is yours and in JSON, the most used format in the industry." - **David Bonilla** - Founder_
</Regularquote>

<div className='flex flex-wrap justify-center items-center gap-4 w-full'>
    <img className='w-full md:w-full lg:w-3/5 xl:w-3/5 2xl:w-3/5 px-20' src='/img/posts/2024/manfred-case-study/period.webp'/>
</div>

## Solution
Recognizing the need for a dynamic and developer-friendly approach, Manfred turned to JSON Schema as the cornerstone of their platform by creating the [MAC](https://github.com/getmanfred/mac), a standard open source format to define and share resumes. By offering developers their data in JSON format and providing the JSON Schema (the MAC) as the map to use it, Manfred transformed the way professionals interacted with their own information. This innovative approach not only differentiated Manfred from competitors but also provided actionable, structured data, fostering a new level of transparency in the recruitment process.

<Regularquote>
  _"After trying different formats, we realized that by using JSON jointly with JSON Schema we would have the flexibility to be able to do anything." - **David Bonilla** - Founder_
</Regularquote>

<div className='flex flex-wrap justify-center items-center gap-4 w-full'>
    <img className='w-full md:w-full lg:w-3/5 xl:w-3/5 2xl:w-3/5 px-20 pt-10' src='/img/posts/2024/manfred-case-study/MAC_Structure.webp'/>
</div>

## Impact
The adoption of JSON Schema had a profound impact on Manfred's operations, contributing to transparency, flexibility, and scalability. Unlike conventional platforms that provide static, non-machine-readable data, Manfred empowered developers to utilize their own data effectively. JSON Schema enabled:

* **Transparency and Accessibility**: Manfred's commitment to open data and JSON Schema allowed developers to access, understand, and navigate their professional data seamlessly. This transparency became a cornerstone of trust between Manfred and its user community.
* **Community Engagement**: Manfred's usage of JSON Schema resonated with the developer community, strengthening its position as a developer-centric platform.
* **Competitive Advantage**: The unique value proposition of providing actionable data in JSON format set Manfred apart in a competitive industry. Developers could now harness the power of their own data, making informed decisions about their careers.
* **Flexibility and Scalability**: JSON Schema provided the necessary flexibility to adapt to evolving business needs and accommodate the rapid growth of Manfred's user base. 

<div className='flex flex-wrap justify-center items-center gap-4 w-full'>
    <img className='w-full md:w-full lg:w-3/5 xl:w-3/5 2xl:w-3/5 px-20' src='/img/posts/2024/manfred-case-study/MAC_Export.webp'/>
</div>

## Key Impact Results
The adoption of JSON Schema enabled third parties to integrate with Manfred easier and faster. Validation and documentation became trivial, for Manfred and their partners.

<Regularquote>
  _“Using JSON Schema proved to be a key decision to provide the platform with the flexibility and scalability to support the Company's growth.” - **David Bonilla** - Founder_
</Regularquote>

## Manfred - The Company
In their own words: 

<p>“Manfred is a recruiting company. Well, to be fair, it's a company that was born to change recruiting, helping the community with more structured, transparent processes and a focus on people.</p>

<p>We help people achieve their professional goals by connecting individuals and companies that share objectives, values, and interests. We search, interview, advise, and present tech profiles to companies. But our mission goes beyond that: we seek to assist them throughout their career journey. We are there, regardless of the stage you're in—whether you're actively looking for a job or not, whether you need advice or are unsure about how to refocus.</p>

<p>Our priority has always been people. We aim to give back to all the developers what they have not given to us. We belong to a very open and collaborative community, but it has a problem: talent acquisition and talent management.</p> 

<p>We don't move boxes, we don't move resources. We <del>move</del> help people.”</p>

<div className='flex flex-wrap justify-center items-center gap-4 w-full'>
    <img className='w-full md:w-full lg:w-3/5 xl:w-3/5 2xl:w-3/5 px-20 pt-10 pb-20' src='/img/posts/2024/manfred-case-study/manfred_team.webp' alt='image'/>
</div>

Transforming the technical recruiting industry with JSON Schema


Probably the most common question that we get is, "How do I model an inheritance hierarchy in JSON Schema?"  And most commonly, our answer to that question is, "You don't."

JSON Schema just isn't [designed](https://modern-json-schema.com/json-schema-is-a-constraint-system) for that.  It's a subtractive system, where more constraints means fewer matches, and data modelling tends to be additive, where more definition means more matches.  The systems are inherently incompatible.

However, if we accept a few concessions, we might just be able to work something out.

## Our Models

To get started, we're going to attempt to model some computer peripherals.  In a strongly typed language, we may model this using a `Peripheral` base class that defines a number of properties (and typically functions) that are common to all peripherals.  Then, each device would be a subclass of this base class.

For our purposes, we're just going to define the `name` property on the base class.  That is, every peripheral needs to have a name.

_I'm going to use TypeScript for the code samples, but the concepts will apply to other languages as well._

```ts
abstract class Peripheral {
  name: string;
  // ...
}
```

Now we can define other peripherals, `Mouse` and `Keyboard`, by inheriting from this base class.

```ts
class Mouse extends Peripheral {
  buttonCount: number;
  wheelCount: number;
  trackingType: "ball" | "optical";
  // ...
}

class Keyboard extends Peripheral {
  keyCount: number;
  mediaButtons: boolean;
  // ...
}
```

This will suffice to get us started.

## Representing Our Models Using Constraints

In JSON Schema, ideally, we'd want to have schemas for each of these.  For peripheral, we might try something like this:

```json
{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "schema:peripheral",
  "type": "object",
  "properties": {
    "name": true
  },
  "required": [ "name" ],
  "additionalProperties": false
}
```

_I'm using a `schema:` URI for the schema identifiers since these schemas aren't accessible anywhere.  It's a recommendation that we're [considering](https://github.com/orgs/json-schema-org/discussions/460) for the upcoming version of JSON Schema.  Let us know if you like this approach._

But that `additionalProperties` keyword causes problems.  Specifically, "inherited" schemas (like what we're going to build for `Mouse`) can't define additional properties, which is something it definitely needs to do.  That just won't work at all, and the solution is simply to omit it.

```json
{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "schema:peripheral",
  "type": "object",
  "properties": {
    "name": true
  },
  "required": [ "name" ]
}
```

But now, _any_ JSON object with a `name` property is validated as a peripheral.  While not quite right, we can live with it.  This gives us our first concession:

> Schemas that model base classes cannot verify that an instance represents a derivation of that class.

Modelling the derivations is pretty straightforward: we model what the derivation defines and add a `$ref` back to the base schema.

```json
{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "schema:mouse",
  "$ref": "schema:peripheral",
  "type": "object",
  "properties": {
    "buttonCount": { "type": "integer" },
    "wheelCount": { "type": "integer" },
    "trackingType": { "enum": [ "ball", "optical" ] }
  },
  "required": [ "buttons", "wheels", "tracking" ],
  "unevaluatedProperties": false
}

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "schema:keyboard",
  "$ref": "schema:peripheral",
  "properties": {
    "keys": { "type": "integer" },
    "mediaButtons": { "type": "boolean" }
  },
  "required": [ "keys", "mediaButtons" ],
  "unevaluatedProperties": false
}
```

For the derived schemas, we can use `unevaluatedProperties` because these don't have any schemas which derive from them.  If the inheritance hierarchy is bigger and these classes serve as bases for others, we'd have to leave the `unevaluatedProperties` off, just like we did for `schema:peripheral`.  Checking for extra properties can only be done for the leaves of the inheritance tree.

Additionally, we use `unevaluatedProperties` instead of `additionalProperties` because we need it to be able to "see inside" of the `$ref` to identify that `name` was evaluated as part of the base schema.  With `additionalProperties`, `name` would be rejected.

That seems pretty simple, and we only had to make a single (and rather easy) concession.

## Adding a Recursive Reference

What if one of our peripherals could itself have other peripherals attached?  Say, a USB hub.

```ts
class UsbHub extends Peripheral {
  connectedDevices: Peripheral[];
  // ...
}
```

Let's try to model that in a schema:

```json
{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "schema:usbhub",
  "$ref": "schema:peripheral",
  "properties": {
    "connectedDevices": {
      "type": "array",
      "items": { "$ref": "schema:peripheral" }
    }
  },
  "required": [ "connectedDevices" ],
  "unevaluatedProperties": false
}
```

This works, but remember that first concession we made?  This schema would allow items that are _anything_ with a string `name` property.  But that doesn't align with the TypeScript models.  The TypeScript model says that `connectedDevices` can only hold types that are derived from `Peripheral`.

While this may be sufficient for some, in my opinion it doesn't work.  I want to ensure that the items in the `connectedDevices` array are only known peripheral types.  To do this, we need another schema.

## Supporting Only Known Derivations

Problem: we want a schema that identifies that some JSON represents _one of_ our known device types.

Solution: define the schema using a `oneOf` that references all of the known device type schemas.

```json
{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "schema:known-peripherals",
  "oneOf": [
    { "$ref": "schema:mouse" },
    { "$ref": "schema:keyboard" },
    { "$ref": "schema:usbhub" }
  ]
}
```

This schema is pretty basic.  It just says, "If the JSON matches one of these devices, then it's a known peripheral."

We can now reference this in `schema:usbhub`.

```json
{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "schema:usbhub",
  "$ref": "schema:peripheral",
  "properties": {
    "connectedDevices": {
      "type": "array",
      "items": { "$ref": "schema:known-peripherals" }
    }
  },
  "required": [ "connectedDevices" ],
  "unevaluatedProperties": false
}
```

Now the USB hub _and its connected devices_ can be validated properly.

The catch is that because I can't dynamically add items to the `oneOf`, I can only support devices I know about at dev time.  For most cases, this isn't a problem.  However, if I plan on publishing this in a package to be used by others, it wouldn't support devices that they create.  (I do have a workaround for this, but it's not a good one, so I won't be sharing it here.)  This gives us our second concession:

> If we need references to the base class, we can only support derivations we know about ahead of time.

## An Unexpected Benefit

To determine whether some JSON is a `Mouse` or a `Keyboard` or a `UsbHub`, one might hold all three of those schemas and validate each one of them in turn to determine which one was received.  But our solution to the referencing problem actually gives us a better option.

We know that `schema:known-peripherals` can validate _any_ known peripheral (because we designed it to do that), but if we use a more verbose output format, it can actually tell us which _kind_ of peripheral we got.

First, we identify which `oneOf` subschema passed validation by looking through its child output nodes for a `valid: true`.  We know that's going to be a `$ref` schema (because it's a `oneOf` that contains only `$ref` schemas), which means that the child output node of that `$ref` schema will represent the output of the peripheral schema, which contains the peripheral schema's `$id` URI.

So in a single validation pass, we get whether it's a supported peripheral of any kind, _and_ we can discern what kind it is.  Two birds, one stone.

## So is Inheritance in JSON Schema Possible?

No.

And yes, if we're okay that:

> Schemas that model base classes cannot verify that an instance represents a derivation of that class.

> If we need references to the base class, we can only support derivations we know about ahead of time.

I think these will be acceptable for most people, but I'm also sure that someone will inevitably run into a scenario where this approach won't work.

This is the best I've seen so far at modelling inheritance, and I'm fairly certain that JSON Schema can't get it 100% right without some new functionality.

If you have some other ideas about how to support polymorphism, or if you think polymorphism is overrated and JSON Schema doesn't need to support it, please join the [conversation](https://github.com/json-schema-org/vocab-idl/issues/49) in our IDL Vocabulary repository.

_Cover image by [Gerd Altmann](https://pixabay.com/users/geralt-9301/) on [Pixabay](https://pixabay.com/illustrations/board-drawing-hierarchy-755792/)_


Modelling Inheritance with JSON Schema


## We are so pleased to announce the launch of our new website!

We are excited to announce the launch of our newly designed JSON Schema website!

We have created a modern design with improved site navigation and new information architecture to help you find exactly what you are looking for and better explain the benefits of JSON Schema and its incredible ecosystem.

## A little bit of history

The JSON Schema story started in 2007 when Chris Zyp proposed JSON Schema as a schema language to validate JSON documents. Today, JSON Schema is one of the most used standards in the industry. During all this time, there have been different leading groups, specification releases, site versions, and, most importantly, hundreds of contributions by the JSON Schema Community. However, we have been receiving feedback about different ways to improve and refresh the site experience, and this is why the Community has been working so hard to release a new version of the website to help JSON Schema continue this amazing growth!

<figure className="mt-5">
  <img className="max-w-xs sm:max-w-md md:max-w-lg lg:max-w-xl xl:max-w-2xl mx-auto" src="/img/posts/2023/new-website/site-2016.webp"/>
    <figcaption className="mt-2 mb-10 text-sm text-center text-gray-500">The JSON Schema website back in 2016</figcaption>
</figure>

## A new story

Our new landing page is a modern, single destination with all the resources necessary for the different types of personas visiting the website: JSON Schema users, JSON Schema tooling implementers, and other organizations.

The information architecture will make it easier for everyone to find the resources they need and make it easier for the Community to maintain and evolve our docs as the project grows.

## Who made this possible?

This has been a long journey led by the Community. We want to start by acknowledging all the people who contributed to the previous website versions because this wouldn't have been possible without them. THANKS!

We'd like to recognize and thank everybody who worked on the new site release: [Johannes Linowski](https://github.com/musemind), [Paul Waller](https://github.com/PaulWaller), [Akanksha Kushwaha](https://github.com/aku1310), [Roni Lookwood](https://github.com/mjgutermuth) and [Melinda Gutermuth](https://github.com/mjgutermuth). Thanks for making this happen!

## What is next?

This journey has just started, but now we still have much work to do, and we are actively looking for contributors!

The short term focus of our future work is:
- Continue refining our docs. See this [issue](https://github.com/json-schema-org/community/issues/421) for more details.
- Build the JSON Schema landscape. See this [issue](https://github.com/json-schema-org/community/issues/354) to know more.

Are you interested in getting involved?
Please join our [Slack workspace](https://json-schema.org/slack) and visit the #website channel.

_Cover photo by [Ricardo Rocha](https://unsplash.com/@rcrazy) on [Unsplash](https://unsplash.com/photos/nj1bqRzClq8)_


Announcing our new website

Loïc Lefèvre

How Oracle is Bridging the Gap Between JSON Schema and Relational Databases


> Originally published at [blog.postman.com](https://blog.postman.com/how-postman-uses-json-schema).

<div className="border-2 py-3 px-3 border-blue-400 bg-blue-100 text-blue-700">
Disclaimer: A number of members of the JSON Schema organization are employed by Postman, but this post is not sponsored content.
</div>

The [Postman API Platform](https://blog.postman.com/announcing-postman-v10/) offers a rich set of solutions for every step of the [API lifecycle](https://www.postman.com/api-platform/api-lifecycle/). Through the years, we have witnessed from the front row how JSON Schema entered the scene to become the industry standard for describing and annotating JSON documents. While many alternatives came and went, JSON Schema unquestionably proved to be the robust and extensible foundation behind the API specification movement. According to Postman's [2022 State of the API Report](https://www.postman.com/state-of-api/api-technologies/#api-technologies), an impressive 72% of respondents chose JSON Schema as their preferred API specification.

With JSON Schema being so ingrained in how APIs are developed today, it is hard to find a place in the Postman organization or the Postman API Platform where JSON Schema is not involved in some way or another. And with the modern (and increasing) importance of [API-first](https://www.postman.com/use-cases/api-first-development/) development, the use of API specifications plays a critical role in designing and sharing APIs. JSON Schema is currently the backbone of the most popular API specification technologies in the world, including [OpenAPI](https://www.openapis.org/), [AsyncAPI](https://www.asyncapi.com/), and [RAML](https://raml.org/).

## JSON Schema in the Postman organization

### How Postman uses JSON Schema to build its own APIs

Postman operates a complex distributed system consisting of many dozens of microservices powering the cloud tools that make the Postman API Platform. These microservices make use of JSON Schema to model their interfaces through standalone JSON Schema definitions or OpenAPI and AsyncAPI specifications. Many of these microservices also use JSON Schema under the hood to validate internal data structures and configuration files, and to perform end-to-end testing by either asserting on expected responses or auto-generating input data from JSON Schema definitions. Being a Node.js company, Postman have started efforts to analyze how they can rely on TypeScript annotations to automatically generate OpenAPI and AsyncAPI definitions from our codebases.

Given the importance of API specifications for managing Postman's distributed systems, in 2021 a comprehensive internal research project was conducted to understand the diverse set of JSON Schema definitions introduced by microservices run within the internal Postman platform. The corresponding JSON Schema definitions were analyzed based on characteristics such as reusability, degree of similarity, uniqueness ratio, and type of content. To promote reusability and discoverability, Postman is exploring the idea of a new microservice that acts as a central JSON Schema catalog API.

For analysis purposes, Postman periodically extracts data from every SQL-based database powering the Postman API Platform. To accommodate for schema changes in our services, Postman automatically converts SQL table definitions into JSON Schema definitions, and rows into JSON documents, before aggregating them into Postman's internal data warehouse. Some initial experiments have been conducted to explore generating SQL table definitions out of JSON Schema documents, making the latter the source of truth.

### Use cases of JSON Schema beyond APIs

The use of JSON Schema within the Postman organization is not restricted to its backend services. For example, the popular [Postman Collection](https://www.postman.com/collection/) JSON-based data format is formally defined using JSON Schema. Postman's [Newman](https://learning.postman.com/docs/collections/using-newman-cli/command-line-integration-with-newman/) command-line Collection Runner makes use of JSON Schema to validate the expected output of custom reporters. Postman's in-house cross-platform desktop framework makes use of JSON Schema to validate and annotate profile definitions that declare the various variants of the desktop application (Stable, Canary, etc). Postman also maintains internal [C4](https://c4model.com/) diagrams of the Postman architecture defined using JSON and validated using JSON Schema.

In early 2022, Postman released support for [gRPC and Protocol Buffers](https://blog.postman.com/postman-now-supports-grpc/). Internally, Postman is able to transform Protocol Buffers schema definitions into JSON Schema definitions and back again. The JSON Schema definitions corresponding to the Protocol Buffers schemas are used to provide type-hinting and auto-completion in the gRPC payload composer editor, to validate user input and to generate random data that matches a Protocol Buffers schema for testing purposes. This approach allows Postman to implement the aforementioned features on top of a single unified schema language: JSON Schema.

## JSON Schema in the Postman API Platform

JSON Schema is not only used internally to develop the various components of the Postman API Platform - many of the features offered by Postman directly involve the use of JSON Schema.

### JSON Schema in the context of Postman Collections

The Postman app can be used to [convert a growing number of API specification formats into Postman Collections](https://learning.postman.com/docs/developer/collection-conversion/). As noted earlier in this post, the most popular API specification formats such as OpenAPI, Swagger, and RAML rely on JSON Schema. In many cases, the API specification conversion logic requires generating random JSON documents that match a JSON Schema definition.

When defining a Postman Collection, users may define JavaScript-based test and pre-request scripts that are executed automatically when running the corresponding collection. The JavaScript engine embedded within Postman to run these scripts integrates with the popular [AJV](https://ajv.js.org/) JSON Schema validator. With it, Postman users write scripts that employ JSON Schema validation using a wide range of JSON Schema specification versions.

### JSON Schema in the context of OpenAPI

The Postman app provides a rich [OpenAPI editor](https://learning.postman.com/docs/designing-and-developing-your-api/defining-an-api/) with advanced JSON Schema capabilities. The editor is able to show autocompletion and syntax warnings, and it also highlights [potential areas of improvements](https://learning.postman.com/docs/api-governance/api-definition/api-definition-warnings/) with regards to readability and security for JSON Schema and OpenAPI endpoint definitions. An OpenAPI definition is then used to generate [rich documentation](https://learning.postman.com/docs/publishing-your-api/documenting-your-api/) of the available endpoints and their respective JSON Schema definitions, and to optionally generate matching [server code](https://learning.postman.com/docs/designing-and-developing-your-api/generating-server-code/) written in Go, Java, Python, and Node.js.

An API defined using Postman is more than its API definition. It has surrounding elements such as documentation, tests, mock servers, and monitors. When writing an OpenAPI specification, Postman will [cross-check the integrity of each of these elements](https://learning.postman.com/docs/designing-and-developing-your-api/validating-elements-against-schema/) against the JSON Schema definitions included in the API specification.

### The Postman API Network

A key component of the Postman API Platform is the [Postman API Network](https://www.postman.com/api-network/), the [world's largest registry of public APIs](https://blog.postman.com/postman-public-api-network-is-now-the-worlds-largest-public-api-hub/). This global public registry includes a vast amount of APIs and their corresponding JSON Schema definitions typically maintained by their respective original authors. Some notable examples are the [Slack Web API](https://www.postman.com/slackhq/workspace/slack-api/collection/13509546-993e3b18-d277-4189-8ce5-af45df38e336), the [Docker HUB API](https://www.postman.com/dockerdev/workspace/docker-hub/collection/17990590-9574e087-2a50-4ecf-88b3-55f12a29d99e), and the [Twilio API](https://www.postman.com/twilio/workspace/twilio-api/overview). The Postman API Network is therefore one of the largest datasets of production-grade JSON Schema definitions.

## Postman and JSON Schema

With JSON Schema being a crucial component of the Postman API Platform and the API ecosystem, Postman is honored to be supporting the JSON Schema organization as part of the [OpenJS foundation](https://json-schema.org/blog/posts/json-schema-joins-the-openjsf) by bringing aboard some of its passionate core contributors - [Ben Hutton](https://blog.postman.com/ben-hutton-joins-postman-to-lead-json-schema-strategy/), [Greg Dennis](https://json-schema.org/blog/posts/and-then-there-were-three), [Jason Desrosiers](https://json-schema.org/blog/posts/joining-postman), and [Julian Berman](https://json-schema.org/blog/posts/hello-world-hello-postman) - as Postmanauts. We can't know everything about what the future might hold, but Postman is certain that the future of APIs involves JSON Schema.

Juan Cruz Viotti

How Postman uses JSON Schema


## Challenge

Forms. Legal forms, employment forms, salary forms, all the forms! Hiring from many different countries creates many different requirements, and Remote has to handle them all. Continually evolving laws and regulations in one country is enough to keep up with, let alone close to 100 countries!

"Imagine wanting to hire someone in Argentina, Germany, and Nigeria. Those are all incredibly different countries with different laws and requirements, and we have to handle all of them." - Sandrina Pereira, Staff Frontend Engineer at Remote.com.

<figure class="mt-10">
  <img src="/img/posts/2023/remote-case-study/Remote-Contract details.webp"/>
  <figcaption class="mt-2 mb-10 text-sm text-center text-gray-500">One of the onboarding forms for Netherlands, among more than 500 unique forms to all the countries.</figcaption>
</figure>

When Remote first started,  the company only covered a handful of countries with a few specific forms per country. Scaling to hundreds of country forms with complex requirements is impressive. Did you even realize that the working hours and weekend days are different across countries and employment types? In some countries, the weekend is Friday and Saturday. And there are many other types of schedules besides the traditional 9 to 5. Getting that wrong for someone's salary payment could have huge implications.

Managing and maintaining hardcoded forms in full HTML format (or even frontend components), wasn't going to be viable for the scaling Remote wanted.

"Our hardcoded solution was blocking us from scaling our business. Duplicated code, inconsistent server/client validations, huge JavaScript bundles…" explains Sandrina.

Remote needed a way to configure forms on the server side to be used by the Remote API, including centralized validations that could be shared between the server and the client.

## Solution

JSON Schema was designed specifically for JSON data validation, but that hasn't stopped emergent use cases. While there are several form generation libraries that take JSON Schema as their input, Remote wanted to have total control, and needed the freedom to evolve the solution as required.

"All of that is now being solved with JSON Schemas," continued Sandrina. They worked on JSON Schema for forms and validation initially in their Development Foundations team, and later as part of the company's "Onboarding" vertical. Their onboarding process, which requires a lot of data, has to be the best experience possible to avoid incompletion, without compromising data safety or accuracy.

Remote decided to create their [own form generation tool](https://remote.com/blog/json-schema-forms-guide), allowing them to layer on additional custom keywords in JSON Schema related to the presentation of form fields as needed. "After deep research with proof of concepts, taking into account the critical nature of forms in our core product, we decided to take a leap of faith… " The first version of this library has very recently been open-sourced, so you can try it today! Go check [@remoteoss/json-schema-form](https://github.com/remoteoss/json-schema-form)!

<figure class="mt-10">
  <img class="rounded-lg" src="/img/posts/2023/remote-case-study/blog-image.webp"/>
  <figcaption class="mt-2 mb-10 text-sm text-center text-gray-500">Diagram of how JSON Schema integrates with the server and client.</figcaption>
</figure>

Data doesn't just come via forms though, but also through APIs, which usually includes third parties. "We started this journey as part of the Remote API, a horizontal tool by itself. Gradually it will affect every area of Remote. Ultimately our vision is to have every form that talks to our API powered by a 'headless form' on top of JSON Schemas," Sandrina explains. Getting data boundaries right can be hard, but it pays to do the work. Having the validation on the server reusing it on the client, not only simplifies the development, but also reduces attack surface in terms of security.

As with adding any new technology to your stack, you must critically evaluate if it will be viable long term, and how flexible it will be for any changes in your existing stack. Sandrina and her colleagues did their research…

"Although JSON Schema is not an "official standard", seeing big players adopting and contributing to its maturity is a big point of trust for us. JSON Schema is in active development by its strong community, with a high adoption rate by many companies, including mature libraries in many languages, as the ones we use: JavaScript and Elixir."

<p className="text-2xl my-10">"When looking for a way to describe and validate JSON objects, there was no other solid alternative. JSON Schema was adopted with confidence." - Sandrina Pereira, Staff Frontend Engineer at Remote.com.</p>

## Impact

Using JSON Schema as the SSoT (Single Source of Truth) unlocked a lot of potential. "JSON Schemas were the first step to solving data validation and form generation problems across all levels at Remote," explains Sandrina.

Once you start with JSON Schema, it's not uncommon to discover multiple uses. Remote creates JSON Schemas on the server and uses them to validate API payloads, run periodic validation reports (based on legal requirements), and auto-complete in code. Additionally, the client consumes these schemas to generate visual representations such as forms and tables.

Armed with the SSoT, Remote sees reduced time to create and maintain forms, but also a measurable reduction in support tickets related to inconsistent data and incorrect validation. Most importantly for Remote, they also reduced the time to onboard new international employees and contractors. While other improvements were made at the same time as integrating JSON Schema, they were confident JSON Schema was pivotal.

## Key Impact Results

Adapt and evolve at a large scale, providing services quicker, and onboarding from weeks to days. Here's a list of the biggest results so far:

- Shared validations between the server and client
- Reduced JS bundles and code duplication, replaced by API requests
- Reduced number of tickets reporting inconsistent validations.
- Conclude tickets much quicker
- Create and maintain forms more easily
- Opened the doors to allow the Internal Operations team to build the forms themselves

"JSON Schemas are the connector to keep our data accurate on a large scale, speeding up many of our provided services to our customers. In some cases, the time taken to implement the forms for a new country was reduced from weeks to just a couple of days."

## Remote - The Company

<p className="text-2xl my-10">Remote makes it easy to onboard and pay your international employees and contractors. Remote is a global, fully distributed company in the Human Resources industry, with no physical offices. "Our employees are free to work from their chosen locations around the world!"</p>

Sandrina Pereira: "I'm a Staff Frontend Engineer working in the Onboarding vertical, and we are the ones who have the biggest and more dynamic forms that carry all the data needed to hire new people." Without question, a critical position to enable the value proposition Remote offers.

With many companies having had to provide a means to work remotely where possible, why limit the talent pool to one geographical location?

Remote wants to empower employees too: "Talent is everywhere, but opportunities aren't. Remote makes it possible for employers to hire anybody from anywhere, to present access to opportunities for anyone, to ultimately make better lives for themselves." These are not just words for Remote.

In 2021, Remote launched their [Refugee program](https://remote.com/news/en-GLO/208556-announcing-remote-for-refugees), enabling employers to invest in a cost effective way, while also helping refugees rebuild their lives. Remote has also set up [dedicated resources for displaced Ukrainians](https://remotecom.notion.site/Remote-for-Ukrainians-b9cf24e00ef145b1b96cd3cca919c9b2).

Remote raised a $300 million Round C in 2022, at a validation of over $3 billion.


## Further Benefits

<p className="text-2xl my-10">"JSON Schemas is bringing us data consistency (annotation) and data security (server validations)." - Sandrina Pereira, Staff Frontend Engineer at Remote.com.</p>

Having migrated some core parts of the business, Remote followed to create an interface allowing Operations teams to edit forms themselves, stored as JSON Schema to benefit from validation. Static JSON Schema files have served them well, but ultimately they needed to be able to react faster to change.

<figure class="mt-10">
  <img class="rounded-lg" src="/img/posts/2023/remote-case-study/Remote-Form builder.webp"/>
  <figcaption class="mt-2 mb-10 text-sm text-center text-gray-500">Backoffice tool to build the Country Forms, powered by JSON Schema.</figcaption>
</figure>

Knowledge sharing has also benefited from JSON Schema, with Remote revamping their Knowledge Base. Country specific data such as supported VISA options and onboarding timelines, can be encoded into a data structure, which in turn can be validated both on the server and client side.

Initially unsure how far JSON Schema could go, and despite Remote pushing the barrier of what JSON Schema can do, it has proven to have an important impact on Remote's ability to scale.

"JSON Schema has proven its value as part of our core tooling ecosystem to validate data and generate its visual representations, such as forms and tables, which are a fundamental piece to take our services to a large-scale without compromising accuracy, speed, and most importantly, customer happiness!" We are glad you've found value in JSON Schema Sandrina! Long may it continue.

Thank you to [Sandrina Pereira](https://www.linkedin.com/in/sandrina-p), Staff Frontend Engineer, and [Remote.com](https://remote.com), for allowing and enabling us to share this case study with you. Sandrina would also like to acknowledge and thank her colleagues who kicked-off this journey more than a year ago: [André Albuquerque](https://pt.linkedin.com/in/amalbuquerque), [António Capelo](https://www.linkedin.com/in/antoniocapelo/), [António Silva](https://www.linkedin.com/in/antonio-silva-0a6a72b4/), [Livia Barbosa](https://www.linkedin.com/in/liviaab/), and [João Almeida](https://www.linkedin.com/in/engjoaoalmeida/).


Using JSON Schema at Remote to scale forms and data validations





# How RxDB embraces JSON Schema to build its NoSQL Database

[RxDB](https://rxdb.info/) is a NoSQL database designed primarily for client-side JavaScript applications, focusing on the [offline-first approach](https://rxdb.info/offline-first.html), multi-tab operations, and data replication. It seamlessly integrates with modern web and mobile app frameworks, supports various storage backends, and provides a plugin-based architecture to handle encryption, compression, type generation, and more.

One of RxDB's standout architectural choices is its reliance on JSON Schema to define data models for documents in collections. Instead of inventing a custom schema language, RxDB adopts a standard that is already well-established in the broader JavaScript community. By doing so, RxDB solves several major hurdles that often arise when developers are introduced to new databases:

- **Familiarity**: Most JavaScript developers have already encountered JSON Schema, either via OpenAPI or other tooling. Thus, they can quickly understand and adopt RxDB's schemas without investing time in learning a new schema language.

- **Tooling ecosystem**: JSON Schema boasts a robust set of libraries, validators, and code generators. RxDB leverages these rather than maintaining its own schema validation library, giving teams the freedom to pick what works best for them.

- **Long-term maintainability**: Because JSON Schema is recognized as a standard, future updates and ecosystem support are more reliable than a custom, one-off solution.


By building on JSON Schema, RxDB has a foundation that makes schema design, data validation, and typing straightforward, making it easier for developers to build robust, safe applications in production.


<center>
    <a href="https://rxdb.info/">
        <img src="https://rxdb.info/files/logo/rxdb_javascript_database.svg" alt="JavaScript Database" width="220" />
    </a>
</center>

## How RxDB uses JSON Schema


While RxDB adopts the JSON Schema Core and Validation specifications, it also extends it to introduce RxDB-specific functionality. Like in other NoSQL databases, you can manually define which fields to encrypt, which ones to index, and how to interpret specific fields for queries. RxDB enables these configurations as custom JSON Schema keywords:

- `primaryKey`: Specifies which field in the document serves as the primary key.
- `indexes`: Defines which fields (or combination of fields) RxDB indexes. You can have single-field indexes or compound indexes.
- `version`: Indicates the version of the schema. Whenever you change your schema, you must increment this version so RxDB can handle migrations or other adjustments. This is important because data migration on a client-side database can be tricky when you have many clients out there that update your app at different points in time.
- `encryption`: Specifies which fields should be stored in an encrypted form. This is useful for sensitive data that you do not want to store in plaintext on the client's device.
- `keyCompression`: Can be set to `true` to enable the key-compression plugin.

RxDB supports a [compression plugin](https://rxdb.info/key-compression.html) that uses a "compression table." This is essentially a lookup table derived from your JSON Schema which assigns shorter keys or transforms fields so that the stored data becomes more compact. By analyzing the schema, the plugin understands which fields appear repeatedly and can replace them with shorter tokens. Remarkably, RxDB can still query the data in its compressed form. This leads to performance improvements, especially in environments where local storage space is limited.

Below is a sample RxDB schema that demonstrates how standard JSON Schema vocabularies combine with RxDB's custom extensions:

```ts
const mySchema = {
    primaryKey: 'id',
    type: 'object',
    properties: {
        id: {
            type: 'string',
            maxLength: 100
        },
        name: {
            type: 'string'
        },
        birthdate: {
            type: 'string',
            format: 'date'
        },
        secret: {
            type: 'string'
        }
    },
    additionalProperties: false,
    required: ['name', 'birthdate', 'id'],
    // RxDB specific fields:
    primaryKey: 'id',
    version: 0,
    keyCompression: true,
    indexes: [
        'name', // single-field index
        ['name', 'birthdate'] // compound index
    ],
    encrypted: ['secret']
}
```

## Restrictions

Although RxDB aims to remain fully spec compliant with JSON Schema, it does impose some extra restrictions:

The schema requires top-level field names to match the regex `^[a-zA-Z][a-zA-Z0-9_]*[a-zA-Z0-9]$`. This rule ensures that property names are valid JavaScript identifiers.

At the top level of the schema, `additionalProperties` must be set to `false`. This prevents silently introducing new fields that could potentially clash with built-in RxDocument methods (like `.toJSON()`). By making you list out each property, RxDB ensures you can't unknowingly overwrite or conflict with standard methods.

RxDB does not allow `$ref` to other files or external schema fragments or `$dynamicRef`. The goal is to keep schema loading fast and self-contained. If a schema needs to be composed of multiple parts, you must combine them at build time or otherwise ensure they are merged before passing them to RxDB at runtime. This approach also prevents any network calls or asynchronous fetches that could slow down your application's startup.


## Inferring Document Types with TypeScript

JSON Schema is not just for validation and structural guarantees, it can also help generate or infer TypeScript types. In many projects, developers rely on tools like [json-schema-to-typescript](https://github.com/bcherny/json-schema-to-typescript) to produce `.d.ts` files or interface definitions from schema files at build time. However, that requires a separate build step, which slows down your workflow.

To improve developer experience, RxDB offers a [built-in way](https://rxdb.info/tutorials/typescript.html) to infer the document's TypeScript type from the schema during runtime. This is helpful because you get immediate feedback in your IDE: As soon as you update the schema, TypeScript picks up the changes. You'll see type errors in your code if you attempt to use fields that are no longer valid or if you forget to include newly required fields.

Below is an example on how to infer the TypeScript type of a document from its JSON-schema:

```ts
import {
    toTypedRxJsonSchema,
    ExtractDocumentTypeFromTypedRxJsonSchema,
    RxJsonSchema,
    RxCollection
} from 'rxdb';

const heroSchemaLiteral = {
    title: 'hero schema',
    description: 'describes a human being',
    version: 0,
    keyCompression: true,
    primaryKey: 'passportId',
    type: 'object',
    properties: {
        passportId: {
            type: 'string',
            maxLength: 100
        },
        /* ...other fields... */
    },
    required: ['name', 'birthdate', 'passportId'],
    indexes: ['name', 'birthdate']
} as const;

// Convert the literal object to a typed schema
const schemaTyped = toTypedRxJsonSchema(heroSchemaLiteral);

// Extract the document type from the typed schema
type HeroDocType = ExtractDocumentTypeFromTypedRxJsonSchema<typeof schemaTyped>;

// Create a typed RxJsonSchema
const heroSchema: RxJsonSchema<HeroDocType> = heroSchemaLiteral;

// Example usage: create a typed collection
const myCollection: RxCollection<HeroDocType> = db.heroes;
```


## Different JSON Schema Validators

In order to validate whether a given document complies with the schema, RxDB uses pluggable validators. Historically, RxDB supports/supported:

- [ajv](https://ajv.js.org/)
- [z-schema](https://github.com/zaggino/z-schema)
- [is-my-json-valid](https://github.com/mafintosh/is-my-json-valid) (now deprecated in RxDB because it incorrectly handles large and decimal numbers)

No single validator is perfect; each has its trade-offs in performance, build size, and correctness edge cases. Some libraries rely on `eval()` or `new Function`, which can break in strict Content-Security-Policy (CSP) environments. Others have issues with large integers. RxDB encourages you to pick a validator that matches your app's needs and environment constraints.

Note that RxDB itself does not define a specific JSON Schema dialect. Instead, the range of supported dialects is determined by the JSON Schema validation plugin in use.

One important consideration is that each validator has a unique format for its **error messages**. If your application inspects validation errors and makes decisions (e.g., showing descriptive warnings to the user), be aware that switching validators later can require extensive code changes. In theory this is solved by the JSON Schema specification by the use of the [Standard Output](https://json-schema.org/draft/2020-12/json-schema-core#name-output-formatting) format, but this is not implemented in the listed schema validators.

## Performance Comparison of Validators

Performance is a critical factor in deciding whether to validate documents at runtime, especially in production environments. The following tables illustrate a basic comparison of initialization time (time-to-first-insert) and bulk insertion speed for different validators on two RxDB storages

The RxDB team ran performance benchmarks using two storage options on an Ubuntu 24.04 machine with Chrome version `131.0.6778.85`. The testing machine has 32 core `13th Gen Intel(R) Core(TM) i9-13900HX` CPU.

Dexie Storage (based on IndexedDB in the browser):

| **Dexie Storage** | Time to First insert | Insert 3000 documents |
| ----------------- | :------------------: | --------------------: |
| no validator      |        68 ms         |                213 ms |
| ajv               |        67 ms         |                216 ms |
| z-schema          |        71 ms         |                230 ms |

On Dexie Storage, the difference in time-to-first-insert is negligible, and inserting thousands of documents also shows only a modest increase in latency when using a validator. The overall overhead for 3000 inserts remains fairly small (a difference of tens of milliseconds).


Memory Storage: stores everything in memory for extremely fast reads and writes, with no persistence by default. Often used with the RxDB memory-mapped plugin that processes data in memory an later persists to disc in background:

| **Memory Storage** | Time to First insert | Insert 3000 documents |
| ------------------ | :------------------: | --------------------: |
| no validator       |       1.15 ms        |                0.8 ms |
| ajv                |       3.05 ms        |                2.7 ms |
| z-schema           |        0.9 ms        |                 18 ms |

For the Memory Storage, you see a wider variance. Although z-schema has a faster startup (time-to-first-insert) than ajv, it becomes much slower at continuous inserts (18 ms vs. 2.7 ms). This discrepancy might not matter if you rarely insert documents, but it can become significant if you have high-volume write operations.

Including a validator library also increases your JavaScript bundle size. Here's how it breaks down (minified + gzip):

| **Build Size** (minified+gzip) | Build Size (dexie) | Build Size (memory) |
| ------------------------------ | :----------------: | ------------------: |
| no validator                   |      73103 B       |             39976 B |
| ajv                            |      106135 B      |             72773 B |
| z-schema                       |      125186 B      |             91882 B |

Including a validator can substantially increase your final bundle size. For large single-page applications, an extra 30-50 KB or more of JavaScript could influence startup times, especially for users on slow networks.



## Should JSON Schema Validation Be Used in Production?

Many teams limit JSON Schema validation to development builds to avoid performance overhead in production. However, if your application deals with highly sensitive or mission-critical data, keeping validation enabled ensures data integrity and can prevent costly errors, despite the added CPU and bundle-size costs. Ultimately, the choice depends on your performance targets and  the risk of invalid data.

### Running the Validation in a WebWorker

If you must keep validation enabled in production but you have to ensure that your UI does not lack during validation, you might consider the [RxDB WebWorker plugin](https://rxdb.info/rx-storage-worker.html). This plugin runs the RxDB storage & validation in a separate Web Worker, offloading the main UI thread. While it won't reduce the absolute time spent on validation, it can help maintain a smooth UI by preventing blocking operations on the main thread.


## Learnings

Over time, RxDB has evolved its usage of JSON Schema, learning from real production experiences and feedback from the community. Here are some key takeaways:

- Avoid inlined `required` fields: In older JSON Schema dialects such as Draft 3, it was acceptable to define `"required": true` directly inside the property definition. However, more recent dialects expect required to be declared as an `array` at the parent object level. If you're using a validator based on newer specs, place your required fields in the parent-level array. If you intentionally stick to Draft 3, inlined required is still valid, but it may cause confusion if you switch to a newer validator or tool later on.

- Keep Custom Fields at the Top Level: Originally, RxDB allowed custom definitions (`index`, `encrypted`, etc.) to appear deeply nested. This caused performance hits because the library had to traverse large schema objects to find them. By placing these fields at the top level, RxDB can parse and apply them much faster, improving startup times.

- Error messages are not standardized: Each validator produces a different structure for error messages. If your app logic inspects these errors, you risk partial or complete rewrites if you ever switch validators. Decide early on which validator meets your needs and plan on sticking with it long-term. This might be solved in the future when all validators support the [standard output formatting](https://json-schema.org/draft/2020-12/json-schema-core#name-output-formatting).


## Follow Up

Using JSON Schema in RxDB has greatly simplified data definitions, tooling integration, and type inference. Although some restrictions and performance considerations come into play, the overall developer experience is significantly improved by using a well-known standard rather than reinventing the wheel.

For more information on RxDB, including further details on schema extensions and advanced plugins, check out the [official RxDB documentation](https://rxdb.info/quickstart.html).


Daniel Meyer

How RxDB embraces JSON Schema to build its NoSQL Database


When I first implemented JSON Schema, I took an approach that was probably typical for a C# developer: a blend of object-oriented and procedural programming.  However, over the past year or so, I've had an idea that I just haven't been able to escape.

A schema defines constraints on values in well-known locations inside the instance.  What if I could capture those constraints somehow and model a schema that way?  Then I would only have to do that work once, and when I finally received an instance, I would only have to evaluate each individual constraint.

I tried this several times.  Four, I think.  Each time, I got closer to getting it working, but I would always hit a roadblock that whatever design I had come up with just couldn't overcome.

But over the past couple weeks, I got it!  (And the performance boost was substantial!)

In this post, I'd like to share some of the more abstract JSON Schema analysis things that I've learned during this process.  **This one's for the implementers!**

## What are constraints?

Constraints are the building blocks of JSON Schema.  They're individual requirements that apply to specific locations within JSON data.

In this schema

```json
{
  "type": "object",
  "properties": {
    "foo": { "type": "string" },
    "bar": { "type": "number" }
  }
}
```

we have three constraints:

1. The root instance **must** be an object.
2. If there is a `foo` property, its value **must** be a string.
3. If there is a `bar` property, its value **must** be a number.

Each constraint identifies:

- where in the schema we are, the "schema location"
- how we got there, the "evaluation path"
- the instance location
- a specific requirement by one keyword

Note that none of this actually requires an instance, and we should be able to precalculate a lot of this.

This seems pretty easy to set up.  We can model each of these constraints, and when we get an instance, we test each constraint.  If they all pass, then the validation passes.

Simple, right?  That's what I thought, too.  It gets complicated in multiple ways pretty quickly.

_Please note that I don't present these as problems that need to be overcome when implementing JSON Schema; they're simply mechanics that exist._

## Keywords that have dependencies

The vast majority of keywords operate completely independently.  These are keywords like `type` and `properties`, as we saw above, as well as `maximum`, `minItems`, `title`, `format`, and a bunch of others.

But there are a few keywords that depend on (or interact with) others in order to operate.  These are:

| Keyword | Dependencies |
|:--|:--|
| `additionalProperties` | `properties`<br/>`patternProperties` |
| `contains` | `minContains`<br/>`maxContains` |
| `then`<br/>`else` | `if` |
| `items` | `prefixItems` |
| `unevaluatedItems`\* | `prefixItems`<br/>`items`<br/>`unevaluatedItems` |
| `unevaluatedProperties`\* | `properties`<br/>`patternProperties`<br/>`additionalProperties`<br/>`unevaluatedProperties` |

\* _While most keywords can only find their dependencies among their siblings, the `unevaluated*` keywords also take dependencies on keywords inside subschemas of their siblings (that apply to the same instance location)._

The `if`/`then`/`else` keywords are a good example of keyword interaction.

The procedural approach would use branching logic like

```
if ( ... )
then {
  ...
} else {
  ...
}
```

We evaluate `if`, and the result of that determines whether we evaluate `then` or `else`.

But if we think of these as constraints, then these three present a rather peculiar boolean logic:

```
valid = (if && then) || (!if && else)
```

Note how this differs from the notion that all individual constraints passing implies validation passing.  In this case, if `if` passes, then it doesn't matter whether `else` passes because it's just skipped; conversely, if `if` fails, `then` is skipped.

While the interaction for `if`/`then`/`else` is pretty straightforward to precalculate, doing so for some other keywords, like `additionalProperties`, doesn't work.  For that, you have an additional complication.

## Unknown instance locations

At the top, we defined a constraint as a requirement applied to a specific location.  However some keywords apply their subschemas more generally.  These keywords include:

| Keyword | Instance locations |
|:--|:--|
| `patternProperties` | any property that matches one of its regular expression keys |
| `additionalProperties`<br/>`unevaluatedProperties`<br/>`unevaluatedItems` | any object property not evaluated by one of its dependencies |
| `contains` | any item in an array |
| `items`<br/>`unevaluatedItems` | any array item not evaluated by one of its dependencies |

For all of these, you need the instance to determine what locations are available.  Only then can you complete the constraints.

The strategy here is to build a "constraint template."  The idea is a constraint that has a requirement and some mechanism to determine where that requirement needs to be applied once the available locations are known.  So while we can't build a completed constraint, we can still get a little of the work out of the way.

## Static references

Static references, i.e. `$ref`, can be resolved ahead of time without an instance.  They always point to the same location within the same document, regardless of the instance.  Easy mode.  Sometimes.

What if we have a recursive schema, like a schema that validates a linked list or a binary tree?  In these cases, the recursion stops when the instance no longer has any data that needs validating, i.e. when you read the end of the list or a leaf on the node.  To handle this, we can take the same "constraint template" approach as in the previous section.

_The template solution actually works for a lot of cases.  The real trick is figuring out when you need to use it._

## Dynamic references

Dynamic references, on the other hand, generally boil down to one thing: dynamic scope.  Dynamic scope is the ordered set of resource IDs (generally set by `$id`) that evaluation enters and leaves.  (Think of a stack, pushing when entering a resource, and popping when leaving.)  The dynamic scope is influenced by two things:

- where the evaluation started
- instance data

### Root schema dynamics

Last year I wrote a [post](dynamicref-and-generics) describing how to use `$dynamicRef` to model generic types in languages.  The idea went like this:

1. Start by defining a generic schema, using a `$dynamicRef` to a `$dynamicAnchor` to identify an undefined "type" parameter.
2. Define multiple secondary schemas that reference the generic one and use their own `$dynamicAnchor` to define the "type" parameter: one for each type.

Using this approach, if you start evaluation from the generic schema, evaluation will fail because the "type" isn't defined.  However, starting evaluation from a secondary schema redirects the `$dynamicRef` resolution to the one defined in the secondary schema.  This different resolution can allow an instance to pass validation.

This is a great example of a dynamic reference that _can_ be resolved without an instance present.  You only need the starting point of the evaluation.  Specifically, you need to know where the dynamic scope starts in order to identify the reference target.

### Data-driven dynamics

Another way the dynamic scope can change is through some kind of conditional logic.  [This test](https://github.com/jdesrosiers/JSON-Schema-Test-Suite/blob/06fc43d767e11369be443a60523261a3ce032423/tests/draft2020-12/dynamicRef.json#L390-L475) from the JSON Schema Test Suite is a good example.  It uses the same ideas from from the generics post, but instead of having separate schemas, everything is bundled up into one.

For this case, depending on the value of the `kindOfList` instance property, the items in the array are expected to either be numbers or strings.  Mechanically, this is determined by a set of `if`/`then`/`else` keywords that directs evaluation into either of the `numberList` or `stringList` definitions, both of which define `$dynamicAnchor: itemType` and reference into the `genericList` definition which contains `$dynamicRef: #itemType`.

When `$dynamicRef` is eventually hit, the evaluation had to either go through `numberList` or `stringList`.  This identifies which `$dynamicAnchor` is resolved.

In this case, you can't fully define the constraint until you have the instance because, while you may know the instance location, you don't know the requirements that need to apply.

I wasn't able to find a good strategy to isolate any kind of pre-work for this, and as a result, I still have to calculate all of this at evaluation time.

## Summary

These were the primary gotchas that I found when trying to change my approach to schema evaluation.  JSON Schema static analysis has proven to be a pretty interesting area of study for me, and I hope that I've piqued your interest as well.

If you'd like to find out more about how I ended up implementing all of this, I've got a summary on [blog.json-everything.net](https://blog.json-everything.net/posts/new-json-schema-net/).  It's a bit more functional now than procedural, but still very object-oriented.

There's probably a lot more to explore here, too.  If you think of something, feel free to come find me in [Slack](https://json-schema.org/slack).

_Cover image by [Google DeepMind](https://unsplash.com/@googledeepmind) on [Unsplash](https://unsplash.com/photos/ZJKE4XVlKIA)_


Static Analysis of JSON Schema


Many of you may have noticed a lot of changes happening around the JSON Schema community.  This website has had a facelift, we've started appearing at developer conferences, and we even participated in Google's Summer of Code & Summer of Docs programs.  But many of you may be asking, "What's happening with the spec?"

In this post I'll be providing an update on specification development and our new publication process.

## Publication Process

In a [previous post](./json-schema-joins-the-openjsf), we announced that we were no longer publishing our specifications through the IETF.  At the time, the intent was to join the OpenJS Foundation, however for multiple reasons, the onboarding process was not able to be completed.  As such, currently we are independent.  In the future, we may again seek to join a foundation (we may even try OpenJS again), but for now, independence seems to be the best place for us.

When we last published a specification, we used the IETF's [Internet-Draft Publication Process](https://authors.ietf.org/en/rfc-publication-process).  This meant that our documents were required to retain the "draft" moniker, even though we considered them fully fledged and ready for use in production systems.  Publishing independently, we're no longer bound by this requirement, however it does mean that we can no longer rely on the IETF document publishing infrastructure.  We have to build our own.

The foundation of our publication process is built on several ideals, and I'll cover the major ones in the next few sections.  By focusing on these pillars, we can ensure a safe and easy upgrade path for our users as the specification continues to evolve.

### Minimize Backward-Incompatible Changes Between Releases

One of the primary complaints we received about releasing a new specification over the last few iterations, particularly between Drafts 2019-09 and 2020-12, is the lack of consideration for backward compatibility.

Just to highlight a few of the breaking changes introduced with the latest version:

- `$recursiveRef` / `$recursiveAnchor` became `$dynamicRef` / `$dynamicAnchor`
- Array-form `items` became `prefixItems`
- `additionalItems` was removed; use schema-form `items` now

These changes meant that a schema written for Draft 2019-09 could validate completely differently when processed under Draft 2020-12 rules.  This creates a very poor upgrade experience.

In defending our decision to include these breaking changes, we hid behind our own advice to schema authors to include the `$schema` keyword, which identifies the dialect (specification version), instead of designing a [pit of success](https://blog.codinghorror.com/falling-into-the-pit-of-success/) that allowed them to continue their current behavior in a system that "just works".

**_Moving forward, upgrade compatibility is our #1 concern when developing a new release._**

That doesn't mean that a breaking change will _never_ happen, but we will make every effort to ensure that it doesn't.

### Maximize Forward Compatibility

**_We need to protect ourselves from potential future changes that would harm or prevent backward compatibility._**

Ironically, this means introducing a breaking change.  (I had [posted](./the-last-breaking-change) about this a couple of years ago, and the reception wasn't awesome, which led to a [follow-up post](./custom-annotations-will-continue).)

In short, to preserve future compatibility, we need to ensure that schemas can't contain extra properties which may one day be proposed as legitimate keywords.  This means that extra data in schemas can no longer be acceptable.  To accommodate our users' need to include meta-data and other values in their schemas, the specification will define a convention or namespace for property names that are to be ignored, and we guarantee that no future proposed keyword may follow this convention in order to keep the namespace protected.

As it stands today, the convention is that your property name should start with `x-`.  If you like, you can read about alternative ideas and how we decided on this approach in [the ADR](https://github.com/json-schema-org/json-schema-spec/blob/main/adr/2023-04-sva-prefix.md).

### Formally Define a Feature Proposal Process

In the past, new features would just be added to the specification and released with the next version.  We held it as acceptable that a feature may not be fully finished when we released the specification.  We expected to receive feedback, and the feature would be updated in the next version.

However, this approach puts a large burden on tooling maintainers who want to support multiple versions of the specification, which is basically all of them.  They'd have to support many different behaviors, sometimes for the same keyword.

The perfect example of this is Draft 2019-09's `$recursiveRef` & `$recursiveAnchor`.  The idea was solid and had a lot going for it.  In 2019 when we were working on it, we developed the concept as best we could, but we needed actual user feedback, so we added it to the specification, knowing that it wasn't quite complete.  Prior to releasing the specification, a few of us who had implementations also added it as an optional feature to let our users try it out.  But after the specification was released, we found that many users were confused as to how it should work, and we identified several ways that we could make it better.  Those improvements became `$dynamicRef` & `$dynamicAnchor` in Draft 2020-12.  Even though the "recursive" keywords weren't part of the latest specification, many implementations wanted to support Draft 2020-12 while continuing to support Draft 2019-09.  Furthermore, schemas which contained the "recursive" keywords couldn't be processed under the Draft 2020-12 rules, which takes us back to the backward compatibiility problem.

**_We need a feature proposal process that encourages tool support while also allowing continued feature development._**

By introducing a feature proposal process, all tooling maintainers can add support as they elect to do so.  And because these proposals aren't considered a hard requirement of the specification, we can continue to iterate on them (including breaking changes).  Finally, by actively encouraging tooling support, we can obtain the real-world feedback that we need to ensure that the feature is the best it can be before we officially integrate it into the specification.

## Compatibility with Draft 2020-12

To close this out, and with all of the above in mind, I want to cover what the upcoming release will entail as it stands now.

First, the incompatibilities:

- As mentioned before, we won't be supporting arbitrary properties anymore.  Any extra data you wish to include will need to use the `x-` property name convention.
- The `$dynamicRef` / `$dynamicAnchor` referencing system will be completely disjoint from the `$ref` / `$id` / `$anchor` system.  In Draft 2020-12, `$dynamicRef` could sometimes fall back to a normal `$ref` behavior.  This has been very confusing, so we've just isolated the two referencing systems.  This is _technically_ a breaking change, but we're pretty certain no one ever used `$dynamicRef` expecting it to work like `$ref`.
- `format` will validate by default.  There was a lot of [discussion](https://github.com/json-schema-org/json-schema-spec/issues/1520) on this topic, but the deciding factor was that users generally expect it to validate, and we wanted to meet that expectation.
- Vocabularies are being demoted to a feature proposal.  This was one of those ideas that wasn't completely finished when it was added to the spec in Draft 2019-09.  It's likely that this feature will change substantially before being merged back into the specification.  As a proposal, the feature can still be supported as development continues, but it's not a hard requirement.

Additionally, but not breaking:

- The recommended output format introduced in Draft 2019-09 has been extracted to its own specification.  The two driving factors for this decision were that different output formats could be more beneficial to different consumers and having the output versioned separately from the specification would more easily allow them to evolve independently.
- Many keyword interaction behaviors were defined by the annotations that were produced at various levels of evaluation.  This led to some confusion and unnecessary requirements, so we've loosened the language around this to allow tooling to figure out how they want to implement the behavior rather than prescribing a particular approach.
- General clean-up and clarifications to more accurately define behavior.

Beyond that, we still have a lot to do, but not nearly as much as we had this time last year, and we still have to figure out the actual publication piece, i.e. how we want it on the website.  We have the IETF-published documents mirrored here, and the new ones will likely go up next to those.  It's just another thing we need to do.  You can track our list of items on the [GitHub project board](https://github.com/orgs/json-schema-org/projects/15/views/1).

It's been four years since we published a new version, but we're getting really close, and I'm very excited.  Stay tuned!

_Cover image is of Mt. Taranaki in New Zealand, by [Sophie Turner](https://unsplash.com/@sophie_turner) on [Unsplash](https://unsplash.com/photos/mountain-near-body-of-water-during-daytime-LZVmvKlchM0)._


Moving Toward a Stable Spec


Recently, I've had a few conversations that got me thinking about JSON Schema's place architecturally.  Today I'd like to share some of those thoughts.

## What _is_ JSON Schema?

JSON Schema is a specification that defines a mechanism to describe a JSON value and can be used to validate and annotate the value.  This mechanism is organized into a number of keywords, each of which provides a defined behavior.

A single set of keywords comprises a "schema."

This system takes an input of a schema, which itself is representable as a JSON value, and a JSON value (which we call the "instance") to which the rules in the schema will be applied.  For the purposes of this post, we'll call the application of these rules "evaluation."  (i.e. A schema "evaluates" an instance.)

The output of an evaluation is an aggregation of the individual errors and/or annotations of each of the rules.

However, in general, a system of behaviors is somewhat abstract, and so it's not practically useful.  What we need is a realization of this system in code.  What we need is an implementation.

## Making JSON Schema useful

Before we can make JSON Schema useful, we need to ask who it should be useful for.  To answer _that_ we need to know why JSON Schema exists in the first place.

Well, we visited that in the previous section: a schema evaluates an instance to ensure that the instance complies with all of the rules represented by the schema.  If the instance does comply with the rules in the schema, then we say it is "valid" against that schema.

The reasons to ensure we have valid JSON data could be numerous: checking data before deserializing into programmatic models, checking form inputs before submission, etc.  **These are the needs of applications.**

So _applications_ are the consumers of JSON Schema.

But what about annotations?  Well, annotations are specifically _intended_ for applications so that they can provide additional behavior.  So, yeah, applications are still the consumers here.

But an application can't consume a specification without that specification being realized into code.  That's where implementations come in.

An implementation of JSON Schema is an embodiment of the specification that is directly consumable by an application.

## A nuance

Something I've seen a lot of lately, and I think the source of some of the confusion that has arisen in my discussions, is that an application may be merely an executable wrapper around an implementation.  This tends to give the appearance that the application itself is the implementation, in which case the application would be subject to the requirements of the specification.  But I don't see it that way.  Even in these cases, there exists a distinction between the application and the implementation, even if that distinction is, in practice, really blurry.

_**NOTE** It's important to recognize that this distinction is **actually** blurry.  This is currently an open point of discussion, and nothing has been officially defined in this area._

Applications tend to have three basic components:  an interface (UX or API), some business logic, and data persistence.  **All** applications have an interface.  However, the business logic and data persistence components are optional to the degree that you can have one or the other or both.  (An application with only a UX is generally not very useful.)

An application may only provide an interface over data persistence (e.g. a Postgres web service), meaning that there's no need for any business logic.  Conversely, another application may provide a computational service (e.g. image processing) where there's no need to persist data.

For the recent conversations I've had, this second scenario seems to be the case: an application is created that just evaluates instances against schemas.  But this doesn't mean that the application and implementation *are* the same thing.

Within these applications the implementation, which for these applications _is_ the business logic, is still a separate component from the interface.  And it's important to recognize that JSON Schema as a specification can only cover the implementation part.

## Why any of this matters

It all comes back to what I touched on this in the opening section: JSON Schema needs to define inputs and outputs.  This comprises a minimal API that implementations and applications can use to communicate with each other.

When the line between implementation and application is blurred, it's natural to think that the specification is imposing requirements on how the application communicates with its users.  But that's not the case.

It is impossible for JSON Schema to know the needs of an application's users, and so it's impractical for the specification to attempt to define input and output requirements to which applications must adhere.

Users of different applications have different needs.  Even when you consider two applications that essentially just provide a UX for implementations, say a web app and a CLI, the UX needs of their users are vastly different, despite the two applications doing basically the same thing.

## The scope of the specification

As a result of everything discussed above, it follows that the specification's input and output requirements are only applicable when there is a clear communication seam between an application and a JSON Schema implementation.

The specification recognizes that programming languages and frameworks likely will not be dealing with textual JSON, but rather they will use data models defined within the limitations of that language.  As such, it defines input and output in terms of abstract JSON data and JSON Schema models so that implementations are free to use what they have at their disposal.

Specifically, these requirements only pertain to standalone implementations that are provided as general-use representations of the JSON Schema specification to be consumed by unknown parties.

Applications which have integrated implementations or application/implementation pairs which have specialized contracts need not adhere to these requirements because these arrangements are out of scope of the specification.

_Cover photo by me_ 😁

The Architectural Scope of JSON Schema


As we continue to [move toward a stable specification](https://json-schema.org/blog/posts/future-of-json-schema), we have been analyzing the various components, behaviors, and features of JSON Schema to determine what can be included, what may need modification, and what (if anything) needs to be removed or replaced.  The primary question that we had was whether there were any planned changes that would cause problems for users with existing schemas, i.e. breaking changes.

To answer that, we needed user input.  So we posted across the internet in order to encourage users to participate in a [particular conversation on GitHub](https://github.com/orgs/json-schema-org/discussions/295) around how we treat any supposed breaking changes for the upcoming release.  The response was very much, "If you can help it, please don't break things."  But many people also said they would be less bothered if there was a defined migration path and tooling to help.

We also performed an internal survey of feature stability, involving all of the JSON Schema core team members.  We're pleased to report that the vast majority of Draft 2020-12 can be kept completely as-is.  There are a few keywords and behaviors that could use some tweaking, but mostly in ways that we feel are generally compatible with the current release.

Another common sentiment among both users and team members was that JSON Schema already has a history (and reputation) of including breaking changes when releasing new versions of the specification, meaning that a schema that validates properly with one version may not validate properly with the next.  This perception is one that we need to address.  To do that, **we need to orient ourselves and the specification so that we can promise schemas written to one version will validate consistently with every subsequent version**.

Unfortunately, in order to make that promise, there is one behavior that we absolutely must change in a breaking way: support for unknown keywords.

## What are unknown keywords?

A keyword is "known" if it is defined by a vocabulary listed in the schema's meta-schema (identified by the value in `$schema`).  By recognizing this vocabulary and continuing to process the schema, an implementation declares that it understands how to process all of the keywords that vocabulary defines.  Any keyword which is not defined by a vocabulary listed in the schema's meta-schema is considered "unknown."

All of the currently published versions of JSON Schema instruct implementations to (at a minimum) ignore keywords that are not recognized.  With the introduction of annotations in Draft 2019-09, implementations were given the option to collect the values of unknown keywords and report them to the user in the output.  This behavior was widely supported by the community because it meant that they could annotate and document their schemas without having to worry that their validations would be impacted.  For a schema like

```json
{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "type": "object",
  "properties": {
    "firstName": {
      "type": "string",
      "database-field-id": 1234
    }
  }
}
```

a validator would either ignore `database-field-id` or include it and its value as an annotation in the output.  One can see how this feature would be extremely helpful.  However, this feature also blocks our ability to promise future compatibility.

## Why can't we still support this?

Suppose a user wrote the example schema above while using a validator written to conform to the 2023 version of the specification.  A couple years later, they want to upgrade the validator now that supports the 2025 specification.  However in that 2025 specification, we added `database-field-id` as a keyword, and its value is expected to be a string.  Suddenly the user's schema is no longer valid.  **We've broken a user by adding a new keyword.**

In order to prevent this scenario, we are forced to forbid keywords that are not declared by a listed vocabulary.  We recognize that this will break a lot of people up front.  However we feel that a promise of future-compatible specifications should take precedence over the immediate pain of having to change your schemas yet again (and potentially again perpetually into the future).

## What is being done to soften the blow?

We have an [open discussion](https://github.com/orgs/json-schema-org/discussions/329) on this very topic, and we invite you to weigh in.

Current proposals include things from simple prefixes on unknown keywords as a convention to indicate custom annotations to more complex solutions like inlined and ad-hoc vocabularies bundled into the schema that can define annotation-only keywords, like `title` and `readOnly`.  These options are effectively ways for the schema author to say, "I know these keywords aren't declared by any vocabulary, but I'd really like them in my schema.  Please disregard them."

## Will there be any other breaking changes in the next release?

Hopefully not.  We believe that removing support for unknown keywords will be the only change of any real negative impact, but that doesn't preclude the possibility of other changes that break things in less critical ways.

We recognize that making breaking changes like this is hard on developers, both users and implementors, and so we're doing our best to measure the changes that are needed and seek alternatives to breaks where possible.  We don't see this as an opportunity to go wild with changes.

In any case, we will be open and transparent about any changes that we make, breaking and otherwise.

## Sum up

It looks like we have to break some schemas with the next version.  However by doing so, we are able to promise that we won't in the future.

We have to break it in order to fix it.

_Cover photo by [Ken Suarez](https://unsplash.com/@kensuarez) on [Unsplash](https://unsplash.com/photos/4IxPVkFGJGI)_


The Last Breaking Change


## Challenge

With multiple government clients, Tyler Technologies faces the continual challenge of meeting stringent differing requirements defined in law. Building custom rules for each client was very time-consuming, and Tyler Technologies were seeking to create a multi-tenancy application solution that could be customised for each client without needing custom code.

Tyler's multi-tenant application solution needed an easy way to define custom data and validation rules, giving power to the client, rather than having to make heavy investment in custom development work.

"This way we can spend our time providing business value and innovating the application instead of handling all the custom work manually." - Andres Moreno, Lead Software Engineer at Tyler Technologies.

## Solution

JSON Schema allowed Tyler Technologies to create and maintain different model definitions for each client in a simple and understandable format.

JSON Schema was chosen as a viable solution for defining data structures. Tyler found wide adoption of the standard across the industry, partial familiarity from using OpenAPI, and broadness of open-source tools providing rich functionality.

"With JSON Schema, Formly, and AJV we were able to get the functionality we needed to render and validate fields dynamically for each tenant without requiring a developer to do it!" - Andres Moreno

Formly allows for the dynamic generation of forms from JSON Schema, enabling each client to have unique forms based on their unique definitions. AJV allowed for the validation of data from those same forms on the server-side.

"We are focused on API-First development, so we needed a way to allow validation on the server-side for when our API was used directly rather than through the generated forms," Andres continued.

Further, AJV has options to take advantage of annotations such as the `default` keyword, filling in missing attributes, and optionally filtering out data not defined in the schema. These options allowed Tyler Technologies to provide additional value to customers, catering to broader requirements.

## Impact

The effort required to develop and deliver custom work for customer-specific requirements was reduced. Existing schemas can now be easily updated using APIs, without the need to wait for a full deployment cycle.

A new custom field would need to be fully identified and get approval, sent to the development team, verified by QA, and finally deployed to the production environment. "This process can span multiple days or even weeks," explains Andres.

Rather than a full development cycle and release process, updating a JSON Schema takes just a few minutes by those that know JSON Schema, dramatically reducing the time to resolution of bugs related to custom data structures.

## Key Impact Results
<Bigquote>
Showing added value to clients in minutes, rather than days or weeks.
</Bigquote>

"From what used to be a multi-day/week process, to being able to get something in front of the client within minutes," says Andres.

## Tyler Technologies

<p className="text-2xl my-10">Tyler Technologies is a leading provider of end-to-end information management solutions and services for local governments. In recent years they've been playing a key role in the digital transformation of the public sector.</p>

<div className="container flex flex-col sm:flex-row sm:space-x-4 space-y-4 sm:space-y-0 place-content-between mb-4">
  <figure className="group m-auto">
    <img className="flex-1 " src="/img/posts/2022/tyler-technologies-case-study/2187563_0051.webp" />
  </figure>
  <figure className="group m-auto">
    <img className="flex-1" src="/img/posts/2022/tyler-technologies-case-study/2187563_0278.webp" />
  </figure>
</div>

"We empower the public sector to create smarter, safer, and stronger communities. Our team is providing the most innovative solutions to the processes each of our customers have without reinventing the wheel; We choose to assemble instead of building everything ourselves."

Custom development work for each client is a big investment, and a more cost-effective and performant approach was required.

## Benefits in development

<p className="text-2xl my-10">"We are very confident that JSON Schema has solved the challenges that we had faced as we started these projects" - Andres Moreno</p>

Initially using Formly's proprietary configuration format to dynamically generate forms, Tyler needed validation of the resulting data on the server-side. Creating different configurations for the client-side and server-side, which needed to match, wasn't going to give Tyler the scalability they needed.

While investigating, they discovered that Formly also supported configuration using JSON Schema. The same JSON Schema could be used for validation on the client and server sides. With minimal research and prototyping, it became clear using JSON Schema was going to be useful.

After demonstrating how JSON Schema could be easily used to build an extensible application, other teams at Tyler saw the appeal of using a single configuration to both dynamically generate forms and perform validation on the submitted data.

JSON Schema is now a core concern when developing existing and new applications.

"We are very confident that JSON Schema has solved the challenges that we had faced as we started these projects. We can say with certainty that we are going to continue using JSON Schema as the core of the application for any future applications that are built using the cloud infrastructure that we have put in place," explains Andres.

Thank you to Andres Moreno, Lead Software Engineer, and Tyler Technologies, for allowing and enabling us to share this case study with you.


How Tyler Technologies reduced its client feedback loop with JSON Schema

Understanding JSON Schema Lexical and Dynamic Scopes


Starting with the release of OpenAPI 3.1, the dialect of JSON Schema used in
OpenAPI documents is configurable. By default, you get the OpenAPI 3.1 Schema
dialect, but you can choose to use draft 2020-12 or any other dialect if you
choose. This brings up the question of how to validate an OpenAPI 3.1 document
if one of its components (JSON Schema) is open-ended. In this article, we'll
cover how to configure the default JSON Schema dialect of an OpenAPI 3.1
document and how to validate that document, including JSON Schemas, no matter
which dialect(s) you choose to use.

## What is a JSON Schema dialect?

Because not everyone is familiar with the term "dialect" in this context, let's
take a moment to define it before moving on. A JSON Schema dialect is any unique
incarnation of JSON Schema. This includes any official release of JSON Schema
such as draft-07 or draft 2020-12, but it also includes custom versions of JSON
Schema. OpenAPI has effectively had three dialects of JSON Schema introduced
with 2.0, 3.0, and 3.1. JSON Schema Dialects are compatible with the core
architecture of JSON Schema but may add keywords, remove keywords, or modify the
behavior of keywords.

## The OpenAPI 3.1 Schema dialect

By default, schemas in OpenAPI 3.1 are assumed to use OpenAPI 3.1's custom JSON
Schema dialect. This dialect includes full support for all draft 2020-12
features plus a few additional keywords and `format` values.

## Validating with the default dialect

There are two schemas available for validating OpenAPI 3.1 documents.
`https://spec.openapis.org/oas/3.1/schema` includes all the constraints for
validating the document except for schemas. You aren't expected to validate your
OpenAPI documents against this schema by itself. Think of this schema as an
abstract schema that is intended to be extended to include schema validation
support for the JSON Schema dialect you're using.

That's why there's also `https://spec.openapis.org/oas/3.1/schema-base` that
extends the abstract schema with validation support for the OpenAPI 3.1 Schema
dialect. If you're using plain out-of-the-box OpenAPI 3.1, this is the schema
you want to validate your document against. If you want to use a different
dialect, keep reading to see how to extend the main schema to get validation
support for your chosen dialect.

This is made possible by dynamic references which were added in JSON Schema
2020-12. The details of how dynamic references work is out of scope for this
article, but we'll cover enough for you to make your own concrete schemas for
any dialect you choose to use in your OpenAPI 3.1 documents.

### Examples

These examples use
[@hyperjump/json-schema](https://www.npmjs.com/package/@hyperjump/json-schema)
to validate OpenAPI documents. Beware that dynamic references are a relatively
new feature of JSON Schema and many validators don't yet support them, or have
limited support, or have bugs.

#### Without schema validation
```javascript
import { validate } from "@hyperjump/json-schema/openapi-3-1";

const validateOpenApi = await validate("https://spec.openapis.org/oas/3.1/schema");

const example = YAML.parse(await readFile("./example.openapi.json"));
const result = validateOpenApi(example);
console.log(result);
```

#### With OpenAPI Schema dialect schema validation
```javascript
import { validate } from "@hyperjump/json-schema/openapi-3-1";

(async function () {
  const validateOpenApi = await validate("https://spec.openapis.org/oas/3.1/schema-base");

  const example = YAML.parse(await readFile("./example.openapi.json"));
  const result = validateOpenApi(example);
  console.log(result);
}());
```

### How does it work?

To get an idea about how this works, let's take a look at a few selections from
of the OpenAPI 3.1 schemas.

This is where the Schema Object is defined. The `$dynamicAnchor` declares this
sub-schema to be something that can be effectively overridden by another schema.
If it's not overridden, the default behavior is to validate that the value is an
object or a boolean. No other validation is performed on the schema.

```yaml
$defs:
  schema:
    $dynamicAnchor: meta
    type:
      - object
      - boolean
```

When something in this schema wants to reference the Schema Object, instead of
referencing `#/$defs/schema` like normal, it uses a dynamic reference to the
"meta" dynamic anchor set in the previous selection. Now instead of always
resolving to `#/$defs/schema`, another schema can potentially override where it
resolves to.

```yaml
$defs:
  components:
    type: object
    properties:
      schemas:
        type: object
        additionalProperties:
          $dynamicRef: '#meta'
```

## Validating Schema Objects against the default dialect

With those vague building blocks in mind let's derive a schema that "extends"
the abstract schema to create a schema that validates Schema Objects using the
default dialect meta-schema.

The first step is to include the abstract schema.

```yaml
$schema: 'https://json-schema.org/draft/2020-12/schema'

$ref: 'https://spec.openapis.org/oas/3.1/schema/latest'
```

Then we need to add a `$dynamicAnchor` that matches the one in the abstract
schema to override where dynamic references to "meta" will resolve to. From
there we can reference the meta schema for the default dialect.

```yaml
$schema: 'https://json-schema.org/draft/2020-12/schema'

$ref: 'https://spec.openapis.org/oas/3.1/schema/latest'

$defs:
  schema:
    $dynamicAnchor: meta
    $ref: 'https://spec.openapis.org/oas/3.1/dialect/base'
```

That's enough to get the Schema Object validation we were after, but there are a
few loose ends we'll want to tie up as well. The `jsonSchemaDialect` field in
the OpenAPI 3.1 document can be used to change the dialect used. Since this
schema only supports the default dialect, we want to restrict people from
changing that to something else. If they need to change it, they'll need a
different schema to validate against. We also don't want people using the
`$schema` keyword to change the dialect of individual schemas.

```yaml
$schema: 'https://json-schema.org/draft/2020-12/schema'

$ref: 'https://spec.openapis.org/oas/3.1/schema'
properties:
  jsonSchemaDialect:
    $ref: '#/$defs/dialect'

$defs:
  dialect:
    const: 'https://spec.openapis.org/oas/3.1/dialect/base'
  schema:
    $dynamicAnchor: meta
    $ref: 'https://spec.openapis.org/oas/3.1/dialect/base'
    properties:
      $schema:
        $ref: '#/$defs/dialect'
```

With that, we have exactly what you'll find in the official
`https://spec.openapis.org/oas/3.1/schema-base` schema.

## Supporting multiple dialects

With the adoption of JSON Schema 2020-12 came support for the `$id` and
`$schema` keywords, which together allows us to override the default JSON Schema
dialect for a schema. Let's assume we have an OpenAPI 3.1 document where we use
JSON Schema 2020-12 by default, but we also have some legacy JSON Schema
draft-07 schemas that we want to use as well.

```yaml
jsonSchemaDialect: 'https://json-schema.org/draft/2020-12/schema'
components:
  schemas:
    foo:
      type: object
      properties:
        foo:
          $ref: '#/components/schemas/baz'
      unevaluatedProperties: false
    bar:
      $id: './schemas/bar'
      $schema: 'http://json-schema.org/draft-07/schema#'
      type: object
      properties:
        bar:
          $ref: '#/definitions/number'
      definitions:
        number:
          type: number
    baz:
      type: string
```

### What's going on here

First, we use the `jsonSchemaDialect` field to set the default dialect for the
document. By setting the default dialect to JSON Schema 2020-12, by default,
schema will not understand the keywords added in the OpenAPI 3.1 vocabulary such
as `discriminator`. Only standard JSON Schema 2020-12 keywords will be
recognized.

The `/components/schemas/foo` schema is understood to be interpreted as JSON
Schema 2020-12 because that's what we set to be the default.

The `/components/schemas/bar` schema changes the dialect of that schema to be
JSON Schema draft-07. There are a couple of things working together to make this
possible. The `$schema` keyword sets the dialect for the schema, but `$schema`
is only allowed at the root of the document it appears in. That's why we also
need to include the `$id` keyword. The `$id` keyword effectively makes that
schema a separate document with its own identifier and that location as the
root. It's an independent document embedded inside the OpenAPI 3.1 document. You
can think of it like an iframe in HTML.

A consequence of this is that `/components/schemas/bar` can't use a local
reference like `#/components/schemas/foo` to reference another schema in
`/components/schemas` because it's now technically in a different document.
There are two ways to get around this. One option is to use an external
reference to the OpenAPI 3.1 document, such as
`myapi.openapi.yml#/components/schemas/foo`. The other option is to give
`/components/schemas/foo` an `$id` as well and reference that instead,
`./schemas/foo`.

### Validating

Now that we understand how this works, let's derive the schema to validate an
OpenAPI 3.1 document with JSON Schema 2020-12 as the default dialect and JSON
Schema draft-07 as an allowed alternative.

```yaml
$schema: 'https://json-schema.org/draft/2020-12/schema'

$ref: 'https://spec.openapis.org/oas/3.1/schema'
properties:
  jsonSchemaDialect:
    const: 'https://json-schema.org/draft/2020-12/schema'
required:
  - jsonSchemaDialect

$defs:
  schema:
    $dynamicAnchor: meta
    properties:
      $schema:
        enum:
          - 'https://json-schema.org/draft/2020-12/schema'
          - 'http://json-schema.org/draft-07/schema#'
    allOf:
      - if:
          properties:
            $schema:
              const: 'https://json-schema.org/draft/2020-12/schema'
        then:
          $ref: 'https://json-schema.org/draft/2020-12/schema'
      - if:
          type: object
          properties:
            $schema:
              const: 'http://json-schema.org/draft-07/schema#'
          required:
            - $id
            - $schema
        then:
          $ref: 'http://json-schema.org/draft-07/schema'
```

The first change is that the `jsonSchemaDialect` field is now required because
we are no longer using the default.

Next, we have to update the schema definition to only allow `$schema` values for
the dialects we want to allow.

The first `if`/`then` will validate the schema as a JSON Schema 2020-12 schema
if there is no `$schema` keyword used, or `$schema` is set to JSON Schema
2020-12. Of course it's unnecessary to use `$schema` in this case, but it is
allowed.

The second `if`/`then` will validate the schema as a JSON Schema draft-07 schema
if there is an `$id` and a `$schema` indicating draft-07.

You can extend this pattern for any number of dialects you want to support.

Photo by <a href="https://unsplash.com/@gonchifacello?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Gonzalo Facello</a> on <a href="https://unsplash.com/collections/8799947/json-schema-posts/c9d98f0b917af1758a609d674392502f?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a>


Validating OpenAPI and JSON Schema


## Challenge

The Internet of Things (IoT) is the network of physical "smart" objects that
exchange data with other devices over the Internet. While communication and
interoperability are by definition the crux of the Internet of Things, the
emergence of custom or proprietary solutions results in devices that cannot
talk to each other due to differences in data interchange mechanisms.

To integrate these disparate devices, developers must work with a growing set
of protocols, serialization formats and API specifications. This results in
repetitive, non-scalable and error-prone work that is difficult to automate.

While technologies like OpenAPI and AsyncAPI largely solve this problem in the
context of Web APIs, they fall short for describing networks of non-HTTP and
multi-protocol devices, and do not consider different modes of interaction
based on their meaning in the physical world.

## Solution

To solve these problems, the W3C Web of Things works on providing standardized
building blocks that make use of JSON Schema.

JSON Schema is used for validating descriptions of network-facing capabilities
of physical entities called [Thing Descriptions][thing-description], and to
model and describe data sent by Internet of Things consumers and producers in a
multi-protocol manner.

The W3C Web of Things specifications are on board with JSON Schema since Draft
4, and even the first draft versions already catered to data models of Internet
of Things devices.  "Currently, we are on JSON Schema Draft 7 and expect to
move to JSON Schema
[2020-12](https://json-schema.org/specification-links.html#understanding-draft-names-and-numbers)
or a newer one when starting our new charter in February 2023," said Ege
Korkan, Research Scientist at Siemens and W3C Specifications editor.

"We observe JSON Schema becoming more and more stable over the years and
Postman's support to the community gives us even more confidence on it,"
continued Ege Korkan.

JSON Schema is delivering exciting features, and more importantly these
features are becoming more and more consistent across implementations.  For
example, Ege Korkan added, "JSON Schema vocabularies are very promising and we
plan to explore them further in conjunction with Semantic Web technologies."

## Impact

The W3C Web of Things relies on JSON-LD for linking vocabularies and ontologies
that already exist on the web, such as [SAREF][#saref], [Units of
Measure][#units-of-measure] and [Schema.org][#schema-org]. Being based on the
JSON data model, JSON Schema fits well with JSON-LD when integrating both
technologies as part of a single specification.

Adopting JSON Schema means that the W3C Web of Things does not need to invest
effort on inventing another schema language. As a consequence of the popularity
of JSON Schema, existing tooling can be often reused instead of implementing
custom parsers, validators, code generators and UI generators for all the
programming languages that the community might be interested in.

JSON Schema has proven to be a foundational block for creating higher-level
tooling that is specific to the W3C Web of Things. As a notable example, the
community has developed a tool called
[testbench](https://github.com/tum-esi/testbench) that makes use of JSON Schema
definitions inside Thing Descriptions to generate matching payloads for the
purpose of stress and penetration testing.

Given the growing popularity of using JSON Schema to generate web-based forms,
W3C Web of Things specifications can be used to automatically generate
dashboards and user interfaces to interact with the objects modeled by Thing
Descriptions.

## Key Impact Results

The ultimate metric of success for a standard like the W3C Web of Things is
adoption, and community adoption is not something you can just "engineer".  Ege
Korkan commented, "since we adopt JSON Schema, developers who are familiar
with it have a smaller learning curve when learning the W3C Web of Things
standard."

"During my PhD times, my MSc engineering students had to become familiar with
the W3C Web of Things. These students typically took 1 week to go through my
[tutorial](https://github.com/tum-esi/wot-tutorial). If they knew JSON Schema,
that 1 week would become 2 days", said Ege Korkan.

## W3C Web of Things

The World Wide Web Consortium (W3C) is the standardization organization behind
most of the standards used in the Web. The W3C Web of Things Working and
Interest Groups work on the standardization of Web of Things at W3C.

The Web of Things groups aim to create an interoperability layer so that
different Internet of Things platforms, protocols and standards can operate
together.

The work started as a discussion in the Web of Things Community Group in
2013-2014.  It proceeded to become an Interest Group in 2015 that
collected the use cases and defined the standards to be worked on. Since 2016,
the working group is working on different standards on the Web of Things with
the first publications of the Thing Description and Architecture
recommendations in 2019 and more that will be published by the end of this
year.

At the time of this writing, the Working Group consists of 96 participants
representing 36 organizations and the Interest Group consists of 131
participants representing 48 organizations. 

![W3C Web of Things Examples](/img/posts/2022/w3c-wot-case-study/wot-examples.webp)

## Getting Started

The W3C Web of Things Working Group invites you to leverage your existing JSON
Schema knowledge to play with the Internet of Things through the use of the W3C
Web of Things specifications and tools. You might already have a smart device
next to you, just that it does not come with a Thing Description!

If you write your own Thing Description, you can programatically interact with
your device by using the [Node.js Web of Things
implementation](https://github.com/eclipse/thingweb.node-wot), or play with it
through the
[Node-RED](https://www.npmjs.com/package/node-red-contrib-web-of-things)
integration.

To learn more, head over to the
[documentation](https://www.w3.org/WoT/documentation/), watch the [introductory
videos](https://www.w3.org/WoT/videos/), have a look at the many examples
provided as part of the [Thing Description specification](#thing-description)
and try them out on the [Thing Description
Playground](http://plugfest.thingweb.io/playground/) app.

Thank you to Ege Korkan, Web of Things Researcher at Siemens AG, Munich, the
W3C Web of Things Working and Interest Groups and Coralie Mercier from the W3C
Marketing and Communications Team for allowing and enabling us to share this
case study with you.

Cover image: [Hello WoT](https://bind.systems/tags/web-of-things/) © 2022 by desertmonitor OÜ is licensed under CC BY-ND 4.0

[thing-description]: https://www.w3.org/TR/2020/REC-wot-thing-description-20200409/
[saref]: https://saref.etsi.org
[units-of-measure]: https://bioportal.bioontology.org/ontologies/UO
[schema-org]: https://schema.org


How the W3C Web of Things brings JSON Schema to the Internet of Things


I grant you, 7 days isn't an ideal length of time to do an analysis over, but this is what we have for now. Previously, we had basic website analytics, but that only tells a fraction of the story. We run the website on GitHub pages with Cloudflare in front, which provides 7 days of analytics, critically, including direct access traffic.

The JSON Schema website hosts the official meta-schemas. If you're not sure what that means, meta-schemas are JSON Schemas for JSON Schema. Why is this interesting? Well, we host all historical versions of the meta-schema, not just the most recent. And while we don't expect code to be making requests on a regular basis to download the meta-schemas, the number of requests strongly suggests that they do.

Why now? In addition to providing meta-schemas at versioned URLs, the web site has, and continues to, provide the latest meta-schema at `/schema`. We currently advise people not to be accessing this! We put in a delayed redirect with a message warning people to avoid it, but realistically, we expect most access will be via code and not in a browser. (This means things are likely broken. More on that later.)

Maybe now we can remove hosting that URL? For reasons out of the scope of this article, using it really isn't a great idea, and will probably produce results you don't expect. As mentioned, web analytics won't tell us what's happening outside of a browser, so we needed something else… We were already using Cloudflare, so decided to submit for the open source free pro tier plan… while we wait, the cost isn't huge, so I decided we should look at the data now.

The data in this article is a seven-day snapshot taken at 2023-07-21.
(At the end of the article, I'll share an image from this snapshot, and one from 2023-07-31 for comparison.)

- **Caveat 1**: Clients should not be making requests to meta-schemas as part of their production code. It may be appropriate as part of a CI/CD system, but this is also unlikely. Instead, clients should store a local cached copy. Any noticeable number of requests strongly suggests little or no caching.

- **Caveat 2**: The traffic analytics data is based on a sample of the total traffic. Each stat uses a different percentage. Anywhere from 0.16%, to 16.67%. I guess there's a lot of numbers to crunch and logs to keep to do a true analysis. You can ask for logs to be piped to a third party… but you need the enterprise plan. Regardless, the insights are still pretty interesting.

- **Caveat 3**: The "top" of each category of data is limited to 15 items in the UI. I think we could use their API to access additional data, but that's something I couldn't figure out in 5 minutes, so let's use what we have today.

Oh yes, and we are limited to the last SEVEN days worth of data.

## What are we sharing here?

It's best if we start with some splashy numbers, then go into what is exactly being accessed and why that's interesting, and finally just some interesting discoveries.

### The big numbers

51.92 million requests

211.47GB of data transfer

1.05 million page views

1.01 million visits

### All the meta-schemas!

About 51 million of those requests are made from something that doesn't declare a browser, client, or bot name. We see Chrome and Firefox take the top spots, and lower down we see CURL and various indexing bots.

In terms of what paths are accessed, and therefore what meta-schemas, I found the results interesting and insightful.

The top result, at 11.82 million, is `/draft-07/schema`. This suggests that draft-07 JSON Schema is the most popular passed to misbehaving implementations. (Most well behaved implementations won't go download the meta-schemas, because they already have it! This includes the most popular implementations!)

<figure className="mt-10">
  <img className="mx-auto" src="/img/posts/2023/website-analytics-snapshot-2023/Screenshot 2023-07-21 at 12.12.31.webp"/>
  <figcaption className="mt-2 mb-10 text-sm text-center text-gray-500">Web traffic snapshot 2023-07-21."</figcaption>
</figure>

Draft-04 doesn't appear till number 4 in the list. Ranks number 2 and 3 are taken by Draft 2019-09, but not as we might expect… the files there, `links` and `hyper-schema` are for Hyper Schema, as opposed to the primary validation use case of JSON Schema.

## Someone's accessing Hyper Schema?

Remember, we previously published JSON Hyper Schema in addition to our core offering. Hyper Schema is a hypermedia specification. Specifically, in relation to APIs, [Hypermedia as the engine of application state (HATEOAS)](https://en.wikipedia.org/wiki/HATEOAS). (For those interested, further reading: [Richardson Maturity Model](https://martinfowler.com/articles/richardsonMaturityModel.html)). We decided to pause JSON Hyper Schema, as we didn't have the resources to champion progressing the specification, around the same time as wrapping up draft 2020-12. (We can ascribe the difference between `hyper-schema` and `links` to the sample used. It's reasonable to assume both were accessed sequentially.)

Where are these requests for Hyper Schema meta-schemas coming from? While the physical location is uninteresting, we can look at the user agent provided. The top two, which covers 7.5 million requests, doesn't really help… `python-requests`. At most, we can see that version 2.27 and 2.28 are roughly in equal use. After that, we jump down to 635k requests, from something identifying as `neptune-client`. Curiosity piqued!

They call it an "Experiment tracking tool and model registry". This is to do with Machine Learning (ML) Ops (MLOps), built for data scientists and ML engineers. I know very little about ML, so I asked my colleague to share his take on the [Neptune.ai](https://neptune.ai/) service, who knows vastly more about this space…

"Piece of software that, when you tweak something about a machine learning model, let's you run some calculations that tell you how your changes affect how good the model is" - [Julian Berman](https://mastodon.social/@JulianWasTaken)

Well, that's dumbed down enough for me! Thanks Julian!

I guess we should reach out to them to find out about their use case for Hyper Schema!

## OK, so who's still accessing `/schema`?

Accessing the latest meta-schema published using this unversioned URI is not advised. We started making people aware of this about four years ago, and you'll see [a message with your redirection](https://github.com/json-schema-org/json-schema-org.github.io/blob/fb3b1e681e273738f38cb1438b0635b6d4ad5383/_layouts/redirect.html#L13) if using a browser.

The vast majority of requests to this have no referring source, so that's not very helpful. Turning to user agent, we see the vast majority of requests have a user agent of "". *sigh*. Yup! blank! This is poor form, and I had a suspicion about what might be causing this.

When you look at the adjacent data, seeing over half of the remaining top 15 user agents, it's pretty clear that most of these requests are coming from IDEs and code editors. We see InteliJ IDEA, WebStorm, and multiple versions of each. But, what we don't see is VSCode.

Sure enough, VSCode, as far as we can tell, [doesn't provide a user agent](https://github.com/microsoft/vscode/issues/188398). At least, when making requests related to JSON Schema. While we love VSCode and the baked-in support for JSON Schema, there are a number of things we wish were a little better, and this is one of them.

Visual Studio provides a user agent (and we'll get to that analysis later), but given Visual Studio Code does not, we can't differentiate between requests it is making, and any others which aren't providing a user agent. (We generally see a user agent is given.)

It's important to note here that, due to the limitations of hosting the site on GitHub, the redirect used for `/schema` is an HTML based header redirect as opposed to using a HTTP 302 status code like you might expect. This means that realistically, it's unlikely any code making these requests is actually getting what they want. (The redirection technique is almost exclusivly followed by browsers, and not developer tools, such as CURL, as it is non-standard). Sure enough, if we try this in Visual Studio Code, we see an "Unable to parse content" error. It expects JSON in the response, but the website does not oblige.

## Who is accessing draft-04?

The total number of requests to the draft-04 meta-schema is 7.21 million in the last 7 days.

The physical locations showed one surprise, with Peru taking the lead over the UK and Russia. Next the USA with 5.73 million of those 7.21 million requests.

This time, the user agent data is much more useful. We don't see the empty agent string till slot number 10, accounting for only 78k requests. At the top we see Java! I'm making an educated guess here, and saying this is related to the Java implementation by Francis Galiegue. They were one of the early core team members of JSON Schema, and created a Java implementation, which hasn't upgraded support since draft-04.

From this, we can speculate that many Java services in production using JSON Schema, on AWS, don't have a reason to move away from obsolete implementations of JSON Schema. Maybe they are using a more modern version of JSON Schema itself, and the implementation is handling them incorrectly… but without testing, I'm just making educated guesses at this point.

With Java taking the lions share, 4.11 million, we come next to Mojolicious (a Perl framework), followed by Visual Studio, with 526k. Now that's much more interesting. I also note here, that Visual Studio provides a much more helpful user agent string. Look:

`Visual Studio/17.6 (JSON Editor; ASP.NET and Web Tools/17.6.326.62524)`

If we account for multiple versions of Visual Studio, we are easily going over 1 million here. They take 8 of the top 15 spots. We already have some lines of communication open with people working on Visual Studio. The same cannot be said for Visual Studio Code. I'm hopeful we can correct that at some point.

## Latest three JSON Schema versions?

Let's look at accessing paths with `draft-07`, `2019-09`, and `2020-12`.

For draft-07, the path `/draft-07/schema` was accessed 11.66 million times. The next largest, at 1.8k, is the release notes… as in a HTML file on the website (oh, look, there's a website there too!).

For 2019-09, we see 20.87 million requests. Why the jump? Both `links` and `hyper-schema` received over 8 million requests each. As mentioned previously, it's really hard to attribute those requests, but let's see what happens when we exclude them. Now we see 4.05 million… not bad! Although, we have to consider, with 2019-09 we broke up the meta-schema into multiple files, specifically the individual vocabularies. We see just above 500k requests for each of the vocabularies, with slightly more for the general purposes meta-schema, and core.

The user agents for 2019-09, with hyper-schema URLs filtered out, are predominantly Java.

What do we expect to see for 2020-12 then? I was curious too.

We see 7.42 million, and none of those are related to Hyper Schema.

Of that, 1.53 million to `.../meta/core`, and in the next place, the validation vocabulary meta-schema, with only 577.84k. If this is accurate, it's a little concerning. We would expect `/draft/2020-12/schema` to be the top path accessed, but it doesn't appear till position 8, with 472.46k.

The traffic pattern is very spikey, with a repeating time based pattern. It's probably some cron job running. If we exclude the top IP address, we lose all the spikes, strongly suggesting this bursting traffic is coming from some Amazon controlled AWS servers.

We'll reach out to AWS and see if we can get any more specific details. It's likely not a DoS attack, but it's a signal of something going wrong.

## ASN, OSI, what?

The top two ASN sources are Amazon controlled, totalling just over 39 million requests.

For those of you who aren't network engineers, ASN is an [Autonomous System Number](https://www.cloudflare.com/en-gb/learning/network-layer/what-is-an-autonomous-system/), used for routing of packets over the internet, making sure they take the likely fastest route. If you cast your mind back to when you learnt about the OSI model (yeah, I don't remember much of that either), AKA the [Open Systems Interconnection model](https://www.cloudflare.com/en-gb/learning/ddos/glossary/open-systems-interconnection-model-osi/), these ASNs are used as part of the transport layer.

Point is, we can tell without needing to look up IP address owners, that almost 80% of our traffic is coming from AWS!

## Country

Given how much traffic is coming from AWS, it's not a surprise that about 43 million of those requests are coming from the US and Ireland. After those, Finland, India, and Netherlands make up the 1 mil club. Following on, Germany and Russia, and then several countries from Asia, all above 200k.

## Source

A staggering 51.12 million of those 51.92 million requests, are not declaring a reference. The following top three are json-schema.org, Google, and then, very interestingly, localhost!

Weirdly, 1.54k requests declare a referrer of some defunct Uber settlement, just above Duck Duck Go.

## What does this all mean?

Access of meta-schemas is unlikely to correlate with usage percentages, but it is at least an indicator of use to an extent.
Looking at this data has meant we are reaching out to AWS to see if we can get any details.
It has prompted us to reach out to IDE makers, although doing so was already on our list.
We have extended our known platform users. It's nice when people use a proper agent string.

Given we only have access to a 7 day history, there's not much we can tell over a longer time.
While traditional analytics are useful, and we do use them, they can't tell us about accessing JSON files directly.

While some of these numbers are interesting to see, I'm yet to be convinced that they provide much utility today.

---

Here's the side by side traffic for paths from 2023-07-21 and 2023-07-31.

<div className="grid md:grid-cols-2 gap-2">
<figure className="mt-10">
  <img src="/img/posts/2023/website-analytics-snapshot-2023/Screenshot 2023-07-21 at 12.12.31.webp"/>
  <figcaption className="mt-2 mb-10 text-sm text-center text-gray-500">Web traffic snapshot 2023-07-21.</figcaption>
</figure>

<figure className="mt-10">
  <img src="/img/posts/2023/website-analytics-snapshot-2023/Screenshot 2023-07-31 at 10.39.20.webp"/>
  <figcaption className="mt-2 mb-10 text-sm text-center text-gray-500">Web traffic snapshot 2023-07-31.</figcaption>
</figure>
</div>

---

Got any thoughts, comments, feedback on this article? Got any ideas why we are seeing this data? Have some ideas for futher research or analysis? Please, let's discuss this further: https://github.com/orgs/json-schema-org/discussions/480

Photo by <a href="https://unsplash.com/@isaacmsmith?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Isaac Smith</a> on <a href="https://unsplash.com/photos/6EnTPvPPL6I?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a>


How Oracle is Bridging the Gap Between JSON Schema and Relational Databases

Welcome to the JSON Schema Blog!