Jsonrepair: Repair Invalid JSON Documents

Repair invalid JSON documents.

Try it out in a minimal demo: https://josdejong.github.io/jsonrepair/

Use it in a full-fledged application: https://jsoneditoronline.org

Read the background article "How to fix JSON and validate it with ease"

The following issues can be fixed:

• Add missing quotes around keys
• Add missing escape characters
• Add missing commas
• Add missing closing brackets
• Repair truncated JSON
• Replace single quotes with double quotes
• Replace special quote characters like “...” with regular double quotes
• Replace special white space characters with regular spaces
• Replace Python constants None, True, and False with null, true, and false
• Strip trailing commas
• Strip comments like /* ... */ and // ...
• Strip fenced code blocks like ```json and ```
• Strip ellipsis in arrays and objects like [1, 2, 3, ...]
• Strip JSONP notation like callback({ ... })
• Strip escape characters from an escaped string like {\"stringified\": \"content\"}
• Strip MongoDB data types like NumberLong(2) and ISODate("2012-12-19T06:01:17.171Z")
• Concatenate strings like "long text" + "more text on next line"
• Turn newline delimited JSON into a valid JSON array, for example:
  { "id": 1, "name": "John" } { "id": 2, "name": "Sarah" }

The jsonrepair library has streaming support and can handle infinitely large documents.

Note that in the lib folder, there are builds for ESM, UMD, and CommonJs.

Use the jsonrepair function using an ES modules import:

Use the streaming API in Node.js:

Use in CommonJS (not recommended):

Use with UMD in the browser (not recommended):

Use in Python via PythonMonkey.

1. Install jsonrepair via npm install jsonrepair

2. Install PythonMonkey via pip install pythonmonkey

3. Use the libraries in a Python script:

   import pythonmonkey jsonrepair = pythonmonkey.require('jsonrepair').jsonrepair json = "[1,2,3," repaired = jsonrepair(json) print(repaired) # [1,2,3]

You can use jsonrepair as a function or as a streaming transform. Broken JSON is passed to the function, and the function either returns the repaired JSON, or throws an JSONRepairError exception when an issue is encountered which could not be solved.

The streaming API is availabe in jsonrepair/stream and can be used in a Node.js stream. It consists of a transform function that can be used in a stream pipeline.

The option chunkSize determines the size of the chunks that the transform outputs, and is 65536 bytes by default. Changing chunkSize can influcence the performance.

The option bufferSize determines how many bytes of the input and output stream are kept in memory and is also 65536 bytes by default. This buffer is used as a "moving window" on the input and output. This is necessary because jsonrepair must look ahead or look back to see what to fix, and it must sometimes walk back the generated output to insert a missing comma for example. The bufferSize must be larger than the length of the largest string and whitespace in the JSON data, otherwise, and error is thrown when processing the data. Making bufferSize very large will result in more memory usage and less performance.

When jsonrepair is installed globally using npm, it can be used on the command line. To install jsonrepair globally:

Usage:

Options:

Example usage:

Similar libraries:

• https://github.com/RyanMarcus/dirty-json

When implementing a fix or a new feature, it important to know that there are currently two implementations:

• src/regular This is a non-streaming implementation. The code is small and works for files up to 512MB, ideal for usage in the browser.
• src/streaming A streaming implementation that can be used in Node.js. The code is larger and more complex, and the implementation uses a configurable bufferSize and chunkSize. When the parsed document contains a string or number that is longer than the configured bufferSize, the library will throw an "Index out of range" error since it cannot hold the full string in the buffer. When configured with an infinite buffer size, the streaming implementation works the same as the regular implementation. In that case this out of range error cannot occur, but it makes the performance worse and the application can run out of memory when repairing large documents.

Both implementations are tested against the same suite of unit tests in src/index.test.ts.

Scripts:

Released under the ISC license.