JSON v2 in Go 1.27: encoding/json/v2 and jsontext

Go 1.27 ships two new packages for JSON, and on the body {"status":"WL","status":"CNF"} the new one disagrees with the old: encoding/json returns CNF, encoding/json/v2 an error. encoding/json/v2 is a revised API for turning Go values into JSON and back, with stricter defaults, and encoding/json/jsontext reads and writes JSON as a stream of tokens.

You will call the v2 API, see what its defaults do to real payloads and which options bring v1 back, give types you do not own a JSON form, and stream documents of any size. All of the programs were run on Go 1.27.1; below each one is the output of that run.

Two packages: v2 maps Go values, jsontext reads the syntax

encoding/json/v2 does the old package’s jobs, with new names for some of them. Marshal returns bytes and MarshalWrite writes to an io.Writer; Unmarshal reads bytes and UnmarshalRead reads an io.Reader to its end. Every one of them takes options, so each call chooses its own behaviour. encoding/json/jsontext is the layer beneath: it handles JSON as a sequence of tokensJSON token: One lexical item of JSON text: a string, a number, true, false, null, or one of the delimiters of an object or an array. Colons and commas follow from the grammar, so encoding/json/jsontext has no tokens for them. and raw values, and its state machine refuses anything that would not be valid JSON. Go 1.25 shipped both as an experiment behind GOEXPERIMENT=jsonv2, and Go 1.27 adds them to the standard library and runs encoding/json itself on the new code. Step through how they call each other:

The usual imports are json "encoding/json/v2" and, where both are needed, jsonv1 "encoding/json". Here is a booking through the four calls:

// v2_api.go: the encoding/json/v2 calls on one booking: Marshal and MarshalWrite, Unmarshal and UnmarshalRead.
package main

import (
	"encoding/json/jsontext"
	json "encoding/json/v2"
	"fmt"
	"os"
	"strings"
)

type Booking struct {
	PNR       string   `json:"pnr"`
	Passenger string   `json:"passenger"`
	FarePaise int64    `json:"fare_paise"`
	Seats     []string `json:"seats"`
}

func main() {
	b := Booking{PNR: "4521876390", Passenger: "Asha Rao", FarePaise: 129900, Seats: []string{"B2-34"}}

	out, err := json.Marshal(b)
	fmt.Printf("Marshal: %s err=%v\n", out, err)

	fmt.Println("MarshalWrite, indented:")
	if err := json.MarshalWrite(os.Stdout, b, jsontext.WithIndent("  ")); err != nil {
		fmt.Println("write:", err)
	}
	fmt.Println()

	var back Booking
	err = json.Unmarshal(out, &back)
	fmt.Printf("Unmarshal: %+v err=%v\n", back, err)

	err = json.UnmarshalRead(strings.NewReader(`{"pnr":"4521876391"}`+"\n"), &back)
	fmt.Printf("UnmarshalRead: pnr=%s err=%v\n", back.PNR, err)

	err = json.UnmarshalRead(strings.NewReader(`{"pnr":"4521876392"} {"pnr":"4521876393"}`), &back)
	fmt.Printf("UnmarshalRead, two values: err=%v\n", err)
}

Marshal: {"pnr":"4521876390","passenger":"Asha Rao","fare_paise":129900,"seats":["B2-34"]} err=<nil>
MarshalWrite, indented:
{
  "pnr": "4521876390",
  "passenger": "Asha Rao",
  "fare_paise": 129900,
  "seats": [
    "B2-34"
  ]
}
Unmarshal: {PNR:4521876390 Passenger:Asha Rao FarePaise:129900 Seats:[B2-34]} err=<nil>
UnmarshalRead: pnr=4521876391 err=<nil>
UnmarshalRead, two values: err=jsontext: invalid character '{' after top-level value after offset 21

MarshalWrite took jsontext.WithIndent, an option of the syntax layer, on that call alone. UnmarshalRead read the reader to its end, and the last line shows what that buys: two values in one input are an error, with the byte offset, where a v1 Decoder would have returned the first value and said nothing.

The stricter defaults, on five inputs

By default jsontext follows RFC 7493, the I-JSONI-JSON: The Internet JSON message format of RFC 7493: JSON that is UTF-8 throughout, never writes a member name twice in one object, and avoids numbers beyond what an IEEE 754 double holds exactly. profile of JSON, which forbids names written twice and requires valid UTF-8, and v2 changes rules of its own for names, nil values and struct tags. The program sends the same inputs through v1, through v2, and through v2 with the one option that restores v1’s behaviour:

// one_payload.go: the same inputs through encoding/json (v1) and encoding/json/v2, and the option that brings back v1.
package main

import (
	jsonv1 "encoding/json"
	"encoding/json/jsontext"
	json "encoding/json/v2"
	"errors"
	"fmt"
	"strings"
)

type Ticket struct {
	PNR    string   `json:"pnr"`
	Status string   `json:"status"`
	Seats  []string `json:"seats"`
}

type Badly struct {
	Coach string `json:"coach,omitempty,"` // a stray comma at the end of the tag
}

func decode(title, in string, option json.Options) {
	fmt.Println(title, strings.ReplaceAll(in, "\xff", `\xff`)) // show the bad byte as \xff
	var a, b, c Ticket
	errA := jsonv1.Unmarshal([]byte(in), &a)
	errB := json.Unmarshal([]byte(in), &b)
	errC := json.Unmarshal([]byte(in), &c, option)
	fmt.Printf("   v1:            %+v err=%v\n", a, errA)
	fmt.Printf("   v2:            %+v err=%v\n", b, errB)
	fmt.Printf("   v2 + option:   %+v err=%v\n", c, errC)
}

func main() {
	decode("1. duplicate names", `{"status":"WL","status":"CNF"}`, jsontext.AllowDuplicateNames(true))
	decode("2. name in another case", `{"PNR":"4521876390"}`, json.MatchCaseInsensitiveNames(true))
	decode("3. a byte that is not UTF-8", "{\"status\":\"C\xffNF\"}", jsontext.AllowInvalidUTF8(true))

	fmt.Println("4. a nil slice")
	t := Ticket{PNR: "4521876390"}
	a, _ := jsonv1.Marshal(t)
	b, _ := json.Marshal(t)
	c, _ := json.Marshal(t, json.FormatNilSliceAsNull(true))
	fmt.Printf("   v1:            %s\n   v2:            %s\n   v2 + option:   %s\n", a, b, c)

	fmt.Println("5. a malformed tag")
	x, errX := jsonv1.Marshal(Badly{})
	_, errY := json.Marshal(Badly{})
	z, errZ := json.Marshal(Badly{}, jsonv1.ReportErrorsWithLegacySemantics(true))
	fmt.Printf("   v1:            %s err=%v\n", x, errX)
	var se *json.SemanticError
	if errors.As(errY, &se) {
		fmt.Printf("   v2:            refused: %v\n", se.Err)
	}
	fmt.Printf("   v2 + option:   %s err=%v\n", z, errZ)
}

1. duplicate names {"status":"WL","status":"CNF"}
   v1:            {PNR: Status:CNF Seats:[]} err=<nil>
   v2:            {PNR: Status:WL Seats:[]} err=jsontext: duplicate object member name "status"
   v2 + option:   {PNR: Status:CNF Seats:[]} err=<nil>
2. name in another case {"PNR":"4521876390"}
   v1:            {PNR:4521876390 Status: Seats:[]} err=<nil>
   v2:            {PNR: Status: Seats:[]} err=<nil>
   v2 + option:   {PNR:4521876390 Status: Seats:[]} err=<nil>
3. a byte that is not UTF-8 {"status":"C\xffNF"}
   v1:            {PNR: Status:C�NF Seats:[]} err=<nil>
   v2:            {PNR: Status: Seats:[]} err=jsontext: invalid UTF-8 within "/status" after offset 12
   v2 + option:   {PNR: Status:C�NF Seats:[]} err=<nil>
4. a nil slice
   v1:            {"pnr":"4521876390","status":"","seats":null}
   v2:            {"pnr":"4521876390","status":"","seats":[]}
   v2 + option:   {"pnr":"4521876390","status":"","seats":null}
5. a malformed tag
   v1:            {} err=<nil>
   v2:            refused: Go struct field Coach has malformed `json` tag: invalid trailing ',' character
   v2 + option:   {} err=<nil>

The errors say where they happened: within "/status" is a JSON PointerJSON Pointer: A path to one value inside a JSON document, defined by RFC 6901: member names and array indexes separated by slashes, such as /orders/1/paise for the paise of the second order. encoding/json/v2 names places in its errors this way., the path of the member at fault. Case 2 is the quiet one, since v2 found no field for PNR and skipped it like any unknown member. Case 5’s option comes from encoding/json, where the options that exist for v1’s sake live, and it also brings back v1’s types of error, so fixing the tag is the better cure. Four more defaults matter once you write v2 code:

• omitempty drops a field that would be written as null, "", {} or [], so it keeps 0 and false.
• time.Duration has no default JSON form, so marshalling one is an error until the caller picks a form.
• Maps are written in no fixed order unless json.Deterministic(true) is set.
• <, > and & are written as they are, unless jsontext.EscapeForHTML(true) is set.

And v2 gives three kinds of control:

• WithMarshalers and MarshalToFunc give one call a JSON form for a type, without changing the type.
• MarshalJSONTo and UnmarshalJSONFrom are a type’s own v2 methods, which write to and read from jsontext directly.
• A jsontext.Decoder reads a document one token or one value at a time, however long the document is.

JSON forms for types you do not own, and for your own Pro: opens with a Pro, Premium or Ultimate pass

Streaming large documents as tokens, and errors that say where Pro: opens with a Pro, Premium or Ultimate pass

Practise

These questions cover the two packages and their defaults, starting with the idea you know least:

Interview questions

Warm-up (2 to 5 years): How do encoding/json/v2 and encoding/json/jsontext relate to encoding/json in Go 1.27?

encoding/json/v2 is a new major version of the JSON API: Marshal, MarshalWrite, Unmarshal, UnmarshalRead and the rest, each with options and stricter defaults. jsontext is the syntax layer under it, an Encoder and a Decoder of tokens and raw values. Since Go 1.27, encoding/json calls v2 with DefaultOptionsV1, so its behaviour is unchanged apart from the text of some errors. Nothing forces a move: encoding/json stays under the Go 1 compatibility promise.

Three longer questions of the module bank build on this lesson:

Key takeaways

• encoding/json/v2 maps Go values to JSON with Marshal, MarshalWrite, Unmarshal and UnmarshalRead, each taking options, and encoding/json/jsontext reads and writes the syntax as tokens and raw values.
• Since Go 1.27 encoding/json runs on v2 with DefaultOptionsV1, and v2’s own defaults are stricter: names written twice, invalid UTF-8 and malformed tags are errors, and names match exactly.
• Each of v1’s ways with a payload comes back with one option, such as jsontext.AllowDuplicateNames or json.FormatNilSliceAsNull.