JSON, UTF-8 and base64

An order with the id 9007199254740993 goes out as JSON and comes back as 9007199254740992 — one less, silently, with no error anywhere. The same program, the same library, two compilation targets.

dart:convert is two functions and a handful of codecs; the care goes into what they hand you. You will encode a nested order, read the types that come back, measure that big-integer difference between the virtual machine and a JavaScript build, then write a model by hand that refuses a bad body with the field that failed, and convert bytes with UTF-8 and base64 well enough that a character split across two reads still arrives whole.

What jsonEncode and jsonDecode really hand you

Serialisationserialisation: Turning a value in memory into a form that can be stored or sent, such as JSON text, and back again; in Dart the outward step is usually a toJson method. starts with a method, not a library call. jsonEncode knows maps, lists, strings, numbers, booleans and null; for anything else it calls toJson() and encodes whatever that returns. The run below encodes an order of two items, prints it indented, and then reads it back.

import 'dart:convert';

class Item {
  Item(this.sku, this.qty);
  final String sku;
  final int qty;
  Map<String, Object?> toJson() => {'sku': sku, 'qty': qty};
}

class Order {
  Order(this.id, this.items);
  final String id;
  final List<Item> items;
  Map<String, Object?> toJson() => {'id': id, 'items': items};
}

void main() {
  final order = Order('A-4181', [Item('tea-250g', 2), Item('jaggery-1kg', 1)]);

  final compact = jsonEncode(order);
  print('compact: $compact');
  print('characters of the text: ${compact.length}');

  print('indented:');
  print(const JsonEncoder.withIndent('  ').convert(order));

  final back = jsonDecode(compact);
  print('decoded type: ${back.runtimeType}');
  final map = back as Map<String, Object?>;
  print('id: ${map['id']}  items is: ${map['items'].runtimeType}');
  final items = map['items']! as List<Object?>;
  print('first item: ${items.first}  type: ${items.first.runtimeType}');
  print('qty read without a check: ${(items.first as Map<String, Object?>)['qty']}');
}

compact: {"id":"A-4181","items":[{"sku":"tea-250g","qty":2},{"sku":"jaggery-1kg","qty":1}]}
characters of the text: 82
indented:
{
  "id": "A-4181",
  "items": [
    {
      "sku": "tea-250g",
      "qty": 2
    },
    {
      "sku": "jaggery-1kg",
      "qty": 1
    }
  ]
}
decoded type: _Map<String, dynamic>
id: A-4181  items is: List<dynamic>
first item: {sku: tea-250g, qty: 2}  type: _Map<String, dynamic>
qty read without a check: 2

Two things in that output decide how the rest of your code looks. The key order is the order your map wrote, because JSON objects have no order of their own, so a comparison with another system’s output compares parsed values rather than strings. And the decoded value is a _Map<String, dynamic> holding a List<dynamic>: jsonDecode returns Object?, so items.first['qty'] is an unchecked claim about a body a stranger sent.

JsonEncoder.withIndent(' ') is the same encoder with a two-space indent, useful for a file a person reads or a fixture you want to diff. For a long response you have already received, our JSON formatter does the same job in the browser.

What happens to a number on the way through

JSON has one number type and Dart has two, so decoding has to choose. The rule is the text: a body with no decimal point and no exponent becomes an int, anything else a double.

import 'dart:convert';

void main() {
  for (final text in ['1', '1.0', '1e3', '2500', '0.1', '9007199254740993']) {
    final value = jsonDecode(text);
    print('jsonDecode("$text") -> $value  (${value.runtimeType})');
  }

  final body = jsonDecode('{"amount_paise": 49900, "rate": 1.0}') as Map<String, Object?>;
  print('amount_paise is int: ${body['amount_paise'] is int}');
  print('rate is int: ${body['rate'] is int}  rate is double: ${body['rate'] is double}');
  print('both are num: ${body.values.every((v) => v is num)}');

  final asNum = body['rate']! as num;
  print('read as num, used as double: ${asNum.toDouble()}');
  print('re-encoded: ${jsonEncode(body)}');
}

jsonDecode("1") -> 1  (int)
jsonDecode("1.0") -> 1.0  (double)
jsonDecode("1e3") -> 1000.0  (double)
jsonDecode("2500") -> 2500  (int)
jsonDecode("0.1") -> 0.1  (double)
jsonDecode("9007199254740993") -> 9007199254740993  (int)
amount_paise is int: true
rate is int: false  rate is double: true
both are num: true
read as num, used as double: 1.0
re-encoded: {"amount_paise":49900,"rate":1.0}

That is why a field you read as int throws the day the sender starts writing 1.0, and why value as num with toDouble() is the tolerant way to read a quantity that may be whole. The run also re-encodes the map and shows 1.0 surviving as 1.0: double keeps its decimal point on the way out, so a round trip does not quietly turn money into an integer.

The second measurement is the one in the first line of this lesson. The same program, run on the virtual machine and then compiled to JavaScript and run by Node.js, disagrees about a large integer.

from session import files, run

# The same encoding work on the Dart VM and compiled to JavaScript, run by Node.js.
files("web/ids.dart=ids.dart")
run(
    """
dart run ids.dart
dart compile js -O2 -o build/ids.js ids.dart > /dev/null
node build/ids.js
"""
)

$ dart run ids.dart
id read back: 9007199254740993
id is int: true
re-encoded: {"id":9007199254740993,"paise":49900}
utf8 bytes: 18
base64: 4KSo4KSu4KS44KWN4KSk4KWH
$ dart compile js -O2 -o build/ids.js ids.dart > /dev/null
$ node build/ids.js
id read back: 9007199254740992
id is int: true
re-encoded: {"id":9007199254740992,"paise":49900}
utf8 bytes: 18
base64: 4KSo4KSu4KS44KWN4KSk4KWH

On the virtual machine the id reads back as 9007199254740993. In the JavaScript build it reads back as 9007199254740992, and is int is still true, because a web build represents integers as JavaScript numbers, which are doubles with 53 bits of precision. Dart’s own documentation on number representation describes that difference; what the run adds is the cost in one concrete case: an identifier that is one unit wrong, with no exception and no log line. Send identifiers larger than 2^53 as strings, as many payment and ledger systems do, and keep whole numbers that must be exact in int fields that never go near a web build. The UTF-8 and base64 lines of the same run are identical on both targets, which is the part of dart:convert you can rely on everywhere.

The depth of this lesson goes in two directions from here. One is a model written by hand, where fromJson checks every field with a pattern and refuses a bad body by naming the field that failed, and toJson keeps the round trip honest. The other is the byte side: UTF-8 and base64 measured on real text, and converters bound to a stream so that a character cut in half between two reads still arrives whole.

Practise with a body you do not trust

Build this in a console project: dart create -t console orders, then work in lib/ and bin/.

1. Write toJson() for an order with a list of items, print it compact and with JsonEncoder.withIndent(' '), and check that the compact form is 82 characters.
2. Decode that text and print runtimeType for the map, the list and the first item.
3. Decode '{"amount_paise": 49900, "rate": 1.0}', read both values as num, and throw a FormatException naming the field when a value is not a number.
4. Encode a map holding 9007199254740993, compile it with dart compile js, run it with node, and compare.

A typed model written by hand, with errors that name the field Pro: opens with a Pro, Premium or Ultimate pass

From text to bytes: UTF-8, base64 and a character cut in half Pro: opens with a Pro, Premium or Ultimate pass

Interview questions

Three questions from the module’s bank carry this further:

Key takeaways

• jsonEncode calls toJson() on anything it does not know, so that method defines the shape you send.
• jsonDecode returns Object?, so every field read afterwards is an unchecked claim until your code proves it.
• A body without a decimal point decodes to int and one with a decimal point or an exponent to double.
• A web build holds integers as doubles with 53 bits, so an identifier above 2^53 travels safely only as a string.
• Code units and bytes are different limits, and a converter bound to a stream keeps an unfinished character or line safe across two reads.