PL/SQL & T-SQL (Oracle and SQL Server) Module 8 – Production T-SQL: concurrency, security, new features and tests
SQL Server 2025: JSON, regex and new functions
Store JSON in the SQL Server 2025 json type, build it from rows, clean and check text with the REGEXP functions, and see which features need level 170.
What you will learn
- Store and query JSON with the json type, JSON_VALUE, JSON_QUERY, OPENJSON and FOR JSON
- Build JSON from rows with JSON_OBJECTAGG and JSON_ARRAYAGG, and choose what a NULL becomes
- Validate and clean text with REGEXP_LIKE, REGEXP_REPLACE and the other REGEXP functions
- Check which new features need a database compatibility level before you depend on them
Before you start
What you’ll learn · Before you start
What you will learn
- Store and query JSON with the json type, JSON_VALUE, JSON_QUERY, OPENJSON and FOR JSON
- Build JSON from rows with JSON_OBJECTAGG and JSON_ARRAYAGG, and choose what a NULL becomes
- Validate and clean text with REGEXP_LIKE, REGEXP_REPLACE and the other REGEXP functions
- Check which new features need a database compatibility level before you depend on them
Before you start
On this page
A partner sends 8 customer rows, and only 4 of them can be loaded: mobile numbers arrive as +91-98470-11111 or
98450 12345, one vehicle number is too short, and one preferences value is a bare JSON string. SQL Server 2025 can
check all of it in T-SQL, with a json type that refuses what is not JSON and functions for
regular expression: A pattern that describes a set of strings, such as ten digits that start with 6 to 9. SQL Server 2025's REGEXP functions test, extract, count and replace matches of patterns written in RE2 syntax..
You will store and read JSON in the new type, see how rows become JSON, clean and test the import’s text with the REGEXP functions, and check which of these features depend on the database’s compatibility level.
SQL Server 2025 stores JSON in a type that refuses what is not JSON
Until SQL Server 2025, JSON lived in nvarchar(max) columns, and nothing stopped a broken document unless a CHECK
with ISJSON was added. The new json data type: The SQL Server 2025 type for JSON documents. It stores a parsed binary form, accepts only a JSON object or array at the top level and refuses other text with error 13609. stores documents in a binary format
and accepts only a JSON object or array at the top level. Microsoft’s page names the refusals: a JSON scalar such as
"email", or an empty string, fails with error 13609, and so does text that is not JSON at all. A duplicate key is not
an error: the type keeps the first value. The type works at every compatibility level.
Six functions read JSON or build it:
| Function | What it gives you |
|---|---|
JSON_VALUE |
one scalar from a path, such as $.lang |
JSON_QUERY |
an object or an array from a path |
OPENJSON |
rows and columns from a document |
FOR JSON |
JSON text from a query’s rows |
JSON_OBJECTAGG |
one object from many rows’ keys and values |
JSON_ARRAYAGG |
one array from many rows’ values |
Checkpoint
Check: The json type and the JSON functions
The REGEXP functions clean and check text inside the database
SQL Server 2025’s regular expressions follow RE2, Google’s library, and come in two kinds. REGEXP_LIKE returns
true or false, so it goes where T-SQL expects a condition: WHERE, CASE WHEN or a CHECK constraint.
REGEXP_REPLACE, REGEXP_SUBSTR, REGEXP_INSTR and REGEXP_COUNT return a value, and REGEXP_MATCHES and
REGEXP_SPLIT_TO_TABLE return rows. Step through one row of the import:
The drawing shows the whole scene; its 6 steps are listed below.
Every step as text
- The row arrives as text. Row 2 of the import carries its mobile number as +91-98470-11111, with a country code, hyphens and a plus sign that no column should keep.
- Only the digits stay. REGEXP_REPLACE with the pattern of every character that is not a digit leaves 919847011111, twelve digits.
- The shape is checked. REGEXP_LIKE tests the digits against an optional 91 or 0 followed by ten digits that start with 6 to 9. Row 2 fits, and its number is 9847011111.
- The preferences are JSON. The preferences text goes into a json value, which accepts only an object or an array. Row 2 sends an object, so it is stored.
- The clean row is loaded. Every check has passed, so the cleaned values are inserted. The raw text stays in the staging table, so the cleaning can be checked later.
- A failing row is set aside. Row 7 reduces to 1234567890, which starts with 1, so it fails the shape check. It goes to the rejects with the reason, and nobody guesses a number for it.
The model applies the same three teaching shapes to all 8 rows, with the json type’s rules for the preferences. Its patterns use only what RE2 and Python read alike; press Edit and add a row of your own:
"""Clean and check a partner's import of 8 customer rows, the way the lesson's T-SQL does it.
Each mobile number keeps only its digits and may start with 91 or 0; what is left must be 10 digits that start with
6, 7, 8 or 9. Vehicle numbers lose their spaces and hyphens and must have the shape of two letters, one or two
digits, up to three letters and four digits. PIN codes keep their digits and must be six digits that do not start
with 0. These are teaching shapes, not official validation rules. The preferences must be JSON that SQL Server 2025's
json type accepts: an object or an array at the top level; a duplicate key keeps its first value.
The patterns use only what RE2, the library behind SQL Server's REGEXP functions, and Python's re both read the same
way: anchors, character classes, groups and counted repeats. Press Edit and add a row of your own to ROWS.
"""
import json
import re
MOBILE = r"^(91|0)?([6-9][0-9]{9})$"
VEHICLE = r"^[A-Z]{2}[0-9]{1,2}[A-Z]{0,3}[0-9]{4}$"
PIN = r"^[1-9][0-9]{5}$"
ROWS = [ # (row, mobile as sent, vehicle as sent, PIN as sent, preferences as sent)
(1, "98450 12345", "KA 01 AB 1234", "560001", '{"lang":"kn","sms":true}'),
(2, "+91-98470-11111", "KL07CD4321", "682001", '{"lang":"ml","sms":false}'),
(3, "0522 2611234", "UP32 EF 0001", "226001", '{"lang":"hi"}'),
(4, "9840012345", "TN 09 GH 5678", "600001", '{"lang":"ta","sms":true,"sms":false}'),
(5, "98300-55555", "WB-02-AK-7777", "700001", '"email"'),
(6, "+91 99999 00000", "DL 3C AB 1234", "110001", '{"lang":"en","sms":true}'),
(7, "12345 67890", "MH 12 DE 1433", "411001", "[]"),
(8, "8888 777 666", "GJ 1 KL 99", "380 001", "{}"),
]
def first_key_wins(pairs):
out = {}
for key, value in pairs:
out.setdefault(key, value)
return out
def refuse_constant(name):
raise ValueError(f"{name} is not JSON")
def as_json(text):
"""The json type's rules: an object or an array at the top level, duplicate keys keep the first value."""
try:
value = json.loads(text, object_pairs_hook=first_key_wins, parse_constant=refuse_constant)
except ValueError:
return None, "not valid JSON"
if not isinstance(value, (dict, list)):
return None, "a JSON scalar, not an object or an array"
return json.dumps(value, separators=(",", ":")), None
loaded, rejected, changed = 0, 0, 0
print(f"{'row':<4}{'mobile':<12}{'vehicle':<12}{'PIN':<8}{'preferences':<27}result")
for row, mobile_raw, vehicle_raw, pin_raw, prefs_raw in ROWS:
reasons = []
digits = re.sub(r"[^0-9]", "", mobile_raw)
m = re.search(MOBILE, digits)
mobile = m.group(2) if m else None
if mobile is None:
reasons.append("mobile")
vehicle = re.sub(r"[ -]", "", vehicle_raw).upper()
if not re.search(VEHICLE, vehicle):
reasons.append("vehicle")
pin = re.sub(r"[^0-9]", "", pin_raw)
if not re.search(PIN, pin):
reasons.append("PIN")
prefs, why = as_json(prefs_raw)
if prefs is None:
reasons.append(f"preferences ({why})")
changed += (mobile not in (None, mobile_raw)) + (vehicle != vehicle_raw) + (pin != pin_raw)
changed += prefs is not None and prefs != prefs_raw
result = "load" if not reasons else "reject: " + ", ".join(reasons)
loaded += not reasons
rejected += bool(reasons)
print(f"{row:<4}{mobile or '-':<12}{vehicle:<12}{pin:<8}{(prefs or '-'):<27}{result}")
print(f"{loaded} rows to load, {rejected} to reject; cleaning changed {changed} values of the rows") Output
row mobile vehicle PIN preferences result
1 9845012345 KA01AB1234 560001 {"lang":"kn","sms":true} load
2 9847011111 KL07CD4321 682001 {"lang":"ml","sms":false} load
3 - UP32EF0001 226001 {"lang":"hi"} reject: mobile
4 9840012345 TN09GH5678 600001 {"lang":"ta","sms":true} load
5 9830055555 WB02AK7777 700001 - reject: preferences (a JSON scalar, not an object or an array)
6 9999900000 DL3CAB1234 110001 {"lang":"en","sms":true} load
7 - MH12DE1433 411001 [] reject: mobile
8 8888777666 GJ1KL99 380001 {} reject: vehicle
4 rows to load, 4 to reject; cleaning changed 14 values of the rows
Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 import_check.py
Runs on this device, in your browser. The first run downloads Python (about 13.5 MB), which is kept for the next runs.
Your run, in this browser
Cleaning changed 14 values without a guess. Rows 3 and 7 are not mobile numbers once their digits are read, row 8’s
vehicle number has two digits where four belong, and row 5’s preferences are a JSON string, not an object. Row 4
sends sms twice and keeps the first value. A regex tester helps to try a pattern on more
samples before it goes into a constraint.
Checkpoint
Check: Cleaning and checking with REGEXP
Most new features work at any level, a few need level 170
The compatibility level: A setting of each SQL Server database that decides which Transact-SQL behaviours and features it uses; new databases get 170 on SQL Server 2025, 160 on 2022 and 150 on 2019. More in “PL/SQL and T-SQL: what they are and where they run” decides some of what a 2025 server offers a database. Microsoft’s pages say which features need a level and which do not:
| Feature | Level it needs |
|---|---|
REGEXP_LIKE, REGEXP_MATCHES, REGEXP_SPLIT_TO_TABLE |
170 |
REGEXP_REPLACE, REGEXP_SUBSTR, REGEXP_INSTR, REGEXP_COUNT |
any |
json and vector types, JSON_OBJECTAGG, JSON_ARRAYAGG |
any |
OPENJSON |
130 or more |
EDIT_DISTANCE, JARO_WINKLER_SIMILARITY |
preview: PREVIEW_FEATURES on |
The other additions are small and useful: the double-pipe operator joins strings, PRODUCT multiplies a column’s
values, CURRENT_DATE returns today’s date, UNISTR turns escapes such as \20B9 into characters, BASE64_ENCODE
writes binary as text, and SUBSTRING may leave out its length. The two similarity functions measure how alike two
strings are. One older function belongs beside them for Indian data: FORMAT with the culture en-IN writes numbers
with Indian digit grouping.
Checkpoint
Check: Which features need level 170
JSON in depth: the type against nvarchar, reads, and JSON built from rows Pro: opens with a Pro, Premium or Ultimate pass
The json type changes what a database can promise about its documents, and the read and build functions decide how much work each query does. This part works through the lesson’s events table and compares the two ways to store JSON.
In this part:
- The json type against nvarchar(max) with ISJSON
- Reading the events: JSON_VALUE, JSON_QUERY and OPENJSON WITH
- Building JSON for an API: aggregates and the NULL clause
3 runnable examples (Python, SQL) · about 5 minutes
Opening this part with your pass…
This part opens with any MySmartCoPilot pass.
- Pro pass · 1 week₹79$2.99
- 30 days₹179$5.99
- 1 year₹1,699$45
One payment; a pass never renews. The price shown is the total you pay: no fees or charges are added at checkout.
Passes can’t be bought right now, so this part can’t be opened at the moment. The free part above stands on its own. If you have a pass, enter your licence key.
It also opens every other in-depth part in Learn and every Pro tool. Sales are final, except where the law gives you a right to withdraw, for example in the EU and the UK (refund policy).
Regex and the new functions in practice, with the import cleaned in T-SQL Pro: opens with a Pro, Premium or Ultimate pass
The model’s rules become one set-based statement on SQL Server 2025. This part writes it, shows what a regular expression can and cannot do for an index, and puts the smaller new functions to work.
In this part:
- The import cleaned and checked in one statement
- What RE2 refuses, and when a pattern can seek an index
- The smaller additions, each on a lab value
2 runnable examples (Python, SQL) · about 5 minutes
Opening this part with your pass…
This part opens with any MySmartCoPilot pass.
- Pro pass · 1 week₹79$2.99
- 30 days₹179$5.99
- 1 year₹1,699$45
One payment; a pass never renews. The price shown is the total you pay: no fees or charges are added at checkout.
Passes can’t be bought right now, so this part can’t be opened at the moment. The free part above stands on its own. If you have a pass, enter your licence key.
It also opens every other in-depth part in Learn and every Pro tool. Sales are final, except where the law gives you a right to withdraw, for example in the EU and the UK (refund policy).
Practise: clean, check and read the import
Each question shows a value from the import, a function or a level. Decide what SQL Server 2025 does with it:
Practice that adapts to you
Practise this lesson
8 questions about 5 ideas of this lesson. Each answer is explained at once, and what you get right is kept in this browser.
Interview questions
Three questions follow, about a regex check that fails on one server, a choice of type for an events table and PIN codes taken from addresses typed as plain text; write your answer before you open each first step:
-
Mid to senior
A validation procedure that uses REGEXP_LIKE works on your development database, but on the staging server, also SQL Server 2025, its deployment fails with error 195. The same release uses REGEXP_REPLACE and a json column without trouble. Explain the difference, and say what you would do this week and what before the next release.
-
Mid to senior
A new events table on SQL Server 2025 will receive about 2 million JSON documents a day from mobile apps, and reports read three values from each by path. A colleague proposes nvarchar(max) with an ISJSON check, as on the old servers. Would you use the json type instead? Defend the choice and say what you would check first.
-
Mid to senior
A nightly job receives 200,000 shop addresses as plain text and must store each shop's PIN code in its own column. Some addresses hold no PIN code, some hold two, and some write it as 411 001. How would you extract it on SQL Server 2025, and where do rows without exactly one go?
Flashcards · 8 cards
Recall it, then check
Say the answer to yourself before you look: recalling it is what makes it stick.
-
What does SQL Server 2025's json type accept and refuse?
It stores documents in a binary format and accepts only a JSON object or array at the top level: a JSON scalar such as a bare string, an empty string, or text that is not JSON fails with error 13609. A duplicate key is not an error, the type keeps the first value, and the type works at every compatibility level.
-
Which function reads one scalar from JSON, which reads a subtree, and which turns a document into rows?
JSON_VALUE returns one scalar from a path such as $.lang, JSON_QUERY returns an object or an array from a path, and OPENJSON returns rows and columns from a document.
-
Which three things build JSON from rows?
FOR JSON turns a query's rows into JSON text, JSON_OBJECTAGG builds one object from many rows' keys and values, and JSON_ARRAYAGG builds one array from many rows' values.
-
Which REGEXP function is a condition, and what do the others return?
REGEXP_LIKE returns true or false, so it goes where T-SQL expects a condition: WHERE, CASE WHEN or a CHECK constraint. REGEXP_REPLACE, REGEXP_SUBSTR, REGEXP_INSTR and REGEXP_COUNT return a value, and REGEXP_MATCHES and REGEXP_SPLIT_TO_TABLE return rows. All follow RE2.
-
Which of the 2025 features need compatibility level 170?
Only REGEXP_LIKE, REGEXP_MATCHES and REGEXP_SPLIT_TO_TABLE. REGEXP_REPLACE, REGEXP_SUBSTR, REGEXP_INSTR and REGEXP_COUNT work at any level, as do the json and vector types, JSON_OBJECTAGG and JSON_ARRAYAGG; OPENJSON needs 130 or more.
-
Which two string similarity functions does the lesson use, and what switches them on?
EDIT_DISTANCE and JARO_WINKLER_SIMILARITY, two of SQL Server 2025's four fuzzy string matching functions, which measure how alike two strings are; all four are previews and need the PREVIEW_FEATURES setting on.
-
Name four smaller additions of SQL Server 2025.
The double-pipe operator joins strings, PRODUCT multiplies a column's values, CURRENT_DATE returns today's date, UNISTR turns escapes into characters, BASE64_ENCODE writes binary as text, and SUBSTRING may leave out its length.
-
What does the lesson's import do with a row whose number or code does not match its pattern?
It keeps the row with the reason it failed rather than guessing a value: digits that do not make a mobile number, a vehicle code with two digits where four belong, and a preferences value that is a JSON string instead of an object are all reported, not repaired.
Key takeaways
- SQL Server 2025’s json type accepts only a JSON object or array, refuses everything else with error 13609 and keeps the first value of a duplicate key.
REGEXP_LIKEis a condition, the other REGEXP functions return values or rows, and all of them follow RE2.- Only
REGEXP_LIKEand the two table-valued REGEXP functions need compatibility level 170; the json type works at every level. - Clean an import with patterns and keep the rows that fail with their reasons, rather than guessing a value.
Terms in this lesson
- json data type
- The SQL Server 2025 type for JSON documents. It stores a parsed binary form, accepts only a JSON object or array at the top level and refuses other text with error 13609.
- regular expression
- A pattern that describes a set of strings, such as ten digits that start with 6 to 9. SQL Server 2025's REGEXP functions test, extract, count and replace matches of patterns written in RE2 syntax.
References
- JSON data type (Microsoft)
- JSON data in SQL Server (Microsoft)
- JSON_VALUE (Transact-SQL) (Microsoft)
- JSON_QUERY (Transact-SQL) (Microsoft)
- OPENJSON (Transact-SQL) (Microsoft)
- Format query results as JSON with FOR JSON (Microsoft)
- JSON_OBJECTAGG (Transact-SQL) (Microsoft)
- JSON_ARRAYAGG (Transact-SQL) (Microsoft)
- ISJSON (Transact-SQL) (Microsoft)
- CREATE JSON INDEX (Transact-SQL) (Microsoft)
- Regular expressions in SQL Server (Microsoft)
- REGEXP_LIKE (Transact-SQL) (Microsoft)
- REGEXP_REPLACE (Transact-SQL) (Microsoft)
- REGEXP_MATCHES (Transact-SQL) (Microsoft)
- REGEXP_SPLIT_TO_TABLE (Transact-SQL) (Microsoft)
- RE2 syntax (Google)
- PRODUCT (Transact-SQL) (Microsoft)
- || (String concatenation) (Transact-SQL) (Microsoft)
- UNISTR (Transact-SQL) (Microsoft)
- CURRENT_DATE (Transact-SQL) (Microsoft)
- BASE64_ENCODE (Transact-SQL) (Microsoft)
- SUBSTRING (Transact-SQL) (Microsoft)
- EDIT_DISTANCE (Transact-SQL) (Microsoft)
- JARO_WINKLER_SIMILARITY (Transact-SQL) (Microsoft)
- ALTER DATABASE SCOPED CONFIGURATION (Transact-SQL) (Microsoft)
- FORMAT (Transact-SQL) (Microsoft)
- Vector data type (Microsoft)
- ALTER DATABASE compatibility level (Transact-SQL) (Microsoft)
- What's new in SQL Server 2025 (Microsoft)
- Database engine events and errors (13000 to 13999) (Microsoft)
- Database engine events and errors (0 to 999) (Microsoft)
Related tools
Report a problem with this lesson
Kept only in this browser. Your Learn progress