Subinterpreters with concurrent.interpreters

Since Python 3.14, the concurrent.interpreters module lets one process run several interpreters, each a complete and isolated copy of the Python runtime with its own modules, its own global variables and its own GIL. Code in two interpreters can therefore compute on two cores at the same moment, as two processes would, while everything stays inside one process. Nothing is shared by default: arguments and results cross between interpreters as copies, cheap ones for strings, numbers and tuples and pickled ones for most other objects, and a queue carries data from one interpreter to another. An interpreter runs code only when a thread calls into it, so parallel work needs one thread per interpreter, which InterpreterPoolExecutor arranges for you. The 3.14 release notes list the present limits: slow start-up, extra memory for each interpreter, few ways to share objects, and many third-party extension modules that do not support it yet.

One process, several interpreters

Each box in the diagram is a separate world. Importing a module in interpreter 1 does not import it in interpreter 2, and a change to sys.path or to a global variable in one is invisible in the others, because each has its own copy of sys, of builtins and even of __main__. Since Python 3.12 each interpreter also has its own GIL, which is what lets two of them compute at the same time. The isolation is the point of the design: two interpreters can only meet where your code passes something between them. It is not a security boundary, though. They share one address space, and the documentation warns against using them to keep code away from data it must not read.

Creating an interpreter and calling into it

interpreters.create() makes a new, idle interpreter. exec() runs source code in it, and call() runs a function in it and returns the result; both run in the current thread and return when the code is done:

"""Create an interpreter, run code in it, call a function in it, and see what it keeps to itself."""
import sys
import textwrap
from concurrent import interpreters

sys.stdout.reconfigure(line_buffering=True)  # every interpreter has its own sys.stdout: show each line at once
print(f"top of the file runs as {__name__}")
visits = []

SOURCE = textwrap.dedent("""
    from concurrent import interpreters
    print("exec() runs in", __name__, "of interpreter", interpreters.get_current().id, flush=True)
""")


def visit(who):
    """Record a visit in this interpreter's own copy of the list."""
    visits.append(who)
    return f"{who}: {len(visits)} visit(s) in this interpreter's list"


if __name__ == "__main__":
    print(visit("main"))
    interp = interpreters.create()
    interp.exec(SOURCE)
    print(interp.call(visit, "sub"))
    print(interp.call(visit, "sub"))
    print(visit("main"))
    print("interpreters now:", [i.id for i in interpreters.list_all()])
    interp.close()
    print("after close():", [i.id for i in interpreters.list_all()])

top of the file runs as __main__
main: 1 visit(s) in this interpreter's list
exec() runs in __main__ of interpreter 1
top of the file runs as <fake __main__>
sub: 1 visit(s) in this interpreter's list
sub: 2 visit(s) in this interpreter's list
main: 2 visit(s) in this interpreter's list
interpreters now: [0, 1]
after close(): [0]

The exec() code ran in the new interpreter’s own __main__, as interpreter 1. The first call() showed something easy to miss: to run visit(), the new interpreter imported this script under the name <fake __main__>, so its top line ran a second time there. That is why the code that creates interpreters sits under if __name__ == "__main__":, exactly as with process pools. Each interpreter then counted visits in its own list: two in the main interpreter and two in interpreter 1, never four in one place. An interpreter keeps its modules and variables between calls, so the second call found the list that the first one had filled. close() destroyed the interpreter, leaving only the main one, 0.

Threads, processes or interpreters

Interpreters sit between threads and processes on every row. Several tools move data between them, each with one job. A queue from create_queue() carries objects from one interpreter to another. A memoryview shares the bytes of a buffer, so writes in one interpreter are seen in the other. prepare_main() binds values into an interpreter’s __main__ before you run code there. call_in_thread() runs a call in a new thread of its own, which is how several interpreters compute at once. An uncaught exception in another interpreter reaches you as ExecutionFailed. And InterpreterPoolExecutor puts all of it behind the executor interface of concurrent.futures.

Where interpreters are not available, the module is simply missing. WebAssembly builds of Python, such as the one behind the Run buttons of these lessons, have no concurrent.interpreters, and Python 3.13 and older have no public module for it at all. Code that must run everywhere tries the import and falls back to threads or processes.

Check yourself

Moving data between interpreters, with each rule shown Pro: opens with a Pro, Premium or Ultimate pass

Interpreters in parallel: speed, start-up and memory, measured Pro: opens with a Pro, Premium or Ultimate pass

Interview questions

Key takeaways

• concurrent.interpreters, new in Python 3.14, runs isolated interpreters inside one process, each with its own modules, globals and GIL.
• exec() and call() run in the calling thread; call_in_thread() or InterpreterPoolExecutor gives parallel work.
• Arguments and results cross as copies; queues and memoryviews are the few things that are shared.
• Interpreters import your script again to find its functions, so creation code goes under the main guard.
• Their current limits are start-up time, memory per interpreter and extension modules that do not support them yet.