Python 3.13.0b1: exec() does not populate locals()

[hroncok]

Bug report

Bug description:

x.py

xxx = 118888

readx.py

def f():
    with open("x.py", encoding="utf-8") as f:
        exec(compile(f.read(), "x.py", "exec"))
    return locals()["xxx"]

print(f())

shell

$ python3.12 readx.py
118888

$ python3.13 readx.py
Traceback (most recent call last):
  File ".../readx.py", line 6, in <module>
    print(f())
          ~^^
  File ".../readx.py", line 4, in f
    return locals()["xxx"]
           ~~~~~~~~^^^^^^^
KeyError: 'xxx'

This breaks e.g. pillow 10.3.0 which has:

def get_version():
    version_file = "src/PIL/_version.py"
    with open(version_file, encoding="utf-8") as f:
        exec(compile(f.read(), version_file, "exec"))
    return locals()["__version__"]

In https://github.com/python-pillow/Pillow/blob/10.3.0/setup.py#L23

CPython versions tested on:

3.13

Operating systems tested on:

Linux

Linked PRs

• gh-118888: Further PEP 667 docs updates #119893
• [3.13] gh-118888: Further PEP 667 docs updates (gh-119893) #119894

[Eclips4]

Bisected to b034f14
cc @gaogaotiantian

[gaogaotiantian]

This is an expected and intentional behavior change due to PEP 667. locals() now has a clear semantic when called inside a function - a snapshot of the local variables, and xxx (or __version__) is not one of them.

I won't even consider this is "breaking" as the docs clearly states:

modifications to the default locals dictionary should not be attempted. Pass an explicit locals dictionary if you need to see effects of the code on locals after function exec() returns.

So this is an illegal usage that happens to work in a favored way to begin with.

If you want the result of the local changes, pass in an explicit dictionary:

def get_version():
    version_file = "src/PIL/_version.py"
    d = {}
    with open(version_file, encoding="utf-8") as f:
        exec(compile(f.read(), version_file, "exec"), globals(), d)
    return d["__version__"]

I'm aware that this might be a bit inconvenience to the library maintainers, but this is the right way to go and we are making efforts to make locals() more consistent and predictable.

[terryjreedy]

It it true that the 3.12 docs say that readx.py should not be expected to work. But it did then and previously, even though not now. What's New 3.13 only says

PEP 667: FrameType.f_locals when used in a function now returns a write-through proxy to the frame’s locals, rather than a dict. See the PEP for corresponding C API changes and deprecations.

From this, I would not expect changes in how locals() behaves, in particular in the effect of exec bindings. I think this should be mention.

Even the Python subsection of the PEP's Back Compatibility section says nothing. It only mentions a couple of things that do not change.

[gaogaotiantian]

We are aware that the docs are not fully ready for beta 1, but this behavior is described in detail in locals. https://docs.python.org/3.13/library/functions.html#locals . We can add some notes to exec or eval or whatsnews, but the key change is actually on the locals() function, which the first version of docs is written for.

[hugovk]

Can this be closed now the docs were updated in #119201?

[terryjreedy]

@ncoghlan Does #119201 make this obsolete?

[ncoghlan]

Yeah, the function level snapshot behaviour is now covered in the What's New porting guide: https://docs.python.org/3.13/whatsnew/3.13.html#changes-in-the-python-api

It is also mentioned in a versionchanged note on exec itself: https://docs.python.org/3.13/library/functions.html#exec

The general write-up of PEP 667 also mentions locals() first before covering FrameType.f_locals: https://docs.python.org/3.13/whatsnew/3.13.html#whatsnew313-locals-semantics

The most minimal change to fix this kind of exec invocation is to pass explicit target namespaces as suggested in #118888 (comment) (this exec call is already implicitly being called with separate globals and locals namespaces, so explicitly calling it that way won't change the behaviour of the executed code)

Alternatively, for the examples given, https://docs.python.org/3/library/runpy.html#runpy.run_path is a better tool when the task is "run the Python file at this location and return its top level namespace" (it will respect Python source encoding declarations properly, while explicitly opening the files as utf-8 ignores them).