Use interned versions of string constants if they're already present

[efimov-mikhail]

Feature or enhancement

Proposal:

For now intern_constants function in Objects/codeobject.c uses should_intern_string to check if that string should be interned. And do nothing if should_intern_string returned 0. Maybe it'll be better to check if this string already have interned variant and use it if present? Because there's no need to keep equal strings in memory. The small downside is some time needed for this checking.

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

No response

Linked PRs

• gh-140328: Use interned versions of string constants if they're already present #140688

[efimov-mikhail]

For now some behavior is surpising.
For example, 'utf-8' is interned static string because _Py_DECLARE_STR(utf_8, "utf-8") is used in Python/sysmodule.c.
But if we use 'utf-8' in REPL, we get other result (on main):

>>> import sys
>>> sys.getrefcount('utf-8')
2
>>> sys.getrefcount('str')
3221225472
>>> is_interned = lambda s: s is sys.intern(('_' + s + '_')[1:-1])
>>> is_interned('utf-8')
False
>>> is_interned('str')
True

[albertedwardson]

Maybe it'll be better to check if this string already have interned variant and use it if present?

I’d like to open a PR with these changes and benchmark their impact on performance and memory usage

[efimov-mikhail]

Feel free to make this PR, @abebus. But I can't ensure you that it will be accepted, we need to hear another opinions.

[zahlman]

I see that should_intern_string is defined in codeobject.c itself, and there doesn't appear to be mention of it in any header. Would it make sense, then, to put the logic directly within should_intern_string, rather than in intern_constants?