The best new features in Python 3.8

Python 3.8 is the latest version of the popular language for everything from scripting and automation to machine learning and web development. Now available in an official beta release, Python 3.8 brings a number of slick syntax changes, memory sharing, more efficient serialization and deserialization, revamped dictionaries, and much more.

Naturally, Python 3.8 ushers in all manner of performance improvements as well. The overall result is a faster, more concise, more consistent, and more modern Python. Here’s what’s new and most significant in Python 3.8.

Assignment expressions 

The single most visible change in Python 3.8 is assignment expressions, which use what is known as the walrus operator ( := ). Assignment expressions allow a value to be assigned to a variable, even a variable that doesn’t exist yet, in the context of an expression rather than as a stand-alone statement.

while (line := file.readline()) != "end":
    print(chunk)

In this example the variable line is created if it doesn’t exist, then assigned the value from file.readline(). Then line is checked to see if it equates to "end". If not, the next line is read, stored in line, tested, and so on.

Assignment expressions follow the tradition of comprehensible terseness in Python that includes list comprehensions. Here, the idea is to cut down on some of the tedious boilerplate that tends to appear in certain Python programming patterns. The above snippet, for instance, would normally take far more than two lines of code to express.

Positional-only parameters 

A new syntax for function definitions, positional-only parameters, lets developers force certain arguments to be positional only. This removes any ambiguity about which arguments in a function definition are positional and which are keyword arguments.

Positional-only parameters make it possible to define scenarios where, for instance, a function accepts any keyword argument but can also accept one or more positionals. This is often the case with Python built-ins, so giving Python developers a way to do this themselves reinforces consistency in the language.

An example from Python’s documentation:

def pow(x, y, z=None, /):
    r = x**y
    if z is not None:
        r %= z
    return r

The / separates positional from keyword arguments; in this example, all of the arguments are positional. In previous versions of Python, z would be considered a keyword argument. Given the above function definition, pow(2, 10) and pow(2, 10, 5) are valid calls, but pow(2, 10, z=5) is not.

F-string debugging support

The f-string format provides a convenient (and more performant) way to print text and computed values or variables in the same expression:

x = 3 
print(f'{x+1}')

This would yield 4.

Adding an = to the end of an f-string expression prints the text of the f-string expression itself, followed by the value:

x = 3
print (f'{x+1=}')

This would yield x+1=4.

Multiprocessing shared memory

With Python 3.8, the multiprocessing module now offers a SharedMemory class that allows regions of memory to be created and shared between different Python processes.

In previous versions of Python, data could be shared between processes only by writing it out to a file, sending it over a network socket, or serializing it using Python’s pickle module. Shared memory provides a much faster path for passing data between processes, allowing Python to more efficiently use multiple processors and processor cores.

Shared memory segments can be allocated as raw regions of bytes, or they can use immutable list-like objects that store a small subset of Python objects—numeric types, strings, byte objects, and the None object.

Typing module improvements

Python is dynamically typed, but supports the use of type hints via the typing module to allow third-party tools to verify Python programs. Python 3.8 adds new elements to typing to make more robust checks possible:

• The final decorator and Final type annotation indicate that the decorated/annotated objects should not be overridden, subclassed, or reassigned at any point.
• The Literal type restricts expressions to a specific value or list of values, not necessarily of the same type.
• The TypedDict type lets you create dictionaries where the values associated with certain keys are restricted to one or more specific types. Note that these restrictions are limited to what can be determined at compile time, not at run time.

New version of the pickle protocol

Python’s pickle module provides a way to serialize and deserialize Python data structures—for instance, to allow a dictionary to be saved as-is to a file and reloaded later. Different versions of Python support different levels of the pickle protocol, with more recent versions supporting a broader range of capabilities and more efficient serialization.

Version 5 of pickle, introduced with Python 3.8, provides a new way to pickle objects that implement Python’s buffer protocol, such as bytes, memoryviews, or NumPy arrays. The new pickle cuts down on the number of memory copies that have to be made for such objects.

External libraries like NumPy and Apache Arrow support the new pickle protocol in their Python bindings. The new pickle is also available as an add-on for Python 3.6 and Python 3.7 from PyPI.

Reversible dictionaries

Dictionaries in Python were totally rewritten in Python 3.6, using a new implementation contributed by the PyPy project. In addition to being faster and more compact, dictionaries now have inherent ordering for their elements; they’re ordered as they are added, much as with lists. Python 3.8 allows reversed() to be used on dictionaries.

Runtime audit hooks

Runtime audit hooks, as described in PEP 578, allows functions performed by the CPython runtime to be visible to Python code by way of instrumentation functions. This way, application performance monitoring tools can see far more of what's happening inside Python applications as they run, on the function-call level.

Initialization configuration

PEP 587 allows the CPython runtime fine-grained control over its startup options by way of C APIs. This way, Python's startup behavior can be customized far more precisely than before -- for instance, if you're using Python in an embedded way, or as a standalone runtime for an app distribution.

Performance improvements

• Many built-in methods and functions have been sped up by 20% to 50%, as many of them were unnecessarily converting arguments passed to them.
• A new opcode cache can speed up certain instructions in the interpreter. However, the only currently implemented speed-up is for the LOAD_GLOBAL opcode, now 40% faster. Similar optimizations are planned for later versions of Python.
• File copying operations, such as shutil.copyfile() and shutil.copytree(), now use platform-specific calls and other optimizations to speed up operations.
• Newly created lists are now, on average, 12% smaller than before, thanks to optimizations that make use of the length of the list constructor object if it is known beforehand.
• Writes to class variables on new-style classes (e.g., class A(object)) are much faster in Python 3.8.
• operator.itemgetter() and collections.namedtuple() also have new speed optimizations.

Python C API and CPython improvements

In recent versions of Python, major work has gone into refactoring the C API used in CPython, the reference implementation of Python written in C. So far that work has yielded only incremental changes, but they’re adding up:

• A new C API for Python Initialization Configuration allows tighter control and more detailed feedback for Python’s initialization routines. This makes it easier to embed a Python runtime into an application, and to pass startup arguments to Python programmatically. This new API is also meant to ensure that all of Python’s configuration controls have a single, consistent home, so that future changes (like Python’s new UTF-8 mode) are easier to slot in.
• Another new C API for CPython, the “vectorcall” calling protocol, allows for far faster calls to internal Python methods without the overhead of creating temporary objects to handle the call. The API is still unstable, but has been made provisionally available. The plan is to finalize it as of Python 3.9.
• Python runtime audit hooks provide two APIs in the Python runtime for hooking events and making them observable to outside tools like testing frameworks or logging and auditing systems.

Where to download Python 3.8

You can download the Python 3.8 beta from the Python Software Foundation.