By design, Python puts convenience, readability, and ease of use ahead of performance. But that doesn’t mean you should settle for slow Python code. There is probably something you can do to speed it up.
Among the tools available for profiling the performance of Python code, the simplest is the timeit
module. timeit
is used to measure the speed of small snippets of code — a few lines, a function — by executing the code thousands or even millions of times and reporting how long those executions took to complete.
timeit
is most useful for comparing two or three different ways to do something and seeing which is the fastest. For instance, a loop that runs for thousands of iterations is a common Python bottleneck. If you can find a way to speed up the implementation of that loop — say, by using Python built-ins instead of handwritten code — you could get a measurable performance improvement.
A simple Python timeit example
Here is a simple example of how timeit
works:
def f1(): for n in range(100): pass def f2(): n=0 while n<100: n+=1 if __name__ == "__main__": import timeit print (timeit.timeit(f1, number=100000)) print (timeit.timeit(f2, number=100000))
This program compares the performance of two ways to iterate through a loop 100 times: by using Python’s built-in range
function (f1
), and by incrementing a variable (f2
). timeit
runs each of these approaches 100,000 times, and provides a total runtime at the end for each. By default, timeit
uses one million runs, but this example shows how you can set the number of runs to any figure that seems appropriate.
The results (from an Intel i7-3770K processor):
0.1252315
0.45453989999999994
Clearly the range
approach is much faster, by a factor of about 3.75. This isn’t surprising; using a Python built-in typically yields better performance than manually manipulating Python objects.
Use Python timeit by passing a string
Another way to use timeit
is to pass a string that is evaluated as a Python program:
import timeit
print (timeit.timeit('for n in range(100):pass'))
This can also be done from the command line:
python -m timeit "for n in range(100):pass"
On the whole, though, it’s easier to use the technique shown above, since you don’t need to awkwardly shoehorn your code into a text string.
Python timeit tips
As useful as timeit
is, bear in mind these caveats about how to use it.
Avoid using timeit for whole-program profiling
Nothing says you can’t time a whole program with timeit
. A simple 10-line script, for instance, isn’t a bad candidate for being profiled this way.
But there are better tools for that job — for instance, Python’s cProfile
module, which generates far more detailed statistics about your entire program’s performance. timeit
works best with a single component or code snippet — again, a function or a few lines of code. Anything more than that will typically generate results that are too noisy and inconsistent to give you any meaningful performance information.
Also, if the program you’re profiling takes many minutes to complete, timeit
won’t be of much use. For one, it will take too long to run the code more than a few times, so the timings gleaned will be very crude. For two, other tools are better suited to the job.
Perform multiple timeit runs on different machines
Programs don’t run at the same speed every time. Modern computing environments introduce a lot of uncertainty — competition with other programs for resources, cache behaviors, scheduling, and so on. timeit
tries to compensate for this by executing the code ad infinitum, but it is still a good idea to aggregate multiple trials. You should run a timeit
profile many times, toss out the worst and best scores, and average the rest.
Finally, it also helps to run the same test on different systems: how will something disk-bound behave on an SSD versus a conventional spinning hard drive? As with any other question about performance — don’t guess, test.