Python⇒Speed: Massive memory overhead: Numbers in Python and how NumPy helps

Let’s say you want to store a list of integers in Python:

list_of_numbers = []
for i in range(1000000):
    list_of_numbers.append(i)

Those numbers can easily fit in a 64-bit integer, so one would hope Python would store those million integers in no more than ~8MB: a million 8-byte objects.

In fact, Python uses more like 35MB of RAM to store these numbers. Why? Because Python integers are objects, and objects have a lot of memory overhead.

Let’s see what’s going on under the hood, and then how using NumPy can get rid of this overhead.

Read more...

from Planet Python
via read more