bpo-33234 Improve list() pre-sizing for inputs with known lengths (GH-9846)

The list() constructor isn't taking full advantage of known input
lengths or length hints. This commit makes the constructor
pre-size and not over-allocate when the input size is known (the
input collection implements __len__). One on the main advantages is
that this provides 12% difference in memory savings due to the difference
between overallocating and allocating exactly the input size.

For efficiency purposes and to avoid a performance regression for small
generators and collections, the size of the input object is calculated using
__len__ and not __length_hint__, as the later is considerably slower.
This commit is contained in:
Pablo Galindo 2018-10-28 20:16:26 +00:00 committed by GitHub
parent 569d12f448
commit 372d705d95
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
3 changed files with 51 additions and 0 deletions

View file

@ -1,5 +1,6 @@
import sys
from test import list_tests
from test.support import cpython_only
import pickle
import unittest
@ -157,5 +158,13 @@ class ListTest(list_tests.CommonTest):
with self.assertRaises(TypeError):
(3,) + L([1,2])
@cpython_only
def test_preallocation(self):
iterable = [0] * 10
iter_size = sys.getsizeof(iterable)
self.assertEqual(iter_size, sys.getsizeof(list([0] * 10)))
self.assertEqual(iter_size, sys.getsizeof(list(range(10))))
if __name__ == "__main__":
unittest.main()