Commit graph

330 commits

Author SHA1 Message Date
Raymond Hettinger
69492dab07 Factor-out the common code for setting a KeyError. 2013-09-02 15:59:26 -07:00
Raymond Hettinger
a35adf5b09 Instead of XORed indicies, switch to a hybrid of linear probing and open addressing.
Modern processors tend to make consecutive memory accesses cheaper than
random probes into memory.

Small sets can fit into L1 cache, so they get less benefit.  But they do
come out ahead because the consecutive probes don't probe the same key
more than once and because the randomization step occurs less frequently
(or not at all).

For the open addressing step, putting the perturb shift before the index
calculation gets the upper bits into play sooner.
2013-09-02 03:23:21 -07:00
Raymond Hettinger
6c3c1ccd1b Update copyright. 2013-08-31 21:34:24 -07:00
Raymond Hettinger
95c0d67581 Further reduce the cost of hash collisions by inspecting an additional nearby entry. 2013-08-31 21:27:08 -07:00
Raymond Hettinger
afe890923f Tighten-up the lookkey() logic and beautify the code a bit.
Use less code by moving many of the steps from the initial
lookup into the main search loop.

Beautify the code but keep the overall logic unchanged.
2013-08-28 20:59:31 -07:00
Antoine Pitrou
9d95254bb7 Issue #18772: fix the gdb plugin after the set implementation changes 2013-08-24 21:07:07 +02:00
Raymond Hettinger
bfc1e1a9cd Add the same dummy type that is used in dictionaries. 2013-08-23 03:22:15 -05:00
Raymond Hettinger
fcf3b500ba Issue 18797: Remove unneeded refcount adjustments for dummy objects.
It suffices to keep just one reference when the object is created.
2013-08-22 08:20:31 -07:00
Raymond Hettinger
5bb1b1dd6f Hoist the global dummy lookup out of the inner loop for set_merge(). 2013-08-21 01:34:18 -07:00
Raymond Hettinger
929cbac307 Remove a redundant hash table probe (this was artifact from an earlier draft of the patch). 2013-08-20 23:03:28 -07:00
Raymond Hettinger
ae9e616a00 Issue 18772: Restore set dummy object back to unicode and restore the identity checks in lookkey().
The Gdb prettyprint plugin depended on the dummy object being displayable.
Other solutions besides a unicode object are possible.  For now, get it
back up and running.

The identity checks in lookkey() need to be there to prevent the dummy
object from leaking through Py_RichCompareBool() into user code in the
rare circumstance where the dummy's hash value exactly matches the hash
value of the actual key being looked up.
2013-08-20 22:28:24 -07:00
Raymond Hettinger
3c0a4f5def Issue18771: Reduce the cost of hash collisions for set objects. 2013-08-19 07:36:04 -07:00
Raymond Hettinger
07351a0449 Remove the else-clause because the conditions are no longer mutually exclusive. 2013-08-17 02:39:46 -07:00
Raymond Hettinger
237b34b074 Use a known unique object for the dummy entry.
This lets us run PyObject_RichCompareBool() without
first needing to check whether the entry is a dummy.
2013-08-17 02:31:53 -07:00
Raymond Hettinger
8ad3919577 Hoist the global "dummy" lookup outside of the reinsertion loop. 2013-08-15 02:18:55 -07:00
Antoine Pitrou
9ed5f27266 Issue #18722: Remove uses of the "register" keyword in C code. 2013-08-13 20:18:52 +02:00
Raymond Hettinger
c629d4c9a2 Replace outdated optimization with clearer code that compiles better.
Letting the compiler decide how to optimize the multiply by five
gives it the freedom to make better choices for the best technique
for a given target machine.

For example, GCC on x86_64 produces a little bit better code:

Old-way (3 steps with a data dependency between each step):

    shrq    $5, %r13
    leaq    1(%rbx,%r13), %rax
    leaq    (%rax,%rbx,4), %rbx

New-way (3 steps with no dependency between the first two steps
         which can be run in parallel):

    leaq    (%rbx,%rbx,4), %rax     # i*5
    shrq    $5, %r13                # perturb >>= PERTURB_SHIFT
    leaq    1(%r13,%rax), %rbx      # 1 + perturb + i*5
2013-08-05 22:24:50 -07:00
Raymond Hettinger
c86d7e989c Silence compiler warning for an unused declaration 2013-08-04 12:00:36 -07:00
Antoine Pitrou
5e946bacef Fix compilation warning with gcc 4.8 (unused typedef) 2013-06-18 23:28:18 +02:00
Gregory P. Smith
c2176e46d7 Fix the internals of our hash functions to used unsigned values during hash
computation as the overflow behavior of signed integers is undefined.

NOTE: This change is smaller compared to 3.2 as much of this cleanup had
already been done.  I added the comment that my change in 3.2 added so that the
code would match up.  Otherwise this just adds or synchronizes appropriate UL
designations on some constants to be pedantic.

In practice we require compiling everything with -fwrapv which forces overflow
to be defined as twos compliment but this keeps the code cleaner for checkers
or in the case where someone has compiled it without -fwrapv or their
compiler's equivalent.

Found by Clang trunk's Undefined Behavior Sanitizer (UBSan).

Cleanup only - no functionality or hash values change.
2012-12-10 18:32:53 -08:00
Gregory P. Smith
27cbcd6241 Fix the internals of our hash functions to used unsigned values during hash
computation as the overflow behavior of signed integers is undefined.

In practice we require compiling everything with -fwrapv which forces overflow
to be defined as twos compliment but this keeps the code cleaner for checkers
or in the case where someone has compiled it without -fwrapv or their
compiler's equivalent.

Found by Clang trunk's Undefined Behavior Sanitizer (UBSan).

Cleanup only - no functionality or hash values change.
2012-12-10 18:15:46 -08:00
Ezio Melotti
0e1af282b8 Fix typo. 2012-09-28 16:43:40 +03:00
David Malcolm
49526f48fc Issue #14785: Add sys._debugmallocstats() to help debug low-level memory allocation issues 2012-06-22 14:55:41 -04:00
Antoine Pitrou
a701388de1 Rename _PyIter_GetBuiltin to _PyObject_GetBuiltin, and do not include it in the stable ABI. 2012-04-05 00:04:20 +02:00
Kristján Valur Jónsson
31668b8f7a Issue #14288: Serialization support for builtin iterators. 2012-04-03 10:49:41 +00:00
Antoine Pitrou
093ce9cd8c Issue #6695: Full garbage collection runs now clear the freelist of set objects.
Initial patch by Matthias Troffaes.
2011-12-16 11:24:27 +01:00
Benjamin Peterson
1cebc207ea merge 3.2 2011-10-30 14:24:59 -04:00
Benjamin Peterson
2b50a01d11 remove unused variable 2011-10-30 14:24:44 -04:00
Petri Lehtinen
c34f5c256a Fix the return value of set_discard (issue #10519) 2011-10-30 14:35:39 +02:00
Petri Lehtinen
e0aa803714 Fix the return value of set_discard (issue #10519) 2011-10-30 14:35:12 +02:00
Petri Lehtinen
7c5e34d8a3 Avoid unnecessary recursive function calls (#closes #10519) 2011-10-30 13:57:45 +02:00
Petri Lehtinen
5acc27ebe4 Avoid unnecessary recursive function calls (closes #10519) 2011-10-30 13:56:41 +02:00
Martin v. Löwis
bd928fef42 Rename _Py_identifier to _Py_IDENTIFIER. 2011-10-14 10:20:37 +02:00
Martin v. Löwis
1ee1b6fe0d Use identifier API for PyObject_GetAttrString. 2011-10-10 18:11:30 +02:00
Martin v. Löwis
d63a3b8beb Implement PEP 393. 2011-09-28 07:41:54 +02:00
Mark Dickinson
57e683e53e Issue #1621: Fix undefined behaviour in bytes.__hash__, str.__hash__, tuple.__hash__, frozenset.__hash__ and set indexing operations. 2011-09-24 18:18:40 +01:00
Brian Curtin
dfc80e3d97 Replace Py_NotImplemented returns with the macro form Py_RETURN_NOTIMPLEMENTED.
The macro was introduced in #12724.
2011-08-10 20:28:54 -05:00
Victor Stinner
4f2dab5c33 Revert my commit 7ba176c2f558: "Avoid useless "++" at the end of functions
Warnings found by the Clang Static Analyzer."

Most people prefer ++ at the end of functions.
2011-05-27 16:46:51 +02:00
Victor Stinner
a1a807b6ef set_repr(): handle correctly PyUnicode_FromUnicode() error (MemoryError)
Bug found by the Clang Static Analyzer.
2011-05-26 14:24:30 +02:00
Victor Stinner
97e561ef24 Avoid useless "++" at the end of functions
Warnings found by the Clang Static Analyzer.
2011-05-26 13:53:47 +02:00
Éric Araujo
bbcfc1f2d9 Merge from 3.1.
The fix was already committed to 3.2, but I merged two small changes
recommended by Raymond while I was working on the 2.7 patch to ease
future merges.
2011-03-23 03:43:22 +01:00
Éric Araujo
48049911d6 Fix obscure set crashers (#8420). Backport of d56b3cafb1e6, reviewed by Raymond. 2011-03-23 02:08:07 +01:00
Antoine Pitrou
715f3cd10d Issue #8685: Speed up set difference a - b when source set a is
much larger than operand `b`.  Patch by Andrew Bennetts.
2010-11-30 22:23:20 +00:00
Antoine Pitrou
fbb1c6191c Follow up to #9778: fix regressions on 64-bit Windows builds 2010-10-23 17:37:54 +00:00
Georg Brandl
00da4e0b5a Remove unneeded casts to hashfunc. 2010-10-18 07:32:48 +00:00
Benjamin Peterson
8f67d0893f make hashes always the size of pointers; introduce Py_hash_t #9778 2010-10-17 20:54:53 +00:00
Georg Brandl
2d444496b3 Reindent. 2010-09-03 10:52:55 +00:00
Raymond Hettinger
faf7b7f4ec Issue 8420: Fix obscure set crashers. 2010-09-03 10:00:50 +00:00
Daniel Stutzbach
928d4eeee8 Removed an extraneous semicolon 2010-09-02 15:06:03 +00:00
Antoine Pitrou
f72006f442 Merged revisions 84146-84147,84150 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r84146 | antoine.pitrou | 2010-08-17 19:55:07 +0200 (mar., 17 août 2010) | 4 lines

  Issue #9612: The set object is now 64-bit clean under Windows.
........
  r84147 | antoine.pitrou | 2010-08-17 20:30:06 +0200 (mar., 17 août 2010) | 3 lines

  Fix <deque iterator>.__length_hint__() under 64-bit Windows.
........
  r84150 | antoine.pitrou | 2010-08-17 21:33:30 +0200 (mar., 17 août 2010) | 3 lines

  Clean some 64-bit issues. Also, always spell "ssize_t" "Py_ssize_t".
........
2010-08-17 19:39:39 +00:00