Commit graph

310 commits

Author SHA1 Message Date
Armin Rigo
7ccbca93a2 Forward-port of r52136,52138: a review of overflow-detecting code.
* unified the way intobject, longobject and mystrtoul handle
  values around -sys.maxint-1.

* in general, trying to entierely avoid overflows in any computation
  involving signed ints or longs is extremely involved.  Fixed a few
  simple cases where a compiler might be too clever (but that's all
  guesswork).

* more overflow checks against bad data in marshal.c.

* 2.5 specific: fixed a number of places that were still confusing int
  and Py_ssize_t.  Some of them could potentially have caused
  "real-world" breakage.

* list.pop(x): fixing overflow issues on x was messy.  I just reverted
  to PyArg_ParseTuple("n"), which does the right thing.  (An obscure
  test was trying to give a Decimal to list.pop()... doesn't make
  sense any more IMHO)

* trying to write a few tests...
2006-10-04 12:17:45 +00:00
Raymond Hettinger
a0c95fa4d8 Fix endcase for str.rpartition() 2006-09-04 15:32:48 +00:00
Neal Norwitz
17753ecbfa Patch #1541585: fix buffer overrun when performing repr() on
a unicode string in a build with wide unicode (UCS-4) support.

This code could be improved, so add an XXX comment.
2006-08-21 22:21:19 +00:00
Marc-André Lemburg
3a457790c7 Correct an accidentally removed previous patch. 2006-08-14 12:57:27 +00:00
Marc-André Lemburg
040f76b79c Slightly revised version of patch #1538956:
Replace UnicodeDecodeErrors raised during == and !=
compares of Unicode and other objects with a new
UnicodeWarning.

All other comparisons continue to raise exceptions.
Exceptions other than UnicodeDecodeErrors are also left
untouched.
2006-08-14 10:55:19 +00:00
Neal Norwitz
8a87f5d37e Patch #1538606, Patch to fix __index__() clipping.
I modified this patch some by fixing style, some error checking, and adding
XXX comments.  This patch requires review and some changes are to be expected.
I'm checking in now to get the greatest possible review and establish a
baseline for moving forward.  I don't want this to hold up release if possible.
2006-08-12 17:03:09 +00:00
Neal Norwitz
e1fdb32ff2 Handle allocation failures gracefully. Found with failmalloc.
Many (all?) of these could be backported.
2006-07-21 05:32:28 +00:00
Martin v. Löwis
d825143be1 Patch #1455898: Incremental mode for "mbcs" codec. 2006-06-14 05:21:04 +00:00
Neal Norwitz
de4c78a1d7 Initialize the type object so pychecker can't crash the interpreter. 2006-06-13 08:28:19 +00:00
Georg Brandl
90e27d38f5 Apply perky's fix for #1503157: "/".join([u"", u""]) raising OverflowError.
Also improve error message on overflow.
2006-06-10 06:40:50 +00:00
Georg Brandl
242508160e RFE #1491485: str/unicode.endswith()/startswith() now accept a tuple as first argument. 2006-06-09 18:45:48 +00:00
Georg Brandl
9f16760666 Repair refleaks in unicodeobject. 2006-06-04 21:46:16 +00:00
Martin v. Löwis
3f767795f6 Patch #1359618: Speed-up charmap encoder. 2006-06-04 19:36:28 +00:00
Fredrik Lundh
60d8b18831 needforspeed: stringlib refactoring: changed find_obj to find_slice,
to enable use from stringobject
2006-05-27 15:20:22 +00:00
Fredrik Lundh
c2d29c5a6d needforspeed: replace improvements, changed to Py_LOCAL_INLINE
where appropriate
2006-05-27 14:58:20 +00:00
Martin v. Löwis
2e3f6b77d5 Revert bogus change committed in 46432 to this file. 2006-05-27 11:07:49 +00:00
Andrew Dalke
e0df762719 fixed typo 2006-05-27 11:04:36 +00:00
Fredrik Lundh
2d23d5bf2e needforspeed: more stringlib refactoring 2006-05-27 10:05:10 +00:00
Martin v. Löwis
d004fc810a Patch 1494554: Update numeric properties to Unicode 4.1. 2006-05-27 08:36:52 +00:00
Neal Norwitz
d1b6cd7bfb Fix Coverity warnings.
- Check the correct variable (str_obj, not str) for NULL
 - sep_len was already verified it wasn't 0
2006-05-27 05:21:30 +00:00
Andrew M. Kuchling
07bbfc6a51 Comment typo 2006-05-26 19:51:10 +00:00
Fredrik Lundh
e6e43c867d needforspeed: stringlib refactoring: use stringlib/find for string find 2006-05-26 19:48:07 +00:00
Fredrik Lundh
c816281304 needforspeed: use a macro to fix slice indexes 2006-05-26 19:33:03 +00:00
Fredrik Lundh
ce4eccb0c4 needforspeed: stringlib refactoring: use stringlib/find for unicode
find
2006-05-26 19:29:05 +00:00
Fredrik Lundh
58b5e84d52 needforspeed: stringlib refactoring, continued. added count and
find helpers; updated unicodeobject to use stringlib_count
2006-05-26 19:24:53 +00:00
Fredrik Lundh
9c0e9c089c needspeed: rpartition documentation, tests, and a bug fixes.
feel free to add more tests and improve the documentation.
2006-05-26 18:24:15 +00:00
Fredrik Lundh
b3167cbcd7 needforspeed: added rpartition implementation 2006-05-26 18:15:38 +00:00
Fredrik Lundh
b947948c61 needforspeed: stringlib refactoring (in progress) 2006-05-26 17:22:38 +00:00
Fredrik Lundh
a50d201bd9 needforspeed: stringlib refactoring (in progress) 2006-05-26 17:04:58 +00:00
Fredrik Lundh
95e2a91615 use Py_LOCAL also for string and unicode objects 2006-05-26 11:38:15 +00:00
Fredrik Lundh
f2c0dfdb13 needforspeed: use Py_ssize_t for the fastsearch counter and skip
length (thanks, neal!).  and yes, I've verified that this doesn't
slow things down ;-)
2006-05-26 10:27:17 +00:00
Fredrik Lundh
450277fef5 needforspeed: use METH_O for argument handling, which made partition some
~15% faster for the current tests (which is noticable faster than a corre-
sponding find call).  thanks to neal-who-never-sleeps for the tip.
2006-05-26 09:46:59 +00:00
Fredrik Lundh
06a69dd8ff needforspeed: partition implementation, part two.
feel free to improve the documentation and the docstrings.
2006-05-26 08:54:28 +00:00
Andrew Dalke
b552c4d848 Code had returned an ssize_t, upcast to long, then converted with PyInt_FromLong.
Now using PyInt_FromSsize_t.
2006-05-25 18:03:25 +00:00
Fredrik Lundh
0c71f88fc9 needforspeed: check for overflow in replace (from Andrew Dalke) 2006-05-25 16:46:54 +00:00
Fredrik Lundh
347ee277aa needforspeed: refactored the replace code slightly; special-case
constant-length changes; use fastsearch to locate the first match.
2006-05-24 16:35:18 +00:00
Fredrik Lundh
d5e0dc51cf needforspeedindeed: use fastsearch also for __contains__ 2006-05-24 15:11:01 +00:00
Fredrik Lundh
6471ee4f18 needforspeed: use "fastsearch" for count and findstring helpers. this
results in a 2.5x speedup on the stringbench count tests, and a 20x (!)
speedup on the stringbench search/find/contains test, compared to 2.5a2.

for more on the algorithm, see:

    http://effbot.org/zone/stringlib.htm

if you get weird results, you can disable the new algoritm by undefining
USE_FAST in Objects/unicodeobject.c.

enjoy /F
2006-05-24 14:28:11 +00:00
Fredrik Lundh
240bf2a8e4 use Py_ssize_t for string indexes (thanks, neal!) 2006-05-24 10:20:36 +00:00
Fredrik Lundh
7763351808 return 0 on misses, not -1. 2006-05-23 19:47:35 +00:00
Fredrik Lundh
b63588c188 needforspeed: use append+reverse for rsplit, use "bloom filters" to
speed up splitlines and strip with charsets; etc.  rsplit is now as
fast as split in all our tests (reverse takes no time at all), and
splitlines() is nearly as fast as a plain split("\n") in our tests.
and we're not done yet... ;-)
2006-05-23 18:44:25 +00:00
Fredrik Lundh
833bf9422e needforspeed: fixed unicode "in" operator to use same implementation
approach as find/index
2006-05-23 10:12:21 +00:00
Tim Peters
1bacc641a0 unicode_repeat(): Change type of local to Py_ssize_t,
since that's what it should be.
2006-05-23 05:47:16 +00:00
Tim Peters
286085c781 PyUnicode_Join(): Recent code changes introduced new
compiler warnings on Windows (signed vs unsigned mismatch
in comparisons).  Cleaned that up by switching more locals
to Py_ssize_t.  Simplified overflow checking (it can _be_
simpler because while these things are declared as
Py_ssize_t, then should in fact never be negative).
2006-05-22 19:17:04 +00:00
Fredrik Lundh
8a8e05a2b9 needforspeed: use memcpy for "long" strings; use a better algorithm
for long repeats.
2006-05-22 17:12:58 +00:00
Fredrik Lundh
f1d60a5384 needforspeed: speed up unicode repeat, unicode string copy 2006-05-22 16:29:30 +00:00
Fredrik Lundh
763b50f9d9 docstring tweaks: count counts non-overlapping substrings, not
total number of occurences
2006-05-22 15:35:12 +00:00
Neal Norwitz
1004a5339a Patch #1488312, Fix memory alignment problem on SPARC in unicode. Will backport 2006-05-15 07:17:23 +00:00
Thomas Wouters
715a4cdea2 Use %zd instead of %i as format character (in call to PyErr_Format) for
Py_ssize_t argument.
2006-04-16 22:04:49 +00:00
Martin v. Löwis
5cb6936672 Make Py_BuildValue, PyObject_CallFunction and
PyObject_CallMethod aware of PY_SSIZE_T_CLEAN.
2006-04-14 09:08:42 +00:00