mirrors/cpython
1
0
Fork 0
mirror of https://github.com/python/cpython.git synced 2025-11-04 03:44:55 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 9
cpython/Lib/test/test_parser.py
Christian Heimes b186d0084c Merged revisions 61431,61433-61436,61439,61444,61449-61450,61453,61458,61465,61468,61471-61474,61480,61483-61484,61488,61495-61496,61498,61503-61504,61507,61509-61510,61515-61518 via svnmerge from 
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r61431 | vinay.sajip | 2008-03-16 22:35:58 +0100 (So, 16 Mär 2008) | 1 line

  Clarified documentation on use of shutdown().
........
  r61433 | mark.summerfield | 2008-03-17 09:28:15 +0100 (Mo, 17 Mär 2008) | 5 lines

  Added a footnote to each pointing out that for XML output if an encoding
  string is given it should conform to the appropriate XML standards---for
  example, "UTF-8" is okay, but "UTF8" is not.
........
  r61434 | eric.smith | 2008-03-17 12:01:01 +0100 (Mo, 17 Mär 2008) | 7 lines

  Issue 2264: empty float presentation type needs to have at least one digit past the decimal point.

  Added "Z" format_char to PyOS_ascii_formatd to support empty float presentation type.
  Renamed buf_size in PyOS_ascii_formatd to more accurately reflect it's meaning.
  Modified format.__float__ to use the new "Z" format as the default.
  Added test cases.
........
  r61435 | eric.smith | 2008-03-17 13:14:29 +0100 (Mo, 17 Mär 2008) | 2 lines

  Reformated lines > 79 chars.
  Deleted unused macro ISXDIGIT.
........
  r61436 | jeffrey.yasskin | 2008-03-17 15:40:53 +0100 (Mo, 17 Mär 2008) | 13 lines

  Allow Gnu gcc's to build python on OSX by removing -Wno-long-double,
  -no-cpp-precomp, and -mno-fused-madd from configure.
   * r22183 added -no-cpp-precomp, which
     http://gcc.gnu.org/ml/gcc/2005-12/msg00368.html claims hasn't been
     needed since gcc-3.1.
   * r25607 added -Wno-long-double to avoid a warning in
     Include/objimpl.h (issue 525481). The long double is still there,
     but OSX 10.4's gcc no longer warns about it.
   * r33666 fixed issue 775892 on OSX 10.3 by adding -mno-fused-madd,
     which changed the sign of some float 0s. Tim Peters said it wasn't
     a real issue anyway, and it no longer causes test failures.
  Fixes issue #1779871.
........
  r61439 | martin.v.loewis | 2008-03-17 17:31:57 +0100 (Mo, 17 Mär 2008) | 2 lines

  Add Trent Nelson.
........
  r61444 | travis.oliphant | 2008-03-17 18:36:12 +0100 (Mo, 17 Mär 2008) | 1 line

  Add necessary headers to back-port new buffer protocol to Python 2.6
........
  r61449 | gregory.p.smith | 2008-03-17 19:48:05 +0100 (Mo, 17 Mär 2008) | 8 lines

  Force zlib.crc32 and zlib.adler32 to return a signed integer on all platforms
  regardless of the native sizeof(long) used in the integer object.

  This somewhat odd behavior of returning a signed is maintained in 2.x for
  compatibility reasons of always returning an integer rather than a long object.

  Fixes Issue1202 for Python 2.6
........
  r61450 | neal.norwitz | 2008-03-17 20:02:45 +0100 (Mo, 17 Mär 2008) | 3 lines

  Use a buffer large enough to ensure we don't overrun, even if the value
  is outside the range we expect.
........
  r61453 | steven.bethard | 2008-03-17 20:33:11 +0100 (Mo, 17 Mär 2008) | 1 line

  Document unicode.isnumeric() and unicode.isdecimal() (issue2326)
........
  r61458 | neal.norwitz | 2008-03-17 21:22:43 +0100 (Mo, 17 Mär 2008) | 5 lines

  Issue 2321: reduce memory usage (increase the memory that is returned
  to the system) by using pymalloc for the data of unicode objects.

  Will backport.
........
  r61465 | martin.v.loewis | 2008-03-17 22:55:30 +0100 (Mo, 17 Mär 2008) | 2 lines

  Add David Wolever.
........
  r61468 | gregory.p.smith | 2008-03-18 01:20:01 +0100 (Di, 18 Mär 2008) | 3 lines

  Fix the IOError message text when opening a file with an invalid filename.
  Error reported by Ilan Schnell.
........
  r61471 | brett.cannon | 2008-03-18 02:00:07 +0100 (Di, 18 Mär 2008) | 2 lines

  Convert test_strftime, test_getargs, and test_pep247 to use unittest.
........
  r61472 | jeffrey.yasskin | 2008-03-18 02:09:59 +0100 (Di, 18 Mär 2008) | 2 lines

  Fix build on platforms that don't have intptr_t. Patch by Joseph Armbruster.
........
  r61473 | brett.cannon | 2008-03-18 02:50:25 +0100 (Di, 18 Mär 2008) | 2 lines

  Convert test_dummy_threading and test_dbm to unittest.
........
  r61474 | brett.cannon | 2008-03-18 02:58:56 +0100 (Di, 18 Mär 2008) | 2 lines

  Move test_extcall to doctest.
........
  r61480 | brett.cannon | 2008-03-18 04:46:22 +0100 (Di, 18 Mär 2008) | 2 lines

  test_errno was a no-op test; now it actually tests things and uses unittest.
........
  r61483 | brett.cannon | 2008-03-18 05:09:00 +0100 (Di, 18 Mär 2008) | 3 lines

  Remove our implementation of memmove() and strerror(); both are in the C89
  standard library.
........
  r61484 | brett.cannon | 2008-03-18 05:16:06 +0100 (Di, 18 Mär 2008) | 2 lines

  The output directory for tests that compare against stdout is now gone!
........
  r61488 | jeffrey.yasskin | 2008-03-18 05:29:35 +0100 (Di, 18 Mär 2008) | 2 lines

  Block the "socket.ssl() is deprecated" warning from test_socket_ssl.
........
  r61495 | jeffrey.yasskin | 2008-03-18 05:56:06 +0100 (Di, 18 Mär 2008) | 4 lines

  Speed test_thread up from 51.328s to 0.081s by reducing its sleep times. We
  still sleep at all to make it likely that all threads are active at the same
  time.
........
  r61496 | jeffrey.yasskin | 2008-03-18 06:12:41 +0100 (Di, 18 Mär 2008) | 4 lines

  Speed up test_dict by about 10x by only checking selected dict literal sizes,
  instead of every integer from 0 to 400. Exhaustive testing wastes time without
  providing enough more assurance that the code is correct.
........
  r61498 | neal.norwitz | 2008-03-18 06:20:29 +0100 (Di, 18 Mär 2008) | 1 line

  Try increasing the timeout to reduce the flakiness of this test.
........
  r61503 | brett.cannon | 2008-03-18 06:43:04 +0100 (Di, 18 Mär 2008) | 2 lines

  Improve the error message for a test that failed on the S-390 Debian buildbot.
........
  r61504 | jeffrey.yasskin | 2008-03-18 06:45:40 +0100 (Di, 18 Mär 2008) | 3 lines

  Add a -S/--slow flag to regrtest to have it print the 10 slowest tests with
  their times.
........
  r61507 | neal.norwitz | 2008-03-18 07:03:46 +0100 (Di, 18 Mär 2008) | 1 line

  Add some info to the failure messages
........
  r61509 | trent.nelson | 2008-03-18 08:02:12 +0100 (Di, 18 Mär 2008) | 1 line

  Issue 2286: bump up the stack size of the 64-bit debug python_d.exe to 2100000.  The default value of 200000 causes a stack overflow at 1965 iterations of r_object() in marshal.c, 35 iterations before the 2000 limit enforced by MAX_MARSHAL_STACK_DEPTH.
........
  r61510 | trent.nelson | 2008-03-18 08:32:47 +0100 (Di, 18 Mär 2008) | 5 lines

  The behaviour of winsound.Beep() seems to differ between different versions of Windows when there's either:
      a) no sound card entirely
      b) legacy beep driver has been disabled
      c) the legacy beep driver has been uninstalled
  Sometimes RuntimeErrors are raised, sometimes they're not.  If _have_soundcard() returns False, don't expect winsound.Beep() to raise a RuntimeError, as this clearly isn't the case, as demonstrated by the various Win32 XP buildbots.
........
  r61515 | martin.v.loewis | 2008-03-18 13:20:15 +0100 (Di, 18 Mär 2008) | 2 lines

  norwitz-amd64 (gentoo) has EREMOTEIO.
........
  r61516 | martin.v.loewis | 2008-03-18 13:45:37 +0100 (Di, 18 Mär 2008) | 2 lines

  Add more Linux error codes.
........
  r61517 | martin.v.loewis | 2008-03-18 14:05:03 +0100 (Di, 18 Mär 2008) | 2 lines

  Add WSA errors.
........
  r61518 | martin.v.loewis | 2008-03-18 14:16:05 +0100 (Di, 18 Mär 2008) | 2 lines

  Note that the stderr output of the test is intentional.
........
2008-03-18 15:15:01 +00:00

483 lines
15 KiB
Python
Raw Blame History

import parser
import unittest
import sys
from test import test_support
#
# First, we test that we can generate trees from valid source fragments,
# and that these valid trees are indeed allowed by the tree-loading side
# of the parser module.
#
class RoundtripLegalSyntaxTestCase(unittest.TestCase):
def roundtrip(self, f, s):
st1 = f(s)
t = st1.totuple()
try:
st2 = parser.sequence2st(t)
except parser.ParserError as why:
self.fail("could not roundtrip %r: %s" % (s, why))
self.assertEquals(t, st2.totuple(),
"could not re-generate syntax tree")
def check_expr(self, s):
self.roundtrip(parser.expr, s)
def check_suite(self, s):
self.roundtrip(parser.suite, s)
def test_yield_statement(self):
self.check_suite("def f(): yield 1")
self.check_suite("def f(): yield")
self.check_suite("def f(): x += yield")
self.check_suite("def f(): x = yield 1")
self.check_suite("def f(): x = y = yield 1")
self.check_suite("def f(): x = yield")
self.check_suite("def f(): x = y = yield")
self.check_suite("def f(): 1 + (yield)*2")
self.check_suite("def f(): (yield 1)*2")
self.check_suite("def f(): return; yield 1")
self.check_suite("def f(): yield 1; return")
self.check_suite("def f():\n"
" for x in range(30):\n"
" yield x\n")
self.check_suite("def f():\n"
" if (yield):\n"
" yield x\n")
def test_expressions(self):
self.check_expr("foo(1)")
self.check_expr("[1, 2, 3]")
self.check_expr("[x**3 for x in range(20)]")
self.check_expr("[x**3 for x in range(20) if x % 3]")
self.check_expr("[x**3 for x in range(20) if x % 2 if x % 3]")
self.check_expr("list(x**3 for x in range(20))")
self.check_expr("list(x**3 for x in range(20) if x % 3)")
self.check_expr("list(x**3 for x in range(20) if x % 2 if x % 3)")
self.check_expr("foo(*args)")
self.check_expr("foo(*args, **kw)")
self.check_expr("foo(**kw)")
self.check_expr("foo(key=value)")
self.check_expr("foo(key=value, *args)")
self.check_expr("foo(key=value, *args, **kw)")
self.check_expr("foo(key=value, **kw)")
self.check_expr("foo(a, b, c, *args)")
self.check_expr("foo(a, b, c, *args, **kw)")
self.check_expr("foo(a, b, c, **kw)")
self.check_expr("foo + bar")
self.check_expr("foo - bar")
self.check_expr("foo * bar")
self.check_expr("foo / bar")
self.check_expr("foo // bar")
self.check_expr("lambda: 0")
self.check_expr("lambda x: 0")
self.check_expr("lambda *y: 0")
self.check_expr("lambda *y, **z: 0")
self.check_expr("lambda **z: 0")
self.check_expr("lambda x, y: 0")
self.check_expr("lambda foo=bar: 0")
self.check_expr("lambda foo=bar, spaz=nifty+spit: 0")
self.check_expr("lambda foo=bar, **z: 0")
self.check_expr("lambda foo=bar, blaz=blat+2, **z: 0")
self.check_expr("lambda foo=bar, blaz=blat+2, *y, **z: 0")
self.check_expr("lambda x, *y, **z: 0")
self.check_expr("(x for x in range(10))")
self.check_expr("foo(x for x in range(10))")
def test_simple_expression(self):
# expr_stmt
self.check_suite("a")
def test_simple_assignments(self):
self.check_suite("a = b")
self.check_suite("a = b = c = d = e")
def test_simple_augmented_assignments(self):
self.check_suite("a += b")
self.check_suite("a -= b")
self.check_suite("a *= b")
self.check_suite("a /= b")
self.check_suite("a //= b")
self.check_suite("a %= b")
self.check_suite("a &= b")
self.check_suite("a |= b")
self.check_suite("a ^= b")
self.check_suite("a <<= b")
self.check_suite("a >>= b")
self.check_suite("a **= b")
def test_function_defs(self):
self.check_suite("def f(): pass")
self.check_suite("def f(*args): pass")
self.check_suite("def f(*args, **kw): pass")
self.check_suite("def f(**kw): pass")
self.check_suite("def f(foo=bar): pass")
self.check_suite("def f(foo=bar, *args): pass")
self.check_suite("def f(foo=bar, *args, **kw): pass")
self.check_suite("def f(foo=bar, **kw): pass")
self.check_suite("def f(a, b): pass")
self.check_suite("def f(a, b, *args): pass")
self.check_suite("def f(a, b, *args, **kw): pass")
self.check_suite("def f(a, b, **kw): pass")
self.check_suite("def f(a, b, foo=bar): pass")
self.check_suite("def f(a, b, foo=bar, *args): pass")
self.check_suite("def f(a, b, foo=bar, *args, **kw): pass")
self.check_suite("def f(a, b, foo=bar, **kw): pass")
self.check_suite("@staticmethod\n"
"def f(): pass")
self.check_suite("@staticmethod\n"
"@funcattrs(x, y)\n"
"def f(): pass")
self.check_suite("@funcattrs()\n"
"def f(): pass")
def test_class_defs(self):
self.check_suite("class foo():pass")
self.check_suite("class foo(object):pass")
def test_import_from_statement(self):
self.check_suite("from sys.path import *")
self.check_suite("from sys.path import dirname")
self.check_suite("from sys.path import (dirname)")
self.check_suite("from sys.path import (dirname,)")
self.check_suite("from sys.path import dirname as my_dirname")
self.check_suite("from sys.path import (dirname as my_dirname)")
self.check_suite("from sys.path import (dirname as my_dirname,)")
self.check_suite("from sys.path import dirname, basename")
self.check_suite("from sys.path import (dirname, basename)")
self.check_suite("from sys.path import (dirname, basename,)")
self.check_suite(
"from sys.path import dirname as my_dirname, basename")
self.check_suite(
"from sys.path import (dirname as my_dirname, basename)")
self.check_suite(
"from sys.path import (dirname as my_dirname, basename,)")
self.check_suite(
"from sys.path import dirname, basename as my_basename")
self.check_suite(
"from sys.path import (dirname, basename as my_basename)")
self.check_suite(
"from sys.path import (dirname, basename as my_basename,)")
def test_basic_import_statement(self):
self.check_suite("import sys")
self.check_suite("import sys as system")
self.check_suite("import sys, math")
self.check_suite("import sys as system, math")
self.check_suite("import sys, math as my_math")
def test_pep263(self):
self.check_suite("# -*- coding: iso-8859-1 -*-\n"
"pass\n")
def test_assert(self):
self.check_suite("assert alo < ahi and blo < bhi\n")
def test_position(self):
# An absolutely minimal test of position information. Better
# tests would be a big project.
code = "def f(x):\n return x + 1\n"
st1 = parser.suite(code)
st2 = st1.totuple(line_info=1, col_info=1)
def walk(tree):
node_type = tree[0]
next = tree[1]
if isinstance(next, tuple):
for elt in tree[1:]:
for x in walk(elt):
yield x
else:
yield tree
terminals = list(walk(st2))
self.assertEqual([
(1, 'def', 1, 0),
(1, 'f', 1, 4),
(7, '(', 1, 5),
(1, 'x', 1, 6),
(8, ')', 1, 7),
(11, ':', 1, 8),
(4, '', 1, 9),
(5, '', 2, -1),
(1, 'return', 2, 4),
(1, 'x', 2, 11),
(14, '+', 2, 13),
(2, '1', 2, 15),
(4, '', 2, 16),
(6, '', 2, -1),
(4, '', 2, -1),
(0, '', 2, -1)],
terminals)
#
# Second, we take *invalid* trees and make sure we get ParserError
# rejections for them.
#
class IllegalSyntaxTestCase(unittest.TestCase):
def check_bad_tree(self, tree, label):
try:
parser.sequence2st(tree)
except parser.ParserError:
pass
else:
self.fail("did not detect invalid tree for %r" % label)
def test_junk(self):
# not even remotely valid:
self.check_bad_tree((1, 2, 3), "<junk>")
def test_illegal_yield_1(self):
# Illegal yield statement: def f(): return 1; yield 1
tree = \
(257,
(264,
(285,
(259,
(1, 'def'),
(1, 'f'),
(260, (7, '('), (8, ')')),
(11, ':'),
(291,
(4, ''),
(5, ''),
(264,
(265,
(266,
(272,
(275,
(1, 'return'),
(313,
(292,
(293,
(294,
(295,
(297,
(298,
(299,
(300,
(301,
(302, (303, (304, (305, (2, '1')))))))))))))))))),
(264,
(265,
(266,
(272,
(276,
(1, 'yield'),
(313,
(292,
(293,
(294,
(295,
(297,
(298,
(299,
(300,
(301,
(302,
(303, (304, (305, (2, '1')))))))))))))))))),
(4, ''))),
(6, ''))))),
(4, ''),
(0, ''))))
self.check_bad_tree(tree, "def f():\n return 1\n yield 1")
def test_illegal_yield_2(self):
# Illegal return in generator: def f(): return 1; yield 1
tree = \
(257,
(264,
(265,
(266,
(278,
(1, 'from'),
(281, (1, '__future__')),
(1, 'import'),
(279, (1, 'generators')))),
(4, ''))),
(264,
(285,
(259,
(1, 'def'),
(1, 'f'),
(260, (7, '('), (8, ')')),
(11, ':'),
(291,
(4, ''),
(5, ''),
(264,
(265,
(266,
(272,
(275,
(1, 'return'),
(313,
(292,
(293,
(294,
(295,
(297,
(298,
(299,
(300,
(301,
(302, (303, (304, (305, (2, '1')))))))))))))))))),
(264,
(265,
(266,
(272,
(276,
(1, 'yield'),
(313,
(292,
(293,
(294,
(295,
(297,
(298,
(299,
(300,
(301,
(302,
(303, (304, (305, (2, '1')))))))))))))))))),
(4, ''))),
(6, ''))))),
(4, ''),
(0, ''))))
self.check_bad_tree(tree, "def f():\n return 1\n yield 1")
def test_a_comma_comma_c(self):
# Illegal input: a,,c
tree = \
(258,
(311,
(290,
(291,
(292,
(293,
(295,
(296,
(297,
(298, (299, (300, (301, (302, (303, (1, 'a')))))))))))))),
(12, ','),
(12, ','),
(290,
(291,
(292,
(293,
(295,
(296,
(297,
(298, (299, (300, (301, (302, (303, (1, 'c'))))))))))))))),
(4, ''),
(0, ''))
self.check_bad_tree(tree, "a,,c")
def test_illegal_operator(self):
# Illegal input: a $= b
tree = \
(257,
(264,
(265,
(266,
(267,
(312,
(291,
(292,
(293,
(294,
(296,
(297,
(298,
(299,
(300, (301, (302, (303, (304, (1, 'a'))))))))))))))),
(268, (37, '$=')),
(312,
(291,
(292,
(293,
(294,
(296,
(297,
(298,
(299,
(300, (301, (302, (303, (304, (1, 'b'))))))))))))))))),
(4, ''))),
(0, ''))
self.check_bad_tree(tree, "a $= b")
def test_malformed_global(self):
#doesn't have global keyword in ast
tree = (257,
(264,
(265,
(266,
(282, (1, 'foo'))), (4, ''))),
(4, ''),
(0, ''))
self.check_bad_tree(tree, "malformed global ast")
class CompileTestCase(unittest.TestCase):
# These tests are very minimal. :-(
def test_compile_expr(self):
st = parser.expr('2 + 3')
code = parser.compilest(st)
self.assertEquals(eval(code), 5)
def test_compile_suite(self):
st = parser.suite('x = 2; y = x + 3')
code = parser.compilest(st)
globs = {}
exec(code, globs)
self.assertEquals(globs['y'], 5)
def test_compile_error(self):
st = parser.suite('1 = 3 + 4')
self.assertRaises(SyntaxError, parser.compilest, st)
def test_compile_badunicode(self):
st = parser.suite('a = "\\U12345678"')
self.assertRaises(SyntaxError, parser.compilest, st)
st = parser.suite('a = "\\u1"')
self.assertRaises(SyntaxError, parser.compilest, st)
class ParserStackLimitTestCase(unittest.TestCase):
"""try to push the parser to/over it's limits.
see http://bugs.python.org/issue1881 for a discussion
"""
def _nested_expression(self, level):
return "["*level+"]"*level
def test_deeply_nested_list(self):
# XXX used to be 99 levels in 2.x
e = self._nested_expression(93)
st = parser.expr(e)
st.compile()
def test_trigger_memory_error(self):
e = self._nested_expression(100)
print("Expecting 's_push: parser stack overflow' in next line",
file=sys.stderr)
self.assertRaises(MemoryError, parser.expr, e)
def test_main():
test_support.run_unittest(
RoundtripLegalSyntaxTestCase,
IllegalSyntaxTestCase,
CompileTestCase,
ParserStackLimitTestCase,
)
if __name__ == "__main__":
test_main()
Reference in a new issue View git blame Copy permalink