gh-126500: test_ssl: Don't stop ThreadedEchoServer on OSError in ConnectionHandler; rely on __exit__ (GH-126503)
If `read()` in the ConnectionHandler thread raises `OSError` (except `ConnectionError`),
the ConnectionHandler shuts down the entire ThreadedEchoServer,
preventing further connections.
It also does that for `EPROTOTYPE` in `wrap_conn`.
As far as I can see, this is done to avoid the server thread getting stuck,
forgotten, in its accept loop. However, since 2011 (5b95eb90a7)
the server is used as a context manager, and its `__exit__` does `stop()` and `join()`.
(I'm not sure if we *always* used `with` since that commit, but currently we do.)
Make sure that the context manager *is* used, and remove the `server.stop()`
calls from ConnectionHandler.
(cherry picked from commit c9cda1608e)
Co-authored-by: Petr Viktorin <encukou@gmail.com>
gh-126499: test_ssl: Don't assume err.reason is a string (GH-126501)
The skipping machinery called `getattr(err, "reason", "")` on an arbitrary
exception. As intermittent Buildbot failures show, sometimes it's set
to None.
Convert it to string for this specific check.
(cherry picked from commit 78ad7e6322)
Co-authored-by: Petr Viktorin <encukou@gmail.com>
It is also no longer use __len__().
(cherry picked from commit 812360fddd)
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* gh-108834: regrtest reruns failed tests in subprocesses (#108839)
When using --rerun option, regrtest now re-runs failed tests
in verbose mode in fresh worker processes to have more
deterministic behavior. So it can write its final report even
if a test killed a worker progress.
Add --fail-rerun option to regrtest: exit with non-zero exit code
if a test failed pass passed when re-run in verbose mode (in a
fresh process). That's now more useful since tests can pass
when re-run in a fresh worker progress, whereas they failed
when run after other tests when tests are run sequentially.
Rename --verbose2 option (-w) to --rerun. Keep --verbose2 as a
deprecated alias.
Changes:
* Fix and enhance statistics in regrtest summary. Add "(filtered)"
when --match and/or --ignore options are used.
* Add RunTests class.
* Add TestResult.get_rerun_match_tests() method
* Rewrite code to serialize/deserialize worker arguments as JSON
using a new WorkerJob class.
* Fix stats when a test is run with --forever --rerun.
* If failed test names cannot be parsed, log a warning and don't
filter tests.
* test_regrtest.test_rerun_success() now uses a marker file, since
the test is re-run in a separated process.
* Add tests on normalize_test_name() function.
* Add test_success() and test_skip() tests to test_regrtest.
(cherry picked from commit 31c2945f14)
* gh-108834: regrtest --fail-rerun exits with code 5 (#108896)
When the --fail-rerun option is used and a test fails and then pass,
regrtest now uses exit code 5 ("rerun) instead of 2 ("bad test").
(cherry picked from commit 1170d5a292)
* gh-108416: Mark slow but not CPU bound test methods with requires_resource('walltime') (GH-108480)
(cherry picked from commit 1e0d62793a)
* Manually sync Lib/test/libregrtest/ from main
---------
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* In preauth tests of test_ssl, explicitly break reference cycles
invoving SingleConnectionTestServerThread to make sure that the
thread is deleted. Otherwise, the test marks the environment as
altered because the threading module sees a "dangling thread"
(SingleConnectionTestServerThread). This test leak was introduced
by the test added for the fix of issue gh-108310.
* Use support.SHORT_TIMEOUT instead of hardcoded 1.0 or 2.0 seconds
timeout.
* SingleConnectionTestServerThread.run() catchs TimeoutError
* Fix a race condition (missing synchronization) in
test_preauth_data_to_tls_client(): the server now waits until the
client connect() completed in call_after_accept().
* test_https_client_non_tls_response_ignored() calls server.join()
explicitly.
* Replace "localhost" with server.listener.getsockname()[0].
(cherry picked from commit 592bacb6fc)
Co-authored-by: Victor Stinner <vstinner@python.org>
Instances of `ssl.SSLSocket` were vulnerable to a bypass of the TLS handshake
and included protections (like certificate verification) and treating sent
unencrypted data as if it were post-handshake TLS encrypted data.
The vulnerability is caused when a socket is connected, data is sent by the
malicious peer and stored in a buffer, and then the malicious peer closes the
socket within a small timing window before the other peers’ TLS handshake can
begin. After this sequence of events the closed socket will not immediately
attempt a TLS handshake due to not being connected but will also allow the
buffered data to be read as if a successful TLS handshake had occurred.
Co-authored-by: Gregory P. Smith [Google LLC] <greg@krypto.org>
gh-106687: _ssl: use uint64_t for SSL options (GH-106700)
SSL_CTX_get_options() uses uint64_t for options:
https://www.openssl.org/docs/man3.1/man3/SSL_CTX_get_options.html
Fix this compiler warning on Windows with MSC:
conversion from 'uint64_t' to 'long', possible loss of data
(cherry picked from commit ad95c7253a)
Co-authored-by: Victor Stinner <vstinner@python.org>
In PEM, we need to parse until error and then suppress `PEM_R_NO_START_LINE`, because PEM allows arbitrary leading and trailing data. DER, however, does not. Parsing until error and suppressing `ASN1_R_HEADER_TOO_LONG` doesn't quite work because that error also covers some cases that should be rejected.
Instead, check `BIO_eof` early and stop the loop that way.
Automerge-Triggered-By: GH:Yhg1s
GH-25309 enabled SSL_OP_IGNORE_UNEXPECTED_EOF by default, with a comment
that it restores OpenSSL 1.1.1 behavior, but this wasn't quite right.
That option causes OpenSSL to treat transport EOF as the same as
close_notify (i.e. SSL_ERROR_ZERO_RETURN), whereas Python actually has
distinct SSLEOFError and SSLZeroReturnError exceptions. (The latter is
usually mapped to a zero return from read.) In OpenSSL 1.1.1, the ssl
module would raise them for transport EOF and close_notify,
respectively. In OpenSSL 3.0, both act like close_notify.
Fix this by, instead, just detecting SSL_R_UNEXPECTED_EOF_WHILE_READING
and mapping that to the other exception type.
There doesn't seem to have been any unit test of this error, so fill in
the missing one. This had to be done with the BIO path because it's
actually slightly tricky to simulate a transport EOF with Python's fd
based APIs. (If you instruct the server to close the socket, it gets
confused, probably because the server's SSL object is still referencing
the now dead fd?)
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Christian Heimes <christian@python.org>
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Fixes https://github.com/python/cpython/issues/89051
This appears to be a typo. It causes try_protocol_combo to try to turn
on SSL 3.0 when testing PROTOCOL_SSLv23 (aka PROTOCOL_TLS), which
doesn't make any sense. Fix it to be PROTOCOL_SSLv3.
Without this, try_protocol_combo is actually setting
context.minimum_version to SSLv3 when called as
try_protocol_combo(ssl.PROTOCOL_TLS, ssl.PROTOCOL_TLS, True)
One would think this causes a no-ssl3 OpenSSL build to fail, but OpenSSL
forgot to make SSL_CTX_set_min_proto_version(SSL3_VERSION) does not
notice no-ssl3, so this typo has gone undetected. But we should still
fix the typo because, presumably, a future version of OpenSSL will
remove SSL 3.0 and do so more thoroughly, at which point this will
break.
Remove the ssl.wrap_socket() function, deprecated in Python 3.7:
instead, create a ssl.SSLContext object and call its
sl.SSLContext.wrap_socket() method. Any package that still uses
ssl.wrap_socket() is broken and insecure. The function neither sends
a SNI TLS extension nor validates server hostname. Code is subject to
CWE-295 : Improper Certificate Validation.
Remove dead code related to ssl.PROTOCOL_SSLv2. ssl.PROTOCOL_SSLv2
was already removed in Python 3.10.
In test_ssl, @requires_tls_version('SSLv2') always returned False.
Extract of the removed code: "OpenSSL has removed support for SSLv2".
Add methods enterContext() and enterClassContext() in TestCase.
Add method enterAsyncContext() in IsolatedAsyncioTestCase.
Add function enterModuleContext().
- skip doctest that changes depending on target system
- skip doctest that only fails on CI
- substitute in values that change depending on target system
Undo rejected PEP-663 changes:
- restore `repr()` to its 3.10 status
- restore `str()` to its 3.10 status
New changes:
- `IntEnum` and `IntFlag` now leave `__str__` as the original `int.__str__` so that str() and format() return the same result
- zero-valued flags without a name have a slightly changed repr(), e.g. `repr(Color(0)) == '<Color: 0>'`
- update `dir()` for mixed-in types to return all the methods and attributes of the mixed-in type
- added `_numeric_repr_` to `Flag` to control display of unnamed values
- enums without doc strings have a more comprehensive doc string added
- `ReprEnum` added -- inheriting from this makes it so only `__repr__` is replaced, not `__str__` nor `__format__`; `IntEnum`, `IntFlag`, and `StrEnum` all inherit from `ReprEnum`
Remove the asyncore and asynchat modules, deprecated in Python
3.6: use the asyncio module instead.
Remove the smtpd module, deprecated in Python 3.6: the aiosmtpd
module can be used instead, it is based on asyncio.
* Remove asyncore, asynchat and smtpd documentation
* Remove test_asyncore, test_asynchat and test_smtpd
* Rename Lib/asynchat.py to Lib/test/support/_asynchat.py
* Rename Lib/asyncore.py to Lib/test/support/_asyncore.py
* Rename Lib/smtpd.py to Lib/test/support/_smtpd.py
* Remove DeprecationWarning from private _asyncore, _asynchat and
_smtpd modules
* _smtpd: remove deprecated properties
Instead of explicitly enumerate test classes for run_unittest()
use the unittest ability to discover tests. This also makes these
tests discoverable and runnable with unittest.
load_tests() can be used for dynamic generating tests and adding
doctests. setUpModule(), tearDownModule() and addModuleCleanup()
can be used for running code before and after all module tests.
