Commit graph

225 commits

Author SHA1 Message Date
Miss Islington (bot)
571ec56b65
[3.11] gh-105375: Improve error handling in sqlite3 collation callback (GH-105412) (#105441)
Check for error after each call to PyUnicode_FromStringAndSize().
(cherry picked from commit a24a780d93)

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-06-07 21:22:01 +00:00
Erlend E. Aasland
681d5028bd
[3.11] gh-100370: fix OverflowError in sqlite3.Connection.blobopen for 32-bit builds (#103902) (#104285) 2023-05-08 08:41:34 +00:00
Erlend Egeberg Aasland
5d3f2fbf37
[3.11] gh-95132: Correctly relay *args and **kwds from sqlite3.connect to factory (GH-95146) (#95158)
This PR partially reverts gh-24421 (PR) and fixes the remaining concerns
given in gh-93044 (issue):

- keyword arguments are passed as positional arguments to factory()
- if an argument is not passed to sqlite3.connect(), its default value
  is passed to factory()

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>.
(cherry picked from commit a3d4d15f53)

Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
2022-07-23 10:23:19 +02:00
Miss Islington (bot)
8b068c4b0c
gh-93925: Improve clarity of sqlite3 commit/rollback, and close docs (GH-93926)
Co-authored-by: CAM Gerlach <CAM.Gerlach@Gerlach.CAM>
(cherry picked from commit 6446592c89)

Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
2022-06-19 12:44:48 -07:00
Miss Islington (bot)
f384a8e477
gh-92434: Silence compiler warning in Modules/_sqlite/connection.c on 32-bit systems (GH-93090)
(cherry picked from commit d8395eb38d)

Co-authored-by: neonene <53406459+neonene@users.noreply.github.com>
2022-06-10 01:37:38 -07:00
Miss Islington (bot)
6ec050f633
gh-91922: Fix sqlite connection on nonstardard locales and paths (GH-92926)
(cherry picked from commit d853758092)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2022-05-20 02:42:48 -07:00
Erlend Egeberg Aasland
d9ec553194
gh-89289: Fix compiler warning in _sqlite/connection.c (#92258) 2022-05-03 22:21:56 +02:00
Erlend Egeberg Aasland
c278474df9
gh-89289: Harden sqlite3.Connection init (#92214)
- Make sure SQLite resources are freed if database open fails
- Remove unneeded branches if init is aborted
2022-05-03 12:18:11 -06:00
Erlend Egeberg Aasland
1d4a9a45b7
gh-92206: Improve scoping of sqlite3 register cursor helper (#92212) 2022-05-03 06:33:24 -06:00
Erlend Egeberg Aasland
721aa96540
gh-89301: Fix regression with bound values in traced SQLite statements (#92053) 2022-05-02 08:14:35 -06:00
Erlend Egeberg Aasland
c06a4ffe81
gh-69093: improve sqlite3.Connection.blobopen() error handling (GH-91571)
Unless sqlite3_blob_open() returns SQLITE_MISUSE, the error code and
message are available on the connection object. This means we have to
handle SQLITE_MISUSE error messages explicitly.
2022-04-15 09:27:39 -07:00
Erlend Egeberg Aasland
ee475430d4
gh-69093: Support basic incremental I/O to blobs in sqlite3 (GH-30680)
Authored-by: Aviv Palivoda <palaviv@gmail.com>
Co-authored-by: Erlend E. Aasland <erlend.aasland@innova.no>
Co-authored-by: palaviv <palaviv@gmail.com>
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2022-04-14 17:02:56 -07:00
Erlend Egeberg Aasland
9ebcece82f
gh-79097: Add support for aggregate window functions in sqlite3 (GH-20903) 2022-04-11 17:55:59 -07:00
Erlend Egeberg Aasland
a7551247e7
bpo-41930: Add support for SQLite serialise/deserialise API (GH-26728)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
2022-04-05 07:15:25 -07:00
Erlend Egeberg Aasland
4674fd4e93
bpo-44859: Raise more accurate exceptions in sqlite3 (GH-27695)
* Improve exception compliance with PEP 249
* Raise InterfaceError instead of ProgrammingError for SQLITE_MISUSE.
  If SQLITE_MISUSE is raised, it is a sqlite3 module bug. Users of the
  sqlite3 module are not responsible for using the SQLite C API correctly.
* Don't overwrite BufferError with ValueError when conversion to BLOB fails.
* Raise ProgrammingError instead of Warning if user tries to execute() more
  than one SQL statement.
* Raise ProgrammingError instead of ValueError if an SQL query contains null characters.
* Make sure `_pysqlite_set_result` raises an exception if it returns -1.
2022-03-16 22:58:25 -07:00
Erlend Egeberg Aasland
e801e88744
bpo-45138: Revert GH-28240: Expand traced SQL statements (GH-31788)
This reverts commit d1777515f9.

Automerge-Triggered-By: GH:JelleZijlstra
2022-03-09 09:39:49 -08:00
Erlend Egeberg Aasland
d1777515f9
bpo-45138: Expand traced SQL statements in sqlite3 trace callback (GH-28240) 2022-03-08 18:46:40 -08:00
Erlend Egeberg Aasland
4d95fa1ac5
bpo-46878: Purge 'non-standard' from sqlite3 docstrings (GH-31612) 2022-03-07 20:18:41 -08:00
Erlend Egeberg Aasland
88567a9970
bpo-46874: Speed up sqlite3 user-defined aggregate 'step' method (GH-31604) 2022-03-03 22:54:36 +09:00
Erlend Egeberg Aasland
b2077117d1
bpo-46541: Replace _Py_IDENTIFIER with _Py_ID in sqlite3 (GH-31351) 2022-02-17 00:24:44 +09:00
Eric Snow
81c72044a1
bpo-46541: Replace core use of _Py_IDENTIFIER() with statically initialized global objects. (gh-30928)
We're no longer using _Py_IDENTIFIER() (or _Py_static_string()) in any core CPython code.  It is still used in a number of non-builtin stdlib modules.

The replacement is: PyUnicodeObject (not pointer) fields under _PyRuntimeState, statically initialized as part of _PyRuntime.  A new _Py_GET_GLOBAL_IDENTIFIER() macro facilitates lookup of the fields (along with _Py_GET_GLOBAL_STRING() for non-identifier strings).

https://bugs.python.org/issue46541#msg411799 explains the rationale for this change.

The core of the change is in:

* (new) Include/internal/pycore_global_strings.h - the declarations for the global strings, along with the macros
* Include/internal/pycore_runtime_init.h - added the static initializers for the global strings
* Include/internal/pycore_global_objects.h - where the struct in pycore_global_strings.h is hooked into _PyRuntimeState
* Tools/scripts/generate_global_objects.py - added generation of the global string declarations and static initializers

I've also added a --check flag to generate_global_objects.py (along with make check-global-objects) to check for unused global strings.  That check is added to the PR CI config.

The remainder of this change updates the core code to use _Py_GET_GLOBAL_IDENTIFIER() instead of _Py_IDENTIFIER() and the related _Py*Id functions (likewise for _Py_GET_GLOBAL_STRING() instead of _Py_static_string()).  This includes adding a few functions where there wasn't already an alternative to _Py*Id(), replacing the _Py_Identifier * parameter with PyObject *.

The following are not changed (yet):

* stop using _Py_IDENTIFIER() in the stdlib modules
* (maybe) get rid of _Py_IDENTIFIER(), etc. entirely -- this may not be doable as at least one package on PyPI using this (private) API
* (maybe) intern the strings during runtime init

https://bugs.python.org/issue46541
2022-02-08 13:39:07 -07:00
Erlend Egeberg Aasland
3eb3b4f270
bpo-43853: Expand test suite for SQLite UDF's (GH-27642) 2022-01-26 08:26:16 -08:00
Erlend Egeberg Aasland
9d6a239a34
bpo-44092: Don't reset statements/cursors before rollback (GH-26026)
In SQLite versions pre 3.7.11, pending statements would block a rollback.  This is no longer the case, so remove the workaround.
2022-01-03 19:02:39 +00:00
Erlend Egeberg Aasland
c4a69a4ad0
bpo-45828: Use unraisable exceptions within sqlite3 callbacks (FH-29591) 2021-11-29 15:22:32 +00:00
Erlend Egeberg Aasland
0920b61a0c
bpo-45512: Use Argument Clinic to set sqlite3 isolation level (GH-29593) 2021-11-18 18:18:09 +09:00
Dong-hee Na
e002bbc6cc
bpo-45512: Simplify manage isolation level (GH-29562) 2021-11-17 21:47:02 +09:00
Erlend Egeberg Aasland
9d6215a54c
bpo-45126: Harden sqlite3 connection initialisation (GH-28227) 2021-11-16 15:53:35 +01:00
Erlend Egeberg Aasland
822c3dcce3
bpo-45512: Raise exception if sqlite3.Connection.__init__ is called with bad isolation level (#29561)
* bpo-45512: Raise sqlite3.Connection.__init__ is called with bad isolation level

* Also explicitly test allowed isolation levels

* Use subTest for better error messages if something goes wrong

* Update Lib/test/test_sqlite3/test_dbapi.py

Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>

Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
2021-11-15 21:55:38 +09:00
Erlend Egeberg Aasland
b567b9d74b
bpo-45512: Simplify isolation_level handling in sqlite3 (GH-29053) 2021-11-15 16:50:59 +09:00
Christian Heimes
e9594f6747
bpo-45731: Handle --enable-loadable-sqlite-extensions in configure (GH-29434) 2021-11-06 10:30:37 +01:00
Erlend Egeberg Aasland
b6b38a8226
bpo-45243: Add support for setting/getting sqlite3 connection limits (GH-28463) 2021-11-01 22:50:53 +00:00
Erlend Egeberg Aasland
e2e62b3808
bpo-45581: Raise MemoryError in sqlite3.connect if SQLite signals memory error (GH-29171) 2021-10-29 22:21:58 +02:00
Erlend Egeberg Aasland
8f24b7dbcb
bpo-42064: Convert sqlite3 global state to module state (GH-29073) 2021-10-27 13:12:21 +02:00
Erlend Egeberg Aasland
09c04e7f0d
bpo-42064: Add module backref to sqlite3 callback context (GH-28242) 2021-10-19 15:44:45 +02:00
Erlend Egeberg Aasland
cfb1df3b71
bpo-44991: Normalise function and collation callback naming (GH-28209) 2021-10-12 13:38:49 +02:00
Victor Stinner
d943d19172
bpo-45439: Move _PyObject_CallNoArgs() to pycore_call.h (GH-28895)
* Move _PyObject_CallNoArgs() to pycore_call.h (internal C API).
* _ssl, _sqlite and _testcapi extensions now call the public
  PyObject_CallNoArgs() function, rather than _PyObject_CallNoArgs().
* _lsprof extension is now built with Py_BUILD_CORE_MODULE macro
  defined to get access to internal _PyObject_CallNoArgs().
2021-10-12 08:38:19 +02:00
Victor Stinner
ce3489cfdb
bpo-45439: Rename _PyObject_CallNoArg() to _PyObject_CallNoArgs() (GH-28891)
Fix typo in the private _PyObject_CallNoArg() function name: rename
it to _PyObject_CallNoArgs() to be consistent with the public
function PyObject_CallNoArgs().
2021-10-12 00:42:23 +02:00
Erlend Egeberg Aasland
771a546713
bpo-45040: Simplify sqlite3 transaction control functions (GH-28019) 2021-09-19 23:51:36 +01:00
Erlend Egeberg Aasland
c78d5ca380
bpo-45126: Fix ref. leak in sqlite3.Connection.__init__ (GH-28231) 2021-09-12 21:27:42 +09:00
Erlend Egeberg Aasland
979336de34
bpo-42064: Pass module state to trace, progress, and authorizer callbacks (GH-27940)
- add print-or-clear traceback helper
- add helpers to clear and visit saved contexts
- modify callbacks to use the new callback_context struct
2021-09-07 15:06:17 +02:00
Erlend Egeberg Aasland
0474d06008
bpo-44991: Normalise sqlite3 callback naming (GH-28088)
- all callbacks are now named xxx_callback
- normalise callable naming in set_*() functions
- normalise context argument naming in callbacks

The sqlite code is being "touched" in bpo-42064 (and related issues);
this style change makes it easier to work with and review.
2021-09-07 13:43:44 +02:00
Petr Viktorin
01dea5f12b
bpo-42064: Offset arguments for PyObject_Vectorcall in the _sqlite module (GH-27931)
This allows e.g. methods to be called efficiently by providing
space for a "self" argument; see PY_VECTORCALL_ARGUMENTS_OFFSET docs.
2021-08-31 14:34:44 +02:00
Erlend Egeberg Aasland
001ef4600f
bpo-44991: Make GIL handling more explicit in sqlite3 callbacks (GH-27934)
- acquire the GIL at the very start[1]
- release the GIL at the very end

[1] The trace callback performs a sanity check before acquiring the GIL

Automerge-Triggered-By: GH:encukou
2021-08-31 05:18:43 -07:00
Erlend Egeberg Aasland
a3c11cebf1
bpo-27334: Fix reference leak introduced by GH-26202 (GH-27942) 2021-08-25 15:57:54 +02:00
Erlend Egeberg Aasland
7ecd3425d4
bpo-27334: roll back transaction if sqlite3 context manager fails to commit (GH-26202)
Co-authored-by: Luca Citi
Co-authored-by: Berker Peksag <berker.peksag@gmail.com>
2021-08-25 11:59:42 +01:00
Erlend Egeberg Aasland
9ed523159c
bpo-42064: Pass module state to sqlite3 UDF callbacks (GH-27456)
- Establish common callback context struct
- Convert UDF callbacks to fetch module state from callback context
2021-08-24 05:24:09 -07:00
Erlend Egeberg Aasland
243b6c3b8f
bpo-44079: Strip superfluous statement cache from sqlite3.Connection (GH-25998) 2021-08-19 00:37:53 +01:00
Serhiy Storchaka
0eec6276fd
bpo-44859: Improve error handling in sqlite3 and and raise more accurate exceptions. (GH-27654)
* MemoryError is now raised instead of sqlite3.Warning when
  memory is not enough for encoding a statement to UTF-8
  in Connection.__call__() and Cursor.execute().
* UnicodEncodeError is now raised instead of sqlite3.Warning when
  the statement contains surrogate characters
  in Connection.__call__() and Cursor.execute().
* TypeError is now raised instead of ValueError for non-string
  script argument in Cursor.executescript().
* ValueError is now raised for script containing the null
  character instead of truncating it in Cursor.executescript().
* Correctly handle exceptions raised when getting boolean value
  of the result of the progress handler.
* Add many tests covering different corner cases.

Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
2021-08-08 08:49:44 +03:00
Serhiy Storchaka
7d747f26e6
bpo-44839: Raise more specific errors in sqlite3 (GH-27613)
MemoryError raised in user-defined functions will now preserve
its type. OverflowError will now be converted to DataError.
Previously both were converted to OperationalError.
2021-08-06 21:28:47 +03:00
Erlend Egeberg Aasland
8f010dc920
bpo-44822: Don't truncate strs with embedded NULL chars returned by sqlite3 UDF callbacks (GH-27588) 2021-08-05 10:22:08 +03:00