Commit graph

533 commits

Author SHA1 Message Date
Simon Charette
83c9765f45 Refs #33766 -- Removed sql.Query.build_filtered_relation_q().
It was a copy of sql.Query._add_q that avoided JOIN updates.
2023-04-24 08:32:49 +02:00
Simon Charette
d660cee5bc Fixed #33766 -- Resolved FilteredRelation.condition at referencing time.
The previous implementation resolved condition at Join compilation time
which required introducing a specialized expression resolving mode to
alter the join reuse logic solely during that phase.

FilteredRelation.condition is now resolved when the relation is first
referenced which maintains the existing behavior while allowing the
removal of the specialized resolving mode and address an issue where
conditions couldn't spawn new joins.
2023-04-24 08:32:44 +02:00
Simon Charette
9daf8b4109 Fixed #34464 -- Fixed queryset aggregation over group by reference.
Regression in 59bea9efd2.

Refs #28477.

Thanks Ian Cubitt for the report.
2023-04-07 06:57:32 +02:00
Simon Charette
87c63bd8df Fixed #34458 -- Fixed QuerySet.defer() crash on attribute names.
Thanks Andrew Cordery for the report.

Regression in b3db6c8dcb.
2023-04-05 05:38:10 +02:00
Simon Charette
0e1aae7a5f Fixed #34450 -- Fixed multi-valued JOIN reuse when filtering by expressions.
Thanks Roman Odaisky for the report.
2023-04-04 14:35:21 +02:00
Mariusz Felisiak
3afdc9e9b4
Refs #29799 -- Added field instance lookups to suggestions in FieldErrors.
Bug in cd1afd553f.
2023-03-28 19:18:48 +02:00
Simon Charette
cb13792938
Fixed #34437 -- Made values() resolving error mention selected annotations.
While the add_fields() call from set_values() does trigger validation it
does so after annotations are masked resulting in them being excluded
from the choices of valid options surfaced through a FieldError.
2023-03-25 20:22:45 +01:00
David Wobrock
d6b6e5d0fd Fixed #28553 -- Fixed annotation mismatch with QuerySet.values()/values_list() on compound queries.
Co-authored-by: Matthias Kestenholz <mk@feinheit.ch>
2023-03-24 06:09:27 +01:00
Liyang Zhang
f9f9215d3e
Fixed some typos in comments, docstrings, and tests. 2023-03-20 08:07:23 +01:00
Mariusz Felisiak
23e8868862
Refs #34233 -- Used str.removeprefix()/removesuffix(). 2023-01-18 19:11:18 +01:00
Simon Charette
dd68af62b2 Fixed #34176 -- Fixed grouping by ambiguous aliases.
Regression in b7b28c7c18.

Refs #31377.

Thanks Shai Berger for the report and reviews.

test_aggregation_subquery_annotation_values_collision() has been
updated as queries that are explicitly grouped by a subquery should
always be grouped by it and not its outer columns even if its alias
collides with referenced table columns. This was not possible to
accomplish at the time 10866a10 landed because we didn't have compiler
level handling of colliding aliases.
2023-01-09 10:52:51 +01:00
Simon Charette
0ff46591ac Refs #33308 -- Deprecated support for passing encoded JSON string literals to JSONField & co.
JSON should be provided as literal Python objects an not in their
encoded string literal forms.
2022-12-01 19:14:00 +01:00
Simon Charette
1297c0d0d7 Fixed #31679 -- Delayed annotating aggregations.
By avoiding to annotate aggregations meant to be possibly pushed to an
outer query until their references are resolved it is possible to
aggregate over a query with the same alias.

Even if #34176 is a convoluted case to support, this refactor seems
worth it given the reduction in complexity it brings with regards to
annotation removal when performing a subquery pushdown.
2022-11-23 17:46:07 +01:00
Simon Charette
b181cae2e3 Refs #25307 -- Replaced SQLQuery.rewrite_cols() by replace_expressions().
The latter offers a more generic interface that doesn't require
specialized expression types handling.
2022-11-14 09:05:32 +01:00
Simon Charette
1771998c09 Reduced nesting inside Query.get_aggregation(). 2022-11-14 09:01:21 +01:00
Simon Charette
10037130c1
Refs #28477 -- Fixed handling aliased annotations on aggregation.
Just like when using .annotate(), the .alias() method will generate the
necessary JOINs to resolve the alias even if not selected.

Since these JOINs could be multi-valued non-selected aggregates must be
considered to require subquery wrapping as a GROUP BY is required to
combine duplicated tuples from the base table.

Regression in 59bea9efd2.
2022-11-14 05:45:33 +01:00
Simon Charette
a9d2d8d1c3 Refs #28477 -- Reduced complexity of aggregation over qualify queries. 2022-11-11 08:46:08 +01:00
Simon Charette
59bea9efd2 Fixed #28477 -- Stripped unused annotations on aggregation.
Also avoid an unnecessary pushdown when aggregating over a query that doesn't
have aggregate annotations.
2022-11-09 13:22:14 +01:00
Nick Pope
d3cb91db87 Used more augmented assignment statements.
Identified using the following command:

$ git grep -I '\(\<[_a-zA-Z0-9]\+\>\) *= *\1 *[-+/*^%&|<>@]'
2022-10-31 12:30:13 +01:00
Simon Charette
c6350d594c Refs #30158 -- Removed alias argument for Expression.get_group_by_cols().
Recent refactors allowed GROUP BY aliasing allowed for aliasing to be
entirely handled by the sql.Query.set_group_by and compiler layers.
2022-10-06 12:04:00 +02:00
Simon Charette
3d734c09ff Refs #33992 -- Refactored subquery grouping logic.
This required moving the combined queries slicing logic to the compiler
in order to allow Query.exists() to be called at expression resolving
time.

It allowed for Query.exists() to be called at Exists() initialization
time and thus ensured that get_group_by_cols() was operating on the
terminal representation of the query that only has a single column
selected.
2022-10-06 11:38:03 +02:00
Simon Charette
04518e310d Refs #33308 -- Enabled explicit GROUP BY and ORDER BY aliases.
This ensures explicit grouping from using values() before annotating an
aggregate function groups by selected aliases if supported.

The GROUP BY feature is disabled on Oracle because it doesn't support it.
2022-10-06 09:34:31 +02:00
David Sanders
4771a1694b Fixed #34012 -- Made QuerySet.order_by() apply transforms on related fields for models with Meta.ordering.
This makes QuerySet.order_by() no longer ignore trailing transforms for
models with Meta.ordering. As a consequence, FieldError is raised in
such cases for non-existent fields.

Thanks to Klaas van Schelven for the report and Mariusz Felisiak for the
review and advice.
2022-10-04 08:19:34 +02:00
Mariusz Felisiak
ce6230aa97
Fixed #34015 -- Allowed filtering by transforms on relation fields. 2022-09-22 00:17:04 +02:00
DevilsAutumn
32797e7fbf Fixed #33975 -- Fixed __in lookup when rhs is a queryset with annotate() and alias().
This fixes clearing selected fields.
2022-09-09 08:37:46 +02:00
Simon Charette
b3db6c8dcb Fixed #21204 -- Tracked field deferrals by field instead of models.
This ensures field deferral works properly when a model is involved
more than once in the same query with a distinct deferral mask.
2022-08-30 08:43:53 +02:00
Nick Pope
769d7cce4a Used AND, OR, XOR constants instead of hard-coded values. 2022-07-27 07:55:09 +02:00
Ipakeev
425718726b
Fixed #33816 -- Fixed QuerySet.only() after select_related() crash on proxy models. 2022-07-04 06:37:36 +02:00
Mariusz Felisiak
c48b34e26d Refs #32786 -- Made query clear ordering when ordered combined queryset is used in subquery on Oracle. 2022-06-27 06:21:31 +02:00
非法操作
19dc3f0f96 Fixed typo in Query.clone()'s docstring. 2022-05-13 06:56:20 +02:00
marcperrinoptel
4282fd468f Fixed #33655 -- Removed unnecessary constant from GROUP BY clause for QuerySet.exists(). 2022-04-26 06:19:18 +02:00
mgaligniana
0ad5316f22 Fixed #24296 -- Made QuerySet.exists() clear selected columns for not sliced distinct querysets. 2022-04-12 08:18:22 +02:00
Mariusz Felisiak
6723a26e59 Fixed CVE-2022-28347 -- Protected QuerySet.explain(**options) against SQL injection on PostgreSQL. 2022-04-11 08:59:58 +02:00
Mariusz Felisiak
93cae5cb2f Fixed CVE-2022-28346 -- Protected QuerySet.annotate(), aggregate(), and extra() against SQL injection in column aliases.
Thanks Splunk team: Preston Elder, Jacob Davis, Jacob Moore,
Matt Hanson, David Briggs, and a security researcher: Danylo Dmytriiev
(DDV_UA) for the report.
2022-04-11 08:59:33 +02:00
Mariusz Felisiak
d7eb500338 Removed unnecessary Query.get_loaded_field_names_cb() and Query.deferred_to_data()'s callback argument. 2022-03-31 10:54:59 +02:00
Mariusz Felisiak
0a3c6fe6b2 Refs #24020 -- Removed redundant Query.get_loaded_field_names().
get_loaded_field_names() is no longer called in multiple places
(see 0c7633178f) and it's redundant
with SQLCompiler.deferred_to_columns().
2022-03-31 10:54:59 +02:00
Mariusz Felisiak
fac662f479
Fixed #33598 -- Reverted "Removed unnecessary reuse_with_filtered_relation argument from Query methods."
Thanks lind-marcus for the report.

This reverts commit 0c71e0f9cf.

Regression in 0c71e0f9cf.
2022-03-30 07:31:56 +02:00
Gagaro
bf524d229f Refs #30581 -- Allowed sql.Query to be used without model. 2022-03-16 09:33:16 +01:00
Keryn Knight
24cc51f8fb Refs #27624 -- Optimized Query.clone() a bit. 2022-03-03 08:38:03 +01:00
Keryn Knight
6d5709ce7d Refs #27624 -- Optimized sql.Query creation by moving immutable/singleton attributes to class attributes. 2022-03-03 08:36:29 +01:00
Mariusz Felisiak
7119f40c98 Refs #33476 -- Refactored code to strictly match 88 characters line length. 2022-02-07 20:37:05 +01:00
django-bot
9c19aff7c7 Refs #33476 -- Reformatted code with Black. 2022-02-07 20:37:05 +01:00
Mariusz Felisiak
c5cd878382
Refs #33476 -- Refactored problematic code before reformatting by Black.
In these cases Black produces unexpected results, e.g.

def make_random_password(
    self,
    length=10,
    allowed_chars='abcdefghjkmnpqrstuvwxyz' 'ABCDEFGHJKLMNPQRSTUVWXYZ' '23456789',
):

or

cursor.execute("""
SELECT ...
""",
    [table name],
)
2022-02-03 11:20:46 +01:00
Mariusz Felisiak
30a0144134
Fixed #29338 -- Allowed using combined queryset in Subquery.
Thanks Eugene Kovalev for the initial patch, Simon Charette for the
review, and Chetan Khanna for help.
2022-01-17 18:01:07 +01:00
Ömer Faruk Abacı
81739a45b5 Fixed #33319 -- Fixed crash when combining with the | operator querysets with aliases that conflict. 2021-12-08 21:16:24 +01:00
Ömer Faruk Abacı
f1bfdff690 Refs #33319 -- Added comment about keys/values assertion in Query.change_aliases(). 2021-12-08 21:16:24 +01:00
Erik Cederstrand
547656c850 Refs #25265 -- Allowed customizing Query's datastructure classes. 2021-12-07 13:36:50 +01:00
Simon Charette
e5a92d400a Fixed #33282 -- Fixed a crash when OR'ing subquery and aggregation lookups.
As a QuerySet resolves to Query the outer column references grouping logic
should be defined on the latter and proxied from Subquery for the cases where
get_group_by_cols is called on unresolved expressions.

Thanks Antonio Terceiro for the report and initial patch.
2021-12-02 07:23:33 +01:00
Jonny Park
6fa2930573 Refs #24121 -- Added __repr__() to BaseDatabaseWrapper, JoinPromoter, and SQLCompiler. 2021-11-19 07:57:02 +01:00
Keryn Knight
a697424969 Fixed #32996 -- Cached PathInfos on relations.
PathInfo values are ostensibly static over the lifetime of the object
for which they're requested, so the data can be memoized, quickly
amortising the cost over the process' duration.
2021-11-03 11:27:04 +01:00