Merge pull request #1544 from evildmp/ticket_20920_rebase

Fixed #20920 -- Consolidated F() and Q() documentation

docs/ref/models/fields.txt

@@ -1062,11 +1062,12 @@ define the details of how the relation works.
     only allows the choice of related objects with a ``pub_date`` before the
     current date to be chosen.
 
-    Instead of a dictionary this can also be a :class:`~django.db.models.Q`
-    object for more :ref:`complex queries <complex-lookups-with-q>`. However,
-    if ``limit_choices_to`` is a :class:`~django.db.models.Q` object then it
-    will only have an effect on the choices available in the admin when the
-    field is not listed in ``raw_id_fields`` in the ``ModelAdmin`` for the model.
+    Instead of a dictionary this can also be a :class:`Q object
+    <django.db.models.Q>` for more :ref:`complex queries
+    <complex-lookups-with-q>`. However, if ``limit_choices_to`` is a :class:`Q
+    object <django.db.models.Q>` then it will only have an effect on the
+    choices available in the admin when the field is not listed in
+    ``raw_id_fields`` in the ``ModelAdmin`` for the model.
 
 .. attribute:: ForeignKey.related_name
 

docs/ref/models/index.txt

@@ -12,3 +12,4 @@ Model API reference. For introductory material, see :doc:`/topics/db/models`.
    options
    instances
    querysets
+   queries

docs/ref/models/instances.txt

@@ -342,6 +342,8 @@ only.
 
 Using ``update_fields`` will force an update similarly to ``force_update``.
 
+.. _ref-models-field-updates-using-f-expressions:
+
 Updating attributes based on existing fields
 --------------------------------------------
 
@@ -356,35 +358,21 @@ achieve this is to do something like::
 If the old ``number_sold`` value retrieved from the database was 10, then
 the value of 11 will be written back to the database.
 
-This sequence has a standard update problem in that it contains a race
-condition. If another thread of execution has already saved an updated value
-after the current thread retrieved the old value, the current thread will only
-save the old value plus one, rather than the new (current) value plus one.
-
-The process can be made robust and slightly faster by expressing the update
-relative to the original field value, rather than as an explicit assignment of
-a new value. Django provides :ref:`F() expressions <query-expressions>` for
-performing this kind of relative update. Using ``F()`` expressions, the
-previous example is expressed as::
+The process can be made robust, :ref:`avoiding a race condition
+<avoiding-race-conditions-using-f>`, as well as slightly faster by expressing
+the update relative to the original field value, rather than as an explicit
+assignment of a new value. Django provides :class:`F expressions
+<django.db.models.F>` for performing this kind of relative update. Using
+:class:`F expressions <django.db.models.F>`, the previous example is expressed
+as::
 
     >>> from django.db.models import F
     >>> product = Product.objects.get(name='Venezuelan Beaver Cheese')
     >>> product.number_sold = F('number_sold') + 1
     >>> product.save()
 
-This approach doesn't use the initial value from the database. Instead, it
-makes the database do the update based on whatever value is current at the time
-that the :meth:`~Model.save()` is executed.
-
-Once the object has been saved, you must reload the object in order to access
-the actual value that was applied to the updated field::
-
-    >>> product = Products.objects.get(pk=product.pk)
-    >>> print(product.number_sold)
-    42
-
-For more details, see the documentation on :ref:`F() expressions
-<query-expressions>` and their :ref:`use in update queries
+For more details, see the documentation on :class:`F expressions
+<django.db.models.F>` and their :ref:`use in update queries
 <topics-db-queries-update>`.
 
 Specifying which fields to save

docs/ref/models/queries.txt

@@ -0,0 +1,127 @@
+=====================
+Query-related classes
+=====================
+
+.. currentmodule:: django.db.models
+
+This document provides reference material for query-related tools not
+documented elsewhere.
+
+``F()`` expressions
+===================
+
+.. class:: F
+
+An ``F()`` object represents the value of a model field. It makes it possible
+to refer to model field values and perform database operations using them
+without actually having to pull them out of the database into Python memory.
+
+Instead, Django uses the ``F()`` object to generate a SQL expression that
+describes the required operation at the database level.
+
+This is easiest to understand though an example. Normally, one might do
+something like this::
+
+    # Tintin filed a news story!
+    reporter = Reporters.objects.get(name='Tintin')
+    reporter.stories_filed += 1
+    reporter.save()
+
+Here, we have pulled the value of ``reporter.stories_filed`` from the database
+into memory and manipulated it using familiar Python operators, and then saved
+the object back to the database. But instead we could also have done::
+
+    from django.db.models import F
+    reporter = Reporters.objects.get(name='Tintin')
+    reporter.stories_filed = F('stories_filed') + 1
+    reporter.save()
+
+Although ``reporter.stories_filed = F('stories_filed') + 1`` looks like a
+normal Python assignment of value to an instance attribute, in fact it's an SQL
+construct describing an operation on the database.
+
+When Django encounters an instance of ``F()``, it overrides the standard Python
+operators to create an encapsulated SQL expression; in this case, one which
+instructs the database to increment the database field represented by
+``reporter.stories_filed``.
+
+Whatever value is or was on ``reporter.stories_filed``, Python never gets to
+know about it - it is dealt with entirely by the database. All Python does,
+through Django's ``F()`` class, is create the SQL syntax to refer to the field
+and describe the operation.
+
+.. note::
+
+   In order to access the new value that has been saved in this way, the object
+   will need to be reloaded::
+
+       reporter = Reporters.objects.get(pk=reporter.pk)
+
+As well as being used in operations on single instances as above, ``F()`` can
+be used on ``QuerySets`` of object instances, with ``update()``. This reduces
+the two queries we were using above - the ``get()`` and the
+:meth:`~Model.save()` - to just one::
+
+    reporter = Reporters.objects.filter(name='Tintin')
+    reporter.update(stories_filed=F('stories_filed') + 1)
+
+We can also use :meth:`~django.db.models.query.QuerySet.update()` to increment
+the field value on multiple objects - which could be very much faster than
+pulling them all into Python from the database, looping over them, incrementing
+the field value of each one, and saving each one back to the database::
+
+    Reporter.objects.all().update(stories_filed=F('stories_filed) + 1)
+
+``F()`` therefore can offer performance advantages by:
+
+* getting the database, rather than Python, to do work
+* reducing the number of queries some operations require
+
+.. _avoiding-race-conditions-using-f:
+
+Avoiding race conditions using ``F()``
+--------------------------------------
+
+Another useful benefit of ``F()`` is that having the database - rather than
+Python - update a field's value avoids a *race condition*.
+
+If two Python threads execute the code in the first example above, one thread
+could retrieve, increment, and save a field's value after the other has
+retrieved it from the database. The value that the second thread saves will be
+based on the original value; the work of the first thread will simply be lost.
+
+If the database is responsible for updating the field, the process is more
+robust: it will only ever update the field based on the value of the field in
+the database when the :meth:`~Model.save()` or ``update()`` is executed, rather
+than based on its value when the instance was retrieved.
+
+Using ``F()`` in filters
+------------------------
+
+``F()`` is also very useful in ``QuerySet`` filters, where they make it
+possible to filter a set of objects against criteria based on their field
+values, rather than on Python values.
+
+This is documented in :ref:`using F() expressions in queries
+<using-f-expressions-in-filters>`
+
+Supported operations with ``F()``
+---------------------------------
+
+As well as addition, Django supports subtraction, multiplication, division,
+and modulo arithmetic with ``F()`` objects, using Python constants,
+variables, and even other ``F()`` objects.
+
+``Q()`` objects
+===============
+
+.. class:: Q
+
+A ``Q()`` object, like an :class:`~django.db.models.F` object, encapsulates a
+SQL expression in a Python object that can be used in database-related
+operations.
+
+In general, ``Q() objects`` make it possible to define and reuse conditions.
+This permits the :ref:`construction of complex database queries
+<complex-lookups-with-q>` using ``|`` (``OR``) and ``&`` (``AND``) operators;
+in particular, it is not otherwise possible to use ``OR`` in ``QuerySets``.