Fixed #28586 -- Added model field fetch modes.

May your database queries be much reduced with minimal effort.

co-authored-by: Andreas Pelme <andreas@pelme.se>
co-authored-by: Simon Charette <charette.s@gmail.com>
co-authored-by: Jacob Walls <jacobtylerwalls@gmail.com>

docs/topics/db/fetch-modes.txt

@@ -0,0 +1,138 @@
+===========
+Fetch modes
+===========
+
+.. versionadded:: 6.1
+
+.. module:: django.db.models.fetch_modes
+
+.. currentmodule:: django.db.models
+
+When accessing model fields that were not loaded as part of the original query,
+Django will fetch that field's data from the database. You can customize the
+behavior of this fetching with a **fetch mode**, making it more efficient or
+even blocking it.
+
+Use :meth:`.QuerySet.fetch_mode` to set the fetch mode for model
+instances fetched by a ``QuerySet``:
+
+.. code-block:: python
+
+    from django.db import models
+
+    books = Book.objects.fetch_mode(models.FETCH_PEERS)
+
+Fetch modes apply to:
+
+* :class:`~django.db.models.ForeignKey` fields
+* :class:`~django.db.models.OneToOneField` fields and their reverse accessors
+* Fields deferred with :meth:`.QuerySet.defer` or :meth:`.QuerySet.only`
+* :ref:`generic-relations`
+
+Available modes
+===============
+
+.. admonition:: Referencing fetch modes
+
+    Fetch modes are defined in ``django.db.models.fetch_modes``, but for
+    convenience they're imported into :mod:`django.db.models`. The standard
+    convention is to use ``from django.db import models`` and refer to the
+    fetch modes as ``models.<mode>``.
+
+Django provides three fetch modes. We'll explain them below using these models:
+
+.. code-block:: python
+
+    from django.db import models
+
+
+    class Author(models.Model): ...
+
+
+    class Book(models.Model):
+        author = models.ForeignKey(Author, on_delete=models.CASCADE)
+        ...
+
+…and this loop:
+
+.. code-block:: python
+
+    for book in books:
+        print(book.author.name)
+
+…where ``books`` is a ``QuerySet`` of ``Book`` instances using some fetch mode.
+
+.. attribute:: FETCH_ONE
+
+Fetches the missing field for the current instance only. This is the default
+mode.
+
+Using ``FETCH_ONE`` for the above example would use:
+
+* 1 query to fetch ``books``
+* N queries, where N is the number of books, to fetch the missing ``author``
+  field
+
+…for a total of 1+N queries. This query pattern is known as the "N+1 queries
+problem" because it often leads to performance issues when N is large.
+
+.. attribute:: FETCH_PEERS
+
+Fetches the missing field for the current instance and its "peers"—instances
+that came from the same initial ``QuerySet``. The behavior of this mode is
+based on the assumption that if you need a field for one instance, you probably
+need it for all instances in the same batch, since you'll likely process them
+all identically.
+
+Using ``FETCH_PEERS`` for the above example would use:
+
+* 1 query to fetch ``books``
+* 1 query to fetch all missing ``author`` fields for the batch of books
+
+…for a total of 2 queries. The batch query makes this mode a lot more efficient
+than ``FETCH_ONE`` and is similar to an on-demand call to
+:meth:`.QuerySet.prefetch_related` or
+:func:`~django.db.models.prefetch_related_objects`. Using ``FETCH_PEERS`` can
+reduce most cases of the "N+1 queries problem" to two queries without
+much effort.
+
+The "peer" instances are tracked in a list of weak references, to avoid
+memory leaks where some peer instances are discarded.
+
+.. attribute:: RAISE
+
+Raises a :exc:`~django.core.exceptions.FieldFetchBlocked` exception.
+
+Using ``RAISE`` for the above example would raise an exception at the access of
+``book.author`` access, like:
+
+.. code-block:: python
+
+    FieldFetchBlocked("Fetching of Primary.value blocked.")
+
+This mode can prevent unintentional queries in performance-critical
+sections of code.
+
+.. _fetch-modes-custom-manager:
+
+Make a fetch mode the default for a model class
+===============================================
+
+Set the default fetch mode for a model class with a
+:ref:`custom manager <custom-managers>` that overrides ``get_queryset()``:
+
+.. code-block:: python
+
+    from django.db import models
+
+
+    class BookManager(models.Manager):
+        def get_queryset(self):
+            return super().get_queryset().fetch_mode(models.FETCH_PEERS)
+
+
+    class Book(models.Model):
+        title = models.TextField()
+        author = models.ForeignKey("Author", on_delete=models.CASCADE)
+
+        objects = BookManager()

docs/topics/db/index.txt

@@ -13,6 +13,7 @@ Generally, each model maps to a single database table.
 
    models
    queries
+   fetch-modes
    aggregation
    search
    managers

docs/topics/db/optimization.txt

@@ -196,28 +196,46 @@ thousands of records are returned. The penalty will be compounded if the
 database lives on a separate server, where network overhead and latency also
 play a factor.
 
-Retrieve everything at once if you know you will need it
-========================================================
+Retrieve related objects efficiently
+====================================
 
-Hitting the database multiple times for different parts of a single 'set' of
-data that you will need all parts of is, in general, less efficient than
-retrieving it all in one query. This is particularly important if you have a
-query that is executed in a loop, and could therefore end up doing many
-database queries, when only one was needed. So:
+Generally, accessing the database multiple times to retrieve different parts
+of a single "set" of data is less efficient than retrieving it all in one
+query. This is particularly important if you have a query that is executed in a
+loop, and could therefore end up doing many database queries, when only one
+is needed. Below are some techniques to combine queries for efficiency.
+
+Use the ``FETCH_PEERS`` fetch mode
+----------------------------------
+
+Use the :attr:`~django.db.models.FETCH_PEERS` fetch mode to make on-demand
+field access more efficient with bulk-fetching. Enable all it for all usage of
+your models :ref:`with a custom manager <fetch-modes-custom-manager>`.
+
+Using this fetch mode is easier than declaring fields to fetch with
+:meth:`~django.db.models.query.QuerySet.select_related` or
+:meth:`~django.db.models.query.QuerySet.prefetch_related`, especially when it's
+hard to predict which fields will be accessed.
 
 Use ``QuerySet.select_related()`` and ``prefetch_related()``
 ------------------------------------------------------------
 
-Understand :meth:`~django.db.models.query.QuerySet.select_related` and
-:meth:`~django.db.models.query.QuerySet.prefetch_related` thoroughly, and use
-them:
+When the :attr:`~django.db.models.FETCH_PEERS` fetch mode is not appropriate or
+efficient enough, use :meth:`~django.db.models.query.QuerySet.select_related`
+and :meth:`~django.db.models.query.QuerySet.prefetch_related`. Understand their
+documentation thoroughly and apply them where needed.
 
-* in :doc:`managers and default managers </topics/db/managers>` where
-  appropriate. Be aware when your manager is and is not used; sometimes this is
-  tricky so don't make assumptions.
+It may be useful to apply these methods in :doc:`managers and default managers
+</topics/db/managers>`. Be aware when your manager is and is not used;
+sometimes this is tricky so don't make assumptions.
 
-* in view code or other layers, possibly making use of
-  :func:`~django.db.models.prefetch_related_objects` where needed.
+Use ``prefetch_related_objects()``
+----------------------------------
+
+Where :attr:`~django.db.models.query.QuerySet.prefetch_related` would be useful
+after the queryset has been evaluated, use
+:func:`~django.db.models.prefetch_related_objects` to execute an extra
+prefetch.
 
 Don't retrieve things you don't need
 ====================================

docs/topics/db/queries.txt

@@ -1702,6 +1702,12 @@ the link from the related model to the model that defines the relationship.
 For example, a ``Blog`` object ``b`` has a manager that returns all related
 ``Entry`` objects in the ``entry_set`` attribute: ``b.entry_set.all()``.
 
+These accessors may be prefetched by the ``QuerySet`` methods
+:meth:`~django.db.models.query.QuerySet.select_related` or
+:meth:`~django.db.models.query.QuerySet.prefetch_related`. If not prefetched,
+access will trigger an on-demand fetch through the model's
+:doc:`fetch mode </topics/db/fetch-modes>`.
+
 All examples in this section use the sample ``Blog``, ``Author`` and ``Entry``
 models defined at the top of this page.