mirrors/django
1
0
Fork 0
mirror of https://github.com/django/django.git synced 2025-08-03 18:38:50 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

Added array field support for PostgreSQL.

Browse source 
The first part of django.contrib.postgres, including model and two form
fields for arrays of other data types.

This commit is formed of the following work:

    Add shell of postgres app and test handling.

    First draft of array fields.

    Use recursive deconstruction.

    Stop creating classes at lookup time.

    Add validation and size parameter.

    Add contained_by lookup.

    Add SimpleArrayField for forms.

    Add SplitArrayField (mainly for admin).

    Fix prepare_value for SimpleArrayField.

    Stop using MultiValueField and MultiWidget.

    They don't play nice with flexible sizes.

    Add basics of admin integration.

    Missing:
    - Tests
    - Fully working js

    Add reference document for django.contrib.postgres.fields.ArrayField.

    Various performance and style tweaks.

    Fix internal docs link, formalise code snippets.

    Remove the admin code for now.

    It needs a better way of handing JS widgets in the admin as a whole
    before it is easy to write. In particular there are serious issues
    involving DateTimePicker when used in an array.

    Add a test for nested array fields with different delimiters.

    This will be a documented pattern so having a test for it is useful.

    Add docs for SimpleArrayField.

    Add docs for SplitArrayField.

    Remove admin related code for now.

    definition -> description

    Fix typo.

    Py3 errors.

    Avoid using regexes where they're not needed.

    Allow passing tuples by the programmer.

    Add some more tests for multidimensional arrays.

    Also fix slicing as much as it can be fixed.

    Simplify SplitArrayWidget's data loading.

    If we aren't including the variable size one, we don't need to search
    like this.
This commit is contained in:
Marc Tamlyn 2014-03-26 16:44:21 +00:00
parent d9d9242505
commit 604162604b
15 changed files with 1272 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

8
docs/ref/contrib/index.txt
View file

@ -31,6 +31,7 @@ those packages have.
gis/index
humanize
messages
postgres/index
redirects
sitemaps
sites
@ -122,6 +123,13 @@ messages
See the :doc:`messages documentation </ref/contrib/messages>`.
postgres
========
A collection of PostgreSQL specific features.
See the :doc:`contrib.postgres documentation </ref/contrib/postgres/index>`.
redirects
=========

228
docs/ref/contrib/postgres/fields.txt Normal file
View file

@ -0,0 +1,228 @@
PostgreSQL specific model fields
================================
All of these fields are available from the ``django.contrib.postgres.fields``
module.
.. currentmodule:: django.contrib.postgres.fields
ArrayField
----------
.. class:: ArrayField(base_field, size=None, **options)
A field for storing lists of data. Most field types can be used, you simply
pass another field instance as the :attr:`base_field
<ArrayField.base_field>`. You may also specify a :attr:`size
<ArrayField.size>`. ``ArrayField`` can be nested to store multi-dimensional
arrays.
.. attribute:: base_field
This is a required argument.
Specifies the underlying data type and behaviour for the array. It
should be an instance of a subclass of
:class:`~django.db.models.Field`. For example, it could be an
:class:`~django.db.models.IntegerField` or a
:class:`~django.db.models.CharField`. Most field types are permitted,
with the exception of those handling relational data
(:class:`~django.db.models.ForeignKey`,
:class:`~django.db.models.OneToOneField` and
:class:`~django.db.models.ManyToManyField`).
It is possible to nest array fields - you can specify an instance of
``ArrayField`` as the ``base_field``. For example::
from django.db import models
from django.contrib.postgres.fields import ArrayField
class ChessBoard(models.Model):
board = ArrayField(
ArrayField(
CharField(max_length=10, blank=True, null=True),
size=8),
size=8)
Transformation of values between the database and the model, validation
of data and configuration, and serialization are all delegated to the
underlying base field.
.. attribute:: size
This is an optional argument.
If passed, the array will have a maximum size as specified. This will
be passed to the database, although PostgreSQL at present does not
enforce the restriction.
.. note::
When nesting ``ArrayField``, whether you use the `size` parameter or not,
PostgreSQL requires that the arrays are rectangular::
from django.db import models
from django.contrib.postgres.fields import ArrayField
class Board(models.Model):
pieces = ArrayField(ArrayField(models.IntegerField()))
# Valid
Board(pieces=[
[2, 3],
[2, 1],
])
# Not valid
Board(pieces=[
[2, 3],
[2],
])
If irregular shapes are required, then the underlying field should be made
nullable and the values padded with ``None``.
Querying ArrayField
^^^^^^^^^^^^^^^^^^^
There are a number of custom lookups and transforms for :class:`ArrayField`.
We will use the following example model::
from django.db import models
from django.contrib.postgres.fields import ArrayField
class Post(models.Model):
name = models.CharField(max_length=200)
tags = ArrayField(models.CharField(max_length=200), blank=True)
def __str__(self): # __unicode__ on python 2
return self.name
.. fieldlookup:: arrayfield.contains
contains
~~~~~~~~
The :lookup:`contains` lookup is overridden on :class:`ArrayField`. The
returned objects will be those where the values passed are a subset of the
data. It uses the SQL operator ``@>``. For example::
>>> Post.objects.create(name='First post', tags=['thoughts', 'django'])
>>> Post.objects.create(name='Second post', tags=['thoughts'])
>>> Post.objects.create(name='Third post', tags=['tutorial', 'django'])
>>> Post.objects.filter(tags__contains=['thoughts'])
[<Post: First post>, <Post: Second post>]
>>> Post.objects.filter(tags__contains=['django'])
[<Post: First post>, <Post: Third post>]
>>> Post.objects.filter(tags__contains=['django', 'thoughts'])
[<Post: First post>]
.. fieldlookup:: arrayfield.contained_by
contained_by
~~~~~~~~~~~~
This is the inverse of the :lookup:`contains <arrayfield.contains>` lookup -
the objects returned will be those where the data is a subset of the values
passed. It uses the SQL operator ``<@``. For example::
>>> Post.objects.create(name='First post', tags=['thoughts', 'django'])
>>> Post.objects.create(name='Second post', tags=['thoughts'])
>>> Post.objects.create(name='Third post', tags=['tutorial', 'django'])
>>> Post.objects.filter(tags__contained_by=['thoughts', 'django'])
[<Post: First post>]
>>> Post.objects.filter(tags__contained_by=['thoughts', 'django', 'tutorial'])
[<Post: First post>, <Post: Second post>, <Post: Third post>]
.. fieldlookup:: arrayfield.overlap
overlap
~~~~~~~
Returns objects where the data shares any results with the values passed. Uses
the SQL operator ``&&``. For example::
>>> Post.objects.create(name='First post', tags=['thoughts', 'django'])
>>> Post.objects.create(name='Second post', tags=['thoughts'])
>>> Post.objects.create(name='Third post', tags=['tutorial', 'django'])
>>> Post.objects.filter(tags__overlap=['thoughts'])
[<Post: First post>, <Post: Second post>]
>>> Post.objects.filter(tags__overlap=['thoughts', 'tutorial'])
[<Post: First post>, <Post: Second post>, <Post: Third post>]
.. fieldlookup:: arrayfield.index
Index transforms
~~~~~~~~~~~~~~~~
This class of transforms allows you to index into the array in queries. Any
non-negative integer can be used. There are no errors if it exceeds the
:attr:`size <ArrayField.size>` of the array. The lookups available after the
transform are those from the :attr:`base_field <ArrayField.base_field>`. For
example::
>>> Post.objects.create(name='First post', tags=['thoughts', 'django'])
>>> Post.objects.create(name='Second post', tags=['thoughts'])
>>> Post.objects.filter(tags__0='thoughts')
[<Post: First post>, <Post: Second post>]
>>> Post.objects.filter(tags__1__iexact='Django')
[<Post: First post>]
>>> Post.objects.filter(tags__276='javascript')
[]
.. note::
PostgreSQL uses 1-based indexing for array fields when writing raw SQL.
However these indexes and those used in :lookup:`slices <arrayfield.slice>`
use 0-based indexing to be consistent with Python.
.. fieldlookup:: arrayfield.slice
Slice transforms
~~~~~~~~~~~~~~~~
This class of transforms allow you to take a slice of the array. Any two
non-negative integers can be used, separated by a single underscore. The
lookups available after the transform do not change. For example::
>>> Post.objects.create(name='First post', tags=['thoughts', 'django'])
>>> Post.objects.create(name='Second post', tags=['thoughts'])
>>> Post.objects.create(name='Third post', tags=['django', 'python', 'thoughts'])
>>> Post.objects.filter(tags__0_1=['thoughts'])
[<Post: First post>]
>>> Post.objects.filter(tags__0_2__contains='thoughts')
[<Post: First post>, <Post: Second post>]
.. note::
PostgreSQL uses 1-based indexing for array fields when writing raw SQL.
However these slices and those used in :lookup:`indexes <arrayfield.index>`
use 0-based indexing to be consistent with Python.
.. admonition:: Multidimensional arrays with indexes and slices
PostgreSQL has some rather esoteric behaviour when using indexes and slices
on multidimensional arrays. It will always work to use indexes to reach
down to the final underlying data, but most other slices behave strangely
at the database level and cannot be supported in a logical, consistent
fashion by Django.
Indexing ArrayField
^^^^^^^^^^^^^^^^^^^
At present using :attr:`~django.db.models.Field.db_index` will create a
``btree`` index. This does not offer particularly significant help to querying.
A more useful index is a ``GIN`` index, which you should create using a
:class:`~django.db.migrations.operations.RunSQL` operation.

135
docs/ref/contrib/postgres/forms.txt Normal file
View file

@ -0,0 +1,135 @@
PostgreSQL specific form fields and widgets
===========================================
All of these fields and widgets are available from the
``django.contrib.postgres.forms`` module.
.. currentmodule:: django.contrib.postgres.forms
SimpleArrayField
----------------
.. class:: SimpleArrayField(base_field, delimiter=',', max_length=None, min_length=None)
A simple field which maps to an array. It is represented by an HTML
``<input>``.
.. attribute:: base_field
This is a required argument.
It specifies the underlying form field for the array. This is not used
to render any HTML, but it is used to process the submitted data and
validate it. For example::
>>> from django.contrib.postgres.forms import SimpleArrayField
>>> from django import forms
>>> class NumberListForm(forms.Form):
... numbers = SimpleArrayField(forms.IntegerField())
>>> form = NumberListForm({'numbers': '1,2,3'})
>>> form.is_valid()
True
>>> form.cleaned_data
{'numbers': [1, 2, 3]}
>>> form = NumberListForm({'numbers': '1,2,a'})
>>> form.is_valid()
False
.. attribute:: delimiter
This is an optional argument which defaults to a comma: ``,``. This
value is used to split the submitted data. It allows you to chain
``SimpleArrayField`` for multidimensional data::
>>> from django.contrib.postgres.forms import SimpleArrayField
>>> from django import forms
>>> class GridForm(forms.Form):
... places = SimpleArrayField(SimpleArrayField(IntegerField()), delimiter='|')
>>> form = GridForm({'places': '1,2|2,1|4,3'})
>>> form.is_valid()
True
>>> form.cleaned_data
{'places': [[1, 2], [2, 1], [4, 3]]}
.. note::
The field does not support escaping of the delimiter, so be careful
in cases where the delimiter is a valid character in the underlying
field. The delimiter does not need to be only one character.
.. attribute:: max_length
This is an optional argument which validates that the array does not
exceed the stated length.
.. attribute:: min_length
This is an optional argument which validates that the array reaches at
least the stated length.
.. admonition:: User friendly forms
``SimpleArrayField`` is not particularly user friendly in most cases,
however it is a useful way to format data from a client-side widget for
submission to the server.
SplitArrayField
---------------
.. class:: SplitArrayField(base_field, size, remove_trailing_nulls=False)
This field handles arrays by reproducing the underlying field a fixed
number of times.
.. attribute:: base_field
This is a required argument. It specifies the form field to be
repeated.
.. attribute:: size
This is the fixed number of times the underlying field will be used.
.. attribute:: remove_trailing_nulls
By default, this is set to ``False``. When ``False``, each value from
the repeated fields is stored. When set to ``True``, any trailing
values which are blank will be stripped from the result. If the
underlying field has ``required=True``, but ``remove_trailing_nulls``
is ``True``, then null values are only allowed at the end, and will be
stripped.
Some examples::
SplitArrayField(IntegerField(required=True), size=3, remove_trailing_nulls=False)
['1', '2', '3'] # -> [1, 2, 3]
['1', '2', ''] # -> ValidationError - third entry required.
['1', '', '3'] # -> ValidationError - second entry required.
['', '2', ''] # -> ValidationError - first and third entries required.
SplitArrayField(IntegerField(required=False), size=3, remove_trailing_nulls=False)
['1', '2', '3'] # -> [1, 2, 3]
['1', '2', ''] # -> [1, 2, None]
['1', '', '3'] # -> [1, None, 3]
['', '2', ''] # -> [None, 2, None]
SplitArrayField(IntegerField(required=True), size=3, remove_trailing_nulls=True)
['1', '2', '3'] # -> [1, 2, 3]
['1', '2', ''] # -> [1, 2]
['1', '', '3'] # -> ValidationError - second entry required.
['', '2', ''] # -> ValidationError - first entry required.
SplitArrayField(IntegerField(required=False), size=3, remove_trailing_nulls=True)
['1', '2', '3'] # -> [1, 2, 3]
['1', '2', ''] # -> [1, 2]
['1', '', '3'] # -> [1, None, 3]
['', '2', ''] # -> [None, 2]

28
docs/ref/contrib/postgres/index.txt Normal file
View file

@ -0,0 +1,28 @@
``django.contrib.postgres``
===========================
PostgreSQL has a number of features which are not shared by the other databases
Django supports. This optional module contains model fields and form fields for
a number of PostgreSQL specific data types.
.. note::
Django is, and will continue to be, a database-agnostic web framework. We
would encourage those writing reusable applications for the Django
community to write database-agnostic code where practical. However, we
recognise that real world projects written using Django need not be
database-agnostic. In fact, once a project reaches a given size changing
the underlying data store is already a significant challenge and is likely
to require changing the code base in some ways to handle differences
between the data stores.
Django provides support for a number of data types which will
only work with PostgreSQL. There is no fundamental reason why (for example)
a ``contrib.mysql`` module does not exist, except that PostgreSQL has the
richest feature set of the supported databases so its users have the most
to gain.
.. toctree::
:maxdepth: 2
fields
forms