Merge remote-tracking branch 'core/master' into schema-alteration

Conflicts:
	django/db/backends/oracle/base.py
	django/db/backends/postgresql_psycopg2/base.py
	django/db/models/signals.py
	tests/queries/tests.py

docs/ref/class-based-views/base.txt

@@ -79,10 +79,6 @@ View
         you can override the ``head()`` method. See
         :ref:`supporting-other-http-methods` for an example.
 
-        The default implementation also sets ``request``, ``args`` and
-        ``kwargs`` as instance variables, so any method on the view can know
-        the full details of the request that was made to invoke the view.
-
     .. method:: http_method_not_allowed(request, *args, **kwargs)
 
         If the view was called with a HTTP method it doesn't support, this

docs/ref/models/instances.txt

@@ -104,14 +104,9 @@ aren't present on your form from being validated since any errors raised could
 not be corrected by the user.
 
 Note that ``full_clean()`` will *not* be called automatically when you call
-your model's :meth:`~Model.save()` method, nor as a result of
-:class:`~django.forms.ModelForm` validation. In the case of
-:class:`~django.forms.ModelForm` validation, :meth:`Model.clean_fields()`,
-:meth:`Model.clean()`, and :meth:`Model.validate_unique()` are all called
-individually.
-
-You'll need to call ``full_clean`` manually when you want to run one-step model
-validation for your own manually created models. For example::
+your model's :meth:`~Model.save()` method. You'll need to call it manually
+when you want to run one-step model validation for your own manually created
+models. For example::
 
     from django.core.exceptions import ValidationError
     try:
@@ -526,6 +521,25 @@ For example::
   In previous versions only instances of the exact same class and same
   primary key value were considered equal.
 
+``__hash__``
+------------
+
+.. method:: Model.__hash__()
+
+The ``__hash__`` method is based on the instance's primary key value. It
+is effectively hash(obj.pk). If the instance doesn't have a primary key
+value then a ``TypeError`` will be raised (otherwise the ``__hash__``
+method would return different values before and after the instance is
+saved, but changing the ``__hash__`` value of an instance `is forbidden
+in Python`_).
+
+.. versionchanged:: 1.7
+
+  In previous versions instance's without primary key value were
+  hashable.
+
+.. _is forbidden in Python: http://docs.python.org/reference/datamodel.html#object.__hash__
+
 ``get_absolute_url``
 --------------------
 

docs/ref/settings.txt

@@ -1290,11 +1290,22 @@ LANGUAGE_CODE
 
 Default: ``'en-us'``
 
-A string representing the language code for this installation. This should be
-in standard :term:`language format<language code>`. For example, U.S. English
+A string representing the language code for this installation. This should be in
+standard :term:`language ID format <language code>`. For example, U.S. English
 is ``"en-us"``. See also the `list of language identifiers`_ and
 :doc:`/topics/i18n/index`.
 
+:setting:`USE_I18N` must be active for this setting to have any effect.
+
+It serves two purposes:
+
+* If the locale middleware isn't in use, it decides which translation is served
+  to all users.
+* If the locale middleware is active, it provides the fallback translation when
+  no translation exist for a given literal to the user's preferred language.
+
+See :ref:`how-django-discovers-language-preference` for more details.
+
 .. _list of language identifiers: http://www.i18nguy.com/unicode/language-identifiers.html
 
 .. setting:: LANGUAGE_COOKIE_NAME
@@ -2392,7 +2403,7 @@ SESSION_ENGINE
 
 Default: ``django.contrib.sessions.backends.db``
 
-Controls where Django stores session data. Valid values are:
+Controls where Django stores session data. Included engines are:
 
 * ``'django.contrib.sessions.backends.db'``
 * ``'django.contrib.sessions.backends.file'``
@@ -2435,6 +2446,28 @@ Whether to save the session data on every request. If this is ``False``
 (default), then the session data will only be saved if it has been modified --
 that is, if any of its dictionary values have been assigned or deleted.
 
+.. setting:: SESSION_SERIALIZER
+
+SESSION_SERIALIZER
+------------------
+
+Default: ``'django.contrib.sessions.serializers.JSONSerializer'``
+
+.. versionchanged:: 1.6
+
+    The default switched from
+    :class:`~django.contrib.sessions.serializers.PickleSerializer` to
+    :class:`~django.contrib.sessions.serializers.JSONSerializer` in Django 1.6.
+
+Full import path of a serializer class to use for serializing session data.
+Included serializers are:
+
+* ``'django.contrib.sessions.serializers.PickleSerializer'``
+* ``'django.contrib.sessions.serializers.JSONSerializer'``
+
+See :ref:`session_serialization` for details, including a warning regarding
+possible remote code execution when using
+:class:`~django.contrib.sessions.serializers.PickleSerializer`.
 
 Sites
 =====

docs/releases/1.6.txt

@@ -727,6 +727,29 @@ the ``name`` argument so it doesn't conflict with the new url::
 You can remove this url pattern after your app has been deployed with Django
 1.6 for :setting:`PASSWORD_RESET_TIMEOUT_DAYS`.
 
+Default session serialization switched to JSON
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Historically, :mod:`django.contrib.sessions` used :mod:`pickle` to serialize
+session data before storing it in the backend. If you're using the :ref:`signed
+cookie session backend<cookie-session-backend>` and :setting:`SECRET_KEY` is
+known by an attacker, the attacker could insert a string into his session
+which, when unpickled, executes arbitrary code on the server. The technique for
+doing so is simple and easily available on the internet. Although the cookie
+session storage signs the cookie-stored data to prevent tampering, a
+:setting:`SECRET_KEY` leak immediately escalates to a remote code execution
+vulnerability.
+
+This attack can be mitigated by serializing session data using JSON rather
+than :mod:`pickle`. To facilitate this, Django 1.5.3 introduced a new setting,
+:setting:`SESSION_SERIALIZER`, to customize the session serialization format.
+For backwards compatibility, this setting defaulted to using :mod:`pickle`
+in Django 1.5.3, but we've changed the default to JSON in 1.6. If you upgrade
+and switch from pickle to JSON, sessions created before the upgrade will be
+lost. While JSON serialization does not support all Python objects like
+:mod:`pickle` does, we highly recommend using JSON-serialized sessions. See the
+:ref:`session_serialization` documentation for more details.
+
 Miscellaneous
 ~~~~~~~~~~~~~
 

docs/releases/1.7.txt

@@ -266,6 +266,14 @@ Miscellaneous
   equal when primary keys match. Previously only instances of exact same
   class were considered equal on primary key match.
 
+* The :meth:`django.db.models.Model.__eq__` method has changed such that
+  two ``Model`` instances without primary key values won't be considered
+  equal (unless they are the same instance).
+  
+* The :meth:`django.db.models.Model.__hash__` will now raise ``TypeError``
+  when called on an instance without a primary key value. This is done to
+  avoid mutable ``__hash__`` values in containers.
+
 Features deprecated in 1.7
 ==========================
 

docs/topics/http/sessions.txt

@@ -128,8 +128,9 @@ and the :setting:`SECRET_KEY` setting.
 
 .. warning::
 
-    **If the SECRET_KEY is not kept secret, this can lead to arbitrary remote
-    code execution.**
+    **If the SECRET_KEY is not kept secret and you are using the**
+    :class:`~django.contrib.sessions.serializers.PickleSerializer`, **this can
+    lead to arbitrary remote code execution.**
 
     An attacker in possession of the :setting:`SECRET_KEY` can not only
     generate falsified session data, which your site will trust, but also
@@ -256,7 +257,9 @@ You can edit it multiple times.
         in 5 minutes.
 
       * If ``value`` is a ``datetime`` or ``timedelta`` object, the
-        session will expire at that specific date/time.
+        session will expire at that specific date/time. Note that ``datetime``
+        and ``timedelta`` values are only serializable if you are using the
+        :class:`~django.contrib.sessions.serializers.PickleSerializer`.
 
       * If ``value`` is ``0``, the user's session cookie will expire
         when the user's Web browser is closed.
@@ -301,6 +304,72 @@ You can edit it multiple times.
       Removes expired sessions from the session store. This class method is
       called by :djadmin:`clearsessions`.
 
+.. _session_serialization:
+
+Session serialization
+---------------------
+
+.. versionchanged:: 1.6
+
+Before version 1.6, Django defaulted to using :mod:`pickle` to serialize
+session data before storing it in the backend. If you're using the :ref:`signed
+cookie session backend<cookie-session-backend>` and :setting:`SECRET_KEY` is
+known by an attacker, the attacker could insert a string into his session
+which, when unpickled, executes arbitrary code on the server. The technique for
+doing so is simple and easily available on the internet. Although the cookie
+session storage signs the cookie-stored data to prevent tampering, a
+:setting:`SECRET_KEY` leak immediately escalates to a remote code execution
+vulnerability.
+
+This attack can be mitigated by serializing session data using JSON rather
+than :mod:`pickle`. To facilitate this, Django 1.5.3 introduced a new setting,
+:setting:`SESSION_SERIALIZER`, to customize the session serialization format.
+For backwards compatibility, this setting defaults to
+using :class:`django.contrib.sessions.serializers.PickleSerializer` in
+Django 1.5.x, but, for security hardening, defaults to
+:class:`django.contrib.sessions.serializers.JSONSerializer` in Django 1.6.
+Even with the caveats described in :ref:`custom-serializers`, we highly
+recommend sticking with JSON serialization *especially if you are using the
+cookie backend*.
+
+Bundled Serializers
+^^^^^^^^^^^^^^^^^^^
+
+.. class:: serializers.JSONSerializer
+
+    A wrapper around the JSON serializer from :mod:`django.core.signing`. Can
+    only serialize basic data types. See the :ref:`custom-serializers` section
+    for more details.
+
+.. class:: serializers.PickleSerializer
+
+    Supports arbitrary Python objects, but, as described above, can lead to a
+    remote code execution vulnerability if :setting:`SECRET_KEY` becomes known
+    by an attacker.
+
+.. _custom-serializers:
+
+Write Your Own Serializer
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Note that unlike :class:`~django.contrib.sessions.serializers.PickleSerializer`,
+the :class:`~django.contrib.sessions.serializers.JSONSerializer` cannot handle
+arbitrary Python data types. As is often the case, there is a trade-off between
+convenience and security. If you wish to store more advanced data types
+including ``datetime`` and ``Decimal`` in JSON backed sessions, you will need
+to write a custom serializer (or convert such values to a JSON serializable
+object before storing them in ``request.session``). While serializing these
+values is fairly straightforward
+(``django.core.serializers.json.DateTimeAwareJSONEncoder`` may be helpful),
+writing a decoder that can reliably get back the same thing that you put in is
+more fragile. For example, you run the risk of returning a ``datetime`` that
+was actually a string that just happened to be in the same format chosen for
+``datetime``\s).
+
+Your serializer class must implement two methods,
+``dumps(self, obj)`` and ``loads(self, data)``, to serialize and deserialize
+the dictionary of session data, respectively.
+
 Session object guidelines
 -------------------------
 
@@ -390,14 +459,15 @@ An API is available to manipulate session data outside of a view::
     >>> from django.contrib.sessions.backends.db import SessionStore
     >>> import datetime
     >>> s = SessionStore()
-    >>> s['last_login'] = datetime.datetime(2005, 8, 20, 13, 35, 10)
+    >>> # stored as seconds since epoch since datetimes are not serializable in JSON.
+    >>> s['last_login'] = 1376587691
     >>> s.save()
     >>> s.session_key
     '2b1189a188b44ad18c35e113ac6ceead'
 
     >>> s = SessionStore(session_key='2b1189a188b44ad18c35e113ac6ceead')
     >>> s['last_login']
-    datetime.datetime(2005, 8, 20, 13, 35, 0)
+    1376587691
 
 In order to prevent session fixation attacks, sessions keys that don't exist
 are regenerated::
@@ -543,8 +613,11 @@ behavior:
 Technical details
 =================
 
-* The session dictionary should accept any pickleable Python object. See
-  the :mod:`pickle` module for more information.
+* The session dictionary accepts any :mod:`json` serializable value when using
+  :class:`~django.contrib.sessions.serializers.JSONSerializer` or any
+  pickleable Python object when using
+  :class:`~django.contrib.sessions.serializers.PickleSerializer`. See the
+  :mod:`pickle` module for more information.
 
 * Session data is stored in a database table named ``django_session`` .
 

docs/topics/i18n/translation.txt

@@ -1550,14 +1550,17 @@ should be used -- installation-wide, for a particular user, or both.
 
 To set an installation-wide language preference, set :setting:`LANGUAGE_CODE`.
 Django uses this language as the default translation -- the final attempt if no
-other translator finds a translation.
+better matching translation is found through one of the methods employed by the
+locale middleware (see below).
 
-If all you want to do is run Django with your native language, and a language
-file is available for it, all you need to do is set :setting:`LANGUAGE_CODE`.
+If all you want is to run Django with your native language all you need to do
+is set :setting:`LANGUAGE_CODE` and make sure the corresponding :term:`message
+files <message file>` and their compiled versions (``.mo``) exist.
 
 If you want to let each individual user specify which language he or she
-prefers, use ``LocaleMiddleware``. ``LocaleMiddleware`` enables language
-selection based on data from the request. It customizes content for each user.
+prefers, then you also need to use use the ``LocaleMiddleware``.
+``LocaleMiddleware`` enables language selection based on data from the request.
+It customizes content for each user.
 
 To use ``LocaleMiddleware``, add ``'django.middleware.locale.LocaleMiddleware'``
 to your :setting:`MIDDLEWARE_CLASSES` setting. Because middleware order

docs/topics/testing/overview.txt

@@ -328,7 +328,8 @@ Some of the things you can do with the test client are:
   everything from low-level HTTP (result headers and status codes) to
   page content.
 
-* Test that the correct view is executed for a given URL.
+* See the chain of redirects (if any) and check the URL and status code at
+  each step.
 
 * Test that a given request is rendered by a given Django template, with
   a template context that contains certain values.
@@ -337,8 +338,8 @@ Note that the test client is not intended to be a replacement for Selenium_ or
 other "in-browser" frameworks. Django's test client has a different focus. In
 short:
 
-* Use Django's test client to establish that the correct view is being
-  called and that the view is collecting the correct context data.
+* Use Django's test client to establish that the correct template is being
+  rendered and that the template is passed the correct context data.
 
 * Use in-browser frameworks like Selenium_ to test *rendered* HTML and the
   *behavior* of Web pages, namely JavaScript functionality. Django also