with merge confs fixed

README.md

@@ -396,7 +396,10 @@ In addition, Cotton enables you to navigate around some of the limitations with
 
 ## Caching
 
-Cotton components are cached whilst in production (`DEBUG = False`). The cache's TTL is for the duration of your app's lifetime. So on deployment, when the app is normally restarted, caches are cleared. During development, changes are detected on every component render. This feature is a work in progress and some refinement is planned.
+Cotton is optimal when used with Django's cached.Loader. If you use <a href="https://django-cotton.com/docs/quickstart">automatic configuration</a> then the cached loader will be automatically applied. This feature has room for improvement, some desirables are:
+
+- Integration with a cache backend to survive runtime restarts / deployments.
+- Cache warming
 
 For full docs and demos, checkout <a href="https://django-cotton.com" target="_blank">django-cotton.com</a>
 
@@ -404,6 +407,11 @@ For full docs and demos, checkout <a href="https://django-cotton.com" target="_b
 
 | Version | Date                                                               | Title and Description                                                                                                                                                        |
 |---------|--------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| v0.9.30 | 2024-08-23                                                         | Fixes an issue where by attribute order defined inside default slots were not being kept, causing template expression issues.
+| v0.9.29 | 2024-08-22                                                         | Added an auto setup script on boot that manage settings.py with fallback. Overhauled caching and adopted django's template caching with custom fallback.                  |
+| v0.9.28 | 2024-08-21                                                         | Reverted cache changes due to cache not recognising file updates.                  |
+| v0.9.27 | 2024-08-21                                                         | Resolved issues with component caching in dev environments.                  |
+| v0.9.26 | 2024-08-19                                                         | We now check if a template contains any cotton syntax before processing it.                  |
 | v0.9.25 | 2024-08-12                                                         | Fix case sensitive placeholder issue in connection with duplicate attribute handling mechanism.                  |
 | v0.9.24 | 2024-08-12                                                         | Fixes whitespace preservation around template expressions used in attribute names.                  |
 | v0.9.23 | 2024-07-21                                                         | Fixed an issue causing closing tags to become mutated, resolved with better whitespace handling.                  |

dev/example_project/example_project/settings.py

@@ -66,20 +66,6 @@ TEMPLATES = [
                 "django.contrib.auth.context_processors.auth",
                 "django.contrib.messages.context_processors.messages",
             ],
-            "loaders": [
-                (
-                    "django.template.loaders.cached.Loader",
-                    [
-                        "django_cotton.cotton_loader.Loader",
-                        "django.template.loaders.filesystem.Loader",
-                        "django.template.loaders.app_directories.Loader",
-                    ],
-                )
-            ],
-            "builtins": [
-                "django.templatetags.static",
-                "django_cotton.templatetags.cotton",
-            ],
         },
     },
 ]

dev/example_project/render_load_test.py

@@ -27,31 +27,17 @@ settings.configure(
                     "django.contrib.auth.context_processors.auth",
                     "django.contrib.messages.context_processors.messages",
                 ],
-                "loaders": [
-                    (
-                        "django.template.loaders.cached.Loader",
-                        [
-                            "django_cotton.cotton_loader.Loader",
-                            "django.template.loaders.filesystem.Loader",
-                            "django.template.loaders.app_directories.Loader",
-                        ],
-                    )
-                ],
-                "builtins": [
-                    "django.templatetags.static",
-                    "django_cotton.templatetags.cotton",
-                ],
             },
         },
     ],
-    COTTON_TEMPLATE_CACHING_ENABLED=True,
+    COTTON_TEMPLATE_CACHING_ENABLED=False,
     DEBUG=False,
 )
 
 django.setup()
 
 
-def template_bench(template_name, iterations=10000):
+def template_bench(template_name, iterations=500):
     start_time = time.time()
     for _ in range(iterations):
         render_to_string(template_name)
@@ -59,7 +45,7 @@ def template_bench(template_name, iterations=10000):
     return end_time - start_time, render_to_string(template_name)
 
 
-def template_bench_alt(template_name, iterations=10000):
+def template_bench_alt(template_name, iterations=500):
     data = list(range(1, iterations))
     start_time = time.time()
     render_to_string(template_name, context={"data": data})

django_cotton/apps.py

@@ -0,0 +1,77 @@
+"""
+django-cotton
+
+App configuration to set up the cotton loader and builtins automatically.
+"""
+
+from contextlib import suppress
+
+import django.contrib.admin
+import django.template
+from django.apps import AppConfig
+from django.conf import settings
+
+
+def wrap_loaders(name):
+    for template_config in settings.TEMPLATES:
+        engine_name = template_config.get("NAME")
+        if not engine_name:
+            engine_name = template_config["BACKEND"].split(".")[-2]
+        if engine_name == name:
+            loaders = template_config.setdefault("OPTIONS", {}).get("loaders", [])
+
+            loaders_already_configured = (
+                loaders
+                and isinstance(loaders, (list, tuple))
+                and isinstance(loaders[0], (tuple, list))
+                and loaders[0][0] == "django.template.loaders.cached.Loader"
+                and "django_cotton.cotton_loader.Loader" in loaders[0][1]
+            )
+
+            if not loaders_already_configured:
+                template_config.pop("APP_DIRS", None)
+                default_loaders = [
+                    "django_cotton.cotton_loader.Loader",
+                    "django.template.loaders.filesystem.Loader",
+                    "django.template.loaders.app_directories.Loader",
+                ]
+                cached_loaders = [("django.template.loaders.cached.Loader", default_loaders)]
+                template_config["OPTIONS"]["loaders"] = cached_loaders
+
+            options = template_config.setdefault("OPTIONS", {})
+            builtins = options.setdefault("builtins", [])
+            builtins_already_configured = (
+                builtins and "django_cotton.templatetags.cotton" in builtins
+            )
+            if not builtins_already_configured:
+                template_config["OPTIONS"]["builtins"].insert(
+                    0, "django_cotton.templatetags.cotton"
+                )
+
+            break
+
+    # Force re-evaluation of settings.TEMPLATES because EngineHandler caches it.
+    with suppress(AttributeError):
+        del django.template.engines.templates
+        django.template.engines._engines = {}
+
+
+class LoaderAppConfig(AppConfig):
+    """
+    This, the default configuration, does the automatic setup of a partials loader.
+    """
+
+    name = "django_cotton"
+    default = True
+
+    def ready(self):
+        wrap_loaders("django")
+
+
+class SimpleAppConfig(AppConfig):
+    """
+    This, the non-default configuration, allows the user to opt-out of the automatic configuration. They just need to
+    add "django_cotton.apps.SimpleAppConfig" to INSTALLED_APPS instead of "django_cotton".
+    """
+
+    name = "django_cotton"

django_cotton/cotton_loader.py

@@ -6,52 +6,44 @@ import re
 
 from django.template.loaders.base import Loader as BaseLoader
 from django.core.exceptions import SuspiciousFileOperation
-from django.template import TemplateDoesNotExist
+from django.template import TemplateDoesNotExist, Origin
-from bs4.formatter import HTMLFormatter
 from django.utils._os import safe_join
 from django.template import Template
-from django.core.cache import cache
-from django.template import Origin
 from django.conf import settings
 from django.apps import apps
 
 from bs4 import BeautifulSoup, MarkupResemblesLocatorWarning
+from bs4.formatter import HTMLFormatter
 
 warnings.filterwarnings("ignore", category=MarkupResemblesLocatorWarning)
 
-# If an update changes the API that a cached version of a template will break, we increment the cache version in order to
-# force the re-rendering of the template
-cache_version = "1"
-
 
 class Loader(BaseLoader):
     is_usable = True
 
     def __init__(self, engine, dirs=None):
         super().__init__(engine)
-        self.cache_handler = CottonTemplateCacheHandler()
         self.cotton_compiler = CottonCompiler()
+        self.cache_handler = CottonTemplateCacheHandler()
         self.dirs = dirs
 
     def get_contents(self, origin):
-        # check if file exists, whilst getting the mtime for cache key
+        cache_key = self.cache_handler.get_cache_key(origin)
-        try:
-            mtime = os.path.getmtime(origin.name)
-        except FileNotFoundError:
-            raise TemplateDoesNotExist(origin)
-
-        cache_key = self.cache_handler.get_cache_key(origin.template_name, mtime)
         cached_content = self.cache_handler.get_cached_template(cache_key)
 
         if cached_content is not None:
             return cached_content
 
         template_string = self._get_template_string(origin.name)
-        compiled_template = self.cotton_compiler.process(template_string, origin.template_name)
 
-        self.cache_handler.cache_template(cache_key, compiled_template)
+        if "<c-" not in template_string and "{% cotton_verbatim" not in template_string:
+            compiled = template_string
+        else:
+            compiled = self.cotton_compiler.process(template_string, origin.template_name)
 
-        return compiled_template
+        self.cache_handler.cache_template(cache_key, compiled)
+
+        return compiled
 
     def get_template_from_string(self, template_string):
         """Create and return a Template object from a string. Used primarily for testing."""
@@ -75,6 +67,10 @@ class Loader(BaseLoader):
 
         return dirs
 
+    def reset(self):
+        """Empty the template cache."""
+        self.cache_handler.reset()
+
     def get_template_sources(self, template_name):
         """Return an Origin object pointing to an absolute path in each directory
         in template_dirs. For security reasons, if a path doesn't lie inside
@@ -113,9 +109,9 @@ class CottonCompiler:
     def __init__(self):
         self.django_syntax_placeholders = []
 
-    def process(self, content, component_key):
+    def process(self, content, template_name):
         content = self._replace_syntax_with_placeholders(content)
-        content = self._compile_cotton_to_django(content, component_key)
+        content = self._compile_cotton_to_django(content, template_name)
         content = self._fix_bs4_attribute_empty_attribute_behaviour(content)
         content = self._replace_placeholders_with_syntax(content)
         content = self._remove_duplicate_attribute_markers(content)
@@ -165,7 +161,7 @@ class CottonCompiler:
 
         return content
 
-    def _compile_cotton_to_django(self, html_content, component_key):
+    def _compile_cotton_to_django(self, html_content, template_name):
         """Convert cotton <c-* syntax to {%."""
         soup = BeautifulSoup(
             html_content,
@@ -177,7 +173,7 @@ class CottonCompiler:
         if cvars_el := soup.find("c-vars"):
             soup = self._wrap_with_cotton_vars_frame(soup, cvars_el)
 
-        self._transform_components(soup, component_key)
+        self._transform_components(soup, template_name)
 
         return str(soup.encode(formatter=UnsortedAttributes()).decode("utf-8"))
 
@@ -296,7 +292,7 @@ class CottonCompiler:
 
             if tag.contents:
                 tag_soup = BeautifulSoup(
-                    tag.decode_contents(),
+                    tag.decode_contents(formatter=UnsortedAttributes()),
                     "html.parser",
                     on_duplicate_attribute=self.handle_duplicate_attributes,
                 )
@@ -357,21 +353,32 @@ class CottonCompiler:
 
 
 class CottonTemplateCacheHandler:
-    """Handles caching of cotton templates so the html parsing is only done on first load of each view or component."""
+    """This mimics the simple template caching mechanism in Django's cached.Loader. Django's cached.Loader is a bit
+    more performant than this one but it acts a decent fallback when the loader is not defined in the Django cache loader.
+
+    TODO: implement integration to the cache backend instead of just memory. one which can also be controlled / warmed
+    by the user and lasts beyond runtime restart.
+    """
 
     def __init__(self):
-        self.enabled = getattr(settings, "COTTON_TEMPLATE_CACHING_ENABLED", True)
+        self.template_cache = {}
-
-    def get_cache_key(self, template_name, mtime):
-        template_hash = hashlib.sha256(template_name.encode()).hexdigest()
-        return f"cotton_cache_v{cache_version}_{template_hash}_{mtime}"
 
     def get_cached_template(self, cache_key):
-        if not self.enabled:
+        return self.template_cache.get(cache_key)
-            return None
 
-        return cache.get(cache_key)
+    def cache_template(self, cache_key, compiled_template):
+        self.template_cache[cache_key] = compiled_template
 
-    def cache_template(self, cache_key, content, timeout=None):
+    def get_cache_key(self, origin):
-        if self.enabled:
+        try:
-            cache.set(cache_key, content, timeout=timeout)
+            cache_key = self.generate_hash([origin.name, str(os.path.getmtime(origin.name))])
+        except FileNotFoundError:
+            raise TemplateDoesNotExist(origin)
+
+        return cache_key
+
+    def generate_hash(self, values):
+        return hashlib.sha1("|".join(values).encode()).hexdigest()
+
+    def reset(self):
+        self.template_cache.clear()

django_cotton/templatetags/_component.py

@@ -1,5 +1,4 @@
 import ast
-from functools import lru_cache
 
 from django import template
 from django.conf import settings
@@ -14,19 +13,6 @@ class CottonIncompleteDynamicComponentException(Exception):
     pass
 
 
-@lru_cache(maxsize=1024)
-def get_cached_template(template_name):
-    """App runtime cache for cotton templates. Turned on only when DEBUG=False."""
-    return get_template(template_name)
-
-
-def render_template(template_name, context):
-    if settings.DEBUG:
-        return get_template(template_name).render(context)
-    else:
-        return get_cached_template(template_name).render(context)
-
-
 def cotton_component(parser, token):
     """
     Template tag to render a cotton component with dynamic attributes.
@@ -79,7 +65,6 @@ class CottonComponentNode(Node):
             for expression_attr in local_named_slots_ctx["ctn_template_expression_attrs"]:
                 attrs[expression_attr] = local_named_slots_ctx[expression_attr]
 
-        # Build attrs string before formatting any '-' to '_' in attr names
         attrs_string = " ".join(f"{key}={ensure_quoted(value)}" for key, value in attrs.items())
         local_ctx["attrs"] = mark_safe(attrs_string)
         local_ctx["attrs_dict"] = attrs
@@ -93,7 +78,7 @@ class CottonComponentNode(Node):
 
         template_path = self._generate_component_template_path(attrs)
 
-        return render_template(template_path, local_ctx)
+        return get_template(template_path).render(local_ctx)
 
     def _build_attrs(self, context):
         """

django_cotton/templatetags/_vars_frame.py

@@ -32,34 +32,30 @@ class CottonVarsFrameNode(template.Node):
         self.kwargs = kwargs
 
     def render(self, context):
-        # Assume 'attrs' are passed from the parent and are available in the context
+        # Retrieve attrs_dict from parent _component
-        component_attrs = context.get("attrs_dict", {})
+        provided_attrs = context.get("attrs_dict", {})
 
         # Initialize vars based on the frame's kwargs and parent attrs
-        vars = {}
+        c_vars = {}
         for key, value in self.kwargs.items():
             # Check if the var exists in component attrs; if so, use it, otherwise use the resolved default
-            if key in component_attrs:
+            if key in provided_attrs:
-                vars[key] = component_attrs[key]
+                c_vars[key] = provided_attrs[key]
             else:
                 # Attempt to resolve each kwarg value (which may include template variables)
                 resolved_value = value.resolve(context)
-                vars[key] = resolved_value
+                c_vars[key] = resolved_value
 
         # Overwrite 'attrs' in the local context by excluding keys that are identified as vars
-        attrs_without_vars = {k: v for k, v in component_attrs.items() if k not in vars}
+        attrs_dict = {k: v for k, v in provided_attrs.items() if k not in c_vars}
 
         # Provide all of the attrs as a string to pass to the component before any '-' to '_' replacing
-        attrs = " ".join(
+        attrs_string = " ".join(f"{k}={ensure_quoted(v)}" for k, v in attrs_dict.items())
-            f"{k}={ensure_quoted(v)}" for k, v in attrs_without_vars.items()
+        context["attrs"] = mark_safe(attrs_string)
-        )
+        context["attrs_dict"] = attrs_dict
-        context["attrs"] = mark_safe(attrs)
-
-        context["attrs_dict"] = attrs_without_vars
 
         # Store attr names in a callable format, i.e. 'x-init' will be accessible by {{ x_init }} when called explicitly and not in {{ attrs }}
-        formatted_vars = {key.replace("-", "_"): value for key, value in vars.items()}
+        formatted_vars = {key.replace("-", "_"): value for key, value in c_vars.items()}
-
         context.update(formatted_vars)
 
         return self.nodelist.render(context)

django_cotton/tests/test_cotton.py

@@ -160,7 +160,11 @@ class InlineTestCase(CottonInlineTestCase):
             """<div class="i-am-component">{{ slot }}</div>""",
         )
 
-        self.create_template("custom_directory_view.html", """<c-custom-directory>Hello, World!</c-custom-directory>""", "view/")
+        self.create_template(
+            "custom_directory_view.html",
+            """<c-custom-directory>Hello, World!</c-custom-directory>""",
+            "view/",
+        )
 
         # Override URLconf
         with self.settings(ROOT_URLCONF=self.get_url_conf(), COTTON_DIR=custom_dir):
@@ -192,7 +196,7 @@ class InlineTestCase(CottonInlineTestCase):
 
             self.assertContains(response, '@click="this=test"')
 
-    def test_dynamic_components(self):
+    def test_dynamic_components_via_string(self):
         self.create_template(
             "cotton/dynamic_component.html",
             """
@@ -208,6 +212,22 @@ class InlineTestCase(CottonInlineTestCase):
 
         self.assertTrue("I am dynamic" in rendered)
 
+    def test_dynamic_components_via_variable(self):
+        self.create_template(
+            "cotton/dynamic_component.html",
+            """
+            <div>I am dynamic<div>
+            """,
+        )
+
+        html = """
+            <c-component :is="is" />
+        """
+
+        rendered = get_rendered(html, {"is": "dynamic-component"})
+
+        self.assertTrue("I am dynamic" in rendered)
+
     def test_dynamic_components_via_expression_attribute(self):
         self.create_template(
             "cotton/subfolder/dynamic_component_expression.html",
@@ -397,3 +417,18 @@ class CottonTestCase(TestCase):
 
         self.assertFalse("</div{% if 1 = 1 %}>" in rendered, "Tag corrupted")
         self.assertTrue("</div>" in rendered, "</div> not found in rendered string")
+
+    def test_conditionals_evaluation_inside_elements(self):
+        html = """
+            <c-test-component>
+                <select>
+                    <option value="1" {% if my_obj.selection == 1 %}selected{% endif %}>Value 1</option>
+                    <option value="2" {% if my_obj.selection == 2 %}selected{% endif %}>Value 2</option>
+                </select>                         
+            </c-test-component>
+        """
+
+        rendered = get_rendered(html, {"my_obj": {"selection": 1}})
+
+        self.assertTrue('<option value="1" selected>Value 1</option>' in rendered)
+        self.assertTrue('<option value="2" selected>Value 2</option>' not in rendered)

docs/docs_project/docs_project/settings.py

@@ -64,7 +64,7 @@ TEMPLATES = [
     {
         "BACKEND": "django.template.backends.django.DjangoTemplates",
         "DIRS": ["docs_project/templates"],
-        "APP_DIRS": False,
+        "APP_DIRS": True,
         "OPTIONS": {
             "context_processors": [
                 "django.template.context_processors.debug",
@@ -72,19 +72,8 @@ TEMPLATES = [
                 "django.contrib.auth.context_processors.auth",
                 "django.contrib.messages.context_processors.messages",
             ],
-            "loaders": [
-                (
-                    "django.template.loaders.cached.Loader",
-                    [
-                        "django_cotton.cotton_loader.Loader",
-                        "django.template.loaders.filesystem.Loader",
-                        "django.template.loaders.app_directories.Loader",
-                    ],
-                )
-            ],
             "builtins": [
                 "django.templatetags.static",
-                "django_cotton.templatetags.cotton",
                 "docs_project.templatetags.force_escape",
                 "docs_project.templatetags.custom_tags",
                 "heroicons.templatetags.heroicons",
@@ -159,6 +148,7 @@ STATIC_ROOT = os.path.join(BASE_DIR, "staticfiles")
 
 DEFAULT_AUTO_FIELD = "django.db.models.BigAutoField"
 
+COTTON_TEMPLATE_CACHING_ENABLED = True
 
 CACHES = {
     "default": {

docs/docs_project/docs_project/templates/cotton/navbar.html

@@ -1,6 +1,6 @@
 <div class="py-5 sticky top-0 md:top-auto bg-amber-50 dark:bg-gray-900 border-opacity-15 z-10" :class="{'bg-white dark:bg-gray-800': isOpen}"
      x-data="{isOpen: false}"
-     x-init="h
+     x-init="
         window.addEventListener('resize', () => {
             if (window.matchMedia('(min-width: 640px)').matches) {
                 isOpen = false;

docs/docs_project/docs_project/templates/quickstart.html

@@ -15,31 +15,46 @@
     <h2>Install cotton</h2>
     <p>Run the following command:</p>
     <c-snippet language="python">pip install django-cotton</c-snippet>
+
     <p>Then update your settings.py:</p>
 
+    <h3>Automatic configuration:</h3>
+
     <c-snippet language="python" label="settings.py">{% cotton_verbatim %}{% verbatim %}
 INSTALLED_APPS = [
-    ...
     'django_cotton',
 ]
+    {% endverbatim %}{% endcotton_verbatim %}</c-snippet>
+
+    <p>This will automatically handle the settings.py adding the required loader and templatetags.</p>
+
+    <h3>Customised configuration</h3>
+
+    <p>If your project requires any non-default loaders or you do not wish Cotton to manage your settings, you should instead provide `django_cotton.apps.SimpleAppConfig` in your INSTALLED_APPS:</p>
+
+    <c-snippet language="python" label="settings.py">{% cotton_verbatim %}{% verbatim %}
+INSTALLED_APPS = [
+    'django_cotton.apps.SimpleAppConfig',
+]
 
 TEMPLATES = [
     {
-        'BACKEND': 'django.template.backends.django.DjangoTemplates',
+        "BACKEND": "django.template.backends.django.DjangoTemplates",
-        'DIRS': ['your_project/templates'], # Add your template directories here
+        ...
-        'APP_DIRS': False,
+        "OPTIONS": {
-        'OPTIONS': {
+            "loaders": [(
-            'loaders': [
+                "django.template.loaders.cached.Loader",
-                'django_cotton.cotton_loader.Loader', # First position
+                [
-                # continue with default loaders, typically:
+                    "django_cotton.cotton_loader.Loader",
-                # "django.template.loaders.filesystem.Loader",
+                    "django.template.loaders.filesystem.Loader",
-                # "django.template.loaders.app_directories.Loader",
+                    "django.template.loaders.app_directories.Loader",
+                ],
+            )],
+            "builtins": [
+                "django_cotton.templatetags.cotton"
             ],
-            'builtins': [
+        }
-                'django_cotton.templatetags.cotton',
+    }
-            ],
-        },
-    },
 ]
     {% endverbatim %}{% endcotton_verbatim %}</c-snippet>
 

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "poetry.core.masonry.api"
 
 [tool.poetry]
 name = "django-cotton"
-version = "0.9.25"
+version = "0.9.31"
 description = "Bringing component based design to Django templates."
 authors = [ "Will Abbott <willabb83@gmail.com>",]
 license = "MIT"