There is a distinction between tests that verify extension-specific
behavior and tests that verify interactions between the database engine
and extensions. Previously, both types of tests were kept in extensions.py.
With this new framework, we can extract the latter type of tests from
extensions.py into TCL. This cleans up extensions.py and provides
compatibility testing with SQLite at no extra cost.
To demonstrate the framework’s usage, tests verifying the handling of
virtual tables were extracted to TCL.
In the future, we may consider moving extension-specific tests to TCL as
well, especially those that have counterparts in SQLite or sqlean.
The side of the binary expression no longer needs to be stored in
`ConstraintInfo`, since the optimizer now guarantees that it is always
on the right. As a result, only the index of the corresponding constraint
needs to be preserved.
This change extends table-valued function support by allowing arguments
to be column references, not only literals.
Virtual tables can now reject a plan by returning an error from
best_index (e.g., when a TVF argument references a table that appears
later in the join order). The planner using this information excludes
invalid plans during join order search.
This change connects virtual tables with the query optimizer.
The optimizer now considers virtual tables during join order search
and invokes their best_index callbacks to determine feasible access
paths.
Currently, this is not a visible change, since none of the existing
extensions return information indicating that a plan is invalid.
The `best_index` implementation now returns a ResultCode along with the
IndexInfo. This allows it to signal specific outcomes, such as errors or
constraint violations. This change aligns better with SQLite’s xBestIndex
contract, where cases like missing constraints or invalid combinations of
constraints must not result in a valid plan.
The `filter` methods for extensions affected by this fix expect arguments
to be passed in a specific order. For example, `generate_series` assumes
that if the `start` argument exists, it is always passed to `filter`
first. If `start` does not exist, then `stop` is passed first — but
`stop` must never come before `start`.
Previously, this was not guaranteed: `best_index` relied on constraints
being passed in the order matching `filter`'s expectations.
Previously, there were two ways to indicate that a constraint should not
be passed to the filter function: setting `argv_index` to `None` or to
a value less than 1. This was redundant, so now only `None` is used.
Additional changes:
- Update IndexInfo documentation to clarify that constraint_usages must
have exact 1:1 correspondence with input ConstraintInfo array. The code
translating constraints into VFilter arguments heavily relies on this.
- Fix best_index implementation in test extension to comply with new
validation requirements by returning usage entry for each constraint
`BTreeTable::to_sql` makes us incompatible with SQLite by losing e.g. the original whitespace provided during the CREATE TABLE command.
For now let's fix our tests by regex-replacing every CREATE TABLE in
the entire repo to have exactly 1 space after the table name in the
CREATE TABLE statement.
as nilskch points out in #1807, Rust 1.88.0 is stricter about alignment
checks.
because rust integers default to `i32`, we were casting a pointer to an
`i32` as a pointer to an `i64` causing a panic when dereferenced due to
misalignment as rust expects it to be 8 byte aligned.
Reviewed-by: Preston Thorpe (@PThorpe92)
Closes#2064
With this change, the following two queries are considered equivalent:
```sql
SELECT value FROM generate_series(5, 50);
SELECT value FROM generate_series WHERE start = 5 AND stop = 50;
```
Arguments passed in parentheses to the virtual table name are now
matched to hidden columns.
Column references are still not supported as table-valued function
arguments. The only difference is that previously, a query like:
```sql
SELECT one.value, series.value
FROM (SELECT 1 AS value) one, generate_series(one.value, 3) series;
```
would cause a panic. Now, it returns a proper error message instead.
Adding support for column references is more nuanced for two main
reasons:
- We need to ensure that in joins where a TVF depends on other tables,
those other tables are processed first. For example, in:
```sql
SELECT one.value, series.value
FROM generate_series(one.value, 3) series, (SELECT 1 AS value) one;
```
the one table must be processed by the top-level loop, and series must
be nested.
- For outer joins involving TVFs, the arguments must be treated as ON
predicates, not WHERE predicates.
In SQLite, the field equivalent to `constraint_usage` (`aConstraintUsage`
from `sqlite3_index_info`) is used to request arguments that are later
passed to the `xFilter` method. In Limbo, this behavior applies to
virtual tables, but not to table-valued functions. Currently, TVFs have
dedicated handling that passes all function arguments to the filter
method and doesn't use information provided in the `constraint_usage`
field.
This commit is a step toward unifying the handling of virtual tables and
TVFs.
