Queries using SQL Query Engine

The SQL Query Engine (SQE) provides targeted optimization for partitioned tables using dynamic partition expansion optimization.

This targeted optimization method first determines whether a given query is structured such that certain partitions in the partitioned table would benefit from specific optimization. If targeted optimization is warranted, the optimizer determines which partitions can benefit from individual optimization; those partitions are then optimized separately. The remaining partitions use the once for all technique.

The optimizer determines whether the query or table environment justifies dynamic expansion based on the following characteristics:

The table is range partitioned and the query involves predicate selection against the range.
The table has an index over one or some of the partitions, but not all (a subpartition spanning index).
The table has relatively few partitions.
Constraint definitions on the table dictate that only certain partitions participate.
Estimated run time exceeds a particular threshold.

If expansion is justified, the optimizer determines the target partitions using existing statistic techniques as appropriate. For example, for range partitioning and predicate selection, the optimizer looks into the statistics or index to determine which main partitions are of interest. When the target partitions are identified, the optimizer rewrites the query. The targeted partitions are redefined in a UNION operation. The remaining partitions remain as a single table instance. That single instance is then added to the UNION operation along with the targeted partitions. As soon as the rewriting is performed, the optimizer uses existing optimization techniques to determine the plan for each UNION piece. At the end of the optimization, the single instance is converted into another UNION operation of its contained partitions. The optimized plan for the single instance is replicated across the UNION subtrees, thus drastically reducing the optimization time for partitioned tables.

The SQL Query Engine also uses logical partition elimination to optimize partitioned tables. This method allows the optimizer to identify potential partition elimination opportunities. The optimizer looks for opportunities where a source table's reduced answer set can be applied to the partition table's definition key with a join predicate. When these opportunities are identified, the optimizer builds logic into the query run time to eliminate partitions based on the result set of the source table.

For example, consider a query in which the optimizer identifies a predicate (in this example, a WHERE clause) involving the partition key of the partition table. If a constraint on a partition limits the key to a range from 0 to 10 and a predicate in the query identifies a key value of 11, the partition can be logically eliminated. Note that the implementation for the partition is still built into the query, but the path is shunted by the up-front processing of the constraint combined with the predicate. This logic is built in for reusability purposes. Another query with a predicate that identifies a key value of 10 can be implemented with this plan as well. If the partition was physically removed from the implementation, the implementation would not be reusable.

Consider this more complicated example:

select *
from sales, timeDim
where sales.DateId = timeDim.DateId
and timeDim.Date > '09/01/2004'

In this example, sales is a range-partitioned table where sales.DateId is identified as the partition key and sales.DateId = timeDim.DateId is identified as a potential reducing predicate. The query is modified (roughly) as:

with sourceTable as (select * from timeDim where  timeDim.Date > '09/01/2004')
select *
from sales, sourceTable
where sales.DateId = sourceTable.DateId
and sales.DateId IN (select DateId from sourceTable)

In this example, the original join survives, but the additional local predicate, sales.DateId IN (select DateId from sourceTable), is added. This new predicate can now be combined with information about the partition definition to produce an answer of which partitions participate. This outcome is produced because of the following conditions:

The existence of a join to the partitioning key.
The join from table can be reduced into a smaller set of join values. These join values can then be used to shunt partitions in the partitioned table.