PartiQL is a language that provides SQL-compatible unified query access across multiple data stores containing structured tabular data (as in SQL), in addition to semi-structured and nested data. PartiQL is already adopted by multiple AWS services (Amazon S3 Select, Amazon Glacier Select, Amazon Redshift Spectrum, Amazon Quantum Ledger Database) and Amazon internal systems. More AWS services will follow along with third party database engines.
The PartiQL syntax and semantics specifies a single language that is compatible among the multiple services that adopt it. While the adopting query engines may not support all features of PartiQL, a database engine that “supports PartiQL” is expected to be consistent with the PartiQL specification in the syntax subset it supports.
To ensure consistency, the PartiQL specification provides the detail on syntax and semantics while the open source PartiQL reference engine allows you to check the PartiQL compliance of an implementation.
The PartiQL specification will expand in a number of directions, via community effort. As PartiQL expands, it is expected to keep conforming to the following PartiQL tenets, which guided its design:
SQL compatibility: PartiQL facilitates adoption by maintaining compatibility with SQL, avoiding the need to rewrite existing SQL and making it easy for developers and business intelligence tools to leverage PartiQL. Existing SQL queries will continue to work in SQL query processors extended to provide PartiQL.
Minimal extensions: PartiQL has a minimum number of extensions over SQL. Extensions are easy to understand, lend themselves to efficient implementation, and compose well with each other and SQL itself. This enables intuitive filtering, joining, aggregation, and windowing on the combination of structured, semi-structured, and nested datasets.
Optional schema and query stability: PartiQL does not require a predefined schema over a dataset. Rows within a table need not have the same schema. The results of a query do not change as schema is added or changed as long as the data itself is the same.
Format independence: PartiQL syntax and semantics are not tied to any particular data format. A query is written identically across underlying data in JSON, Parquet, ORC, CSV, Ion, or other formats. Queries operate on a comprehensive logical type system that maps to diverse underlying formats.
Data store independence: PartiQL syntax and semantics are not tied to a particular underlying data store. Because of its expressiveness, the language is applicable to diverse underlying data stores.
The open source organization and repositories will expand to reflect new PartiQL features as they emerge. Furthermore, we welcome contributions that add application specific functions (eg, feature new function libraries), support for reading particular data formats, query processing optimizations and other features that appeal in particular usage areas.
PartiQL’s evolution will be driven by the PartiQL steering committee. We welcome your feedback and collaboration in PartiQL’s evolution.