Open Source Charter
PartiQL is a language that provides SQL-compatible unified query access across multiple data stores containing structured tabular data (as in SQL), in addition to semi-structured and nested data. PartiQL is already adopted by multiple AWS services (Amazon S3 Select, Amazon Glacier Select, Amazon Redshift Spectrum, Amazon Quantum Ledger Database) and Amazon internal systems. More AWS services will follow along with third party database engines.
The PartiQL syntax and semantics specifies a single language that is compatible among the multiple services that adopt it. While the adopting query engines may not support all features of PartiQL, a database engine that "supports PartiQL" is expected to be consistent with the PartiQL specification in the syntax subset it supports.
The PartiQL specification will expand in a number of directions, via community effort. As PartiQL expands, it is expected to keep conforming to the following PartiQL tenets, which guided its design:
PartiQL facilitates adoption by maintaining compatibility with SQL, avoiding the need to rewrite existing SQL and making it easy for developers and business intelligence tools to leverage PartiQL. Existing SQL queries will continue to work in SQL query processors extended to provide PartiQL.
The data model treats nested data as a fundamental part of the data abstraction. Consequently, the PartiQL query language provides syntax and semantics that comprehensively and accurately access and query nested data, while naturally composing with the standard features of SQL.
PartiQL does not require a predefined schema over a dataset. Rows within a table need not have the same schema. The results of a query do not change as schema is added or changed as long as the data itself is the same.
PartiQL has a minimum number of extensions over SQL. Extensions are easy to understand, lend themselves to efficient implementation, and compose well with each other and SQL itself. This enables intuitive filtering, joining, aggregation, and windowing on the combination of structured, semi-structured, and nested datasets.
PartiQL syntax and semantics are not tied to any particular data format. A query is written identically across underlying data in JSON, Parquet, ORC, CSV, Ion, or other formats. Queries operate on a comprehensive logical type system that maps to diverse underlying formats.
PartiQL syntax and semantics are not tied to a particular underlying data store. Because of its expressiveness, the language is applicable to diverse underlying data stores.
The open source organization and repositories will expand to reflect new PartiQL features as they emerge. Furthermore, we welcome contributions that add application specific functions (eg, feature new function libraries), support for reading particular data formats, query processing optimizations and other features that appeal in particular usage areas.
PartiQL’s evolution will be driven by the partiql-committee@amazon.com. We welcome your feedback and collaboration in PartiQL’s evolution.