Scuemata: A Framework for Evolvable, Composable Data Schema
This talk lays out Scuemata, a CUE-based framework for expressing data schemas that aims directly at these issues of schema evolution and backwards compatibility. It does not seek to innovate in defining schema themselves (beyond what CUE natively offers vis-a-vis other schema languages). Rather, Scuemata defines a logical meta-structure around schemas that allows for:
- All versions of a schema to be declared in a single place, easing analysis for humans and machines
- Enforcement that migrations are written when a particular schema change is backwards-incompatible
- Standard ways of expressing the risks and insufficiency of automated migrations
- Encapsulating the problem of data validation and migration to a boring, standardized process that can be isolated to process boundaries of programs
- Standard interfaces and tools for consuming any scuemata-defined schema, allowing generic tooling to be written for tasks like language translation, API generation, and more
- Standardized composition of scuemata, allowing large schema to be broken down into discrete subcomponents managed by different groups, each evolving independently
- Seamless integration into larger programs and ecosystems. (Scuemata are currently used by Grafana to express schema are complex, pluggable, and compositional, and canonical in three languages.)
Scuemata is based on a premise borne of years of experience with how most developers actually deal with versioning and compatibility in their day-to-day programming. Schema authors generally want to either maintain backwards compatibility, or else define migration paths - but absent simple structures, easy verification, and supportive tooling, doing so is often too onerous and nonstandardized to justify. By providing formalisms that encapsulate the meta-problems of schema evolution, we aim to reduce friction to the point where that calculus can shift.
Mon 18 OctDisplayed time zone: Central Time (US & Canada) change
10:50 - 12:10 | Configuration managementCONFLANG at Zurich E Chair(s): Jürgen Cito TU Wien and Facebook, Marcel van Lohuizen CUE | ||
10:50 5mTalk | Analyzing Infrastructure as Code to Prevent Intra-update Sniping Vulnerabilities CONFLANG Julien Lepiller Yale University Media Attached | ||
10:55 5mTalk | Continuous Configuration Testing CONFLANG Tianyin Xu University of Illinois at Urbana-Champaign Media Attached | ||
11:00 5mTalk | Intra-update Sniping Vulnerabilities in Smart Contracts CONFLANG Mark Santolucito Barnard College, Columbia University, USA, Shmuel Berman Columbia University, Brennen Yu Columbia University, USA, Stella Lessler Media Attached | ||
11:05 5mTalk | Local Expectation Testing for Terraform CONFLANG | ||
11:10 5mTalk | Scuemata: A Framework for Evolvable, Composable Data Schema CONFLANG Sam Boyer Grafana Labs Media Attached | ||
11:15 55mLive Q&A | Configuration management: Q&A and discussion CONFLANG |