SakaQL

A new general-purpose relational programming language

Sufyan Dawoodjee ·

Four animations drawn by SakaQL programs: a ray tracer whose small sphere orbits three larger ones, a rainbow plasma that flows, a Game of Life board that evolves, and a Voronoi diagram whose colored cells drift
Graphics rendered with SakaQL

TL;DR: SQL is bad, so I made a general-purpose relational programming language. It feels like a modern language and has features you won’t find anywhere else. It powers Saka, an upcoming data management product for developers.

SQL bad

SQL is a flawed language with bad ergonomics. Other languages compete for users on their own merits, but not SQL. It comes packaged with databases, and if you don’t like it, tough luck. You can’t even build a better language on top because performance requirements inevitably demand SQL’s low-level control.

People have tried. EdgeQL was ambitious and promising, but it didn’t work out because it asked users to make an unjustifiable tradeoff: get a better language, but risk your data on unproven technology from a startup, lose compatibility with the wider data ecosystem, and burn time learning something new. Contrast this with choosing a new SQL database: try it, and in the worst case, go back to Postgres.

Other relational languages have found success by sticking to a niche: Datalog for recursive queries, Cypher for graph traversal, and CodeQL for code analysis. Outside their niche, they’re irrelevant.

SakaQL good

Despite its dominance in databases, SQL hasn’t made significant inroads in general-purpose programming because its ergonomics are intolerable. Declarative schemas and queries are very useful, but developers collectively reject them if they come with SQL’s baggage.

So what happens if you remove the baggage, layer on ideas stolen from modern programming languages, and throw in a splash of home-grown innovation?

You get SakaQL.

SakaQL is a general-purpose relational programming language. It has a foundation in relational algebra, expression-based syntax, a set-based type system inspired by TypeScript and EdgeQL, and an unusual skepticism of language design conventions.

This is what FizzBuzz looks like:

def fizzbuzz(n! int) =
  1..n
  map i => i -> {
    i modulo 15 == 0 then "FizzBuzz"
    else i modulo 3 == 0 then "Fizz"
    else i modulo 5 == 0 then "Buzz"
    else "\{i}"
  }

proc main(): void = do {
  print(str::join(fizzbuzz(100), "\n"))
}

I hope this conveys my intent: SakaQL should look and feel like a modern language.

Not SQL

It’s great to emulate the things you admire about predecessors. But it’s way more fun to point out their mistakes.

Here are some SakaQL design decisions inspired by SQL’s flaws:

  • No unique syntax for every feature, just shared syntax
  • No queries and clauses, just composable expressions
  • No obtuse name resolution rules, just lexical scopes
  • No split between scalar and table values, just multisets
  • No NULL or three-valued logic, just empty multisets
  • No tables, just records, tuples, and objects
  • No mind-bending correlated query rules, just expressions
  • No CTEs, just variables, functions, and views
  • No obtuse recursion rules, just arbitrary mutual recursion
  • No third-party incremental view maintenance, it’s native
  • No separate procedural language, it’s native

A brief tour of SakaQL

Types and values

Values are typed bags (unordered multisets). Empty bags replace NULL, and bags unnest implicitly, so there are no bags of bags.

// equivalent ways to write "a bag containing integer 7"
7
{7}
{{7}}
int:7
int:{7}

// 'n denotes an element repeated n times
{"a", "b", "b", "c"} // {"a", "b"'2, "c"}
{false'7, true}

// empty bags must have contextual type
int:{}
f64:{}
str:{}
bool:{}

Types are quantified sets with two parts:

  1. A typed, unique, unordered set of allowed values
  2. An integer interval specifying how many values a bag may contain
def a: f64'0 = {} // an empty bag of 64-bit floats
def b: bool'1 = true // a unary bag of booleans
def c: int'{3..7} = {1, 2, 3, 4} // a bag of 3 to 7 integers

!, ?, *, and + respectively denote intervals 1, {0, 1}, {0..inf}, and {1..inf}.

def a* int is syntax sugar for def a: int'*.

def a? f64 = 8.0 // an optional bag of 64-bit floats
def d* i32 = {1, 2} // an n-ary bag of 32-bit signed integers
def e+ str = {"a", "b", "c"} // a non-empty bag of strings

Records, tuples, and maps

Records, tuples, and maps are compound types that can be named with set declarations.

Records group named values.

set Person = (first: str, last: str)

def people: Person'* = {
  (first: "SpongeBob", last: "SquarePants")
  (first: "Patrick",   last: "Star")
}

Tuples group ordered values.

set Point = [f64, f64]
def points: Point'* = {[1, 2], [3, 4], [5, 6]}

Maps are syntax sugar for (k, v) records.

// equivalent
set int_to_str1 = int -> str
set int_to_str2 = (k: int, v: str)

def x: int_to_str1'* = {1 -> "a", 2 -> "b", 3 -> "c"}

Expressions

Expressions transform bags.

def a = 1 + 2 // 3
def b = true or false // true

Arithmetic expressions are pointwise, pairing every value in the first bag with every value in the second bag.

def a = 1 + 2 // 3
def b = {2, 3} * {1, 10} // {2, 3, 20, 30}
def c = "hi \{"mo", "jo"}" // {"hi jo", "hi mo"}

Aggregate functions are setwise expressions.

sum({1, 2, 3}) // 6
count({"a"'3, "b"'6}) // 9
distinct({1, 1, 1, 2, 2, 3}) // {1, 2, 3}

Pipeline expressions

Pipeline expressions are infix expressions that replace SQL clauses.

if and map are filter and projection operators. . refers to the current pipeline element.

def person = {
  (name: "Ann", age: 12)
  (name: "Ann", age: 9)
  (name: "Bo",  age: 30)
  (name: "Cy",  age: 16)
}

def kid_names =
  person
  if .age < 18
  map .name

Alternatively, => syntax names the current pipeline element.

def kid_names =
  person
  if p => p.age < 18
  map p => p.name

by creates a map that groups the input by an expression.

def is_gt3 = {1, 2, 3, 4, 5} by . > 3
// {
//   false -> 1
//   false -> 2
//   false -> 3
//   true -> 4
//   true -> 5
// }

aggv transforms the set of values belonging to each key.

def is_gt3 = {1, 2, 3, 4, 5} by . > 3 aggv sum(.)
// {
//   false -> 6
//   true -> 9
// }

Procedural execution

var declarations store persistent values. proc declarations describe imperative execution and allow side effects like mutation and I/O.

var n! f64 = 0

proc add(x! f64): void = do {
  n = n + x
}

proc reset(): void = do {
  n = 0
}

Declarative expressions

def declarations describe pure computation: they have no side effects and compile to relational algebra. They can reference stored values and other def declarations.

var a! f64 = 3
var b! f64 = 4
def distance = float::sqrt(a * a + b * b)
def cosine = a / distance

Recursion

SakaQL supports recursion, including mutual recursion.

// transitive closure
var edges* int -> int = {1 -> 2, 2 -> 3, 3 -> 4}
def rec paths* int -> int %= edges | {paths -- edges}

// mutually recursive even/odd
def n = 100
def rec even* int %= {
  0
  odd + 1 if . <= n
}
def rec odd* int %= even + 1 if . <= n

Friendlier recursive functions are on the roadmap.

Superpowers

SakaQL has powerful features you won’t find elsewhere.

Incremental view maintenance

Most databases today support only batch execution: every query runs from scratch. If the underlying data changes, no matter how little, the query has to do all that work again. This isn’t a problem for cheap queries, but it rules out expensive queries when latency matters, and latency usually matters.

SakaQL makes expensive queries fast with automatic incremental view maintenance (IVM). def declarations represent relational views. When stored data changes, SakaQL uses IVM to efficiently update dependent views.

Take a moment to admire this animation rendered with SakaQL:

A rainbow plasma that flows, drawn by a SakaQL program
Plasma

This is its code:

var t! f64 = 0
pub proc tick(dt! f64): void = do {t += dt}

pub def width = 128
pub def height = 128
pub def image =
  [0..<width, 0..<height]
  map [x, y] => (
    let v = {
      float::sin(x / 16 + t)
      float::sin(y / 8 - t)
      float::sin({x + y} / 16 + t)
      float::sin(float::sqrt(float(x * x + y * y)) / 8 - t * 2)
    }
    x
    y
    fill: color::hsl(sum(v) * 45 + t * 30, 0.8, 0.5)
  )

The animation is a live view depending on the stored variable t, which represents time. It compiles to an incrementally maintained streaming dataflow: as t changes, the image updates automatically and efficiently.

IVM generalizes. Much of app development is updating views as data changes, and frameworks keep reinventing it with stores, signals, selectors, memoization, and reactivity. SakaQL builds IVM into the language, so you need none of them.

Regular language matching

Like SQL, regular expressions are a terrible implementation of an elegant theory. And like SQL, regular expressions aren’t going anywhere (unfortunately, ugh). The problems:

  • no way to name and reuse a regular expression
  • awkward, unintuitive syntax
  • misfeatures that break performance guarantees

SakaQL has an alternative. Here’s what recognizing an IP address looks like:

reg octet: char'{1..3} = {
  digit
  ("1".."9", digit)
  ("1", digit'2)
  ("2", "0".."4", digit)
  ("25", "0".."5")
}

reg ipv4 = (octet, (".", octet)'3)

// One 16-bit hexadecimal group.
reg h16 = hex'{1..4}

// The final 32 bits: two hexadecimal groups or an IPv4 address.
reg ls32 = {(h16, ":", h16), ipv4}

reg ipv6 = {
  // All eight groups are present.
  ((h16, ":")'6, ls32)

  // One "::" replaces one or more zero groups.
  ("::", (h16, ":")'5, ls32)
  (h16'?, "::", (h16, ":")'4, ls32)
  ((h16, (":", h16)'{0..1})'?, "::", (h16, ":")'3, ls32)
  ((h16, (":", h16)'{0..2})'?, "::", (h16, ":")'2, ls32)
  ((h16, (":", h16)'{0..3})'?, "::", h16, ":", ls32)
  ((h16, (":", h16)'{0..4})'?, "::", ls32)
  ((h16, (":", h16)'{0..5})'?, "::", h16)
  ((h16, (":", h16)'{0..6})'?, "::")
}

reg ip_address = {ipv4, ipv6}

def a = re::full_match("192.168.0.255", ip_address)
def b = re::full_match("2001:db8:0:0:8:800:200c:417a", ip_address)

You didn’t ask, but here’s the equivalent regex.

^(?:(?:(?:25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9]?[0-9])\.){3}(?:25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9]?[0-9])|(?:[0-9A-Fa-f]{1,4}:){7}[0-9A-Fa-f]{1,4}|::(?:[0-9A-Fa-f]{1,4}(?::[0-9A-Fa-f]{1,4}){0,6})?|(?:[0-9A-Fa-f]{1,4}:){1}:(?:[0-9A-Fa-f]{1,4}(?::[0-9A-Fa-f]{1,4}){0,5})?|(?:[0-9A-Fa-f]{1,4}:){2}:(?:[0-9A-Fa-f]{1,4}(?::[0-9A-Fa-f]{1,4}){0,4})?|(?:[0-9A-Fa-f]{1,4}:){3}:(?:[0-9A-Fa-f]{1,4}(?::[0-9A-Fa-f]{1,4}){0,3})?|(?:[0-9A-Fa-f]{1,4}:){4}:(?:[0-9A-Fa-f]{1,4}(?::[0-9A-Fa-f]{1,4}){0,2})?|(?:[0-9A-Fa-f]{1,4}:){5}:(?:[0-9A-Fa-f]{1,4}(?::[0-9A-Fa-f]{1,4}){0,1})?|(?:[0-9A-Fa-f]{1,4}:){6}:(?:[0-9A-Fa-f]{1,4}(?::[0-9A-Fa-f]{1,4}){0,0})?|(?:[0-9A-Fa-f]{1,4}:){7}:|(?:[0-9A-Fa-f]{1,4}:){6}(?:(?:25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9]?[0-9])\.){3}(?:25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9]?[0-9])|::(?:[0-9A-Fa-f]{1,4}:){0,5}(?:(?:25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9]?[0-9])\.){3}(?:25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9]?[0-9])|(?:[0-9A-Fa-f]{1,4}:){1}:(?:[0-9A-Fa-f]{1,4}:){0,4}(?:(?:25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9]?[0-9])\.){3}(?:25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9]?[0-9])|(?:[0-9A-Fa-f]{1,4}:){2}:(?:[0-9A-Fa-f]{1,4}:){0,3}(?:(?:25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9]?[0-9])\.){3}(?:25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9]?[0-9])|(?:[0-9A-Fa-f]{1,4}:){3}:(?:[0-9A-Fa-f]{1,4}:){0,2}(?:(?:25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9]?[0-9])\.){3}(?:25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9]?[0-9])|(?:[0-9A-Fa-f]{1,4}:){4}:(?:[0-9A-Fa-f]{1,4}:){0,1}(?:(?:25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9]?[0-9])\.){3}(?:25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9]?[0-9])|(?:[0-9A-Fa-f]{1,4}:){5}:(?:[0-9A-Fa-f]{1,4}:){0,0}(?:(?:25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9]?[0-9])\.){3}(?:25[0-5]|2[0-4][0-9]|1[0-9]{2}|[1-9]?[0-9]))(?![\s\S])

Simple patterns are more verbose in SakaQL, but there’s a reason we don’t all write K.

Native parsing

Relational programming is bad at string processing, which is a very common need. That’s why SakaQL supports grammar-based parsing inspired by Janet and Raku.

An early SakaQL design generated tokenless, backtracking, error-resilient PEG parsers, but the performance ceiling of this architecture was too low. SakaQL now generates tokenized, predictive, recursive-descent, Pratt parsers.

Here’s an example that parses formulas. First, the lexer classifies tokens:

lexer FormulaTokens = char {
  reg ws = wspace'+ #[trivia]
  reg comment = ("//", -{"\r", "\n"}'*) #[trivia]
  reg number = (digit'+, (".", digit'+)'?)
  reg name = ({alpha, "_"}, {alnum, "_"}'*)
  reg _ = {"+", "-", "*", "/", "^", "(", ")", ","}
}

Then PEG-like rules specify the grammar.

parser FormulaGrammar = FormulaTokens {
  peg Formula = Sum #[entrypoint]

  peg Sum = [Add, Subtract, Product] #[fold] #[pratt]
  peg Add = (Sum, "+", Product)
  peg Subtract = (Sum, "-", Product)

  peg Product = [Multiply, Divide, Unary] #[fold]
  peg Multiply = (Product, "*", Unary)
  peg Divide = (Product, "/", Unary)

  peg Unary = [Negate, Power] #[fold]
  peg Negate = ("-", Unary)

  peg Power = [Raise, Postfix] #[fold]
  peg Raise = (Postfix, "^", Power)

  peg Postfix = [Call, Atom] #[fold]
  peg Call = (Postfix, "(", (Sum, (",", Sum)'*)'?, ")")

  peg Atom = [Number, Name, Group] #[fold]
  peg Number = :number
  peg Name = :name
  peg Group = ("(", Sum, ")") #[fold]
}

You can then parse strings and query resulting parse nodes. This API is experimental; I plan to redesign it for better ergonomics.

mod Formula = compile_parser(FormulaGrammar)

var source! str = "max(0, price * (1 - discount)) + shipping"
def tokens = Formula::tokens::Formula(source)

def identifiers = distinct(
  tokens if .kind == Formula::TokenKind::name
  map token => str::slice(source, token.start, token.end)
)

Unbounded integers

Python supports unbounded integers, which give you intuitive numeric operations at the cost of performance. Other languages offer fixed-size integers at the cost of overflow behavior nobody understands.

SakaQL supports unbounded integers and tracks their possible values at compile time. This enables intuitive behavior without compromising performance.

Built-in fixed-size integers like u8 and i8 are defined as fixed interval subsets of the unbounded integer set:

pub set u8 = int:{0..255}
pub set i8 = int:{-128..127}

Declarations use these sets to constrain their values:

def a! u8 = 200
def b! u8 = 100
def c! i8 = -128
def d! i8 = 500 // Error: int:500 not in int:{-128..127}

The compiler computes the integer interval produced by every expression:

def e = a + b // def e: int:{0..510}
def f = b - a // def f: int:{-255..255}
def g = -c    // def g: int:{-127..128}

Integer overflow is a compile-time type error. SakaQL offers tools to handle overflow intentionally.

def h! u8 = a + b       // Compile time error: int:{0..510} not contained by int:{0..255}
def i! u16 = a + b      // Ok
def j! u8 = expect(e)   // Run time error: int:300 not in int:{0..255}
def k! u8 = wrap(e)     // 44
def l! u8 = saturate(e) // 255

The compiler generates fast code using native machine integers where possible and falls back to unbounded integers otherwise.

Object model

Records and tuples have roughly the same capabilities as SQL tables. If tables were sufficiently expressive, then other programming languages would also rely on tables, but they don’t.

That’s why SakaQL supports an object model. Disclaimer: the object model design is evolving, so the code here may not work in the final design.

The object model defines roles and their properties.

role Person
prop Person::name! str
prop Person::age! int:{0..130}

Relationships are modeled by properties that reference roles.

role Person
prop Person::name! str

role Student
prop Student::advisor? Instructor #[ref_integrity = Prune]

role Instructor

role Course
prop Course::code! str exclusive .
prop Course::instructors+ Instructor unique .
prop Course::students* Student unique . #[ref_integrity = Prune]
prop Course::prerequisites* Course unique .

role StudentCard
prop StudentCard::holder! Student exclusive . #[ref_integrity = Cascade]

Behind the scenes, roles compile to bags of type ref and properties compile to bags of type ref -> value. A ref is a primitive type that uniquely identifies an object instance.

do {
  let bob = ref()
  par {
    Person |= bob
    Person::name |= bob -> "bob"
  }
}

Roles and properties can be referenced in bag and set expressions.

def teaching_students = {Student & Instructor} >> Person::name
def class_sizes = Course map c => (code: c.code, students: count(c.students))
def prerequisites(course! Course)* Course = reachable(course, Course::prerequisites)

There’s plenty of syntax sugar to make mutation easier.

do {
  let ada = Person(name: "Ada")
  Instructor(self: ada)

  let maya = Person(name: "Maya")
  Student(self: maya)
  Instructor(self: maya)
  StudentCard(holder: maya)
  maya.Student::advisor = ada

  let intro = Course(code: "CS101", instructors: ada)
  let advanced = Course(
    code: "CS201", instructors: ada, prerequisites: intro
  )

  intro.students |= maya       // Maya studies CS101.
  advanced.instructors |= maya // Maya co-teaches CS201.
}

There is no inheritance. As the examples above show, an instance can belong to multiple roles at once. This flexibility brings your data model closer to your mental model and makes schema updates less painful.

Fractional indexes

Values in Saka are unordered bags, but a lot of real-world use cases involve ordered sequences.

The recommended way to handle these use cases is to create a map with integer keys. For example, let’s model a queue of people:

var queue* int -> str = {0 -> "mo", 1 -> "kim", 2 -> "jo"}

It’s easy to add someone to the end of the queue.

queue |= 3 -> "aja"

But adding someone between two existing people is difficult.

// insert "bob" between "mo" and "kim"
queue \= {1 -> "kim", 2 -> "jo", 3 -> "aja"} // delete all entries after "mo"
queue |= {1 -> "bob", 2 -> "kim", 3 -> "jo", 4 -> "aja"} // add them back with "bob" first

The ergonomics are bad. Even worse, inserting near the front means rewriting almost the entire bag.

SakaQL’s pos is a native fractional indexing primitive that solves this problem.

A position is an ordering key. We can create one before, after, or between existing positions without changing any of them.

def mo = pos::neutral
def kim = pos::after(mo)
def jo = pos::after(kim)

var queue* pos -> str = {mo -> "mo", kim -> "kim", jo -> "jo"}

It’s still easy to add someone to the end of the queue.

queue |= pos::after(jo) -> "aja"

But now inserting someone between two people is also easy:

queue |= pos::between(mo, kim) -> "bob"

A native fractional indexing primitive helps make mutable ordered sequences fast and correct.

Uniform syntax with contextual semantics

SakaQL comprises multiple integrated sublanguages that share syntax but not semantics. That’s a lot of jargon, so let me explain.

TypeScript has a type language and a value language, and the same symbol can exist in both in the same scope because the two languages occupy independent namespaces. Whether a name refers to a type or a value depends on syntactic position.

// Okay even though Person is both a type and a value
type Person = { name: string; age: number };
const Person: Person = { name: "Mojo Jojo", age: 30 };
const mojo = Person;

TypeScript’s type language resembles its value language, but the two are syntactically independent. For example:

  • keyof Person is valid type syntax but not valid value syntax
  • 1 + 2 is valid value syntax but not valid type syntax

In SakaQL:

  • There are multiple sublanguages and multiple namespaces
  • Different kinds of declarations occupy different namespaces, sometimes more than one
  • One shared grammar accepts all syntax needed by all sublanguages
  • Semantic checking enforces the subset of syntax accepted by each sublanguage
// declarations that don't conflict despite sharing name "a"
def a = {"a", "b"} // bag containing "a" and "b"
set a = {"a", "b"} // set containing "a" and "b"
reg a = {"a", "b"} // pattern matching "a" or "b"

This is unconventional because most languages define distinct syntax for distinct features. Breaking from the norm buys a few things:

  • Predictable syntax. Learn one sublanguage and you can guess how to write features in another.
  • Uniform syntax. Programs look and feel consistent, with fewer exceptions to remember.
  • Less syntax to learn, remember, and specify. SakaQL’s syntax is formally specified in just 250 lines of SakaQL, so every change gets ample attention and consideration.
  • Automatic syntax-based transformations (as done by codemods and macros) have fewer cases to handle.

Where things stand today

Try SakaQL in the browser.

SakaQL is in alpha. The language specification is evolving, so programs that work today won’t necessarily work tomorrow. You’re also likely to encounter bugs.

SakaQL is not yet optimized for performance. There’s a reason the animations at the top of this post are low-resolution. My eventual performance goal is to run DOOM at 60 FPS. Separately, IVM chugs memory because it works by trading space for time. I have experimental ideas on how to mitigate this problem.

SakaQL is closed source. This improves design velocity and quality by allowing me to make and test major changes independently. I don’t plan to open-source SakaQL. I do plan to freely distribute compiler binaries.

Work on SakaQL predates vibecoding. Today AI is used heavily in SakaQL’s design and development.

Project sustainability

It’s been a few decades since you could make a healthy living from building a programming language. That’s sad, because I like money.

In fact, I like money so much that I’m building a data management product around SakaQL. It’s called Saka (whaddya know), and I’ll be selling it. Go to saka.io to check out an early sales pitch.

The idea is that software is built from code and data, but tools for code are way better than tools for data. Saka aims to address this gap by combining:

  • a flexible data model
  • a relational query engine
  • a text file format for data, designed to be checked into git
  • a spreadsheet UI for analysis and editing
  • AI integrations
  • collaboration features

Saka will help wrangle the complexity of software as AI scales it exponentially.

Spreadsheets

I find AI spreadsheet integrations frustratingly lacking. I think a better data model and IVM can be used to build a much better experience.

There are spreadsheet-database hybrids with more rigid data models, but I think their foundation in a separate SQL database holds them back from providing a truly great experience.

My immediate focus with Saka is to build the best data management tool for developers. But my broader vision is for Saka’s technology to enable next-generation spreadsheets for humans and agents.

Interested?

I’m not looking for partners or contributors. But if you have something interesting to say, email me at [email protected].