Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

1. Introduction

1.1 What OGML is

OGML is the language of every openglacier node: clients send it, and the Core plans and runs it. There is no second language for definitions: a view is written in OGML, and so is the query that reads it.

It is a pipeline language. A query starts at a source, and each stage receives the rows of the stage before it, transforms them, and hands its own rows on:

on users                  -- source: every document of `users`
| where age >= 18         -- keeps some rows
| select name, city       -- reshapes each row
| sort city               -- reorders the rows

Read it top to bottom, as a recipe. Each line is one step, and each step can be understood on its own.

1.2 The mental model

ConceptMeaning
DocumentA JSON-like object: fields holding numbers, strings, booleans, null, arrays and nested objects. Every stored document has an _id.
CollectionA named set of documents (orders, shop.orders). It exists from its first write; reading a collection that does not exist returns no rows. There is no schema to declare.
RowA document flowing through a pipeline. A stage may change it (add fields, drop fields), drop it, or make new rows (a group, a join).
StageOne step of a pipeline: where, derive, group, lookup… 28 built-in stages.
ExpressionA formula computed on a row: price * qty, coalesce(nick, name), date_trunc(at, "month"). One evaluator serves every stage.
ViewA named query kept in a Place (define view). It is read like a collection (on view sales.monthly).
Materialized viewA view whose rows are kept in a cache and computed again only when what it reads changes.
PlaceA workspace: its documents carry a _place field, and its queries read only its rows. Views belong to a Place, or are standard views shared by all.

1.3 Design principles

  • One pipeline, read in order. The order you write is the order of meaning. The planner may move work (filters toward the source, for example), but never in a way that changes the result or hides an error.
  • Strict by default. A field that a filter or a formula reads must exist: a typo is an error, not an empty result. Optional fields are said explicitly, with field?, exists() or coalesce().
  • No cross-kind coercion. age == "31" is an error, not false. Conversions are explicit (to_int, to_string).
  • Bounded memory. ogd never loads a whole collection in memory. A query either fits a shape it can run within its budget (spilling to disk where needed), or it is refused before it starts.
  • Transactions. Every write query commits all its changes, or none.
  • Everything is queryable. Views, caches, memory, index statistics and history are collections, read with the same language.

1.4 What the language is not

  • It has no comments, no user-defined functions, no loops.
  • It has no date type: dates are ISO-8601 strings or epoch milliseconds, read by the date functions.
  • It is not SQL: there is no SELECT … FROM … GROUP BY sentence, but there is a stage for each of these ideas.
SQLOGML
SELECT a, b * 2 AS c FROM ton t | select a, b * 2 as c
WHERE x > 1 AND y IN (…)| where x > 1 and y in [...]
GROUP BY k with SUM(v)| group | by k | sum v | end
HAVING s > 10a where after the group
ORDER BY v DESC LIMIT 10| sort v desc | limit 10
JOIN u ON u.id = t.user| lookup u on id == user into u
UNION ALL| union | on other | end
CREATE VIEW v AS …define view v | exec | … | end | end
INSERT, UPDATE, DELETEinsert {…}, | set f = …, | delete
LAG(), SUM() OVER| window | … | lag v | cumsum v | end
PIVOT| pivot | rows … | columns … | values … | end
MongoDBOGML
$matchwhere
$project, $addFields, $unsetselect, derive, drop
$replaceRootroot
$groupgroup
$bucketbucket
$lookuplookup
$unionWithunion
$unwindunwind
$sort, $limit, $skip, $sample, $countsort, limit, skip, sample, count
$setWindowFields, $densifywindow, fill
$out, $mergeinto, load | with merge

1.5 How a query reaches the engine

A client sends the text of a query, with optional parameters, in a query.execute request (chapter 17). The request may name a Place and an App instance: the scope is set by the request, never by the query text.