spanner/spannertest/INTERNALS.md

# spannertest internals

This document describes how spannertest is implemented. It should be considered
a companion to the source code.

## Design philosophy

The purpose of spannertest is to support testing code that uses Cloud Spanner
as a client. It aims to be correct and comprehensive (see README.md for
status), but does not attempt to be performant. Clarity and simplicity of
implementation is a very important property; one should be able to read this
document and the code and have a good understanding of how an SQL database may
operate in principle.

## Overview

There are five sections to spannertest:

* RPC interface (`inmem.go`); this implements the same gRPC interface as Cloud
  Spanner. It handles transitions between the gRPC protobuf types and the types
  used by the rest of the package.
* Top-level DB interface (`db.go`); this has the primitive types such as
  `database`, `table` and `row`, implements schema management, all writes
  (insert/update/delete/etc. including DML), and basic reads (based on keys and
  key ranges).
* Query evaluator (`db_query.go`); this evaluates a `spansql.Query` on a
  database.
* Expression evaluator (`db_eval.go`); this evaluates a `spansql.Expr` in a
  specific context (e.g. on a table row).
* Expression functions (`funcs.go`).

## RPC interface (`inmem.go`)

TODO

## Top-level DB interface (`db.go`)

A `database` contains a set of named tables. It also contains a set of named
indexes, though `spannertest` does not do anything with them.

A `table` contains its own schema (a list of `colInfo` values). The primary key
columns appear first in that list for some implementation conveninence;
otherwise the order is as they appear in the `CREATE TABLE` DDL.

A `table` also has its data, stored as a list of `row` values, each of which is
a list of data values. The rows are stored in primary key order; this aids in
search operations.

A `row` is a list of raw data values, represented as `[]interface{}`. `db.go`
explains the mapping between Spanner types and Go types. These values are used
throughout the other parts of the `spannertest` implementation, particularly in
the expression evaluator.

## Query evaluator (`db_query.go`)

The query evaluator works by transforming a `spansql.Query` into a pipeline of
transformers (the `rowIter` interface). This pipeline implements the SQL
semantics. For instance, `SELECT * FROM X WHERE A > 2` would produce a pipeline
that reads rows from X (`tableIter`), filters them (`whereIter`), and outputs
the full set of columns (`selIter`). See `(*database).Query` and
`(*database.evalSelect)`.

## Expression evaluator (`db_eval.go`)

The expression evaluator walks a `spansql.Expr` in a particular "evaluation
context" to produce an output value. See `evalContext.evalExpr` for the main
entry point.

This also includes the value comparator (`compareVals`), which is used for
evaluating expressions, for ordering results (`ORDER BY`), some filtering
(`SELECT DISTINCT`)