Table of Contents

Overview

This is a description of the internals of the CLI as implemented on Indigo.

The original CLI had many problems and inconsistencies. In part this was no doubt due to poor design and a lack of documentation. This document intends to remedy at least part of that.

Requirements

Tab completion
Help
Consistent syntax
Support both interactive and non-interactive scripting
Reasonably simple to add new commands

High Level Notes

Implemented in Lua
Use library shared with other UI components (web, shell scripts) for executing core functionality.
Keep CLI as close to stateless as possible
Simplify from original implementation
- Commands on a single line
- Limited "modes" and contexts; eventually support "enable" mode for modify operations

The overall process works like:

Input Line Received

Call level 1	Call level 2	Call level 3	Call level 4	Notes
process_line =>	parse_line =>	parse_params =>		Get command table entry with parse table;
			parse positional parameters	Based on parse table entries (numeric keys)
			parse keyword and flag parameters	Based on parse table entries (string keys)
			call command specific parser	Result: table of parsed data
	check parse status			Return if errors or incomplete input
	call command specific handler			called if parse info is complete with no errors

For a command completion request (only available in interactive mode) the process jumps into parse_line directly. Instead of calling the command specific handler, the command line is updated if there is a unique completion; if more than one completions is determined, they are displayed. If the input is not complete, but no completions are available, a help string is displayed to describe what input should follow.

Architecture

CLI architecture decisions inevitably are balancing the trade off of representing the CLI syntax and semantics in data structures versus code. Using data structures allows the application of generic parsing tools, but requires more up front thought; often the resulting framework imposed unnatural structure on the CLI interface. Using code is quicker and dirtier and often more difficult to extend.

In the Indigo CLI, we have chosen to provide a core set of data structures and generic parsing tools, but have tried to limit the requirements on how these are used in the CLI.

Core Parsers

At the root is a set of core parsing functions that take a string and return an object appropriate for handler arguments along with the remainder of the string. Examples include simple integer parsing with min/max checking to parsing a port spec of the form 1-4,5,18 into a list of integers. A core parser returns nil if there is a parse error.

A core parser may accept an empty string as valid input. This occurs when a keyword parameter occurs with no following equal sign (=). The core parser should return true in this case.

A core parser may also return a set of completions. Here is the prototype and description of parameters and return values.

If param_val is returned as nil and completions is nil, an error occurred. If param_val is returned as nil and completions is not nil, then the input was incomplete and completions provides the possible strings to complete the input if known.

param_val, rest_of_line, completions = parser(line)

Parameter	Type	Notes
line	string	The line of input being parsed starting at the object to be parsed
param_val	untyped	The parameter value if successfully parsed. nil otherwise
rest_of_line	string	The part of the line not consumed by this process
completions	table	If param_val is nil, a set of strings that could complete the input so far

Parse Instance

A parse instance is what is required to parse a parameter in the context of a command. It includes the following:

function	type	notes
parser	param_val, rest_of_line, completions = parser(line)	a core parser function
help	string	Text describing the parameter
key	string	A key word to identify the resulting value in a table of parsed values
pargs	untyped, but usually a table	Optional argument to be passed to parser

Parse Table

A parse table is a Lua table whose values are Parse Instances. Both numeric and strings keys are used in these structures. Numeric keys indicate positional parameters. Keyword keys indicate parameters of the form key=value. The keyword index must match the 'key' entry in the parse instance.

Positional parsing ends when:

No more positional parse instances are available in the parse table
An = is found after the next token indicating a keyword parameter
A - is found at the start of the next token
End of line

Thus, positional parameters generally cannot have a value starting with a hyphen.

Keyword parsing does not require an = except to indicate the end of positional input (see above); however, if a keyword key is found an = does not follow, then an empty string is passed to the core parser. If no parser is available and the empty string is the parameter value, it is converted to "true". This follows the convention that the parameter represents a flag. Flags usually start with a hyphen.

parse_params: Parse Parameter List Function

The function parse_params is a utility that accepts an input line and a table of parse instances describing the acceptable parameters. It parses the line according to this table, generates a table of parameter values using the core parsers and returns an indication of success.

The function may also return a table representing possible continuation strings. These are represented as a set by the keys of the table (values being true).

rc, parsed, completions, str = parse_params(line, parse_tab)

line is the current input line to be parsed and parse_tab is a table of parse instance structures.

rc: Return Code	Meaning of Return Code	parsed: Parameter Table	completions: Completion Table	str: Error or help string
0	Success	Contains values returned by core parsers	Optional: Strings that could follow	nil or empty
1	Incomplete	Ignore	Optional: Strings to complete the current token	A description of what's required next if known
-1	Error; unexpected keyword or error in value	Ignore	Ignore	Error description

The table of parse instances may include positional and keyword parameters. Positional parameters are terminated prior to the first '=' keyword parameter. Although value checking of parameters is done during this phase, checking required parameters is the responsibility of the command level parser (see below). Normally, default values for parameters are also set by the command level parser.

Sometimes you want to indicated that there are no keyword parameters (so the remainder of the line can be parsed by other means). To do this, enter a parse instance with the empty string for the key. See cli_config.lua for an example.

Command Parsing

The top level parsing works as follows.

A command parsing table is provided to the command line handler. Each entry is indexed by command and provides a table with the following:

Entry	Type	Notes
command	string	The name of the command (because entries may get passed around as objects)
parse_table	Table of Parse Instances	Optional. The argument passed to parse_params above
help	string	Text describing the command
parser	rv, parsed, continuations, str = parser(command, line, parsed)	Optional. A function called to complete parsing after the parse_table (if present) has been applied
handler	rv, str = handler(command, line, parsed)	A function that may be executed once parsing has completed successfully

The parser arguments are as follows.

Parameter	Type	Notes
command	string	The command parsed from the input line
line	string	The full line of input
parsed	table	The values found by processing so far

The return values for parser are the same as for parse_params above.

Parse events include: tab, '?' or end-of-line.

When a parse event occurs, the parser is called. If the event was tab, completions are shown. If the event was '?', completions and help are shown. If the event is end-of-line and and parsing was successful, then the handler function is called. If parsing was incomplete, this is the equivalent of '?'.

Miscellaneous Notes

There is a validator function validate_parse_table which return nil if the passed parse table is acceptable or a string explaining what's wrong if not. It calls validate_parse_instance on each parse instance.

The key for a key=value parse instance must match the key used to index the instanced in a parse table (at least to pass the above validator).

We use the term "set" to mean a table with only string indexes and whose values are ignored (by convention are set to true). By "list" we mean a traditional array, a table indexed by a consecutive sequence of integers starting at 1 (Lua convention).