Command Language > Preprocessor > Preprocessor introduction

The preprocessor is used to control the production of a revised Command Language (CL) script where all the preprocessor commands have been dealt with.

You can preprocess any file with [CL] [Preprocessor run] from the Main window menu.

When tables are run, the internal preprocessor is used, see notes below for differences from the full CL preprocessor.

The main use of the preprocessor is to save coding with loops to generate repeated commands, and to control the CL script generation from a spreadsheet file. Large projects will also use *INSERT commands to include bits of CL script in separate files.

When running tables with Companion the preprocessor can be included using syntax tables, and the output can be controlled using *ASK commands.

Differences from the full CL preprocessor

The are a few differences between the Companion preprocessor and the one used by CL.

The Companion preprocessor is a Unicode program.

The output from the Companion preprocessor is an STPN file with UTF-8 encoding. Notepad can be used to save in a different encoding.

The global FORMSPEC.STP file must be located in C:\ProgramData\NetMR Ltd\CL. If an inserted CL script is not found, this is also the folder that will be searched for a CL script file with the same name.

The *SET commands %ITEMS, %ROWS, and %COLS are only available for entries and tables created in Companion when running tables in Companion.

There are no preset limits in the Companion preprocessor although warnings may be issued if nested levels get too deep, or things get too long.

Preprocessor basics

The preprocessor reads a CL script and interprets special commands, enclosed in square brackets, for text substitution, insertion and repeat processing.

When running tables no input CL script file is produced, the internally generated script is handed directly to the preprocessor.

In the documentation the commands are shown in upper case. Preprocessor commands are not case sensitive and lower case can be used instead.

After preprocessing, all preprocessor commands and their surrounding square brackets are removed from the output.

If a line of output becomes empty (because it only contained a preprocessor command) then it is not output. Lines within which preprocessor commands were embedded are condensed.

A new line is output wherever it appears except where lines are empty, or in the case of insertion from *DATA sets see *HOLD for ways to prevent this.

Lines that are too long as a result of substitutions are split into two or more output lines.

The *INSERT command allows you to build the output from two or more files. You can break a CL script into logical parts, reuse code by inserting it more than once or build general-purpose CL script files for use with different analysis projects.

There are two insert files that can always included at the start of every CL script if they are present. They are both called formspec.stp and the global one is found in C:\ProgramData\NetMR Ltd\CL and the local one will be found in the project folder.

Spreadsheet files and CSV files can be accessed by the preprocessor.

IMPORTANT: there is no preprocessor comment command. Lines beginning with ! (the standard comment for the compiler) are processed by the preprocessor. You can, however, use the *SKIP command to bypass preprocessor commands.

Except within single quotes ('), blanks within square brackets are ignored by the preprocessor.

Each command should normally appear within its own set of brackets.

Commands fall into two classes: substitutions and others. Commands not beginning with an asterisk (*) or plus sign (+) are substitutions which evaluate the contents of the brackets and substitute the result into the output. This evaluation involves the interpretation of the basic elements of the preprocessor: indices, arithmetic expressions, spreadsheet references and data set references.

Preprocessor substitutions and commands and can be used within commands; this can be referred to as "nested" commands.

Any error in preprocessor commands is fatal and the run will be terminated. If this happens a log file will be produced with full details of where the error occurred.

Commands

Inserting scripts (*INSERT, *PUSH, *POP)

Index usage (*SET, Substitutions)

Interactive (*ASK)

Loops (*DO, *FOR, *END, *SLE, *LAST)

Skip past (*SKIP, *BLOCK, Destination label)

Data sets (*DATA, *DSEP, *LINES)

Databases (*DBOPEN, *DBCLOSE, *DBLOOP, *DBEND, *DBCOLUMN, *DBFINDROWS, *DBLINES)

Subroutines (*SBDEF, *SBEND, *SBCALL)

Controls (*HOLD, *{NO}PPINCSV, *{NO}PPINDB, *{NO}INFO)

If you have problems with a CL script, see also Preprocessor tips which may help.