SaveToken()

Save the global token environment

Syntax

      SaveToken() -> cStaticTokenEnvironment

Returns

<cStaticTokenEnvironment> a binary string encoding the global token environment

Description

The SaveToken() function can be used to store the global token environment for future use or when two or more incremental tokenizers must the nested. Note however that the latter can now be solved with locally stored token environments.

Compliance

SaveToken() is compatible with CT3’s SaveToken(),

Platforms

All

Files

Source is token2.c, library is libct.

Seealso

TOKENINIT(), TOKENEXIT(), TOKENNEXT(), TOKENNUM(), TOKENAT(), RESTTOKEN(), TOKENEND()