CT_RESTTOKEN

 RESTTOKEN()
 Recreates an incremental tokenizer environment
------------------------------------------------------------------------------
 Syntax

     RESTTOKEN(<cTokenEnvironment>) --> cEmptyString

 Argument

     <cTokenEnvironment>  Designates a character string returned by the
     SAVETOKEN() function.

 Returns

     RESTTOKEN() always returns an empty string.

 Description

     The internal environment for the incremental tokenizer can be restored
     using RESTTOKEN().  RESTTOKEN() does the opposite of the SAVETOKEN()
     function.

 Note

     .  <nTokenEnvironment> must originate from the current program
        run; for example, it cannot have been restored from a (.mem) file.

 Examples

     .  Here is an incremental tokenizer.  Text is broken into
        individual lines, and each line is broken into words:

        TOKENINIT(@cTextString, CHR(13) + CHR(10), 2)
        cLine   :=  TOKENNEXT()

        DO WHILE .NOT. TOKENEND()
           cLine  :=  TOKENNEXT(cTextString)
           WORD(cLine)
        ENDDO

     .  The function then breaks the lines into words:

        FUNCTION WORD(cLine)

           cOldEnv  := SAVETOKEN()
           TOKENINIT(@cLine, " .,-:;")

           DO WHILE .NOT. TOKENEND()
              cWord  :=  TOKENNEXT(cLine)
              ? cWord
           ENDDO
           RESTTOKEN(cOldEnv)
           RETURN("")

See Also: SAVETOKEN() TOKENINIT() TOKENNEXT()