Clipper 5.x – Drivers Guide

Chapter 7

DBFNTX Driver Installation and Usage

DBFNTX is the default RDD for Clipper. This new database driver replaces the DBFNTX database driver supplied with earlier versions of Clipper and adds a number of new indexing features. With DBFNTX, you can:

. Create conditional indexes by specifying a FOR condition

. Create indexes using a record scope or WHILE condition, allowing you to INDEX based on the order of another index

. Create both ascending and descending order indexes

. Specify an expression that is evaluated periodically during indexing in order to display an index progress indicator

In This Chapter 

This chapter explains how to install DBFNTX and how to use it in your applications. The following major topics are discussed:

. Overview of the DBFNTX RDD

. New Locking Scheme

. Conditional Indexing

. Installing DBFNTX Driver Files

. Linking the DBFNTX Driver

. Using the DBFNTX Driver

. Compatibility with dBASE III

Overview of the DBFNTX RDD

As an update of the default database driver, DBFNTX is linked into and used automatically by your application unless you compile using the /R option.

New Features

The replaceable driver lets you create and maintain (.ntx) files using features above and beyond those supplied with the previous DBFNTX driver. The new indexing features are supplied in the form of several syntactical additions to the INDEX and REINDEX commands. Specifically you can:

. Specify full record scoping and conditional filtering using the standard ALL, FOR, WHILE, NEXT, REST, and RECORD clauses

. Create an index while another controlling index is still active

. Monitor indexing as each record (or a specified record number interval) is processed using the EVAL and EVERY clauses

. Eliminate separate coding for descending order keys using the DESCENDING clause

Compatibility

Index files (.ntx) created with the original DBFNTX driver are compatible with DBFNTX and can be used in new applications without reindexing. Index files (.ntx) created with this version of DBFNTX will also work with previous Clipper applications provided that you use no FOR, WHILE, <scope>, or DESCENDING clauses.

Important! Indexes produced with DBFNTX using FOR or DESCENDING are incompatible with earlier version (.ntx) files. If you attempt to access them with the original DBFNTX database driver or programs compiled with versions earlier than Clipper 5.2, you will get an unrecoverable runtime error. In Clipper, this generates an “index corrupted” error message, causing the application to terminate.

New Locking Scheme

The DBFNTX database driver implements a new locking scheme to resolve several problems identified in previous versions of Clipper and to prevent potential problems that might arise when running Clipper applications in a network environment. This section discusses these changes and their implications, including compatibility issues.

Lock Time-outs

Problem: Index locking in previous versions of Clipper was handled automatically by the database driver, and had no time-out provision. This created the potential for problems in network environments if a workstation died while holding a lock. If this situation occurred all other workstations waiting for an index lock would appear to freeze while waiting to obtain their lock. This could also happen if a user placed a Clipper application in the background on a multitasking system without sufficient processing time allocated to it. Eventually, most network operation systems would clear a connection that had no activity for a specified period of time. This would free the lock and everything would resume as normal, but frustrated users may have rebooted their machines possibly causing file corruption.

Solution: In Clipper 5.2 the NTX driver will generate a recoverable runtime error if it fails to lock the index after a predetermined number of retries. The default error handler for this system simply returns (.T.) to retry the operation. This emulates the behavior of previous Clipper versions.

Error Handling

Time out handling: The handling of this error is problematic because the lock is issued from various internal index routines. Therefore the only safe recoveries are to retry or quit. Choosing to default from the error or issuing a break will more than likely leave the index in a corrupted state. If either of the options is employed, the application should immediately recreate the index. The preferred way to handle a time out such as this is to alert the user of the situation so they don’t think their machine has hung, and then have the network administrator determine what workstation is causing the problem. When the problem workstation is cleared, the users that have timed out can select retry and continue processing.

NTXERR.PRG: The file NTXERR.PRG contains the source code for the default error handler INIT procedure. This error handler can be modified to allow user-defined error handling for index lock time-outs. Care should be exercised when modifying the error handler as detailed above.

Compatibility: The lock time-out capability when used in conjunction with the default error handler is totally compatibility with previous versions of Clipper. No changes are made to the NTX file structure and no action is required by the developer to activate the time-out functionality.

New Lock Offset

Problem: Index locking, which is transparent to the developer, uses a single-byte semaphore locking system. This semaphore was placed at a virtual offset (beyond the physical end of file) in the index file. In previous versions of Clipper, this offset was located at one billion (1,000,000,000) which was adequate at the time. But many systems today are capable of producing indexes that are large enough to cause the actual data present at the lock offset to become physically locked. This leads to problems when trying to read or write to the data at that offset.

Solution: The solution is to move the offset where locking occurs to a location at a greater offset. We have chosen FFFFFFFF hex, which is the largest offset possible under the DOS operating system. The problem with this solution is that new applications using the index will be locking this new byte while old applications using the same index will lock the old position. Clearly this would cause both applications to fail because each could have a lock on the file at the same time.

To avoid this, the signature of the index (in the index header) is modified to prevent pre-Clipper 5.2 applications from being able to open the index. Clipper 5.2 applications can detect the correct offset to use by the flag in the header and will automatically use the correct one. In Figure 7-1 below, each bit represents a flag:

BIT  7 6 5 4 3 2 1 0

FLAG R R R O P I I C

R Reserved

I Index type - both bits set (NTX)

C Index created with a Condition, condition in header

T Created as a Temporary index

O New Offset for exclusive (semaphore) lock

Figure 7-1: Bit Field for the Signature Byte of a -Clipper 5.2 NTX File

Activation

If Clipper 5.2 automatically modified the signature in the header when it created indexes, programs with automatic reindexing routines would be creating indexes that appeared corrupt to pre-Clipper 5.2 applications. This has an obvious problem with backward compatibility. Therefore, in order to create indexes with the new signature, the developer must link in the module NTXLOCK2.OBJ with the full knowledge that this will create indexes that older applications will not be able to access.

Header Changes

The signature byte of a .NTX file is 6 for an unenhanced NTX index. The inclusion of the NTXLOCK2.OBJ will cause the signature to become 26 hex. (6 hex ORed with 20 hex). See Figure 7-1 for an illustration of all the possible values for the signature byte.

Error Handling

Clipper 5.2 applications will automatically recognize the signature byte of the header, and depending on the signature value, will use the correct index lock location. Applications built with previous versions of Clipper, however, do not have the capability to detect the optional new information in the signature byte. Therefore, when an order application tries to open a file that has been created with the NTXLOCK2.OBJ linked in it will produce a Corruption Detected error.

Compatibility

The new locking location, if used, is not backward compatible with applications compiled with previous versions of Clipper.

Indexes created by applications built with a previous version of Clipper can be used by Clipper 5.2 using the new location and will not be modified unless the index is recreated in application.

Since older applications have no knowledge of the new index locking scheme nor of the significance of the header signature, these applications will assume the index is corrupt and will produce an Index Corrupted error.

Conditional Indexing

Conditional indexes are a feature of the DBFNTX driver. This section discusses this feature of the DBFNTX driver in some detail, giving you specific information about the implementation of conditional indexes. Compatibility issues are also discussed.

Conditional Indexes

Conditional indexes are produced by using a FOR condition in the index creation process. These indexes are made fully maintainable by storing the FOR condition in the index header. This condition is subsequently retrieved and compiled each time the index in opened. During updates, items are added to the index only if they meet the criteria of the condition.

Since older applications do not have the ability to recognize and use the condition stored in the header, they must be prevented from opening the index since they corrupt the index. This is accomplished by modifying the signature of the index (in the index header) preventing pre-Clipper 5.2 applications from being able to open the index. Clipper 5.2 applications can detect the flag in the header and will automatically use the stored FOR condition correctly.

Temporary Indexes

Temporary indexes are produced by using any scoping clause other than the FOR condition in the index creation process. These indexes are not automatically maintainable because the condition is not stored for later use. These indexes can be made maintainable if the condition can be expressed as a FOR condition and is added using the FOR clause. But the main use of temporary indexes is for fast creation of indexes for read- only browses or reports that operate on a subset of the database.

Since older applications would not operate properly with indexes that do not contain all the keys in a given database, they must be prevented from using them. This is accomplished by modifying the index signature to prevent pre-Clipper 5.2 applications from being able to open the index.

Activation

Conditional Indexes

The developer need only specify the FOR condition when creating the index. In doing so he must be fully aware the index will no longer be accessible to pre–Clipper 5.2 applications.

Temporary Indexes

The developer need only specify a scope other than FOR when creating the index. In doing so he must be fully aware the index will no longer be accessible to pre-Clipper 5.2 applications and that the index created is not maintainable.

Header Changes

The signature byte of a .NTX file is 6 for a unenhanced NTX index. If the index is created as a conditional index it will have a signature of 7 hex (6 hex ORed with 1 hex). If the index is created as a temporary index it will have a signature of E hex. (6 hex ORed with 8 hex). See Figure 7-1 for an illustration of all the possible values for the signature byte.

Error Handling

Corruption Detected

Since older applications have no knowledge of the new index features nor how to interpret the additional flags in the header signature, these applications will assume the index is corrupt and will produce an Index Corrupted error.

EOF()

If an index is created with a FOR condition and an attempt is made to update the index with a key that does not match the condition, the update is suppressed and the index is placed at EOF(). This is consistent with the current behavior for indexes created with the unique flag when an update is attempted with a non-unique key.

Also if a navigational action is attempted (SKIP) and the current record is not found in the index, the index will place the record pointer at EOF(). This is true for both conditional and temporary indexes.

Compatibility

Backward Compatibility

If the conditional or temporary indexing features are used the index produced will not be backward compatible with applications compiled with previous versions of Clipper. Indexes that do not use the features, however, will be 100% compatible.

Forward Compatibility

Indexes created by applications built with a previous version of Clipper can be used by Clipper 5.2 and will not be modified unless the index is recreated using either the conditional or temporary index features.

Error Message Produced by Old Applications

Since older applications have no knowledge of the new index locking scheme nor of the significance of the header signature, these applications will assume the index is corrupt and will produce an Index Corrupted error.

Installing DBFNTX Driver Files

DBFNTX is supplied as the file DBFNTX.LIB.

The Clipper installation program installs this driver as the default in the \CLIPPER5\LIB subdirectory on the drive that you specify, so you need not install the driver manually.

Important! Before installing Clipper, you may want to rename the DBFNTX.LIB that currently resides in your \CLIPPER5\LIB directory to DBFNTX.001. The new version, when installed, will overwrite DBFNTX.LIB. If you do not rename or otherwise protect the old version of DBFNTX.LIB, you will lose it.

Linking the DBFNTX Database Driver

Since DBFNTX is the default database driver for Clipper, there are no special instructions for linking. Unless you specify the /R option when you compile, the new driver will be linked into each program automatically if you specify a USE command or DBUSEAREA() function without an explicit request for another database driver. The driver is also linked if you specify an INDEX or REINDEX command with any of the new features.

Using the DBFNTX Database Driver

In applications written for the new DBFNTX driver, you can use the INDEX and REINDEX commands exactly as you have used them in the past. The index files (.ntx) you create and maintain in this way are completely compatible with those created using previous versions of the driver.

Changes to existing code are necessary only if you use the new indexing features. The (.ntx) files you create using the new features will have a slightly different header file and cannot be used by programs linked with a previous version of the driver.

Using (.ntx) and (.ndx) Files Concurrently

You can use (.ntx) and (.ndx) files concurrently in a Clipper program like this:

// (.ntx) file using default DBFNTX driver

USE File1 INDEX File1 NEW

// (.ndx) files using DBFNDX driver

USE File2 VIA "DBFNDX" INDEX File2 NEW

Note, however, that you cannot use (.ntx) and (.ndx) files in the same work area. For example, the following does not work:

USE File1 VIA "DBFNDX" INDEX File1.ntx, File2.ndx

Compatibility with dBASE III PLUS

The default DBFNTX driver makes Clipper programs behave differently than traditional dBASE programs. Some of these differences are discussed below.

Supported Data Types

The DBFNTX database driver supports the following dBASE III PLUS- compatible data types for key expressions:

. Character

. Numeric

. Date

. Logical

Supported Key Expressions

When you create (.ntx) files using the DBFNTX driver, you can use all Clipper or user-defined functions compatible with dBASE III PLUS as well as other functions accepted by the extended Clipper functionality.

Error Handling

The indexing behavior of DBFNTX and DBFNDX in a Clipper application is identical unless otherwise noted. With the default DBFNTX driver, you can handle most errors using BEGIN SEQUENCE…END SEQUENCE as illustrated in the next section.

FIND vs SEEK

In Clipper, you can use the FIND command only to locate keys in indexes where the index key expression is character data type. This differs from dBASE III PLUS where FIND supports character and numeric key values.

Note: In Clipper programs, always use the SEEK command or the DBSEEK() function to search an index for a key value.

The DBFNTX driver lets you recover from data type errors raised during a FIND or SEEK. However, since Error:canDefault, Error:canRetry or Error:canSubstitute are set to false (.F.), you should use BEGIN SEQUENCE…END to handle such SEEK or FIND data type errors. Within the error block for the current operation, issue a BREAK() using the error object that the DBFNTX database driver generates, like this:

bOld := ERRORBLOCK({|oError| BREAK(oError)})
 .
 .
 .
 BEGIN SEQUENCE
     SEEK xVar
 RECOVER USING oError
     // Recovery code END
 .
 .
 .
 ERRORBLOCK(bOld)

There is an extensive discussion of the effective use of the Clipper error system in the Error Handling Strategies chapter of the Programming and Utilities guide.

Sharing Data on a Network

The DBFNTX driver provides file and record locking schemes that are different from dBASE III PLUS schemes. This means that if the same database and index files are open in Clipper and in dBASE III PLUS, Clipper program locks are not visible to dBASE III PLUS and vice versa.

Warning! Database integrity is not guaranteed and index corruption will occur if Clipper and dBASE III PLUS programs attempt to write to a database or index file at the same time. Therefore, concurrent use of the same database (.dbf) and index (.ndx) files by dBASE III PLUS and Clipper programs is strongly discouraged and not supported by Computer Associates.

Summary

In this chapter, you were given an overview of the new features of the default DBFNTX RDD. You learned how to this driver is automatically linked and how to use it in your applications, and were given an overview of the compatiblity issues.