Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- AIR TRANSPORT ASSOCIATION CD-ROM INTERCHANGEABILITY STANDARD-- SFQL DRAFT STANDARD Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- SFQL: STRUCTURED FULL-TEXT QUERY LANGUAGE ATA DRAFT STANDARD 89-9C.SFQL2-R1-1990 Version 2.0 DRAFT June 7, 1990 This is a combined effort of the ATA/AIA Subcommittee 89-9C and contributing vendors Long Beach, CA, USA London, UK Schenectady, NY, USA -------------------------------------------- Structured Full Text Query Language i ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- AUTHORS Committee Chairpersons: Holly Leiby, Boeing Computer Services Beverly Robitaille, Douglas Aircraft Contributing Members (Alphabetical): Paul Cotton, Fulcrum Technologies Sophie Deldon, Aerospatiale Jim Goldenberg, Maxwell Data Management, Inc. Charlie Holland, Context Corporation Mark Millane, EDS Mike Moran, IBM Don Morey, American Airlines Tom Rolander, KnowledgeSet Neil Shapiro, GE Corporate Research & Development John Skipsey, British Airways Larry Stone, TMS Inc. -------------------------------------------- Structured Full Text Query Language ii ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- ACKNOWLEDGEMENTS This specification is based on SFQL 1.0, which was contributed to the ATA/AIA by General Electric in 1988. The committee wishes to express its gratitude to the following people who contributed to this specification: Charlie Hearne (American Airlines) & F. John Bowers (GE), 89-9C Committee Co-chairmen John Anderson (Boeing), 81-8 Committee Chairman, the forerunner of 89-9C Martine Delpech (Aerospatiale) and John Bowers (GE), Sponsors of the first SFQL prototypes. Dr. Neil Shapiro (GE), Elias Diamantopoulos (GE), Sophie Deldon (Aerospatiale), Patrick Mathe (Aerospatiale), Lionel Mondon (Aerospatiale), & Christine Piccou (Aerospatiale), for providing proof of the SFQL concept by building the first SFQL prototypes. Dr. Mike Blaha (GE), Dr. Edward Fox (Virginia Polytech), Dr. Clifford Lynch (University of California), reviewers of SFQL. Russell Copage, Fulcrum Technologies, for contributions to the SFQL 2.0 language and for his efforts in the development of a parser for SFQL 2.0. -------------------------------------------- Structured Full Text Query Language iii ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- TABLE OF CONTENTS List Of Figures................................................ix List of Tables.................................................ix 1. Introductory Concepts.......................................1 2. SFQL Language Specification.................................3 2.1 Scope....................................................3 2.2 Limitations..............................................3 2.3 Data Types...............................................4 2.3.1 Character Strings...................................4 2.3.2 Numbers.............................................4 2.3.3 Dates...............................................5 2.4 Database Model...........................................5 2.4.1 Document-records (SQL Rows).........................5 2.4.2 Collections (SQL Tables)............................6 2.4.3 Tag-Field (SQL Column)..............................6 2.5 Schemas..................................................6 2.6 Root-node................................................6 2.7 Notation.................................................6 2.8 SFQL Statement...........................................7 2.9 SFQL Grammar (Comparable to SQL Common Elements).........7 2.9.1 .........................................7 2.9.1.1 Function........................................7 2.9.1.2 Format..........................................8 2.9.1.3 Syntax Rules....................................8 2.9.2 ...........................................8 2.9.2.1 Function........................................8 2.9.2.2 Format..........................................9 2.9.2.3 Syntax Rules...................................10 2.9.3 ............................................10 2.9.3.1 Function.......................................10 2.9.3.2 Format.........................................10 2.9.3.3 Syntax Rules...................................11 2.9.4 Names..............................................12 2.9.4.1 Function.......................................12 2.9.4.2 Format.........................................12 2.9.4.3 Syntax Rules...................................13 2.9.5 ..............................13 2.9.5.1 Function.......................................13 2.9.5.2 Format.........................................13 2.9.5.3 Syntax Rules...................................13 2.9.6 (SQL ) 13 2.9.6.1 Function.......................................13 2.9.6.2 Format.........................................13 2.9.6.3 Syntax Rules...................................14 2.9.7 ..............................15 2.9.7.1 Function.......................................15 -------------------------------------------- Structured Full Text Query Language v ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- 2.9.7.2 Format.........................................15 2.9.7.3 Syntax Rules...................................16 2.9.8 Search Conditions..................................16 2.9.8.1 Function ......................................16 2.9.8.2 Format.........................................17 2.9.8.3 Syntax Rules...................................19 2.9.9 Cursor Operations..................................20 2.9.9.1 Function.......................................20 2.9.9.2 Format.........................................20 2.9.9.3 Syntax Rules...................................21 2.9.10 Function Definitions..............................22 2.9.10.1 General Functions.............................22 2.9.10.1.1 Function....................................22 2.9.10.1.2 Format......................................22 2.9.10.1.3 Syntax Rules................................22 2.9.10.2 Special Functions.............................22 2.9.10.2.1 Function....................................22 2.9.10.2.2 Format......................................23 2.9.10.2.3 Syntax Rules................................24 2.9.10.3 Action Functions..............................24 2.9.10.3.1 Function....................................24 2.9.10.3.2 Format......................................24 2.9.10.3.3 Syntax Rules................................25 3. Return Data Specification..................................27 3.1 Data Return (SFQL Communications Area)..................27 3.1.1 Status and Data Message Format (SFQLCA)............27 3.1.2 Client/Server Interaction..........................27 3.1.3 SFQLCA Header......................................27 3.1.4 SFQLMSG Area.......................................29 3.1.4.1 Variable Length Record (VLR) Data Return.......29 3.1.4.1.1 Description..................................29 3.1.4.1.2 Field Subrecord (FSR) Format.................32 3.1.4.2 Standard Return Data Types.....................33 3.1.4.2.1 Computer Graphics Metafile (CGM).............34 3.1.4.2.2 Date/Time (DATE1)............................34 3.1.4.2.3 Floating Point (FP)..........................35 3.1.4.2.4 Two-byte Signed Integer (INT16)..............36 3.1.4.2.5 Four-byte Signed Integer (INT32).............36 3.1.4.2.6 Formatted String Data Return (SZ)............36 3.1.4.2.7 Formatted Text Types (TEXT)..................36 3.1.4.2.8 Tagged Image File Format (TIFF)..............36 3.1.4.2.9 Unsigned Four Byte Integer (UINT32)..........36 3.1.4.2.10 Nested Variable Length Record (VLR).........36 3.1.4.2.11 Vendor Extensions (?xxxxxxx)................37 3.1.5 Return of Preliminary SFQLCA.......................38 3.1.6 Error Handling.....................................38 3.1.6.1 Overview.......................................38 3.1.6.2 Error Messages.................................39 3.2 Server Compatibility Model..............................40 3.2.1 Levels of Function.................................40 3.2.2 Discovering Server Capabilities....................40 3.2.3 Discovering Collection Information.................40 3.2.4 Server Differences as Exceptions...................41 4. References.................................................43 -------------------------------------------- Structured Full Text Query Language vi ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- Appendix A SFQL Keywords................................................45 Appendix B Directory of SFQL Functions..................................47 Appendix C SFQLCA Definition............................................65 Appendix D Schema Information Collections...............................71 Appendix E Error Code Directory.........................................79 Appendix F Glossary.....................................................83 Index..........................................................85 -------------------------------------------- Structured Full Text Query Language vii ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- LIST OF ILLUSTRATIONS LIST OF FIGURES Figure 1. Software Independent Approach ................................................................2 Figure 2. Comparison with Relational Model.....................5 Figure 3. Example of an interaction between the client and server. 28 Figure 4. VLR format..........................................31 Figure 5. VLR format illustrating return of textual data......35 Figure 6. VLR format illustrating return of a nested VLR......37 Figure B-1. Document-record versus hit traversal..............55 LIST OF TABLES Table 1. VLR Types............................................29 Table 2. Standard Field Types and Field Type Version..........34 Table C-1. SFQLCA Definition..................................65 Table E-1. SFQLCODE categories and reserved values............79 Table E-2. Standard SFQLCODEs for error conditions............81 Table E-3. Standard SFQLCODEs for exception conditions........82 -------------------------------------------- Structured Full Text Query Language ix ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- 1. INTRODUCTORY CONCEPTS The specification presented here is part of the CD-ROM retrieval standards being prepared by the ATA/AIA Advanced Retrieval Technology (89-9C) committee1. Specifically, it addresses the vendor-independent standardization of information management on removeable media. The current CD-ROM industry standard--ISO 9660--provides a common mechanism for access to directory information and files on a disc. Thus, any program can read data from any of the files on a CD-ROM. ISO 9660 does not provide any information content standards, however. This means that special software is necessary to display or manipulate the information contained in the files on the disc. In order to open the file contents for general use, data format standards (i.e., the organization of data within files) are necessary. Industry groups, such as the ATA/AIA, are currently selecting from the variety of standards available to represent different data types. Standardization of data file formats is not sufficient, however, to produce CD-ROMs that can be used with various software packages. To provide the user rapid access to the vast amount of information that can be placed on a CD-ROM, the information must be indexed. Thus, even with standard data formats, CD-ROMs still contain nonstandard indexes used to search for the data. These indexes are created specifically for a given vendor's end-user retrieval software. One solution to this problem is to standardize the index formats, so that a vendor's software can read another's index. This solution is not very attractive, since it requires vendors to completely rework their existing index/retrieval engines to read the new index format. Further, it constrains vendors from optimizing their retrieval engines, a process which often depends on the storage format of the indexes. The present document describes a different approach to permitting standard access to data and indexes on the disc. Rather than standardize the structure of information stored on the disc, the retrieval engine (which searches the indexes) and the end-user application software (which manages user requests and presents data) are decoupled, and the interface between them is standardized. The decoupling is provided by the two layer design, illustrated in Figure 1. The lower Data/Index Layer (server) has in-depth information about the data and indexes, as represented on the disc. The upper Presentation Layer (client) knows only how to translate user requests into a standard request language, and to present data in standard formats to the user. Thus, rather than understanding the data and index representations on the disc, the Presentation Layer sees a logical view of the disc. As long as the communication between the two layers is standardized, discs with markedly different data/index layers (e.g., from different vendors) can be ____________________ 1 For information on the set of standards applicable to ATA disc interchangeability see ATA 89-9C.CDROMPROFILE-R2-1990. -------------------------------------------- Structured Full Text Query Language 1 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- interchangeably accessed from any end-user (Presentation Layer) retrieval system. There are a number of immediate benefits of disc interchangeability: o A single user interface can be used to access all compliant discs; a compliant disc can be used by all compliant user interfaces. o Information providers can have discs created by any service provider, without impacting the end-user interface; o Information providers can select end-user software without constraining their choice of data preparation service; This layered approach is the easiest path to disc interchangeability, because it does not require reworking the details of existing index (Data/Index Layer) and retrieval (Presentation Layer) software. Instead, a standard interface can be inserted between the existing presentation software and the underlying data/index engine. The proposed standard interface provides a mechanism to search for data on the disc, and to request return data in standard formats. The semantics of the request language and general return data structures are the domain of the Structured Full-Text Query Language (SFQL), which is described in this specification. Figure 1. Software Independent Approach -------------------------------------------- Structured Full Text Query Language 2 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- -------------------------------------------- Structured Full Text Query Language 3 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- 2. SFQL LANGUAGE SPECIFICATION 2.1 Scope This document specifies in detail a proposed interface standard called the Structured Full-Text Query Language (SFQL). SFQL provides convenient access to full-text index information and textual data, in a manner consistent with the way the ANSI standard Structured Query Language (SQL) provides access to data in relational databases (ANSI X3.135-1986; see also Date, 1987). Although SFQL currently focuses on full-text retrieval, the specification has been developed to facilitate an eventual merger of the SFQL and SQL standards. Thus, this specification follows three basic premises: o SFQL will adopt from SQL any constructs it needs to support full-text rather than reinventing them; o All new constructs in this specification are consistent with the SQL syntax style; and o New constructs will not conflict with any known SQL construct. 2.2 Limitations The specification of SFQL, driven by the ATA/AIA, is currently focused on providing software-independent information delivery. This specification is limited to the constructs that are necessary to support information delivery on removeable direct access media. This specification excludes discussion of the following constructs that may eventually be a part of SFQL: o Schema Data Definition Language (SQL DDL) The constructs used to define the database. o Update capability; (UPDATE/INSERT) The constructs used to update or add data. o Transaction capability The constructs used to unitize a set of SFQL instructions, such that they all are executed or none are. This is primarily important when updates are permitted. o Joins The constructs used in SQL to combine two or more tables dynamically are not supported for collections. o Views The construct used in SQL to dynamically form a new logical table from one or all of the attributes (columns) of one or more tables. -------------------------------------------- Structured Full Text Query Language 3 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- o Subselects and Subqueries The SQL constructs used to nest one or more SQL SELECT statements such that the most deeply nested SELECT statements will be completely evaluated and the results of that evaluation used as the basis for the evaluation of the subsequent SELECT. Comparable results are generally possible by expressing the query in terms of Boolean operators. o Modules and procedures Constructs used in SQL to provide simple reuseability and structure. o Embedded SFQL (comparable to Embedded SQL) Constructs to embed SFQL (or SQL) statements directly into a third-generation programming language. A preprocessor converts these into native constructs of the target third-generation language prior to compiling. o Integrity constraints Constructs used in SQL to specify at database creation time the valid range or type of data that is permitted for given data fields. o Security Constructs used to provide and enforce access restrictions on a database. 2.3 Data Types The following data types are supported in SFQL language statements. (See Page 33, "Standard Return Data Types", for information on return data types.) 2.3.1 Character Strings A character string consists of a sequence of characters. The character set in SFQL statements shall comply with the definition as stated in the ISO standard, "8-bit single byte coded graphics sets. Part 1. Latin Alphabets" (ISO 8859-1:1987, 1987). This contrasts with SQL which allows the character set to be implementation defined. 2.3.2 Numbers SFQL supports only signed integer numbers in statements. In contrast to SQL, SFQL does not support floating point values in statements. -------------------------------------------- Structured Full Text Query Language 4 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- 2.3.3 Dates A date is a sequence of integers represented as year, month, and day, separated by hyphens. Dates are enclosed in parenthesis and must be preceded by the keyword "DATE". 2.4 Database Model The database model, intended for full-text databases, is based on a relational DBMS metaphor so that SQL concepts may be applied. Figure 2 illustrates the model. Figure 2. Comparison with Relational Model 2.4.1 Document-records (SQL Rows) A document-record is all or part of a document, analogous to an SQL row. Documents are comprised of a meaningful amount of text, analogous to a paper document. -------------------------------------------- Structured Full Text Query Language 5 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- 2.4.2 Collections (SQL Tables) A collection is a set of document-records, analogous to an SQL table, which may permanently reside in a database, or may be the result of a search. If a collection for a maintenance manual is named "Task", each document-record carries the full text for a task. If a search extracts (via projection) only the "Special_Tool_And_Equipment" field of the "Task" collection, then this results in a new collection where each document-record is the full- text of the "Special_Tool_And_Equipment" field (and any subfields). A collection which results from a search is referred to as search-results. 2.4.3 Tag-Field (SQL Column) Document-records are divided into fields (one or more). A field might be the entire document-record, a specific section of the document-record, such as the abstract or introduction, or it may contain a word or phrase. Fields are typically indicated during data preparation by placing labels in the text, called tags. Thus, in SFQL, fields are referred to as s. Unlike SQL, which does not support nested columns, SFQL permits s to be nested within other s. Further, SFQL permits multiple occurrences of a tag within a document-record. In the relational model, this means that a single column may have more than one value for a single row. Lastly, certain s may be defined in a schema as optional. 2.5 Schemas There are currently no formal schema definition constructs in SFQL, unlike SQL. However, a schema is still necessary. The schema definition language, which is done in SQL using a standard Data Definition Language (DDL), is not included as part of this standard. The requirements of individual servers define the results and the process of data preparation. SFQL contains facilities to permit an application to dynamically determine the s used in a collection (see Appendix D). 2.6 Root-node A root-node is defined as the location of a collection, for example it could be the name of a network node or CD-ROM. The location is logically defined and may span multiple devices or nodes. 2.7 Notation The following grammar is based on the ANSI standard, "Database Language-- SQL" (ANSI X3.135-1986). -------------------------------------------- Structured Full Text Query Language 6 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- The syntactic notation is expressed in Backus Naur Form (BNF), where: o A keyword (a literal portion of the syntax) is indicated by uppercase letters o A production symbol (a symbol for which a production rule using ::= appears in the grammar) is enclosed in angle brackets. o Any character not in angle brackets (excluding the characters below) is a keyword or literal character. o Literal characters which are not printable in this specification are indicated by an underlined textual description. The following additional syntactical notation is used: o Square brackets ([]) are used to indicate optional elements. o Ellipses (...) indicate elements that may be repeated one or more times o Braces ({}) indicate groups of elements o Extensions to the ANSI SQL X3.135-1986 specification are indicated by bold italics on the left-hand side of a production. 2.8 SFQL Statement The grammar of an SFQL statement is as follows (see Appendix A for a alphabetical list of keywords): ::= ; | ; | ; | ; | ; | ; Note that a semicolon is always used to terminate an . 2.9 SFQL Grammar (Comparable to SQL Common Elements) The following grammar is broken down using the same sections presented in the ANSI SQL X3.135-1986 standard. 2.9.1 2.9.1.1 Function To define the terminal symbols of the language. -------------------------------------------- Structured Full Text Query Language 7 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- 2.9.1.2 Format ::= | | ::= 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 ::= | ::= A | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z ::= a | b | c | d | e | f | g | h | i | j | k | l | m | n | o | p | q | r | s | t | u | v | w | x | y | z ::= See Syntax Rules. 2.9.1.3 Syntax Rules A is any character in the implementor-defined character set other than a or a . If the implementor-defined end-of- line indicator is a character, then it is also excluded from (ANSI SQL X3.135-1986 Section 5.1, Rule 1). The s shall include all characters other than s or s that occur in terminal productions of SFQL language, and shall include the percent sign and underscore characters (ANSI SQL X3.135-1986 Section 5.1, Rule 2). 2.9.2 2.9.2.1 Function Specify a nonnull value. -------------------------------------------- Structured Full Text Query Language 8 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- 2.9.2.2 Format ::= | | ::= ' ... ' The specification for permits any ASCII character to be embedded in the string other than the single quote character ('). To embed a single quote, two consecutive single quotes are used. ::= ' ... ' [ ] ::= ESCAPE ::= | A is formed exactly like a , but interpreted somewhat differently. A pattern can contain characters, which are interpreted differently in a than they are in a . The '%' character represents zero or more characters. The '_' character represents any single character. A is distinguished from a by its context. The field WILDCARD in the collection Collections is used to determine where a can be placed (See Appendix D). To embed a without its special meaning, an , defined by the optional , must preface the character. To embed a single quote, two consecutive single quotes are used. ::= [+ | -] A consists of a exact numeric quantity, preceded by an optional sign. -------------------------------------------- Structured Full Text Query Language 9 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- ::= ... ::= DATE ( - - ) SQL does not support . This definition of is based on SQL2, Section 5.3 (ISO/IEC JTC1/SC21 WG3 N986, 1989). ::= ::= ::= ::= ::= _ | % The are used to represent any single character (_), or any sequence of zero or more characters (%). 2.9.2.3 Syntax Rules The data type of a is equivalent to an ANSI SQL character string. The length of a is the number of s that it contains. A single quote character embedded in the character string via two single quotes counts as a single character in both the value and length of the (cf. ANSI SQL X3.135-1986, Section 5.2, Rule 2). The data type of a is integer. The precision of a is the number of digits that it contains (cf. ANSI SQL X3.135-1986, Section 5.2, Rule 4). 2.9.3 2.9.3.1 Function Specify lexical units. 2.9.3.2 Format ::= | -------------------------------------------- Structured Full Text Query Language 10 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- ::= | | ::= [{[]}...] ::= _ ::= | ::= (See Appendix A) ::= | , | ( | ) | < | > | . | = | * | + | - | / | <> | >= | <= ::= { | | }... ::= --[...] A can follow any portion of an and constitutes the rest of the line up to the . ::= ASCII_carriage_return | {ASCII_carriage_return ASCII_line_feed} | ASCII_line_feed SQL leaves this implementation undefined. SFQL uses these definitions to promote interchangeability. ::= space character ::= ' ' An escape character is used to preface another character, which normally has special meaning, to avoid that special interpretation in that context. For example, when a appears in a , an escape character is used to remove the s special meaning. 2.9.3.3 Syntax Rules A , other than a , shall not include a (ANSI SQL X3.135-1986 Section 5.3, Rule 1). Any may be followed by a . A shall be followed by a or a . If the syntax does not allow a to be followed by a , then that shall be followed by a (ANSI SQL X3.135-1986 Section 5.3, Rule 2). -------------------------------------------- Structured Full Text Query Language 11 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- An shall not consist of more than 18 s (ANSI SQL X3.135-1986 Section 5.3, Rule 3). An shall not be identical to a (ANSI SQL X3.135- 1986 Section 5.3, Rule 4). All lower case letters appearing in an identifier shall be treated as though they were effectively converted to the corresponding upper case letter (cf., SQL 3, ISO/IEC JTC1/SC21 WG3, 1989). 2.9.4 Names 2.9.4.1 Function Specify names. 2.9.4.2 Format ::= [ . ] A is equivalent to an SQL . ::= [ : ] This is used to qualify the owner of a collection. (It is equivalent to in SQL except that the optional has been added.) ::= The is included for use in qualifying collections. With removeable media collections (e.g., collections on a CD-ROM), this identifier can be used to specify the disc name which contains the collection. The information can then be used to prompt the user to insert a different . ::= This is a temporary name, which can be equated with a . A can only be defined in the of a . A is valid only during interpretation of the in which it was defined. -------------------------------------------- Structured Full Text Query Language 12 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- ::= 2.9.4.3 Syntax Rules A shall identify a collection defined in the schema (cf. ANSI SQL X3.135-1986 Section 5.4, Rule 5). The scope of a is a . In different scopes, the same may be associated with different collections or with the same collection (cf. ANSI SQL X3.135-1986 Section 5.4, Rule 7). 2.9.5 2.9.5.1 Function Specify one or more values. 2.9.5.2 Format ::= 2.9.5.3 Syntax Rules A specifies a value that is not selected from a collection (cf. ANSI SQL X3.135-1986 Section 5.6, Rule 1). 2.9.6 (SQL ) 2.9.6.1 Function Reference a (column in SQL). 2.9.6.2 Format ::= [ .] | [ .] | [ .] -------------------------------------------- Structured Full Text Query Language 13 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- ::= | The is used to qualify a tag by the hierarchy represented in the schema. Note that the nature of the tagging is not a part of SFQL itself, only the concept. A schema defining the s is necessary for standard retrieval (see "Schemas", on Page 6). ::= -> ::= The is used to qualify an attribute of a tag. ::= => ::= ::= The is equivalent to the in SQL. ::= | ::= DOCUMENT This represents the entire document- record. 2.9.6.3 Syntax Rules A references a named . The meaning of the reference depends on the context. If contains a , then the shall appear within the scope of one or more s or s. If there is more than one such or , then the one with the most local scope is specified. The scope of a or is defined in the . If the is not specified, there shall be exactly one possible qualifier within the scope, and that is implicitly specified (cf. ANSI SQL X3.135-1986 Section 5.7, Rule 3). -------------------------------------------- Structured Full Text Query Language 14 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- 2.9.7 2.9.7.1 Function Specify a collection derived from the search-results of a . 2.9.7.2 Format The select expression is the basic language structure of SFQL. At a minimum, it consists of statements telling SFQL what to select, and from where. Additionally, criterion for selection can be included (), as well as the order in which to return the data ( ). ::= SELECT [ALL | DISTINCT] ::= [{ , }...] | * The is used to specify what information is to be returned from the collection. This is the mechanism for projection in relational databases. ::= | + | - ::= | * | / ::= [+ | -] ::= | | | | | ( ) If the data type of is not number, then the should not include any operators. -------------------------------------------- Structured Full Text Query Language 15 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- ::= [ ] [ ] [ ] ::= FROM [{UNION }...] ::= [] The specifies the collection or collections in which the search is conducted. When more than one collection is specified, if the s are the same for both collections, those fields need to be qualified in the query. ::= WHERE The allows the query to be qualified using predicates. This determines the search- results of the query: that is, which document- records are included. ::= GROUP BY [{ , }...] The s specified must only occur once in the SFQLCA for each document-record in the search- results. ::= HAVING 2.9.7.3 Syntax Rules The applicable priviledges for each specificed in the shall include SELECT (cf. ANSI SQL X3.135-1986 Section 5.25, Rule 1). The "*" is equivalent to a sequence in which each is a that references a of the search-results, and each of the search-results is referenced exactly once (cf. ANSI SQL X3.135-1986 Section 5.25, Rule 4). 2.9.8 Search Conditions 2.9.8.1 Function Specify a condition that is TRUE or FALSE depending on the results of applying boolean operations to specify conditions. -------------------------------------------- Structured Full Text Query Language 16 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- 2.9.8.2 Format ::= | OR ::= | AND ::= [NOT] ::= | ( ) ::= | | | | | ::= For full-text collections, only one in a may represent a . ::= = | <> | < | > | <= | >= ::= [ NOT ] BETWEEN AND The returns TRUE when the falls within the range (alphanumerically or ordinally) between the two strings. For example, whenever the contents of the AUTHOR falls alphabetically between 'BAKER' and 'DARWIN'. All s in the must be atomic. ::= [ NOT ] IN -------------------------------------------- Structured Full Text Query Language 17 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- ::= [{ , }...] ::= value_specification | general_function ::= [NOT] CONTAINS The evaluates to TRUE whenever the evaluates to TRUE when applied to the . ::= | ::= vertical bar character ::= | & ::= [ ~ ] ::= | ( ) ::= | ::= WITHIN OF [ IN_ORDER ] This predicate is used to test for proximity. Proximity is based on a number of s. The condition evaluates TRUE whenever one of the strings in is within the specified distance of one or more of the strings in the second . IN_ORDER modifies the search so that matching document-records will have terms appearing in the same order as specified in the . -------------------------------------------- Structured Full Text Query Language 18 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- ::= | , ::= | The terms in are logically or'd to complete a . Thus: 'A', 'B' & 'C' is equivalent to: ('A' | 'B') & 'C' ::= ::= CHARACTERS | WORDS | SENTENCES | PARAGRAPHS The specifies the distance units for the . ::= [ NOT ] LIKE ::= IS [NOT] NULL 2.9.8.3 Syntax Rules A evaluates to TRUE or FALSE for each document-record to which it is applied. Thus, s are used to determine whether or not (TRUE or FALSE) to include a document-record in the search- results. SFQL predicates may not compare two . Compares between columns in SQL are used to perform joins, which are beyond the current scope of SFQL. -------------------------------------------- Structured Full Text Query Language 19 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- 2.9.9 Cursor Operations 2.9.9.1 Function Cursor operations are used to step through search-results one document- record at a time. They are useful for retrieving data from SFQL in a manner consistent with the operation of third-generation languages, such as C or Pascal. 2.9.9.2 Format ::= DECLARE [ SCROLL ] CURSOR FOR ::= [ ] ::= ::= ORDER BY [ { , }...] | ORDER BY ::= { | } [ASC | DESC ] ::= RELEVANCE | RELEVANCE ( ) The permits the query to specify an order in which to return the data, i.e., so that it may be returned in ascending or descending alphabetical order. The s specified must only occur once in the SFQLCA for each document-record in the search-results. The permits the data to be ordered by relevance, as judged by the server. Each server may have its own method of judging relevance. The optional argument to RELEVANCE is server specific. -------------------------------------------- Structured Full Text Query Language 20 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- ::= OPEN This operation opens a cursor for processing. All expressions in the are evaluated when the cursor is opened. The query is performed and the search-results are bound at this time. If the open is successful, returns a SFQLCA containing the number of hits (possibly zero). If the search exceeds the MAX_SEARCH_TIME or MAX_SEARCH_RECORDS, a PRELIMINARY SFQLCA is returned as if the server had received a SUSPEND statement. ::= CLOSE ::= FETCH [[] FROM ] [] ::= NEXT | PRIOR | FIRST | LAST | { ABSOLUTE | RELATIVE } The has been modified to follow the improved syntax of the ISO Draft Proposed SQL3 Database Language (ISO/IEC JTC1/SC21 WG3 N986, 1989). The modification includes the addition of the . ::= FIELDS FIELDS works in conjunction with the of a SELECT: it returns a subset of the items specified in the . This allows selection of the specific field(s) of data returned from the query on an individual document- record basis. 2.9.9.3 Syntax Rules As per the rules in ANSI SQL X3.135-1986 Sections 8.3 (declare), 8.6 (fetch), 8.8 (open). SCROLL must be specified in the in order to use in a . -------------------------------------------- Structured Full Text Query Language 21 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- 2.9.10 Function Definitions Functions can be used to set server options, return server characteristics, qualify references to schema entities, or convert strings or search- results. There are three classes of functions. General functions are used to convert or qualify information in a used in s or s; special functions are used to return information from the server or from a ; and action functions are used to return or modify server options in s or s. 2.9.10.1 General Functions 2.9.10.1.1 Function The s can be used anywhere a can be used. See Appendix B for a more detailed description. 2.9.10.1.2 Format ::= | | ::= HITS ( , , ) ::= RELEVANCE ( ) ::= THESAURUS ( , ) 2.9.10.1.3 Syntax Rules Functions may not be nested. 2.9.10.2 Special Functions 2.9.10.2.1 Function The s are used only in a . See Appendix B for a more detailed description. -------------------------------------------- Structured Full Text Query Language 22 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- 2.9.10.2.2 Format ::= | | | | | | | | | | | | ::= COLLECTION ( ) ::= DATA ( , , , ) ::= FEATURES ( ) ::= FILTER ( , ) ::= HIT_TEXT ( , , , , ) ::= ::= | ::= INDEX ( , , , , ) ::= BACKWARD | FORWARD ::= LIMITS ( ) ::= MANUFACTURER ( ) ::= MATCH_CODES ( ) ::= RELEVANCE_METHOD ( ) -------------------------------------------- Structured Full Text Query Language 23 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- ::= SEARCH_OPTIMIZATION ( ) ::= SFQLVERSION ( ) ::= STOP_WORDS ( ) 2.9.10.2.3 Syntax Rules Functions may not be nested. 2.9.10.3 Action Functions 2.9.10.3.1 Function The s can only be used in a or a . See Appendix B for a more detailed description. 2.9.10.3.2 Format ::= SET ::= | , ::= | | | | | | | | ::= ON | OFF | TRUE | FALSE | DEFAULT ::= LANGUAGE ( ) | LANGUAGE ( ) ::= MATCH_METHOD ( ) | MATCH_METHOD ( ) ::= MAX_SEARCH_RECORDS ( ) | MAX_SEARCH_RECORDS ( ) -------------------------------------------- Structured Full Text Query Language 24 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- ::= MAX_SEARCH_TIME ( ) | MAX_SEARCH_TIME ( ) ::= RESUME ( ) | RESUME ( ) ::= SERVER_REPORT_TIME ( ) | SERVER_REPORT_TIME ( ) ::= SFQLCA_LENGTH ( ) | SFQLCA_LENGTH ( ) ::= SHOW_MATCHES ( ) | SHOW_MATCHES ( ) ::= SUSPEND ( ) | SUSPEND ( ) 2.9.10.3.3 Syntax Rules The s SUSPEND and RESUME cannot be used in s. Functions may not be nested. Arguments to s are required in s. In s, the arguments are not used. -------------------------------------------- Structured Full Text Query Language 25 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- 3. RETURN DATA SPECIFICATION 3.1 Data Return (SFQL Communications Area) SQL does not have a standard data return format. A standard is necessary, however, in order to use SFQL as the semantics of a client-server communication protocol. The following specification is aimed at providing support for data types returned by both full-text and SQL databases. 3.1.1 Status and Data Message Format (SFQLCA) Data and status return is always in the form of an SFQL Communications Area (SFQLCA). The details of the SFQLCA can be found in Appendix C. The SFQLCA header contains a number of status fields, and is followed by a data area called SFQLMSG. 3.1.2 Client/Server Interaction An SFQLCA is returned in response to all SFQL commands sent to the server. Figure 3 provides an example of the interaction between a client and a server. For SFQL DECLARE, OPEN, CLOSE, and SET commands, the SFQLCA returns status information in the SFQLCA header. In the case of an error, detailed error messages are returned in SFQLMSG. For SELECT and FETCH commands, the SFQLCA returns data one document-record at a time. The data fields that are returned in each data message and the order of these fields depend on the specified in the query. 3.1.3 SFQLCA Header The status field area of the SFQLCA consists of the fields typically found in an SQL Communication Area (SQLCA), plus some additional status information specific to SFQL. Both areas are used by SFQL. See Appendix C for details of the fields. This section describes some of the SFQLCA specific fields. The SFQLBORD field, which is always the first byte of the SFQLCA, indicates the byte order. An 'L' indicates that numeric values in the SFQLCA are given with the least significant byte first; an 'M' indicates most significant byte first. The SFQLVERS field provides version information for the SFQLCA structure. This specification refers to Version 1 of this data structure. The SFQLCABC field indicates the length of the SFQLCA. This field includes the length of the SFQLCA header, which is fixed in length, and the SFQLMSG area, which is variable length. Thus, whereas SQL implementations have a constant SQLCABC, SFQLCABC varies from one SFQLCA return message to another. Note that SFQLCABC reflects the total length of the SFQLCA, which may include padding following the SFQLMSG field. -------------------------------------------- Structured Full Text Query Language 27 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- The SFQLCODE field is used to return a code indicating success, error, or exception in response to each command, as is standard in SQL. If the code is 0, the operation was successful. If the code is negative (< 0), the operation could not be completed and error information is available (described later). If the code is positive, a non 'fatal' exception occurred. That is, the statement was processed, but could not be completed, or completed exactly as requested. Figure 3. Example of an interaction between the client and server. -------------------------------------------- Structured Full Text Query Language 28 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- 3.1.4 SFQLMSG Area The second part of the SFQLCA is the message area, called SFQLMSG. The length of the data in this field is given by SFQLMSGLEN in the SFQLCA header. SFQLMSG may be empty (SFQLMSGLEN = 0) or may contain return data and/or error information. Data and errors are always returned in this area using an SFQL defined format called a Variable Length Record (VLR). This format can return multiple fields of varying types. If the return information cannot fit into the SFQLMSG area because of system constraints on SFQLCA length, the data will be truncated and an exception will be returned with the data. In such cases the VLR data structure integrity will always be maintained. 3.1.4.1 Variable Length Record (VLR) Data Return 3.1.4.1.1 Description The VLR is designed to allow multiple fields of data to be returned in one message without combining them. It consists of a short header followed by one or more Field Subrecords (FSRs). The VLR consists of: o a 16-byte VLR Type o a 4-byte Number Of Subrecords o a 4-byte FSR Offset o one or more FSRs Figure 4 illustrates the repetition of FSRs in the VLR format. The VLR header begins with the type of the VLR. This is a 16-byte ASCII field left-justified and padded with NULLS. The current set of VLR types is given in Table 1. Currently, two types of VLR records are defined, ERROR and DATA. Table 1. VLR Types. VLR Type Field Contents ERROR ERROR_RECORD DATA DATA_RECORD An ERROR VLR may be returned by the server whenever an exception or an unrecoverable error occurs. Error information is supplied in the SFQLCA header; the ERROR VLR provides more detailed information. The DATA VLR is the mechanism by which all requested data is returned. The data is organized into FSRs corresponding to the s in the of the . The DATA VLR contains exactly one FSR for each item listed in the of the -------------------------------------------- Structured Full Text Query Language 29 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- .1 The order of FSRs is not considered to be important in the VLR: the application must check the Field Label and the Field Number of each subrecord before interpreting the Data Field. The Number of Subrecords field indicates the number of FSRs in the VLR. This information is a 4-byte unsigned integer. The FSR Offset field of the VLR header gives the byte offset from the start of the VLR to the first FSR. This field provides an extension area, where future additions to the VLR header will be made while maintaining backwards compatibility. ____________________ 1 If there are multiple instances of a in a single document, the data from these areas is returned in a nested VLR, where each FSR in the nested VLR contains data from one occurrence of the tag_fields. -------------------------------------------- Structured Full Text Query Language 30 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- Figure 4. VLR format. -------------------------------------------- Structured Full Text Query Language 31 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- 3.1.4.1.2 Field Subrecord (FSR) Format A Field Subrecord (FSR) is a variable length data structure which consists of: o a 4-byte Field Position; o an 8-byte Field Type; o an 8-byte Field Type Version; o a 4-byte Field Status; o a 4-byte FSR Offset to the next FSRs Field Position; o a 4-byte Data Offset to the Data Field; o a 4-byte Label Offset to the Field Label; o an M-byte Field Label; o an N-byte Data Field. Field Position is a 4-byte unsigned integer indicating the ordinal position in the corresponding to the current FSR. That is, the first item in the will return an FSR with a Field Position of 1, the second will return 2, etc. Field Type is a string which indicates the type of data contained in Data Field. Table 2 lists the standard types and their designations. Since VLR is one of the valid types, a VLR may be nested inside any FSR. This accommodates conditions where single s in the return complex data items or ERROR_RECORDs. The data is left justified and NULL padded in the field and the unused bytes are filled with NULLs to 8 bytes. Field Type Version is a string which optionally identifies the specific implementation or application profile associated with the Field Type. For example, the version specified by the ATA Specification 100 Rev 29 might be given as "AT100R29". The CALS version could be "1840A". The data is left justified and NULL padded in the field and the unused bytes are filled with NULLs to 8 bytes. Field Status is a four-byte integer which indicates the status of the data returned in the FSR. If the value is zero, it indicates that no status information was returned. All other values are defined to be function- specific. The FSR Offset permits the application to determine where the next FSR begins. The offset is a 4-byte unsigned integer which, when added to the byte position of the start of the Field Position (the beginning of the FSR), points to the next FSRs Field Position. A value of zero indicates that this is the last FSR in the VLR. The Label Offset permits the application to determine where the Field Label begins. The offset is a 4-byte unsigned integer which, when added to the byte position of the start of the Field Position (the beginning of the FSR), points to the Field Label. The Data Offset permits the application to determine where the Data Field begins. The offset is a 4-byte unsigned integer which, when added to the -------------------------------------------- Structured Full Text Query Language 32 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- byte position of the start of the Field Position (the beginning of the FSR), points to the Data Field. Field Label is a variable-length field containing an exact copy of the for the corresponding field position of the , in SZ format. For example, if the FSR represents the data returned from a , the name of the will be in this area. If a function was applied, then the function name and arguments will be in the Field Label, exactly as they appeared in the . The Data Field is a variable length field which contains the actual data for the FSR (the return value for the of the ). The data must be left justified in the field and NULL padded to the next four-byte boundary. The format of the Data Field itself depends on the Field Type. For example, a Field Type of "SZ" indicates that the data field contains a sequence of ASCII characters followed by an ASCII NULL character. Figure 5 illustrates a sample VLR which might be returned in response to the following fetch: FETCH my_cursor FIELDS author, title ; 3.1.4.2 Standard Return Data Types The types of data supported by a server are not defined by SFQL. Instead, SFQL provides a mechanism for the server to return information about the returned data type along with the data in the Field Type and Field Type Version fields of the FSR. SFQL thus requires that a naming standard be applied to data types, so that the client can identify the type of data returned from the server. Table 2 lists the set of standard Field Type designators supported in this version of SFQL. Data format standards are often qualified through application subsets to reduce the implementation burden, or to select from implementation options. The Field Type Version provided in the FSR is used to optionally qualify a subset or a specific version of a standard. Table 2 shows the currently defined values of this field. -------------------------------------------- Structured Full Text Query Language 33 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- Table 2. Standard Field Types and Field Type Version. Field Example Type Field Type Meaning Designator Version CGM AT100R29 Computer Graphics Metafile DATE1 Date/Time--MM/DD/YYYY,00:00:00 FP IEEE Floating point INT16 Two-byte signed integer INT32 Four-byte signed integer SZ ASCII string TEXT AT100R29 Formatted text types TIFF AT100R29 Tagged Image File Format UINT32 Unsigned four byte integer VLR SFQL2.0 Nested Variable Length Record NULL SFQL2.0 No data ?xxxxxxx Vendor extensions The standard data types given in Table 2 are briefly described in the following sections: 3.1.4.2.1 Computer Graphics Metafile (CGM) CGM is a standard format used to define 2-D vector graphics, as adopted by ATA/AIA Specification 100. The revision number is identified by the Field Type Version Identifier, e.g. 'AT100R29'. This field will not contain match-codes. 3.1.4.2.2 Date/Time (DATE1) DATE1 is a NULL-terminated ASCII string of the form: MM/DD/YYYY,00:00:00. Note that the time is in 24-hour format. This field will not contain match-codes. -------------------------------------------- Structured Full Text Query Language 34 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- Figure 5. VLR format illustrating return of textual data. 3.1.4.2.3 Floating Point (FP) FP is an 80-bit floating point number as defined in ANSI/IEEE Std 754- 1985. This field will not contain match-codes. -------------------------------------------- Structured Full Text Query Language 35 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- 3.1.4.2.4 Two-byte Signed Integer (INT16) INT16 is a two-byte integer with the byte order specified in SFQLBORD of the SFQLCA. This field will not contain match-codes. 3.1.4.2.5 Four-byte Signed Integer (INT32) INT32 is a two-byte integer with the byte order specified in SFQLBORD of the SFQLCA. This field will not contain match-codes. 3.1.4.2.6 Formatted String Data Return (SZ) SZ is a NULL-terminated sequence of characters. Character definitions must conform to ISO 8859. The SZ format may contain match-codes as appropriate for the search. 3.1.4.2.7 Formatted Text Types (TEXT) This includes various industry standard text types. The specific industry standard is identified in the Field Type Version Identifier. For example the SFQL Prototype text type is 'ASDP1' (see Shapiro et al., 1989a, 1989b). The TEXT format may contain match-codes as appropriate for the search. 3.1.4.2.8 Tagged Image File Format (TIFF) TIFF is an Aldus/Microsoft format for encapsulating images. The TIFF format to be used is described in the ATA/AIA Specification 100. The revision number is identified by the field_type_version_Identifier, e.g. 'AT100R29'. This field will not contain match-codes. 3.1.4.2.9 Unsigned Four Byte Integer (UINT32) UINT32 is an unsigned 4-byte integer with the byte order specified in SFQLBORD of the SFQLCA. This field will not contain match-codes. 3.1.4.2.10 Nested Variable Length Record (VLR) A nested VLR has the same format as a VLR, but is contained in an FSR of a higher level VLR. See Page 29, "Variable Length Record (VLR) Data Return", for more information. It is used when a returns multiple data items in a single record. For example, if a is represented multiple times in a document-record, and that is listed in the (i.e., projected), the FSR returned for that will contain a nested VLR with data from each instance of the -------------------------------------------- Structured Full Text Query Language 36 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- in the document-record in the nested VLRs FSR. Figure 6 illustrates a nested VLR. Figure 6. VLR format illustrating return of a nested VLR. 3.1.4.2.11 Vendor Extensions (?xxxxxxx) Vendors can add unregistered extensions using the character code '?', followed by one to seven printable ASCII characters. The vendor extensions format may contain match-codes as appropriate for the search. -------------------------------------------- Structured Full Text Query Language 37 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- 3.1.5 Return of Preliminary SFQLCA A preliminary SFQLCA may be returned under a number of circumstances indicted in this specification--for example, in response to a SET SUSPEND() operation. The header of the SFQLCA is interpreted as usual, except that values in the header represent preliminary data and are subject to change if the operation is completed. For example, SFQLROWS represents the number of rows in the search-results at the time the preliminary SFQLCA was sent. If the search was suspended via a SET SUSPEND(), this number of document- records will be available if the cursor is opened without an intervening resume. The preliminary SFQLCA can optionally contain information concerning the status of the operation in the form of a DATA_RECORD in the SFQLMSG area. The standard FSRs that can optionally be provided in the DATA_RECORD are described in the table below. Field Label Type Explanation TOTAL_SFQLROWS UINT32 Expected value of SFQLROWS if the search was run to completion. PERCENT_COMPLETE UINT32 Percentage estimate of the completion status of the operation, where 100% is complete. The preliminary SFQLCA is indicated by a SFQLCODE of 200-203, as described in Appendix E. The code indicates the reason the preliminary SFQLCA was sent. 3.1.6 Error Handling 3.1.6.1 Overview SFQL, like SQL, defines fatal errors (unrecoverable) and exceptions (recoverable errors). The class of error and a numeric identifier are available to the client following each SFQL command in the SFQLCODE field of the SFQLCA. The SFQLCODE value is set as follows: -------------------------------------------- Structured Full Text Query Language 38 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- o In the absence of a fatal error or exception condition the SFQLCODE field is set to 0. o Fatal errors are severe enough that the SFQL statement cannot be executed and only error information is returned. These errors return a negative SFQLCODE value. o Statements resulting in exceptions return a positive SFQLCODE indicating the error. If a server returns data after an exception, it is indicated by a 'T' value in the SFQLCOMP flag, for all subsequently affected requests. This indicates that data returned might be compromised, e.g., it may be incomplete, or the search may have been broadened. This mechanism is similar to SQL (SQLCODE). SQL, however, defines only one standard value of SQLCODE: 100. As in SFQL, this value indicates a fetch issued beyond the last row or a fetch when there were no rows in the search-results. SFQL defines a number of additional standard error and exception codes. These codes are given in Appendix E. Since recovery from errors is an implementation-dependent issue, assignment of the class of these errors is left to the server. That is, any error listed in Appendix E can be classified as unrecoverable or exception, simply by changing the sign of its value. Thus, Error 20000 is Unknown Error (Recoverable), but Error - 20000 is Unknown Error (Unrecoverable). 3.1.6.2 Error Messages To provide the best error information to the user, error messages as well as codes are provided in the SFQLCA. For both exception and fatal error conditions, the 70 character SFQLERRMC is used to pass a brief, end-user oriented message from the server to the client. Additional information can be provided by the following optional FSRs: o SFQL_ERROR. Provides a detailed explanation of any errors together with any part of the SFQL command needed to illustrate where the error occurred. The FSR field position will always be 0. o USER_ERROR. Provides a more detailed explanation oriented towards end-users. The FSR field position will always be 0. All error messages are supplied in the language requested via the LANGUAGE() . Error information can also be returned specifically for any of the s in the through the use of a nested VLR. The nested VLR is placed in the FSR for the and can contain the SFQL_ERROR, USER_ERROR, and return data FSRs for the . The VLR Record Type of the nested VLR will be either ERROR_RECORD or DATA_RECORD, depending on whether or not the nested VLR contains any return data FSRs. -------------------------------------------- Structured Full Text Query Language 39 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- The use of messages in addition to codes gives the client application a mechanism to inform the user intelligently about errors it receives from the server, but does not understand. This allows servers to implement proprietary error messages in addition to the standard ones (codes -10000 to -19999, and 10000 to 19999, are reserved for such proprietary error codes). This scheme also makes it easy to expand the standard message set while maintaining backward compatibility. A client might not recognize an error number when the server represents a newer release of SFQL. The error message displayed to the user, however, will be based on the newer system, the server. This eliminates the need for the user to get new client software whenever a new release of server software is issued. 3.2 Server Compatibility Model 3.2.1 Levels of Function To allow for future expansion, a multileveled standard is proposed. This version of the standard only defines the lowest compatibility level, called Level 1. Level 1 is to be the common level which all retrieval systems must support. The level number of a retrieval engine can be determined by the value returned by the SFQLVERSION function (See Appendix B). 3.2.2 Discovering Server Capabilities A number of functions are provided to determine information about server capabilities. For example: o SFQLVERSION o FEATURES o MANUFACTURER o LIMITS o MATCH_CODES See Appendix B for more details. 3.2.3 Discovering Collection Information SFQL provides schema information collections to determine a particular server's capabilities and schema. These collections are used to supply specific information about the collections managed by the server. They can be retrieved from an application using the standard SFQL query semantics. Appendix D provides the detailed schema for these collections. -------------------------------------------- Structured Full Text Query Language 40 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- 3.2.4 Server Differences as Exceptions A server may offer the capability to simulate an SFQL feature which is not directly supported. Such conditions will be handled using the normal exception handling. That is, the statements resulting in exceptions return a positive SFQLCODE indicating the error, and a message in SFQLERRMC. Further explanatory information can be returned in the VLR using the USER_ERROR and SFQL_ERROR fields. In practice, this will work as follows. When a server receives a declare statement that it does not fully support, the server should return an error as usual, but should set the SFQLCOMP flag. In the ERROR_RECORD, the USER_ERROR field will be used to forward a detailed error to the user indicating the server's ability to perform a "compromised" query (presented to the user at the client's discretion). The SFQL_ERROR field may provide, as always, a developer's oriented message explaining the compromise. Further, all subsequent operations using the "compromised" cursor should return a SFQLCA with the SFQLCOMP flag set. In these operations, SFQLCODE and the SFQL error message facilities can be used to indicate further errors or exceptions. Some example cases where compromises might be made by a server: WHERE clause: o An unsupported is used. The clause may be broadened in scope or excluded from the condition list. o An unknown or unsupported is specified. In this case the search can be conducted using the other search qualifiers, if there are any. Alternately, the search may be broadened to a related tag at the server's discretion. FROM clause: o Several collections were indicated but one of the collections requested is not available. The portion of the query involving that collection might be ignored and the rest of the query executed. o Too many collection names were specified. The first n may be used and all clauses from the other collection(s) may be dropped. o An unknown or unsupported may have been specified. In this case the search can be conducted using the other search qualifiers, if there are any. Alternately, the search may be broadened to a related at the server's discretion. For example, a client may declare a cursor as follows: DECLARE my_cursor CURSOR FOR SELECT DOCUMENT FROM my_database WHERE 'standard' IS WITHIN 1 SENTENCES OF 'SFQL' ; -------------------------------------------- Structured Full Text Query Language 41 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- If a server does not support proximity by SENTENCES but could simulate the operation using an estimate of sentence length and effectively change the query to: DECLARE my_cursor CURSOR FOR SELECT DOCUMENT FROM my_database WHERE 'standard' IS WITHIN 16 WORDS OF 'SFQL' ; then the server would respond to the DECLARE by returning an error but indicating the availability of a compromise via SFQLCOMP and the messages in the ERROR_RECORD. Alternatively, another server might execute the query as follows: DECLARE my_cursor CURSOR FOR SELECT DOCUMENT FROM my_database WHERE 'standard' IS WITHIN 1 PARAGRAPHS OF 'SFQL' ; In both cases, subsequent OPEN, FETCH, and CLOSE operations on this cursor would return SFQLCA's with the SFQLCOMP set. In all other respects, the return information from these subsequent operations would appear normal to the client program. Run-time errors are another example where an exception might be generated. For example, the following conditions: o The user has interrupted the search. The search-results are incomplete. o System or platform limitation. The search-results are incomplete. This guideline allows the server to respond to requests that are not completely within its range of capabilities. The client can choose to accept or reject the compromised data. This allows greater compatibility among search engines without requiring complete Lowest Common Denominator operation by the client. -------------------------------------------- Structured Full Text Query Language 42 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- 4. REFERENCES ANSI X3.135-1986. American National Standard for information systems-- database language--SQL. American National Standards Institute, New York, NY. ATA 89-9C.CDROMPROFILE-R2-1990. Air Transport Association Advanced Retrieval Technology CD-ROM Implementation Profile. Air Transport Association, Washington, DC. ATA 89-9C.FUNCREQ-R2-1990. Air Transport Association Advanced Retrieval Technology Functional Requirements. Air Transport Association, Washington, DC. ATA 89-9C.MMSCHEMA-R3-1990. CD-ROM Interchangeability Standard--Tasked Maintenance Manual Schema. Air Transport Association, Washington, DC. Date, C.J. (1987). A guide to the SQL Standard. Addison-Wesley, Reading, MA. ISO 8859-1:1987. ISO Standard 8859-1:1987, 8 bit single byte coded graphic sets. Part 1: Latin Alphabets. International Standards Organization, New York, NY. ISO/IEC JTC1/SC21 WG3 N986 (1989). Database Language SQL2 and SQL3. ISO- ANSI working draft, American National Standards Institute, New York, NY. Shapiro, N.R. and Bowers, F.J. (1989a). Standardizing the delivery of technical documentation on CD-ROM's. CD Data Report, pp. 21-28. Shapiro, N.R., Diamantopoulos, E., Delpech, M., Deldon, S., Mondon, L., Mathe, P., and Picou, C. (1989b). Specification for Prototype 1 of SFQL. Air Transport Association, Washington, DC. -------------------------------------------- Structured Full Text Query Language 43 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- APPENDIX A: SFQL KEYWORDS ABSOLUTE FOR PRIOR ALL FORWARD PRIVILEGES AND FORTRAN PROCEDURE ANY FOUND PUBLIC AS FROM REAL ASC GO RELATIVE AUTHORIZATION GOTO RELEVANCE AVG GRANT RELEVANCE BACKWARD GROUP RELEVANCE_METHOD BEGIN HAVING RESUME BETWEEN HIT_TEXT ROLLBACK BY HITS SCHEMA CHAR IN SEARCH_OPTIMIZATION CHARACTER IN_ORDER SECTION CHARACTERS INDEX SELECT CHECK INDICATOR SENTENCES CLOSE INSERT SERVER_REPORT_TIME COBOL INT SET COLLECTION INTEGER SFQL COMMIT INTO SFQLCA_LENGTH CONTAINS IS SFQLCODE CONTINUE LANGUAGE SFQLERROR COUNT LAST SHOW_MATCHES CREATE LIKE SMALLINT CURRENT LIMITS SOME CURSOR MANUFACTURER SQL DATA MATCH_CODES SQLCODE DATE MATCH_METHOD SQLERROR DEC MAX SQLVERSION DECIMAL MAX_SEARCH_RECORDS STOP_WORDS DECLARE MAX_SEARCH_TIME SUM DELETE MIN SUSPEND DESC MODULE TABLE DISTINCT NEXT THESAURUS DOCUMENT NOT TO DOUBLE NULL UNION DROP NUMERIC UNIQUE END OF UPDATE ESCAPE ON USER EXEC OPEN VALUES EXISTS OPTION VIEW FEATURES OR WHENEVER FETCH ORDER WHERE FIELDS PARAGRAPHS WITH FILE PASCAL WITHIN FILTER PLI WORDS FIRST PRECISION WORK FLOAT PREVIOUS -------------------------------------------- Structured Full Text Query Language 45 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- APPENDIX B: DIRECTORY OF SFQL FUNCTIONS There are three kinds of SFQL functions: o General Functions o Special Functions o Action Functions B.1. GENERAL FUNCTIONS General Functions specify processing for the server to perform before data is used in a search, or before data is returned. A may form a in a or in a . When a is specified in a , the original function name and arguments are returned in the Field Label field of the FSR. B.1.1 HITS Format. HITS(TAG_FIELD_SPECIFICATION, TERM_OPTION, SCOPE) Description. The HITS function returns the number of query matches in the given of the current document-record, or of the document-records in the current search-results. To retrieve the number of hits throughout the current document-record or search-results, use the DOCUMENT in the . TERM_OPTION specifies the criterion used by the server to count the hits. Valid values are listed in the table below: TERM_OPTION VALUES QUERY_TERMS The number of original query terms matched. HIGHLIGHT_TERMS The number of locations in the text defined by the specified marked by match- codes. HIGHLIGHT_TERMS enables the client to highlight the query terms. HIT_TERMS The number of locations in the text that are considered hits by the server. The criteria for a hit can be different for each server. -------------------------------------------- Structured Full Text Query Language 47 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- SCOPE specifies the target of the function. The RECORD sets the SCOPE to the current document-record, while the TOTAL specifies the document-records in the current search-results. Return format. UINT32. Return value. The number of hits. B.1.2 THESAURUS Format. THESAURUS(WORD, OPERATOR) Description. The THESAURUS function identifies terms deemed by the server to be equivalent to the specified WORD. When used in a , the server returns a list of equivalent terms. When used in a , the server expands the query terms to include equivalent terms before processing the query. The returned data is the same as if the in the original query explicitly referenced all of the equivalent terms in a . OPERATOR specifies the criterion used by the server to determine equivalence. Valid values are listed in the table below. When used in a , The THESAURUS function expands to a list of terms deemed to be equivalent to WORD, using the operator specified (see table below). OPERATOR VALUES SYNONYM Expands WORD to a list of terms with similar meanings. For example, "sofa" may be considered a synonym for "couch." BROADEN Expands WORD to a list of terms that specify a more general form. For example, "vegetable" is a is a more general form of "potato." NARROW Expands WORD to a list of terms that are specific examples. For example, "daisy" is an example of "flower." Return format. Nested VLR (when used in a ). Return value. The Field Label field of each FSR contains an equivalent term. The Data Field is empty. -------------------------------------------- Structured Full Text Query Language 48 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- B.1.3 RELEVANCE Format. RELEVANCE() Description. The RELEVANCE function returns the server-defined relevance rank for the current document-record. The RELEVANCE_METHOD function returns information as to whether higher ranks indicate more or less relevant documents. Return format. UINT32. Return value. The value indicating the relevance rating made by the server. If the server does not support relevance ranking, the same value is returned for all document-records. B.2. SPECIAL FUNCTIONS Special Functions return system information, or modify the data returned for a . A can be used only in a . Each returns a data structure specific to that function. B.2.1 COLLECTION Format. COLLECTION() Description. The COLLECTION function returns the name of the entry in the corresponding to the current search-results. This permits a client to identify the document-records in search-results when more than one collection is searched. For example: DECLARE MY_CURSOR CURSOR FOR SELECT COLLECTION(), ABSTRACT FROM A UNION B WHERE AUTHOR CONTAINS 'SMITH'; FETCH MY_CURSOR; The FETCH command would return the value of the COLLECTION() function for each document-record, thus permitting the client to identify if the document-record is from Collection A or from Collection B. Return format. SZ. Return value. The from which the document-record originated. -------------------------------------------- Structured Full Text Query Language 49 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- B.2.2 DATA Format. DATA(TAG_FIELD_SPECIFICATION, FETCH_ORIENTATION, MAXLENGTH, TAG_LIST) Description. The DATA function returns a block of data (text or graphics) from the . The DATA function has the following arguments: o TAG_FIELD_SPECIFICATION The to return (as per the schema specification), or the DOCUMENT, to retrieve text independently of any . o FETCH_ORIENTATION The values for FETCH_ORIENTATION are given in the table below. FETCH_ORIENTATION VALUES NEXT Return the next block of data PRIOR Return the prior block of data FIRST Return the first block of data LAST Return the last block of data in the TAG_FIELD_SPECIFICATION { ABSOLUTE | RELATIVE } number Return the block of data at the absolute position specified or relative to the current DATA position. The DATA function can be used to get the NEXT or PRIOR block of data following a HIT_TEXT function, although the given must be the same for both functions. If more than one HIT_TEXT function is active on the same , then the result of the NEXT or PRIOR is defined by the server. o MAXLENGTH The number of bytes to return. o TAG_LIST A list indicating which tags to filter. The members of the list are separated by blanks. The keyword ALL may be used in to indicate all tags for the collection. The server removes the named tags and any associated attributes. The DATA function sets the Field Status of the FSR to "1" whenever the data retrieved is the start of the , "2" when it is the end of the , and "3" when it is both the start and the end of the . Otherwise, the Field Status is set to "0" (no status). As -------------------------------------------- Structured Full Text Query Language 50 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- with the cursor for document-records, if a fetch is made out of the range of the (whether by absolute or relative positioning), Field Status is set to 100 and the contents of the Data Field of the FSR are undefined. Note that unlike HIT_TEXT, this function allows the calling program to specify any block of data to be returned. It would be used, for example, to page through the document a block of characters at a time. The block may include match-codes to mark the areas where the search criteria were satisfied. For more information on match-codes, see the MATCH_CODES and SHOW_MATCHES functions. Return format. Dependent on the data type of TAG_FIELD_SPECIFICATION. Return value. A portion of data from the specified. B.2.3 FEATURES Format. FEATURES() Description. The FEATURES function lists optional features, indicating which are supported by the server. The list of these features is provided as a convenience to the client by a server. The server is, however, responsible for attempting to fulfill any and all requests. If the server does not directly support a feature, it should do its best to simulate the feature and return the results with an SFQLCODE exception, warning that results may be compromised. Return format. SZ. Return value. The features are reported by the following sequence of characters: Bytes Values Explanation 1 T/F Supports SUSPEND/RESUME requests 2 T/F Sorts search-results by relevance 3 T/F Supports STOP_WORDS function. (The INDEX function can always be used to determine stop words.) B.2.4 FILTER Format. FILTER(TAG_FIELD_SPECIFICATION, TAG_LIST) -------------------------------------------- Structured Full Text Query Language 51 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- Description. The FILTER function removes tags in the TAG_LIST from the returned text fields. The server removes the named tags and any of their associated attributes. o TAG_FIELD_SPECIFICATION The to return (as per the schema specification), or the DOCUMENT, to retrieve text independently of any . o TAG_LIST A list indicating which tags to filter. The members of the list are separated by blanks. The keyword ALL may be used to indicate all tags for the collection. Return format. TEXT or SZ. Return value. The contents of the , minus any tag markings. B.2.5 HIT_TEXT Format. HIT_TEXT(TAG_FIELD_SPECIFICATION, FETCH_ORIENTATION, MAXLENGTH, JUSTIFY, TAG_LIST) Description. The HIT_TEXT function returns a block containing a hit. -------------------------------------------- Structured Full Text Query Language 52 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- The HIT_TEXT function accepts the following parameters: o TAG_FIELD_SPECIFICATION The to return (as per the schema specification), or the DOCUMENT, to retrieve text independently of any . o FETCH_ORIENTATION The tokens for FETCH_ORIENTATION are given in the table below. FETCH_ORIENTATION VALUES NEXT Return the block of data relative to the next hit in the TAG_FIELD_SPECIFICATION PRIOR Return the block of data relative the prior hit in the TAG_FIELD_SPECIFICATION FIRST Return the block of data surrounding the first hit in the TAG_FIELD_SPECIFICATION LAST Return the block of data surrounding the last hit in the TAG_FIELD_SPECIFICATION { ABSOLUTE | RELATIVE } number Return the block of data at the absolute hit position specified or the relative hit position to the current HIT_TEXT position in the TAG_FIELD_SPECIFICATION o MAXLENGTH The number of bytes to return. o JUSTIFY The portion of text to retrieve relative to the matched text. CENTERED returns the text with approximately half preceding and half following the hit area. Other options are START and END, which return text beginning at the hit and the text concluding with the hit, respectively. o TAG_LIST A list indicating which tags to filter. The members of the list are separated by blanks. The keyword ALL may be used to indicate all tags for the collection. The server removes the named tags and any associated attributes. Figure B-1 illustrates how this function retrieves selected portions of search-results. The HIT_TEXT function sets the Field Status of the FSR to "1" whenever the text retrieved is the start of the , "2" when it is the end of the , and "3" when it is both the start and the end of the -------------------------------------------- Structured Full Text Query Language 53 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- . Otherwise, the Field Status is set to "0" (no status). As with the vertical document cursor, if a fetch is made out of the range of valid hits (whether by absolute or relative positioning), 100 is returned and the contents of the data field of the FSR are undefined. The text may include match-codes to mark the areas of the text where the search criteria were satisfied. The match-codes can be special characters or tags and are determined by the server. For more information on match- codes, see the MATCH_CODES and SHOW_MATCHES functions. Only one active HIT_TEXT or DATA function may be used for each in any given cursor. Return format. Dependent on the data type of TAG_FIELD_SPECIFICATION. Return value. A portion of text from the specified that pertains to the hit. -------------------------------------------- Structured Full Text Query Language 54 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- Figure B-1. Document-record versus hit traversal. B.2.6 INDEX Format. INDEX(TAG_LIST, START_TERM, DIRECTION, LIMIT, OPTIONS) Description. The INDEX function retrieves the terms from the index of the collection specified in the FROM clause. The terms retrieved commence from the specified term, and are returned according to the DIRECTION specified (FORWARD or BACKWARD). A maximum of LIMIT terms are returned. -------------------------------------------- Structured Full Text Query Language 55 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- o TAG_LIST A containing a list of the tags for which to return the index list. o START_TERM The text of the first term. An empty string ('') is specified to start at the beginning (or end, if DIRECTION is BACKWARD) of the list. o DIRECTION The order, relative to the start term, of the return list. The identifiers FORWARD and BACKWARD are the values that may be specified. o LIMIT The maximum number of terms to return. o OPTIONS The OPTIONS parameter is reserved but not currently used. Return format. Nested VLR with a set of ordered unlabeled FSRs. Return value. The nested VLR contains one FSR per term. Inside each FSR is another nested VLR with the following FSRs labeled by the term: o TAG_LIST A containing the list of the tags for which to return the index list. o TERM The text of the term in SZ format. o DOCUMENTS The number of documents in INT32 format. A return value of -1 indicates that this value is not available. o OCCURRENCES The total number of occurrences in all documents in INT32 format. A return value of -1 indicates that this value is not available. A value of 0 indicates that the term was in the stop word list. B.2.7 LIMITS Format. LIMITS() Description. The LIMITS function returns the implementation limits of the server. Return format. Nested VLR Return value. The table below lists the FSRs returned. In each, the Data Field contains the numeric limit that applies. -------------------------------------------- Structured Full Text Query Language 56 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- Field Label Type Explanation OPEN_CURSORS UINT32 Maximum open cursors permitted DECLARED_CURSORS UINT32 Maximum cursor declarations SFQLCA_LENGTH UINT32 Maximum SFQLCA length in kilobytes DOCUMENTS UINT32 Maximum number of document-records permitted in search-results. B.2.8 MANUFACTURER Format. MANUFACTURER() Description. The MANUFACTURER function returns the name of the manufacturer of the server, along with any necessary copyright or trademark notices. Return format. SZ. Return value. The name of the manufacturer and any copyright or trademark notices. B.2.9 MATCH_CODES Format. MATCH_CODES() Description. The MATCH_CODES function returns the codes (match-codes) used by the server to delimit the text matched by the query. Return format. Nested VLR. Return value. The first FSR is an SZ whose Field Label contains "START" and whose Data Field contains the characters that precede matched text (the start match code). The second FSR is an SZ whose Field Label contains "END" and whose Data Field contains the characters that follow the matched text (the end match code). B.2.10 RELEVANCE_METHOD Format. RELEVANCE_METHOD() Description. The RELEVANCE_METHOD function returns information about the method used by the server for relevance ranking. Return format. Nested VLR. -------------------------------------------- Structured Full Text Query Language 57 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- Return value. The server-specific features of relevance ranking are returned in the labeled FSRs shown in the table below.: Field Label Type Explanation DESCRIPTION SZ A description of the technique used by the server for relevance ranking. This is meant for optional presentation to the user by the client. ORDER char {H/L} H indicates that higher relevance rankings suggest a document that better matches the user's request. LOWEST UINT32 Lower end of numeric relevance range. HIGHEST UINT32 Upper end of numeric relevance range. B.2.11 SEARCH_OPTIMIZATION Format. SEARCH_OPTIMIZATION() Description. The SEARCH_OPTIMIZATION function returns a list of search capabilities with associated performance ratings. This function enables the client to select defaults and perform other tuning of client functions to achieve optimum performance. Return format. SZ. Return value. A ranking of the three options, where 1 is the slowest performance and 3 the highest. A return value of 0 means no rating. Bytes Rank Explanation 1 1-3 EXACTCASE 2 1-3 ANYCASE 3 1-3 FUZZY B.2.12 SFQLVERSION Format. SFQLVERSION() Description. The SFQLVERSION function returns the version number of SFQL supported by the server. Return format. SZ. -------------------------------------------- Structured Full Text Query Language 58 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- Return value. A string of the form MAJOR.MINOR.LEVEL, consisting of major (version) number, a minor (release) number, and a functionality level, each separated by periods. The functionality level allows several levels of SFQL standard to be implemented so that servers may support parts of the full standard. B.2.13 STOP_WORDS Format. STOP_WORDS() Description. The STOP_WORDS function returns a list of words contained in the collection that are not indexed. Return format. Nested VLR. Return value. If the collection has stop words, the VLR contains one FSR per term, each labeled with the term name. The FSRs are empty. If the collection has no stop words (all words are indexed), no VLR is returned. B.3. ACTION FUNCTIONS Action functions return on change the current state of a server option. An may be used in a or in a . In a , the functions are used without an argument to return the current setting of the option. In a , the functions change the setting to the argument specified and return the current setting following the change. Each returns a data structure specific to the function. B.3.1 LANGUAGE Format. LANGUAGE() or LANGUAGE(LANGUAGE_NAME) Description. The LANGUAGE function sets the national language in which error messages are returned. The LANGUAGE_NAME argument is a literal string specifying the language, as identified in an appropriate ISO standard to be determined. Return format. SZ. Return value. The language name as identified by an appropriate ISO standard to be determined. B.3.2 MATCH_METHOD Format. MATCH_METHOD() or MATCH_METHOD(TYPE) -------------------------------------------- Structured Full Text Query Language 59 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- Description. The MATCH_METHOD function selects one of the possible types of matching which may be in effect during queries: MATCH METHOD TYPES EXACTCASE matching is case-sensitive: "ME" matches only "ME". ANYCASE matching is not case-sensitive: "ME" matches "ME", "Me", "mE", and "me". FUZZY matching based on all linguistic forms of the query terms: "is" matches "is", "was", "were", and any other form of the verb "to be". DEFAULT matching based on one of the above methods, whichever is preset by the server. Fuzzy matching always implies case-insensitive matching (as if the ANYCASE function had been turned on): "ME" matches "ME", "Me", "mE", and "me". Other characteristics of fuzzy matching are server-defined, but can consist of one or more of the following techniques: verb stemming "is" matches "were" depluralization "mouse" matches "mice" possession flexibility "father" matches "father's" punctuation flexibility "house." matches "house?" transposition and separation "club members excepted" matches "except for club members" The COLLECTIONS collection indicates which of these characteristics are supported for FUZZY matching. For more details, see Appendix D. Return format. SZ. Return value. EXACTCASE, ANYCASE, or FUZZY. B.3.3 MAX_SEARCH_RECORDS Format. MAX_SEARCH_RECORDS() or MAX_SEARCH_RECORDS(NUMBER) Description. The MAX_SEARCH_RECORDS function sets the maximum search limits the server allows for any one search. NUMBERis the maximum number of document records in the search-results. This number must be less than the maximum number of document records allowed in a search-results indicated by the LIMITS function. -------------------------------------------- Structured Full Text Query Language 60 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- If the maximum number of document-records is exceeded, the server sends back a preliminary SFQLCA as if it had received a SUSPEND statement. The client can then choose to continue with the search or discontinue it with the SUSPEND or CLOSE statements. Note that some servers may not be able to RESUME after MAX_SEARCH_RECORDS has caused the search to be suspended. Return format. UINT32. Return value. The maximum number of document-records the server will return in search-results. B.3.4 MAX_SEARCH_TIME Format. MAX_SEARCH_TIME() or MAX_SEARCH_TIME(NUMBER) Description. The MAX_SEARCH_TIME function sets the maximum search time for any one search. NUMBERis the numerical limit to apply in milliseconds. If the maximum time is exceeded, the server sends back a preliminary SFQLCA as if it had received a SUSPEND statement. The client can then choose to continue with the search or discard it with the CLOSE statement. Note that some servers may not be able to RESUME after MAX_SEARCH_TIME has caused the search to be suspended. Return format. UINT32. Return value. The maximum time in milliseconds that the server will conduct a search. B.3.5 RESUME Format. RESUME() or RESUME(CURSOR_NAME) Description. The RESUME function causes the search associated with CURSOR_NAME to be continued from the point it stopped. At the end of the search, an SFQLCA is returned as though an OPEN cursor was executed and was never suspended. If CURSOR_NAME is not supplied, the last search operation is the target of the RESUME function, whether it was a direct-mode (noncursor) or cursor- based search. RESUME cannot be used in a . Return format. The SFQLCA which would have eventually been returned had the search not been SUSPENDED. -------------------------------------------- Structured Full Text Query Language 61 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- B.3.6 SERVER_REPORT_TIME Format. SERVER_REPORT_TIME() or SERVER_REPORT_TIME(INTERVAL) Description. The SERVER_REPORT_TIME function sets the time interval between status reports from the server to the client during a query. When interval is "0" (the initial default), the server does not send a preliminary SFQLCA to the client at any time during a query. When interval is greater than "0", it indicates the number of milliseconds between the return of each preliminary SFQLCA. Return format. UINT32. Return value. The current millisecond value. B.3.7 SFQLCA_LENGTH Format. SFQLCA_LENGTH() or SFQLCA_LENGTH(LENGTH) Description. The SFQLCA_LENGTH function sets the maximum SFQLCA length returned by the SFQL server, expressed in kilobytes. It must be greater than or equal to 1 kilobyte. Return format. UINT32. Return value. The current length. B.3.2 SHOW_MATCHES Format. SHOW_MATCHES() or SHOW_MATCHES(STATE) Description. The SHOW_MATCHES function is used to determine whether or not match-codes are returned in the text. STATE can be the values ON, OFF, TRUE, FALSE, or DEFAULT (where ON and TRUE cause match-codes to be returned, and DEFAULT sets the STATE back to the server-defined value). Return format. SZ. Return value. TRUE or FALSE. B.3.9 SUSPEND Format. SUSPEND() or SUSPEND (CURSOR_NAME) Description. The SUSPEND function requests suspension of the search. After a suspend, the cursor is considered open and documents may be fetched, if there were any hits. -------------------------------------------- Structured Full Text Query Language 62 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- If CURSOR_NAME is not supplied, the last search operation is the target of the SUSPEND request, whether it was a direct-mode (noncursor) or cursor- based search. Return format. A preliminary SFQLCA indicating the search status. -------------------------------------------- Structured Full Text Query Language 63 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- APPENDIX C: SFQLCA DEFINITION Table C-1. SFQLCA Definition NAME Length TYPE DESCRIPTION SFQLBORD 1 char Byte Order. 'L' for Least Significant 'M' for most significant byte first. SFQLPAD1 3 char Reserved. SFQLVERS 4 UINT32 SFQLCA Version Number. SFQL/SQL common fields: SFQLAID 8 char Always "SFQLCA" (used as an ID), padded on the right with blanks. SFQLCABC 4 UINT32 Indicates the length of this SFQLCA in bytes. Its size will vary from implementation to implementation. SFQLCODE 4 INT32 An integer return code. Its value is either: = 0 Execution was successful < 0 An error occurred > 0 The statement executed successfully, but an exception occurred. For example, the value 100 indicates that no data was found to satisfy the request. SFQLERRML 4 UINT32 If there is an error to report, this field must indicate the length of the text of the error message in the SFQLERRMC. If there is no message, this field is set to zero. SFQLERRMC 80 SZ Brief message if there is one. Worded for end-users. Used, for example, to provide a brief explanation of an exception or error code. For serious errors that return an ERROR_RECORD (SFQL only) a more detailed message will be contained in the USER_MESSAGE FSR. Note that developer oriented error messages are still contained in the ERROR_TEXT field of the ERROR_RECORD FSR. SFQLERRP 8 char Not currently in use -------------------------------------------- Structured Full Text Query Language 65 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- SFQLERRD[1] 4 UINT32 This and the following 5 fields are used to indicate the current state of the SFQL server. Not currently in use. SFQLERRD[2] 4 UINT32 Not currently in use. SFQLERRD[3] 4 UINT32 Indicates the number of rows processed by an insert/update operation. Currently used in SQL only. SFQLERRD[4] 4 UINT32 Not currently in use SFQLERRD[5] 4 UINT32 Not currently in use -------------------------------------------- Structured Full Text Query Language 66 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- Table C-1. SFQLCA Definition (cont.) NAME Length TYPE DESCRIPTION SFQLERRD[6] 4 UINT32 Not currently in use SFQLWARN0 1 char When set to "W" indicates a warning has been issued in one of the following 7 fields. SFQLWARN1 1 char When set to "W" indicates a warning. This field means that one or more returned character fields were truncated because the fixed format width was insufficient to fit the entire return data. SFQLWARN2 1 char When set to "W" indicates a warning. This field indicates that one or more null values were ignored in the computation of a function such as MIN or MAX. Currently used in SQL only. SFQLWARN3 1 char When set to "W" indicates a warning. Currently used in SQL only. SFQLWARN4 1 char When set to "W" indicates a warning. Currently used in SQL only. SFQLWARN5 1 char When set to "W" indicates a warning. Currently unused. SFQLWARN6 1 char When set to "W" indicates a warning. Currently used in SQL only. SFQLWARN7 1 char When set to "W" indicates a warning. Currently used in SQL only. SFQLEXT 8 char Not currently in use SFQL Only Fields: SFQLEXT1 1 char When set to "T" this field indicates that the server response includes SFQL-based SQL extensions (the following fields contain valid information). SFQLEXT2 1 char Reserved SFQLEXT3 1 char Reserved SFQLEXT4 1 char Reserved -------------------------------------------- Structured Full Text Query Language 67 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- SFQLROWS 4 UINT32 This field indicates the total number of document-records that contain matches to the query. It is equivalent to the number of SQL rows that can be retrieved from the query. SFQLCPOS 4 UINT32 This field indicates the current cursor position in the hit-set, starting with 1. -------------------------------------------- Structured Full Text Query Language 68 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- Table C-1. SFQLCA Definition (cont.) NAME Length TYPE DESCRIPTION SFQLHPOS 4 UINT32 Position of horizontal cursor (hit position in document record, starting with 1). SFQLCOMP 1 char A value of 'T' indicates that the server accepted a compromised form of the request (otherwise 'F'). All subsequent actions related to the request are also compromised and the results should be qualified to the user as being in need of cautious interpretation. See "Error Handling" on Page 38. SFQLFLAG2 1 char Reserved SFQLFLAG3 1 char Reserved SFQLFLAG4 1 char Reserved SFQLFLAG5 1 char Reserved SFQLFLAG6 1 char Reserved SFQLFLAG7 1 char Reserved SFQLFLAG8 1 char Reserved SFQLEXPAN 80 Reserved SFQLMSGLEN 4 UINT32 Length of the VLR in field SFQLMSG. SFQLMSG n varies This field contains the return message. -------------------------------------------- Structured Full Text Query Language 69 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- APPENDIX D: SCHEMA INFORMATION COLLECTIONS Schema information collections are standard collections that are used to supply specific information about the collections managed by the server. These collections can be retrieved by an application using the standard SFQL query semantics. The schema information collections include the following: COLLECTIONS TAG_FIELDS D.1 SCHEMA COLLECTION The schema collection contains a document-record for each schema known to the server. The collection contains: o SCHEMA_NAME This is an SZ and contains the type of schema that describes data to be found in the collection. The description will follow the official standards numbering scheme for ATA documents. For example, the ATA maintenance manual schema is designated "ATA 89-9C.MMSCHEMA- R3-1990". o DOCUMENT_DESCRIPTION This is an SZ which contains schema- specific data regarding document structure. For example, if the schema is based on SGML, the SGML DTD could be returned in this field. o STYLE This is an SZ which contains schema- specific data regarding document style. For example, if the schema is based on SGML, the SGML DSSSL could be returned in this field. -------------------------------------------- Structured Full Text Query Language 71 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- D.2 COLLECTIONS COLLECTION The collection COLLECTIONS contains a document-record for each collection accessible. The record contains fields describing the server options in effect for the specific collection. An individual document- record in this collection is uniquely identified by the two s, COLLECTION_NAME and TAG_FIELD_NAME. This collection must include the following s: o SCHEMA_NAME This is an SZ and contains the type of schema that describes data to be found in the collection. The description will follow the official standards numbering scheme for ATA documents. For example, the maintenance manual schema might be designated "89-9C.MMSCHEMA-R2- 1990". o COLLECTION_NAME This is an SZ and contains the name of the collection suitable for use in an SFQL . o ANYCASE This is an SZ and contains the value "TRUE" words can be matched without respect to their case. For example, a collection that supports any case matching can match the query term "me" with "ME", "Me", "mE", and "me". If ANYCASE matching is not supported, the value "FALSE" is returned. If ANYCASE match is supported, it can be enabled by selecting ANYCASE in the MATCH_METHOD function. o COLLECTION_REVISION This is an SZ and contains the revision number of the collection. o DEPLURALIZATION This is an SZ and contains the value "TRUE" if word forms are matched when they differ only in number. For example, a collection that supports depluralization can match the query term "mouse" with the word "mice" in the collection. If depluralization is not supported, the value "FALSE" is returned. -------------------------------------------- Structured Full Text Query Language 72 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- If depluralization is supported, it can be enabled by selecting FUZZY in the MATCH_METHOD function. o DESCRIPTION This is an SZ and contains a textual description to explain the contents of the collection suitable for presentation to the end- user. o DESCRIPTIVE_NAME This is an SZ and contains the a name of the collection suitable for presentation to the end-user. o EXACTCASE This is an SZ and contains the value "TRUE" if words can be matched exactly including their case. For example, a collection that supports exact case matching can match the query term "ME" only with "ME" and not with "me". If EXACTCASE is not supported, the value "FALSE" is returned. If EXACTCASE match is supported, it can be enabled by setting MATCH_METHOD to EXACTCASE. o POSSESSIVE_FLEXIBILITY This is an SZ and contains the value "TRUE" if word forms are matched when they differ only by possessive form. For example, a collection that supports possessive flexibility can match the query term "his" with the word "him" in the collection. If possessive flexibility is not supported, the value "FALSE" is returned. If POSSESSIVE__FLEXIBILITY is supported, it can be enabled by setting MATCH_METHOD to FUZZY. o PUNCTUATION_FLEXIBILITY This is an SZ and contains the value "TRUE" if word forms are matched when they differ only by punctuation. For example, a collection that supports punctuation flexibility can match the query term "house." with the word "house?" in the collection. If punctuation flexibility is not supported, the value "FALSE" is returned. If PUNCTUATION_FLEXIBILITY is supported, it can be enabled by setting MATCH_METHOD to FUZZY. -------------------------------------------- Structured Full Text Query Language 73 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- o STOP_WORDS This is an SZ and contains the value "TRUE" if there are words in the collection that were not indexed by the server. If all words were indexed, the value "FALSE" is found in the . If STOP_WORDS are supported, they can be returned by the INDEX function (entries with zero occurrences) or by the STOP_WORDS function. o SYNONYM This is an SZ and contains the value "TRUE" if the THESAURUS function supports the SYNONYM operator. For example, a collection that supports SYNONYM can match the query term "house" with the word "home" in the collection. If the SYNONYM operator is not supported, the value "FALSE" is returned. o BROADEN This is an SZ and contains the value "TRUE" if the THESAURUS function supports the BROADEN operator. For example, a collection that supports BROADEN can match the query term "cat" with the word "animal" in the collection. If the BROADEN operator is not supported, the value "FALSE" is returned. o NARROW This is an SZ and contains the value "TRUE" if the THESAURUS function supports the NARROW operator. For example, a collection that supports NARROW can match the query term "gas" with the word "oxygen" in the collection. If the NARROW operator is not supported, the value "FALSE" is returned. o TRANSPOSITION_AND_SEPARATION This is an SZ and contains the value "TRUE" if phrases are matched when they differ by transposed or separated words. For example, a collection that supports TRANSPOSITION_AND_SEPARATION can match the query term "except for club members" with phrase "club members excepted" in the collection. If TRANSPOSITION_AND_SEPARATION is not supported, the value "FALSE" is returned. -------------------------------------------- Structured Full Text Query Language 74 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- If TRANSPOSITION_AND_SEPARATION is supported, it is turned on and off with the other characteristics of FUZZY matching by the MATCH_METHOD function. o VERB_STEMMING This is an SZ and contains the value "TRUE" if verb forms are matched when they share a common stem. For example, a collection that supports verb stemming can match the query term "is" with the word "were" in the collection. If VERB_STEMMING is not supported, the value "FALSE" is returned. If VERB_STEMMING is supported, it is turned on and off with the other characteristics of FUZZY matching by the MATCH_METHOD function. o WILDCARD This is an SZ containing a three position character map representing the possible position of wildcards. The characters in the map may be: o N Neither wildcard is supported in the position o B Both wildcards are supported in the position o % Only the % wildcard is supported in the position o _ Only the _ wildcard is supported in the position The three positions of the character map represent whether the operation is supported for PREFIX, EMBEDDED, or SUFFIX usage. o PROXIMITY This is a SZ and contains one or more of the following values, separated by blanks: NONE if no proximity operators are supported CHARACTERS if proximity can be expressed in number of characters WORDS if proximity can be expressed in number of server- defined words SENTENCES if proximity can be expressed in number of server- defined sentences PARAGRAPHS if proximity can be expressed in number of server- defined paragraphs -------------------------------------------- Structured Full Text Query Language 75 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- D.3 TAG_FIELDS COLLECTION The collection TAG_FIELDS contains a description of the s accessible by the SFQL server. An individual document- record in this collection is uniquely identified by the three s, SCHEMA_NAME, COLLECTION_NAME, and TAG_FIELD_NAME. This collection must include the following s: o SCHEMA_NAME This is an SZ and contains the type of schema that describes data to be found in the collection. The description will follow the official standards numbering scheme for ATA documents. For example, the maintenance manual schema might be designated "89-9C.MMSCHEMA-R2- 1990". o COLLECTION_NAME This is an SZ and contains the name of the collection suitable for use in a SFQL . o TAG_FIELD_NAME This is an SZ and contains the name of the . TAG_FIELD_NAMEs are unique only within the collection. o ATTRIBUTE_OF This is an SZ which, for attributes, contains the name of the encompassing element. For elements, this is empty. o SYMBOL This is an SZ and contains the ASCII string used to represent the tag in the collection. o FIELD_TYPE This is an SZ and defines the for TAG_FIELD_NAME. The valid values are defined in Table 2 on Page 34. -------------------------------------------- Structured Full Text Query Language 76 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- o FIELD_TYPE_VERSION This is an SZ and contains the field- type-version identifier for the field. The valid values are defined in Table 2 on Page 34. o FIELD_DESCRIPTION This is an SZ and contains a textual description to explain the meaning of the . -------------------------------------------- Structured Full Text Query Language 77 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- APPENDIX E: ERROR CODE DIRECTORY This appendix contains the list of values for the SFQLCA element, SFQLCODE. Table E-1. SFQLCODE categories and reserved values. SFQLCODE DESCRIPTION Normal Condition 0 No errors or exceptions Exception Conditions (SFQLCODE > 0) 100 A FETCH was issued for which there was no row (document-record); OR A SELECT was issued for which no rows were found. 200-203 This SFQLCA is a preliminary SFQLCA. The reason is: 200 A SUSPEND has been executed; 201 MAX_SEARCH_TIME was exceeded; 202 MAX_SEARCH_RECORDS was exceeded; 203 The SERVER_REPORT_TIME interval has elapsed. 210 exception. Problem may be one or more of the following: o UNSUPPORTED-TAG. In this case a nested VLR with an ERROR_RECORD will be returned in place of the missing fields. o UNSUPPORTED-FUNCTION. The results are returned without processing them through the desired function. Any Other Positive Value Exception. A recoverable error was encountered. The data is returned but is of questionable quality. For example, the server may not support the operation (e.g., a compromise) and might return positive 20002. This means that the server tried to accomplish the request. See Table E-3. -------------------------------------------- Structured Full Text Query Language 79 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- Error Conditions (SFQLCODE < 0) Any Negative Value The command was not performed. An ERROR_RECORD may be found in the SFQLMSG area. See Table E-2. -------------------------------------------- Structured Full Text Query Language 80 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- Table E-2. Standard SFQLCODEs for error conditions. SFQLCODE MIN MAX DESCRIPTION -1000 -19999 Reserved for server-specific errors -20000 -20000 Recoverable but unknown error (contact the vendor) -20001 -20001 Unrecoverable unknown error (server exited) -20002 -20002 Operation not supported by this server/operating system -20003 -20003 Command level not supported by server -20004 -20014 Reserved -20015 -20015 does not exist in specified collection -20016 -20016 Collection does not exist -20017 -20017 Volume does not exist -20018 -20018 User does not exist -20019 -20044 Reserved -20045 -20045 Operating system file size limit has been exceeded -20046 -20046 The collection is unreadable -20047 -20047 A file is unreadable -20048 -20048 The disc is unreadable -20049 -20074 Reserved -20075 -20075 Could not complete the query--resource limitations -20076 -20076 Could not complete the query--other reasons -20077 -20102 Reserved -20103 -20103 Max declared cursors exceeded -20104 -20104 Max open cursors exceeded -20105 -20105 Attempt to close an unopened cursor -20106 -20106 Attempt to open an open cursor -20107 -20107 Undefined cursor for FETCH, OPEN, CLOSE -20108 -20208 Reserved -20209 -20209 Syntax Error (unknown cause) -20210 -20210 Unknown clause -20211 -20211 Unknown function -20212 -20212 Bad argument to function -20213 -20213 Wrong number of function arguments -20214 -20214 Missing clause in command -20215 -20215 Missing argument in command clause -20216 -20216 Unknown predicate operator -20217 -20217 Unbalanced parentheses -20218 -20218 Missing semicolon terminator -20219 -20219 Bad argument to command clause -20220 -n Reserved -------------------------------------------- Structured Full Text Query Language 81 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- Table E-3. Standard SFQLCODEs for exception conditions. SFQLCODE MIN MAX DESCRIPTION 1000 19999 Reserved for server-specific errors 20000 20219 See negative value interpretation above; in this case the command was at least partly executed and results are available. 20220 n Reserved -------------------------------------------- Structured Full Text Query Language 82 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- APPENDIX F: GLOSSARY client (SFQL) The user interface software which inputs user requests and which formats and presents data it receives from the SFQL "server." collection A set of document-records, analogous to an SQL table, which may permanently reside in a database or may be the result of a search. depluralization The ability to accept query terms in either singular or plural form and match both forms. document-record All or part of a document, analogous to an SQL row. Documents are comprised of a meaningful amount of text, analogous to a paper document. FSR (Field Subrecord) A subunit of a VLR which contains no more than one of the items specified in the of a SELECT or a FETCH. search-results A set of documents. The client can request documents from this set by using the cursor based FETCH command. match-codes Strings inserted into returned text by the server to indicate where the hits are found. The client can use this information to perform user interface functions, such as highlighting the hits in a different font. message A communication between the client and the SFQL server. SFQL commands are passed as messages from the client to the server. Status and data are passed back as messages from the server to the client. schema A logical representation of data, oriented towards a particular database management system approach, e.g., a data model for a database in SQL or SFQL. server (SFQL) The database engine software which searches and provides information at the request of the client. SFQL Structured Full-Text Query Language. A subset of the SQL language, extended to allow access to full- text databases. -------------------------------------------- Structured Full Text Query Language 83 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- stemming The ability of a server to match verb forms that share a common stem. SZ Data returned as a simple string by the server. Like a VLR, the string may be made up of text from several tag-fields of the collection. Unlike a VLR, however, the fields are not labeled in an SZ record. As such, to find the data in one field versus another, the caller would have to know the length of each field. SQL Structured Query Language. An algebra for manipulating data under the relational model. An invention of IBM, it is now an ANSI standard. It is currently under consideration as an ISO standard. tag A label in the text that is used to identify the start or end of a logical zone, or to indicate formatting in the text. SFQL does not care what tagging scheme is used in the text or even if there is tagging scheme in the text. The Structured Generalized Markup Language is an example of a tagging methodology that is compatible with SFQL. tag-field A field in an SFQL collection analogous to a SQL column. VLR (Variable Length Record) A data structure allowing multiple data objects of varying length to be returned. Each object is in the form of an FSR, and is labeled with the SFQL language element which produced it. For example, if a requested LEFT(author, 6) the FSR header would have this entire function and its argument. Each FSR header also has an offset to the next FSR. -------------------------------------------- Structured Full Text Query Language 84 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- INDEX %..............................................................10 ' '................................................11 <..............................................................17 <=.............................................................17 <>.............................................................17 ..............................................24 ...............................................14 ....................................................14 ....................................................14 .....................................12 ............................................17 ...............................................17 ..............................................17 .................................................17 ...........................................7, 8, 9, 11 ..............................................21 ...................................................23 ......................................12, 14, 16 ...............................................16 .........................................13 ......................................................11 ......................................................17 .........................................17 ...........................................18 ..............................................18 .........................................18 .......................................17, 18 .........................................18, 19 ................................................18 .....................................12, 14, 16 ..........................................13, 20, 21 .....................................20, 21 .........................................................23 ..............................................9, 10 ..............................................10 ...................................................10 ...............................................20 ..............................................11 .................................................8, 10, 11 ....................................................23 .................................................18, 19 ................................................19 ..........................................9, 11 .................................................9 .......................................................15 .....................................................23 ............................................21 -------------------------------------------- Structured Full Text Query Language 85 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- ................................................21 .......................................................23 ..................................................16 .........................................19, 22 ..............................................16 ................................................16 .......................................14 .....................................................23 .........................................................22 .......................................11, 12, 13, 14 .................................................17 ................................................18 ...........................................18 ........................................................23 .....................................................11 .....................................................24 ........................................................8 ..............................................11 ...........................................17, 19 .......................................................23 ................................................8, 9, 13 .............................................8 .................................................23 ..................................................23 .................................................24 ...........................................24 ..............................................25 .................................................10 ......................................................11 ...........................................11 ...........................................17, 19 ...............................................9 ...............................................21 ..............................................20 .................................................15 ...................................................9, 19 .............................................9 ....................................................17 ......................................................15 ..........................................18 ................................................13, 14 ......................................15, 20 ...................................................20 ....................................................22 ...........................................20 .............................................23 .......................................................25 ...............................................12 .........................................16, 17 ..........................................24 ..............................................15, 21 ....................................................11 -------------------------------------------- Structured Full Text Query Language 86 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- ...........................................25 ...............................................24 .....................................................24 ................................................7 ................................................25 ..................................................24 .................................................25 ...........................................20 ........................................................11 .............................................8 .........................................15, 23 .............................................13, 14 ..........................................24 ...................................................24 ..............................................18, 19 .............................................19 ............................................9, 20 ..........................................9, 10 ........................................18 ......................................................25 .........................................15, 16 ...................................................12 ................................................14, 16 ..........................13, 15, 16, 19 .....................................................23 ...............................................23 .................................................13, 14 .........................................................15 ....................................................22 ........................................................10 ...................................................11 ..................................9, 10, 19, 20 .........................................8, 11 .................................15, 17, 21, 33 ......................................13, 15 .............................................15, 16 ..................................................10 >..............................................................17 >=.............................................................17 _..........................................................10, 11 ABSOLUTE...............................................21, 50, 53 Action Functions...........................................24, 59 ANSI............................................................6 ANSI X3.135-1986................................................3 ANYCASE....................................................60, 72 ASC............................................................20 ASCII_carriage_return..........................................11 ASCII_line_feed................................................11 ATTRIBUTE_OF...................................................76 BROADEN....................................................48, 74 CGM............................................................34 Character Strings...............................................4 CHARACTERS.................................................19, 75 Client.........................................................83 -------------------------------------------- Structured Full Text Query Language 87 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- Client/Server Interaction......................................27 CLOSE..........................................................21 COLLECTION.................................................49, 83 Collection Information.........................................40 COLLECTION_NAME............................................72, 76 COLLECTION_REVISION............................................72 Collections.................................................6, 71 COLLECTIONS COLLECTION.........................................72 Column..........................................................6 Communications Area............................................27 Computer Graphics Metafile.....................................34 CURSOR FOR.....................................................20 Cursor Operations..............................................20 DATA...........................................................50 Data Field.................................................32, 33 Data Message Format............................................27 Data Offset....................................................32 Data Return....................................................27 Data Types......................................................4 DATA_RECORD....................................................29 Database Model..................................................5 DATE...........................................................10 Date/Time......................................................34 DATE1..........................................................34 Dates...........................................................5 DECLARE........................................................20 DECLARED_CURSORS...............................................57 DEFAULT........................................................60 DEPLURALIZATION............................................72, 83 DESC...........................................................20 DESCRIPTION................................................58, 73 DESCRIPTIVE_NAME...............................................73 DOCUMENT.......................................................14 Document-record............................................20, 83 Document-records................................................5 DOCUMENT_DESCRIPTION...........................................71 DOCUMENTS......................................................57 Embedded SFQL...................................................4 Embedded SQL....................................................4 Error Handling.................................................38 Error message..................................................27 ERROR_RECORD...................................................29 ESCAPE..........................................................9 EXACTCASE..................................................60, 73 Exception handling.............................................41 Exceptions.....................................................38 FEATURES...................................................40, 51 FETCH..........................................................21 Field Label................................................32, 33 Field Position.................................................32 Field Status...................................................32 Field Subrecord............................................32, 83 Field Type.....................................................32 Field Type Version.............................................32 -------------------------------------------- Structured Full Text Query Language 88 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- Field Type Version Identifier..................................36 FIELD_DESCRIPTION..............................................77 FIELD_TYPE.....................................................76 FIELD_TYPE_VERSION.............................................77 FIELDS.........................................................21 File formats....................................................1 FILTER.........................................................51 FIRST..................................................21, 50, 53 Floating Point.................................................35 Formatted String...............................................36 Formatted Text.................................................36 Four-byte Signed Integer.......................................36 FP.............................................................35 FROM...........................................................16 FSR........................................................32, 83 FSR Offset.....................................................32 FUZZY..........................................................60 General Functions..............................................22 GROUP BY.......................................................16 HAVING.........................................................16 HIGHEST........................................................58 HIGHLIGHT_TERMS................................................47 HIT_TERMS......................................................47 HIT_TEXT.......................................................52 HITS...........................................................47 IN_ORDER.......................................................18 INDEX..........................................................55 INT16..........................................................36 INT32..........................................................36 Integer.....................................................4, 36 Integrity constraints...........................................4 ISO............................................................21 ISO 8859.......................................................36 ISO 9660........................................................1 Joins...........................................................3 Label Offset...................................................32 LANGUAGE.......................................................59 LAST...................................................21, 50, 53 LIKE...........................................................19 Limitations.....................................................3 LIMITS.....................................................40, 56 LOWEST.........................................................58 MANUFACTURER...............................................40, 57 Match-codes................................................36, 83 MATCH_CODES............................................36, 40, 57 MATCH_METHOD...................................................59 MAX_SEARCH_RECORDS.............................................60 MAX_SEARCH_TIME............................................21, 61 Message........................................................83 Modules.........................................................4 Names..........................................................12 NARROW.....................................................48, 74 Nested Variable Length Record..................................36 Nested VLR.....................................................36 -------------------------------------------- Structured Full Text Query Language 89 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- NEXT...................................................21, 50, 53 NOT............................................................19 Notation........................................................6 NULL...........................................................19 Numbers.........................................................4 OF.............................................................18 OPEN...........................................................21 OPEN_CURSORS...................................................57 ORDER..........................................................58 ORDER BY.......................................................20 PARAGRAPHS.................................................19, 75 POSSESSIVE_FLEXIBILITY.........................................73 PRELIMINARY SFQLCA.........................................21, 38 PRIOR..................................................21, 50, 53 Procedures......................................................4 PROXIMITY......................................................75 PUNCTUATION_FLEXIBILITY........................................73 QUERY_TERMS....................................................47 RELATIVE...............................................21, 50, 53 RELEVANCE..................................................20, 49 RELEVANCE_METHOD...............................................57 RESUME.........................................................61 Root-node.......................................................6 Schema.........................................................83 SCHEMA COLLECTION..............................................71 Schema Data Definition Language.................................3 SCHEMA_NAME............................................71, 72, 76 SCHEMA_NAME, COLLECTION_NAME...................................76 Schemas.........................................................6 Search Conditions..............................................16 Search-results.............................................16, 83 SEARCH_OPTIMIZATION............................................58 Security........................................................4 SENTENCES..................................................19, 75 Server.........................................................83 Server Capabilities............................................40 Server Compatibility Model.....................................40 Server Differences.............................................41 SERVER_REPORT_TIME.............................................62 SFQL........................................................2, 83 SFQL Communications Area.......................................27 SFQL Grammar....................................................7 SFQL Statement..................................................7 SFQL_ERROR.....................................................39 SFQLAID........................................................65 SFQLBORD.......................................................65 SFQLCA.....................................................27, 79 SFQLCA Header..................................................27 SFQLCA_LENGTH..............................................57, 62 SFQLCABC.......................................................65 SFQLCODE...................................................65, 79 SFQLCOMP...................................................41, 69 SFQLCPOS.......................................................68 SFQLERRD[1]....................................................66 -------------------------------------------- Structured Full Text Query Language 90 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 ---------------------------------- SFQLERRD[2]....................................................66 SFQLERRD[4]....................................................66 SFQLERRD[5]....................................................66 SFQLERRD[6]....................................................67 SFQLERRMC..............................................39, 41, 65 SFQLERRML......................................................65 SFQLERRP.......................................................65 SFQLEXPAN......................................................69 SFQLEXT........................................................67 SFQLEXT1.......................................................67 SFQLEXT2.......................................................67 SFQLEXT3.......................................................67 SFQLEXT4.......................................................67 SFQLFLAG2......................................................69 SFQLFLAG3......................................................69 SFQLFLAG4......................................................69 SFQLFLAG5......................................................69 SFQLFLAG6......................................................69 SFQLFLAG7......................................................69 SFQLFLAG8......................................................69 SFQLHPOS.......................................................69 SFQLMSG....................................................27, 69 SFQLMSG Area...................................................29 SFQLMSGLEN.....................................................69 SFQLPAD1.......................................................65 SFQLROWS.......................................................68 SFQLVERS.......................................................65 SFQLVERSION................................................40, 58 SFQLWARN0......................................................67 SFQLWARN1......................................................67 SFQLWARN2......................................................67 SFQLWARN3......................................................67 SFQLWARN4......................................................67 SFQLWARN5......................................................67 SFQLWARN6......................................................67 SFQLWARN7......................................................67 SGML...........................................................36 SHOW_MATCHES...................................................62 Special Functions..............................................22 SQL......................................................3, 6, 84 SQL 3..........................................................12 SQL Common Elements.............................................7 SQL3...........................................................21 SQLCODE........................................................39 Standard Return Data Types.....................................33 Status.........................................................27 Stemming.......................................................84 STOP_WORDS.................................................59, 74 STYLE..........................................................71 Subqueries......................................................4 Subselects......................................................4 SUSPEND....................................................21, 62 SYMBOL.........................................................76 SYNONYM....................................................48, 74 -------------------------------------------- Structured Full Text Query Language 91 ........................................................................... Draft Standard 89-9C.SFQL2-R1-1990 SZ.........................................................36, 84 Table...........................................................6 Tag............................................................84 Tag-Field...................................................6, 84 TAG_FIELD_NAME.............................................72, 76 TAG_FIELDS.................................................71, 76 TAG_FIELDS COLLECTION..........................................76 Tagged Image File Format.......................................36 TEXT...........................................................36 THESAURUS......................................................48 TIFF...........................................................36 Transaction capability..........................................3 TRANSPOSITION_AND_SEPARATION...................................74 Two-byte Signed Integer........................................36 UINT32.........................................................36 Unrecoverable error............................................29 Unsigned.......................................................36 Update capability...............................................3 USER_ERROR.....................................................39 Variable Length Record.........................................84 Vendor Extensions..............................................37 VERB_STEMMING..................................................75 Vertical bar character.........................................18 Views...........................................................3 VLR........................................................29, 84 VLR header.....................................................29 WHERE..........................................................16 WILDCARD.......................................................75 WITHIN.........................................................18 WORDS......................................................19, 75 -------------------------------------------- Structured Full Text Query Language 92