A "Distributed Database" client is a fairly common class of real estate software that is preferred by many real estate professionals. This class of client software fundamentally requires the ability to maintain an up-to-date, local copy of the entire (or a cross section) of the MLS database in order to support off-line processing and searching. Currently the RETS standard does not specifically deal with this type of functionality. Under the current standard, a "DDB" client must make certain assumptions about the server’s schema and/or take advantage of proprietary server implementations in order to provide the fundamental functionality of a distributed client. Furthermore, DDB implementations that use combinations of the traditional RETS Search transaction may be causing serious inefficiencies on the server side.
- A DDB client must make the assumption that there exists a field like "LastTransactionDate" in the RETS servers' schema, in order to fetch data incrementally. However, LastTransactionDate is NOT a required field in the RETS specification.
- A DDB client must make the assumption that there exists a field like "LastImageTransactionDate" in the RETS servers' schema, in order to fetch images incrementally. However, LastImageTransactionDate is NOT a required field in the RETS specification.
- Since deleted records are not returned by a typical RETS query, a DDB client may be extremely inefficient in determining which records have been deleted from the system or reclassified to statuses that fall outside of the DDB criteria.
- When downloading into a distributed database, a DDB client often will violate a server's "maximum search result" limit that has been established by the MLS board. In section 7.4.3 the RETS spec states: 'Client implementers should be aware that some server implementations might not honor the request to disable the limit.' This proposal would require servers that do support the DDB transaction set MUST NOT enforce record limits on the results. Servers would, however, have some other means of limiting DDB requests.
This proposal is intended to address the above difficulties as well as provide a far more efficient means for supporting a DDB client.
3. DDB Transaction Proposal
3.1 Request Format
3.1.1 Search Type and Class
The SearchType and Class arguments specify the data that the server is to search.
SearchType ::= ResourceID
The type of search to perform as discussed in Section 7.1 and defined in the Metadata (see section 11.2.2).
Class :: = 1*32ALPHANUM
This parameter is set to a value that represents the class of data within the SearchType, taken from the Class metadata (section 11.3.1). If the resource represented by the SearchType has no classes, the Class parameter will be ignored by the server and MAY be omitted by the client. If the client does include the Class parameter for a classless search, the value SHOULD be the same as the ResourceID in order to insure forward compatibility.
3.1.2
The specification consists of the query itself together with a designation of the query language.
Query ::= <The query to be executed by the server>
This is the query that specifies the subset of the server’s database that is being maintained in the DDB for this Class. The query is specified in the language described in Section 7.7.
QueryType ::= DMQL2
An enumeration giving the language in which the query is presented. The only valid value for RETS/1.5 is "DMQL2" which indicates the query language described in Section 7.7
3.1.3 Last Successful DDB Update Timestamp
This optional argument specifies the time of the last successful DDB update. If this argument is provided, then the response will only include records that have actually been affected or changed since that timestamp. If this argument is not provided, then the response will include all appropriate records, suitable for the complete initialization of the DDB. It is HIGHLY RECOMMENDED that clients actually provide the LastUpdateDate whenever possible. This will increase efficiency on the server and may avoid server-side throttling mechanisms.
LastUpdateDate ::= <the date of last previous successful DDB update>
3.1.4 Field Delimiter
A server may designate a particular OCTET to be used as a delimiter for separating entries in the DATA returned by the response. The octet should be chosen to avoid the need to escape data within a record. A tab character (an octet with a binary value of 9) is the default delimiter unless another is specified as part of the transaction.
field-delimiter ::= HEX HEX
Response Format
3.2.1 <DDB-ACTIVITY> tag
This tag wraps the DDB transaction response.
Attributes:
- Class - Simply echoes the requested class.
- Date - tells the client the date through which the DDB activity is being reported. This date SHOULD be stored on the client and SHOULD be submitted as the LastUpdateDate in the subsequent DDB transaction request. The server MUST guarantee that if the client submits this date upon its subsequent DDB request, that NO transactions will be missed.
- ReplyCode - Result status of the request. See section 3.2.6.
3.2.2 <DDB-TRANSACTION> tag
This tag may be repeated up to four times, each for different types of transactions. Each instance of this tag will also contain a <DATA> tag, which will contain the appropriate KeyField values. The KeyField values in the <DATA> section will be delimited by the field-delimiter as specified in the request (see section 3.1.4). If a particular section (Type) of DDB activity contains no records, then that instance of the <DDB-TRANSACTION> tag may be omitted completely.
Attributes:
- Count – Identifies how many records (KeyFields) are contained within this section.
- Type – Specifies what type of DDB transaction is being returned:
- ReplyCode - Result status of the request. See section 3.2.6.
DeletedRecord | Records that have been deleted from the system, since the given LastUpdateDate parameter, given the specified Query. |
ChangedRecord | Records that have been modified in the system, since the given LastUpdateDate parameter, given the specified Query. |
ChangedImage | Records that have had changes (additions/modifications/deletions) to their associated images, since the given LastUpdateDate parameter, given t-e specified Query. |
NoLongerMatch | Records that no longer match the criteria specified in the Query, since the given LastUpdateDate parameter. This means records that have “fallen out” of the specified criteria since the provided date. |
3.2.3 Sample DDB Response
<DDB-ACTIVITY ReplyCode = “0” Class = “RESIDENTIAL” Date="Sat, 20 Mar 2002
12:03:38 GMT" >
<DDB-TRANSACTION Type= “DeletedRecord” Count = “4”>
<DATA>123111|123222|123333|123444</DATA>
</DDB-TRANSACTION>
<DDB-TRANSACTION Type= “ChangedRecord” Count = “3”>
<DATA>234111|234222|234333</DATA>
</ DDB-TRANSACTION>
< DDB-TRANSACTION Type = “ChangedImage” Count = “3”>
<DATA>111111|111222|111333</DATA>
</ DDB-TRANSACTION>
< DDB-TRANSACTION Type = “NoLongerMatch” Count = “3”>
<DATA>222111|222222|222333</DATA>
</ DDB-TRANSACTION>
</DDB-ACTIVITY>
|
3.2.4 Limitations
The DDB response will only provide the KeyField of the relevant records. Subsequent interactions would be required to fetch the data or images based on the KeyField values.
3.2.5 Throttling
The use of this new transaction would imply that the client wishes to exceed the search result limitations of the server. The server MUST either allow this, or must fail the transaction altogether. Other forms of throttling would be allowed on the server. For example, a server may restrict the frequency of DDB transactions from a particular client, or may restrict such transactions during certain hours of the day. Perhaps DDB requests that do not provide a LastUpdateDate argument would be more restricted than those that do provide a LastUpdateDate.
Note: See section 3.2.6 for reply codes that reflect throttling. There may be a need for an additional transaction for the client to query the server for details of DDB transaction restrictions.
3.2.6 Reply Codes and Errors
| Reply Code | Meaning |
| 0 | Operation Successful |
| 20801 | Server does not support DDB transaction. |
| 20802 | Unable to complete DDB transaction at this time (throttling restriction, try again later). |
| 20803 | DDB transactions not supported for this resource type. |
| 20804 | Invalid Search Criteria Field. |
| 20805 | No DDB Activity since specified LastUpdateDate. |
|