RETS Working Group Meeting: April 29-30, 2003


Search

The first part of the event was intended to be a meeting of the Compliance Workgroup. However, almost everyone attended the entire meeting, so the compliance topic became one for the entire working group.

Compliance Discussion

Bruce Toback began with a presentation on the current state of the compliance structure, and proposed a model for going forward. The key features of the model:

There will be five kinds of compliance certification:
Client search
This is any client that does not support RETS-compliant upload.
Client broker-load
A client that does support RETS-compliant upload
Server
A server, regardless of functionality.
Site
A site -- that is, a RETS server advertising a particular set of capabilities and metadata.

Note that it's possible for a client to do searches in a RETS-compliant manner but uploads in a proprietary manner. Such a client could qualify for the RETS Search certification but not for the RETS Broker Load certification.

Certification will be via published test suites that offer a pass/fail assessment.
Following the USB and FireWire models, certification will be available either at plugfests (as part of the festivities), or via a certified third party in those instances where a vendor needs certification between plugfests. The certification requirements are precisely the same in either case; the "trusted third party" is trusted only insofar as they are allowed to run the standard test suite and certify compliance in a non-public setting.
Plugfests will be held three times per year, concurrent with working group general meetings.
Compliance-testing servers will be running at rets.org and other places, in addition to being available for in-house testing.
The logo design includes the version of RETS with which the displaying product is compliant.
The retsinfo.com site will maintain a publicly-queriable list of compliant products/versions.
Deadlilne for operation of the compliance program: November 7, 2003, driven by the San Francisco NAR convention. Timeline is as follows:
June 1: Alpha client and server test suites published.
July 14: Final logo usage guidelines published. Alpha site test suite published.
August 11: Beta test suites published; plugfest week.
October 1: Logo program starts; logos awarded.
November 7: The Big Demo
San Francisco demo event to include live, on-line demos of RETS-compliant products, possibly in an adjunct NAR booth.
Two issues are potential roadblocks: the lack of standard names and the the fact that many clients break when metadata changes.

Dealing With Metadata Changes

This topic generated considerable discussion relative to compliance testing. What does it mean to "deal with" metadata changes? The discussion alternatives ranged from "break" to "reconfigure automatically". Some of the alternatives discussed included "a client must load new metadata after a change if the metadata is exposed to the user in any way" (rejected because it's too difficult to decide what's meant by "exposed", "a client will not display incorrect information after a metadata change" (rejected because of the difficulty in deciding whether a particular display was incorrect based on the metadata), and "a client will display an error message if it detects a metadata change it can't deal with" (rejected because it's not clear what it means to display an error message if the client is an automated system), among others.

The alternative agreed to, without much enthusiasm but without dissent, is that a client, upon seeing a metadata change, will not submit a transaction using the old metadata. Many thought this insufficient, but all agreed that this was a minimum necessary criterion. It is also machine-testable, which HI changes are not.

There was a suggestion to have a standard set of changes that would be tested. This was not adopted (though the issue is moot if the changes are part of a test script, which must, according to the certification rules, be published).

One person brought up the issue of metadata changes that occur during a session, and suggested that an error code be added for "metadata changed." There were several minutes' discussion on whether this would be necessary: major changes may well happen only during system downtime, why return an error code to a client that's not interested in metadata changes, whether the code should be returned once or until login, and so on. Several server vendors said that they just terminate the session. No resolution was reached on these issues.

Release Timing

Bruce Toback instigated a discussion on release naming and timing. Right now, the concept of the current release is somewhat fuzzy. The intent is to clean this up in order to offer a well-defined peg for contract language. (One vendor noted that their contracts actually reference the specification document URL rather than a version.)

After some discussion, it was agreed that in the absence of the discovery of a major error in the specification, there would be one official release per year, on November 1. The test suite for that release would be finalized as of September 1 of that year, giving time for a certification cycle before the annual NAR convention.

In addition, change proposals from this meeting forward will include the release version with which they become effective. This allows longer implementation time for more complex changes.

Change Proposals

Proposal 30: Hotsheet

This proposal resulted in considerable discussion, detailed below. The proposal was not approved (no vote was called) because the proposal offered was viewed as too specific to a single architecture and single hotsheet design. As noted below, an ad-hoc workgroup will work on this for the next meeting.

Proposal 31: Update Warning Block

Adopted with 13 in favor, two opposed. The main discussion was centered on the idea that this proposal seems to mandate real-time communication with a user, and some applications may have no user with whom to communicate in real-time. The proposal was adopted for 1.8.

Proposal 32: Distributed Data Base

Adopted unanimously, with minor alterations, with the adoption split into two parts: 1.8 for most of the proposal, but 1.7 for the ServerInformation transaction.

The proposal was modified by the group to use foreign key names instead of IDs in the distributed data base metadata. There was also a general shift to the use of the persistent ID (see RCP028) rather than the system name in future metadata structures.

Proposal 33: Validation Expressions

Adopted unanimously, with one change, effective for RETS 1.8. The one change is the use of the Persistent ID field instead of the SystemName. The group agreed to this change; however, during editing, several issues arose that will require more discussion.

Additional Change Proposal Topics

Several members of the working group noted a need for a media upload transaction. The first suggestion was simply a POST transaction to the GetObject URL, but it was quickly realized that something much richer was needed. There was extensive discussion of several options that can be summarized into two categories:

A separate media upload transaction
Using the current update transaction with a well-known resource type

The group identified a number of issues specific to media upload even if the normal update path is taken. The server may well have a maximum image size, and it would be useful to somehow communicate this to a client. The ServerInformation transaction, added with the distributed data base enhancements, was suggested as a vehicle for this. There may also be a maximum media file size; this, too, should be communicated, as should a list of supported MIME types.

There are other size issues as well: the server may have a policy of modifying nonconforming images (scaling and cropping them to fit the MLS's standard), but the user may not want that. It was suggested that a flag be passed up with the media indicating whether such resizing and cropping would be acceptable, or that the user specify the crop and size areas. The latter was rejected because of its complexity and the fact that people didn't want servers turned into general image processing engines. The flag won out; the semantics state that if a server would have to resize or crop but the user specifies the "no alterations" flag, the server would simply reject the upload transaction. The server would also reject the transaction if the user would allow resizing/cropping but the image was outside the size restrictions and the server does not provide the service.

A related issue is that some servers may not be able to store an image immediately, and this raises error handling concerns. It was agreed that the return status from a media upload should indicate either "rejected", "stored" or "accepted for processing." The last state means that the server could eventually reject the upload once processing was completed. There was no agreement on a standard method for reporting this deferred error, but there was some consensus that existing error reporting mechanisms (for example, automatically-generated emails or special logon messges) would continue to be used.

The group agreed that the upload transaction should accept only one image at a time. Again, this was for simplicity in error handling.

The problem of how to upload and store image data was discussed at some length (leading to the well-known resouce idea). There was a question as to what kinds of data should be included, and how to designate the image number to be used by a particular image. Would the client have to query the system to find the next available number? How does the client specify a replacement rather than an addition? One way to do this is for the client to specify an image number and have the server modify it for an add.

Stuart Schuessler noted that the specification is abiguous with respect to the meaning of photo 0. It is the default picture, but is it separate from the others, or is it a synonym for one of the others? There was no resolution on this question; the solution to have photos numbered 0 to N-1 rather than 1 to N wasn't warmly received except by its proposer.

Such consensus as emerged was around the idea of a media table as a well-known resource that could be searched and edited like any other table. Given the right layout for the table, this provides editing capability (changing the number or description of an image) as well as image deletion. It also could cover a need expressed by two participants, to allow for images or other media that are stored by category (for example, "front entrance", "master bedroom", "view").

Stuart will put together a proposal for an ad-hoc working group formed to work on media upload.

Grouping in Upload

Sergio Del Rio identified a need for some human-interface hints for grouping on upload. This could be accomplished with a metadata addition: a GroupID, such that clients would know that the preferred layout of update items has those fields with the same group ID physically grouped on the screen. There was a question as to whether this information should be in the Update metadata or the Table metadata. If in the Table metadata, the information could be used to generate default layouts for display as well, but this was viewed with some distaste.

There was also a question as to the best way to present the grouping information to the client. One proposal was to have an additional metadata table that contained group IDs together with lists of persistent IDs of fields that belong in the group.

Michael Del Gaudio will put together a change proposal based on these discussions.

Localization

Sergio Del Rio began a discussion on localizing metadata. The business need is that agents in certain areas should see only a subset of metadata; an example is the list of codes for school districts. One vendor said that they put together metadata on the fly based on the user ID at logon, and thus localization isn't a problem. The discussion became more general then, noting that one part of localization, conditional lookup, is solved for update but not for search screen creation. (The example here is city and area, where selection of a city may constrain an area or vice versa.) Paul from Marketlinx will put together a change proposal to address this issue.

Hot Sheet

This hot potato was revisited with users falling into two camps: one, that the functionality approved with the Distributed Data Base proposal is adequate to create hotsheets, and two, that it isn't. Steve from Interealty will put together a white paper showing how hot sheets can be derived from facilities currently available; this will be reviewed on a conference call in the near future.

Updates for Current WC3 Standards

There was further discussion of switching to SOAP, a W3C standard, for enveloping and packaging, rather than the current RETS-only packaging method. As with the prior meeting, there was some sentiment that it's too much, too soon, but there was more push for doing it at this point because of activity in other standards groups. Steve will ask Wantao Zhou to put together a white paper on RETS encapsulation in SOAP; this may be turned into a change proposal at some point in the future. There is a concern that we will discourage adoption if we move too quickly. There is also a concern that we will discourage adoption if we do not move quickly enough. The white paper seemed like the best compromise between fast and slow.

Security and Data Protection

Driven by several up-and-coming issues, including VOWs (virtual office websites) and data protection, there is a need for communicating visibility information in RETS metadata. This would allow a system that is replicating another system to learn of data restrictions and apply them or pass them on as necessary. Currently, there is no means for doing this other than simply not sending the data, and in many applications this is not an option.

Discussions of what should be included in a mechanism for communicating visibility rules led Stuart Schuessler to propose that some metadata fields be made more robust by allowing expressions. This would permit a visibility field to include complex conditions that often reflect real-world situations: for example, a broker is allowed to see an expiration date, as is the listing agent, but nobody else.

Bruce will put together a proposal for future consideration by the group.

Web Site Changes

Several changes were requested for the web site:

An archive of change proposals
Links from the change proposals to the notes of the meeting in which they were voted on

These are being implemented now.

Next Meeting

Per the earlier decision to hold meetings three times per year, the next meeting will be in August. It is tentatively scheduled for August 12-13 in Phoenix.