Chapter 4. Release notes

Prev		Next

4.1.1. Usage statistics and OGSA-DAI
4.1.2. Changes from OGSA-DAI 2.2
4.1.3. Backwards compatibility
4.1.4. Known problems and limitations
4.1.5. Product stability
4.1.6. Status of activities in OGSA-DAI 3.0
4.1.7. Tested platforms and databases
4.1.8. WSRF

4.1. Release 3.0 notes

4.1.1. Usage statistics and OGSA-DAI

OGSA-DAI contains functionality that collects statistics on the use of OGSA-DAI and forwards this to the OGSA-DAI team. This is enabled by default in the release. For information on the statistics collected and how to disable this functionality please see Section 5.1, “Usage statistics”.

4.1.2. Changes from OGSA-DAI 2.2

OGSA-DAI 3.0 is a complete rewrite of OGSA-DAI 2.2. A list of the changes made, why and advice on upgrading applications is given in Appendix N, Upgrading from OGSA-DAI 2.2.

4.1.3. Backwards compatibility

There is no backwards compatibility between OGSA-DAI 3.0 and previous releases of OGSA-DAI.

4.1.4. Known problems and limitations

4.1.4.1. Known problems

Platforms
- OGSA-DAI 3.0 does not compile under Java 1.6. This is due to changes in java.sql APIs (in which both additional methods and classes have been added) from earlier versions of Java. The OGSA-DAI team may release a Java 1.6 patch in the near future.
- The setenv scripts don't set CLASSPATHs correctly under Cygwin.
Interacting with databases
- The standard Oracle JDBC driver does not conform to the JDBC API for insertion of BLOBs and so OGSA-DAI does not currently support the addition of BLOBs to tables in Oracle databases.
- The table-name tag in WebRowSet XML is empty in results produced by Oracle or SQLServer. This is a problem of the JDBC drivers that do not provide table names in the metadata of a java.sql.ResultSet.
- Meta-data from a database, returned by ExtractTableSchema, can be case-sensitive depending on the database used. For example, MySQL might return a table called littleblackbook while DB2 returns LITTLEBLACKBOOK.
- If an error occurs during execution of a relational activity then server-side information may be returned to the client. This is because OGSA-DAI does not, as yet, parse java.sql.SQLExceptions from database drivers to identify problems caused by a client (e.g. a syntactically-incorrect SQL query) and those due to server-side failures (e.g. loss of connection to a database).
- Though this is not OGSA-DAI specific, a user encountered a problem using the standard MySQL driver class org.gjt.mm.mysql.Driver from the mysql-connector-java-5.0.5 driver JAR in conjunction with MySQL version mysql-4.1.20-1.RHEL4.1 running on Red Hat. Even though they had provided the database name as part of the database connection URL, e.g.
```
jdbc:mysql://localhost:3306/dbname
```
  They had to specify the database name in their queries also, e.g.
```
SELECT * FROM dbname.table
```
Performance and scalability
- If using eXist and Globus Toolkit and OGSA-DAI GT in the same Tomcat container, the container may run out of memory. We recommend deployment of eXist in its own Tomcat instance.
- Forwarding or retrieving very large amounts of data to/from an OGSA-DAI service may give rise to java.lang.OutOfMemoryError errors on the OGSA-DAI server. See Chapter 22, Best practice.
- Attempting to get large amounts of data via a synchronous request can cause a server-side java.lang.OutOfMemoryError. This can be avoided by creating a data source and streaming the query data into this (via the WriteToDataSource activity) then pulling the data back to a client via a data source service. See Chapter 22, Best practice.
- If using OGSA-DAI on a cluster with a single storage node then problems will arise when creating many resources (e.g. data sources or data sinks) simultaneously if the storage node does not support atomic file locking. The OGSA-DAI persistence components will attempt to access the resource template file for the resource being created many times simultaneously and will run into problems. This could be addressed by either ensuring that the storage node supports atomic file locking or developing a new OGSA-DAI persistence manager (either by modifying the existing one to support more effective concurrent file access or providing a new one that supports configuration from a database).
Miscellaneous
- The uk.org.ogsadai.common.msgs.DAILogger methods can sometimes log the wrong line numbers. Searching the logs will usually reveal where the problem actually arose.
- If a client does not provide a certificate then using deliverFromGridFTP and deliverToGridFTP in a request the transfer will fail.

4.1.4.2. Limitations

Platforms
- There is no explicit support for Globus Toolkit MDS registration at present.
Security
- Our implementations of inter-service delivery activities (ObtainFromDataSource and DeliverToDataSink) do not at present support secure inter-service communications. This does not however preclude us providing these at a future date.
- Encrypted login files are unsupported.
Activities
- The sub-workflow API by which an activity can spawn a sub-workflow (as used in the SQLBag activity for example) is a proof-of-concept only. We intend to refactor the API for invoking sub-workflows at a future date to make it more usable.
- In the source code there are references to the notion of activity contracts. These are intended to help support the use of activities in different roles e.g. the SQLBag could be exposed as an SQLBag or as an SQLQuery activity. We will elaborate on this concept at a later date (when we have had a chance to experiment).
- Some of our activities, e.g.
```
uk.org.ogsadai.activity.sql.SQLBagActivity
uk.org.ogsadai.activity.sql.SQLResilientActivity
uk.org.ogsadai.activity.pipeline.AutomaticTee
```
  use hard-coded activity names to identify the activity or activities in sub-workflows they splice into workflows. This means that you should take care when renaming activities exposed by resources, especially block activities. This problem will be most likely manifested as an unsupported activity error.
- Creation or deletion of databases within DB2 is not currently supported.
- Creation or deletion of databases within Oracle is not currently supported.
- SQL Server IMAGE column type is not currently supported.
- deliverFromGridFTP and deliverToGridFTP do not allow the setting of certain GridFTP parameters.
- Client toolkit support for XPathQuery and XQuery activities does not currently parse data in a request status into an org.xmldb.api.base.ResourceSet object.
Deployment
- When building an OGSA-DAI Axis WAR file the service WSDL is customised to include the host and port on which Tomcat is running (which you specify at the command-line when building the WAR).
  While OGSA-DAI WAR files can be dropped into Tomcats running on other hosts or ports and the services will be usable using OGSA-DAI clients, any attempt to use the service WSDL may run into problems as the WSDL will contain the host and port originally specified when the WAR was built. To work round this you can either:
  - Rebuild the WAR file using an OGSA-DAI Axis binary distribution.
  - Or unZIP the WAR file, edit the host and ports encoded in the WSDL in the six schema/ogsadai/axis-specific-wsdl/*/*_service_service.wsdl files and then reZIP the WAR.
  - The JSP pages are prototypes. We may revise and extend the functionality of these at a later date.
Miscellaneous
- BLOBs and CLOBs are currently only held in-memory with consquent implications on memory usage.
- OGSA-DAI data request execution resources and implementations of JDBC, XMLDB and file system data resources do not support use of the destroy operation. It is a no-op.
- The default OGSA-DAI resource and activity managers
```
uk.org.ogsadai.activity.SimpleActivityManager
uk.org.ogsadai.resource.SimpleResourceManager
```
  cache activity specifications and resources in in-memory hash tables. They do not check for changes in any persisted configuration associated with the activity specifications or resources once the associated objects are in the cache.
- The default OGSA-DAI file-based resource persistence and configuration manager uk.org.ogsadai.persistence.file.resource.SimpleFileResourceStateDAO only handles the persistence and restoration of resource properties or session state that are Strings.
- The WSRF operation SetResourceProperty is not supported by OGSA-DAI services.
- The OGSA-DAI GT wrapper class uk.org.ogsadai.service.gt.resource.GTResourceProperty methods
```
add(X)
clear()
get(int)
remove(Object)
set(int, Object)
```
  are currently not supported. These would need to convert from XML fragments into objects.
- There is no cleanup if client-service communication times out. Service will just reply as usual.
- The uk.org.ogsadai.common.BinaryLob class contains unimplemented methods which throw java.lang.UnsupportedOperationException.
- Conversion of java.sql.ResultSet to XML WebRowSet returns empty key-column and map properties in the properties element.

4.1.5. Product stability

The following APIs may be assumed to be held stable for the forseeable future (12 months). Changes will only be made if a fundamental problem is encountered for a majority of users.

Activity API - API for activity developers.
Activity framework API
Client toolkit API
Common components configuration API
OGSA-DAI context
Persistence manager API
Presentation layer JNDI API
Presentation layer WSDL. However we will release new presentation layers with different WSDL descriptions.
Resource APIs
Security context API

The following components may be subject to change in light of requirements for more functionality or bug fixes. We may release bug fixed components or new versions of existing components. These will not incur API changes however. Note also that these are components that implement OGSA-DAI extensibility points so are designed to be pluggable and replaceable in this way.

Authorizer implementations. These are presentation layer and application specific so are intended to be replacable. We may release new ones for different applications or demonstrating use of different third-party authorization infrastructures.
Activity implementations
Activity manager implementation
Common components configuration manager implementation
Persistence manager implementation. The current file-based implementation is simple and lightweight and may not be scalable.
Presentation layers. We do not envisage changing existing presentation layers but may release new presentation layers for different applications or platforms.
Resource manager implementation

The following APIs and components may be viewed as "alpha" versions which will be subject to change:

JSP pages. We may release new versions. This has no implications on any other components.
Sub-workflow API. This may be engineered to be more usable in a future release. This will possible expose a client toolkit-like API for use in activities that create workflows.

4.1.6. Status of activities in OGSA-DAI 3.0

This list outlines the activities supported in OGSA-DAI 3.0. Prior to release activities were subject to review. This review covered:

Both server and client toolkit activity conformance to the activity specification of Karasavvas, K. Atkinson, M.P. and Hume, A.C. OGSA-DAI - Redesigned and New Activities. http://www.ogsadai.org.uk/documentation/ogsadai3.0/RedesignedAndNewActivitiesV1.9.pdf
- Conformance to input names.
- Conformance to input types and allowed values.
- Conformance to output types.
Iteration model used.
Validation of activity inputs.
Error handling and robustness.
Test coverage of both the activity and its client toolkit counterpart.
User documentation of the activity inputs, outputs, types, input ordering, behaviour, class names, activity extension interfaces implemented and compliant resources in Appendix H, Activities in OGSA-DAI 3.0.

	Note
	This review was not completed in time for the release. It is intended that at a suitable point after completion of this review updated user documentation and any updated activities (as a consequence of identified bugs, improvements to error handling, robustness or behaviour or to make conformant with the specification) will be released.

Name	Server-Side Class Sub-packages of `uk.org.ogsadai.activity.`	Client-Side Class Sub-packages of `uk.org.ogsadai.client.toolkit.activities.`	Reviewed?
Block Activities	`block.`	`block.`
ByteArraysResize	`ByteArraysResizeActivity`	`ByteArraysResize`	Yes
CharArraysResize	`CharArraysResizeActivity`	`CharArraysResize`	Yes
ControlledRepeat	`ControlledRepeatActivity`	`ControlledRepeat`	Yes
ListConcatenate	`ListConcatenateActivity`	`ListConcatenate`	Yes
ListControlledRepeat	`ListControlledRepeatActivity`	`ListControlledRepeat`	Yes
ListMultiply	`ListMultiplyActivity`	`ListMultiply`	-
Split	`SplitActivity`	`Split`	-
Tee	`TeeActivity`	`Tee`	Yes
ListRemove	`ListRemoveActivity`	`ListRemove`	Yes
Delivery Activities	`delivery.`	`delivery.`
DeliverToDataSink	`DeliverToDataSinkActivity`	`DeliverToDataSink`	Yes
DeliverToFTP	`DeliverToFTPActivity`	`DeliverToFTP`	Yes
DeliverToGFTP	`DeliverToGFTPActivity`	`DeliverToGFTP`	Yes
DeliverToHTTP	`DeliverToHTTPActivity`	-	-
DeliverToRequestStatus	`DeliverToRequestStatusActivity`	`DeliverToRequestStatus`	Yes
DeliverToSession	`DeliverToSessionActivity`	`DeliverToSession`	-
DeliverToSMTP	`DeliverToSMTPActivity`	`DeliverToSMTP`	Yes
ObtainFromFTP	`ObtainFromFTPActivity`	`ObtainFromFTP`	Yes
ObtainFromGFTP	`ObtainFromGFTPActivity`	`ObtainFromGFTP`	Yes
ObtainFromHTTP	`ObtainFromHTTPActivity`	`ObtainFromHTTP`	-
ObtainFromSession	`ObtainFromSessionActivity`	`ObtainFromSession`	-
ObtainFromDataSource	`ObtainFromDataSourceActivity`	`ObtainFromDataSource`	Yes
ReadFromDataSink	`ReadFromDataSinkActivity`	`ReadFromDataSink`	Yes
WriteToDataSource	`WriteToDataSourceActivity`	`WriteToDataSource`	Yes
File Activities	`file.`	`file.`
ReadFromFile	`ReadFromFileActivity`	`ReadFromFile`	Yes
ListDirectory	`ListDirectoryActivity`	`ListDirectory`	Yes
Management Activities	`management.`	`management.`
CreateDataSink	`CreateDataSinkActivity`	`CreateDataSink`	Yes
CreateDataSource	`CreateDataSourceActivity`	`CreateDataSource`	Yes
CreateResourceGroup	`CreateResourceGroupActivity`	`CreateResourceGroup`	Yes
Relational Activities	`sql.`	`sql.`
ExtractTableSchema	`ExtractTableSchemaActivity`	`ExtractTableSchema`	-
GetAvailableTable	`GetAvailableTableActivity`	`GetAvailableTable`	Yes
SQLBag	`SQLBagActivity`	`SQLBag`	Yes
SQLBulkLoadTuple	`SQLBulkLoadTupleActivity`	`SQLBulkLoadTuple`	Yes
SQLParameterisedQuery	`SQLParameterisedQueryActivity`	`SQLParameterisedQuery`	Yes
SQLParameterisedUpdate	`SQLParameterisedUpdateActivity`	`SQLParameterisedUpdate`	Yes
SQLQuery	`SQLQueryActivity`	`SQLQuery`	Yes
SQLResilient	`SQLResilientActivity`	`SQLResilient`	Yes
SQLUpdate	`SQLUpdateActivity`	`SQLUpdate`	Yes
Transformation Activities	`transform.`	`transform.`
CSVToTuple	`CSVToTupleActivity`	`CSVToTuple`	Yes
NumberToStringActivity	`NumberToStringActivity`	-	-
StringConcatenateActivity	`StringConcatenateActivity`	-	-
StringTokenizeActivity	`StringTokenizeActivity`	-	-
TupleProjection	`TupleProjectionActivity`	`TupleProjection`	Yes
TupleSimpleMerge	`TupleSimpleMergeActivity`	`TupleSimpleMerge`	Yes
TupleSplit	`TupleSplitActivity`	`TupleSplit`	Yes
TupleToCSV	`TupleToCSVActivity`	`TupleToCSV`	Yes
TableMetadataToXMLCharArrays	`TableMetadataToXMLCharArraysActivity`	`TableMetadataToXMLCharArrays`	-
TupleToWebRowSetCharArrays	`TupleToWebRowSetCharArraysActivity`	`TupleToWebRowSetCharArrays`	Yes
WebRowSetCharacterDataToTuple	`WebRowSetCharacterDataToTupleActivity`	`WebRowSetCharacterDataToTuple`	Yes
Utility Activities	`util.`	`util.`
ActivityUserErrorProducer	`ActivityUserErrorProducerActivity`	`ActivityUserErrorProducer`	-
CloneableObjectProducer	`CloneableObjectProducerActivity`	`CloneableObjectProducer`	-
Counter	`CounterActivity`	`Counter`	-
DQPPattern	`DQPPatternActivity`	`DQPPattern`	-
Echo	`EchoActivity`	`Echo`	Yes
HashCode	`HashCodeActivity`	`HashCode`	Yes
ReadString	`ReadStringActivity`	`ReadString`	Yes
Sleep	`SleepActivity`	`Sleep`	Yes
Timestamp	`TimestampActivity`	`Timestamp`	Yes
XSLTransform	`XSLTransformActivity`	`XSLTransform`	Yes
XMLDB Activities	`xmldb.`	`xmldb.`
XMLListCollections	`XMLListCollectionsActivity`	`XMLListCollections`	Yes
XMLListResources	`XMLListResourcesActivity`	`XMLListResources`	Yes
XPathQuery	`XPathQueryActivity`	`XPathQuery`	Yes
XQuery	`XQueryActivity`	`XQuery`	Yes

Table 4.1. Status of activities in OGSA-DAI 3.0

4.1.7. Tested platforms and databases

OGSA-DAI 3.0 has been tested as follows. All tests are run on Linux Red Hat 9 under Java 1.4.2_12 and using Apache ANT 1.6.5.

Release	Unit tests	Database specific tests	Release build tests	Client-server tests
OGSA-DAI 3.0 GT 4.0.5	Yes	Yes	Yes	Yes - this includes security-related tests

Table 4.2. Tested platforms and databases

Database-specific tests have been run upon all the databases listed in Chapter 3, Data resource products. Client-server tests were run using MySQL, eXist and file system resources.

4.1.8. WSRF

OGSA-DAI services are compliant with the version of WSRF currently bundled with Globus Toolkit 4.0.5. From the Globus Toolkit user doc core facts ( http://www.globus.org/toolkit/docs/4.0/common/javawscore/JavaWSCoreFacts.html) these are

"...the 2004/06 OASIS WSRF and WSN working draft specifications (with minor fixes to the 1.2-draft-01 published schemas and with the March 2004 version of the WS-Addressing specification)"

Prev		Next
Chapter 3. Data resource products	Home	Chapter 5. Usage statistics