Prev   Next

Chapter 4. Release notes

4.1. Release 3.0 notes
4.1.1. Usage statistics and OGSA-DAI
4.1.2. Changes from OGSA-DAI 2.2
4.1.3. Backwards compatibility
4.1.4. Known problems and limitations
4.1.5. Product stability
4.1.6. Status of activities in OGSA-DAI 3.0
4.1.7. Tested platforms and databases

4.1. Release 3.0 notes

4.1.1. Usage statistics and OGSA-DAI

OGSA-DAI contains functionality that collects statistics on the use of OGSA-DAI and forwards this to the OGSA-DAI team. This is enabled by default in the release. For information on the statistics collected and how to disable this functionality please see Section 5.1, “Usage statistics”.

4.1.2. Changes from OGSA-DAI 2.2

OGSA-DAI 3.0 is a complete rewrite of OGSA-DAI 2.2. A list of the changes made, why and advice on upgrading applications is given in Appendix N, Upgrading from OGSA-DAI 2.2.

4.1.3. Backwards compatibility

There is no backwards compatibility between OGSA-DAI 3.0 and previous releases of OGSA-DAI.

4.1.4. Known problems and limitations

4.1.4.1. Known problems

  • Platforms
    • OGSA-DAI 3.0 does not compile under Java 1.6. This is due to changes in java.sql APIs (in which both additional methods and classes have been added) from earlier versions of Java. The OGSA-DAI team may release a Java 1.6 patch in the near future.
    • The setenv scripts don't set CLASSPATHs correctly under Cygwin.
  • Interacting with databases
    • The standard Oracle JDBC driver does not conform to the JDBC API for insertion of BLOBs and so OGSA-DAI does not currently support the addition of BLOBs to tables in Oracle databases.
    • The table-name tag in WebRowSet XML is empty in results produced by Oracle or SQLServer. This is a problem of the JDBC drivers that do not provide table names in the metadata of a java.sql.ResultSet.
    • Meta-data from a database, returned by ExtractTableSchema, can be case-sensitive depending on the database used. For example, MySQL might return a table called littleblackbook while DB2 returns LITTLEBLACKBOOK.
    • If an error occurs during execution of a relational activity then server-side information may be returned to the client. This is because OGSA-DAI does not, as yet, parse java.sql.SQLExceptions from database drivers to identify problems caused by a client (e.g. a syntactically-incorrect SQL query) and those due to server-side failures (e.g. loss of connection to a database).
    • Though this is not OGSA-DAI specific, a user encountered a problem using the standard MySQL driver class org.gjt.mm.mysql.Driver from the mysql-connector-java-5.0.5 driver JAR in conjunction with MySQL version mysql-4.1.20-1.RHEL4.1 running on Red Hat. Even though they had provided the database name as part of the database connection URL, e.g. 
      jdbc:mysql://localhost:3306/dbname
      They had to specify the database name in their queries also, e.g. 
      SELECT * FROM dbname.table
  • Performance and scalability
    • Forwarding or retrieving very large amounts of data to/from an OGSA-DAI service may give rise to java.lang.OutOfMemoryError errors on the OGSA-DAI server. See Chapter 22, Best practice.
    • Attempting to get large amounts of data via a synchronous request can cause a server-side java.lang.OutOfMemoryError. This can be avoided by creating a data source and streaming the query data into this (via the WriteToDataSource activity) then pulling the data back to a client via a data source service. See Chapter 22, Best practice.
    • If using OGSA-DAI on a cluster with a single storage node then problems will arise when creating many resources (e.g. data sources or data sinks) simultaneously if the storage node does not support atomic file locking. The OGSA-DAI persistence components will attempt to access the resource template file for the resource being created many times simultaneously and will run into problems. This could be addressed by either ensuring that the storage node supports atomic file locking or developing a new OGSA-DAI persistence manager (either by modifying the existing one to support more effective concurrent file access or providing a new one that supports configuration from a database).
  • Miscellaneous
    • The uk.org.ogsadai.common.msgs.DAILogger methods can sometimes log the wrong line numbers. Searching the logs will usually reveal where the problem actually arose.
    • If a client does not provide a certificate then using deliverFromGridFTP and deliverToGridFTP in a request the transfer will fail.

4.1.4.2. Limitations

  • Platforms
    • Security
      • Our implementations of inter-service delivery activities (ObtainFromDataSource and DeliverToDataSink) do not at present support secure inter-service communications. This does not however preclude us providing these at a future date.
      • Encrypted login files are unsupported.
    • Activities
      • The sub-workflow API by which an activity can spawn a sub-workflow (as used in the SQLBag activity for example) is a proof-of-concept only. We intend to refactor the API for invoking sub-workflows at a future date to make it more usable.
      • In the source code there are references to the notion of activity contracts. These are intended to help support the use of activities in different roles e.g. the SQLBag could be exposed as an SQLBag or as an SQLQuery activity. We will elaborate on this concept at a later date (when we have had a chance to experiment).
      • Some of our activities, e.g. 
        uk.org.ogsadai.activity.sql.SQLBagActivity
uk.org.ogsadai.activity.sql.SQLResilientActivity
uk.org.ogsadai.activity.pipeline.AutomaticTee
        use hard-coded activity names to identify the activity or activities in sub-workflows they splice into workflows. This means that you should take care when renaming activities exposed by resources, especially block activities. This problem will be most likely manifested as an unsupported activity error.
      • Creation or deletion of databases within DB2 is not currently supported.
      • Creation or deletion of databases within Oracle is not currently supported.
      • SQL Server IMAGE column type is not currently supported.
      • deliverFromGridFTP and deliverToGridFTP do not allow the setting of certain GridFTP parameters.
      • Client toolkit support for XPathQuery and XQuery activities does not currently parse data in a request status into an org.xmldb.api.base.ResourceSet object.
    • Deployment

      • When building an OGSA-DAI Axis WAR file the service WSDL is customised to include the host and port on which Tomcat is running (which you specify at the command-line when building the WAR).

        While OGSA-DAI WAR files can be dropped into Tomcats running on other hosts or ports and the services will be usable using OGSA-DAI clients, any attempt to use the service WSDL may run into problems as the WSDL will contain the host and port originally specified when the WAR was built. To work round this you can either:

        • Rebuild the WAR file using an OGSA-DAI Axis binary distribution.
        • Or unZIP the WAR file, edit the host and ports encoded in the WSDL in the six schema/ogsadai/axis-specific-wsdl/*/*_service_service.wsdl files and then reZIP the WAR.
        • The JSP pages are prototypes. We may revise and extend the functionality of these at a later date.
    • Miscellaneous
      • BLOBs and CLOBs are currently only held in-memory with consquent implications on memory usage.
      • OGSA-DAI data request execution resources and implementations of JDBC, XMLDB and file system data resources do not support use of the destroy operation. It is a no-op.

      • The default OGSA-DAI resource and activity managers 

        uk.org.ogsadai.activity.SimpleActivityManager
uk.org.ogsadai.resource.SimpleResourceManager

        cache activity specifications and resources in in-memory hash tables. They do not check for changes in any persisted configuration associated with the activity specifications or resources once the associated objects are in the cache.

      • The default OGSA-DAI file-based resource persistence and configuration manager uk.org.ogsadai.persistence.file.resource.SimpleFileResourceStateDAO only handles the persistence and restoration of resource properties or session state that are Strings.
      • The WSRF operation SetResourceProperty is not supported by OGSA-DAI services.
      • The OGSA-DAI GT wrapper class uk.org.ogsadai.service.gt.resource.GTResourceProperty methods 
        add(X)
clear()
get(int)
remove(Object)
set(int, Object)
        are currently not supported. These would need to convert from XML fragments into objects.
      • There is no cleanup if client-service communication times out. Service will just reply as usual.
      • The uk.org.ogsadai.common.BinaryLob class contains unimplemented methods which throw java.lang.UnsupportedOperationException.
      • Conversion of java.sql.ResultSet to XML WebRowSet returns empty key-column and map properties in the properties element.

    4.1.5. Product stability

    The following APIs may be assumed to be held stable for the forseeable future (12 months). Changes will only be made if a fundamental problem is encountered for a majority of users.

    • Activity API - API for activity developers.
    • Activity framework API
    • Client toolkit API
    • Common components configuration API
    • OGSA-DAI context
    • Persistence manager API
    • Presentation layer JNDI API
    • Presentation layer WSDL. However we will release new presentation layers with different WSDL descriptions.
    • Resource APIs
    • Security context API

    The following components may be subject to change in light of requirements for more functionality or bug fixes. We may release bug fixed components or new versions of existing components. These will not incur API changes however. Note also that these are components that implement OGSA-DAI extensibility points so are designed to be pluggable and replaceable in this way.

    • Authorizer implementations. These are presentation layer and application specific so are intended to be replacable. We may release new ones for different applications or demonstrating use of different third-party authorization infrastructures.
    • Activity implementations
    • Activity manager implementation
    • Common components configuration manager implementation
    • Persistence manager implementation. The current file-based implementation is simple and lightweight and may not be scalable.
    • Presentation layers. We do not envisage changing existing presentation layers but may release new presentation layers for different applications or platforms.
    • Resource manager implementation

    The following APIs and components may be viewed as "alpha" versions which will be subject to change:

    • JSP pages. We may release new versions. This has no implications on any other components.
    • Sub-workflow API. This may be engineered to be more usable in a future release. This will possible expose a client toolkit-like API for use in activities that create workflows.

    4.1.6. Status of activities in OGSA-DAI 3.0

    This list outlines the activities supported in OGSA-DAI 3.0. Prior to release activities were subject to review. This review covered:

    • Both server and client toolkit activity conformance to the activity specification of Karasavvas, K. Atkinson, M.P. and Hume, A.C. OGSA-DAI - Redesigned and New Activities. http://www.ogsadai.org.uk/documentation/ogsadai3.0/RedesignedAndNewActivitiesV1.9.pdf
      • Conformance to input names.
      • Conformance to input types and allowed values.
      • Conformance to output types.
    • Iteration model used.
    • Validation of activity inputs.
    • Error handling and robustness.
    • Test coverage of both the activity and its client toolkit counterpart.
    • User documentation of the activity inputs, outputs, types, input ordering, behaviour, class names, activity extension interfaces implemented and compliant resources in Appendix H, Activities in OGSA-DAI 3.0.

    [Note]Note

    This review was not completed in time for the release. It is intended that at a suitable point after completion of this review updated user documentation and any updated activities (as a consequence of identified bugs, improvements to error handling, robustness or behaviour or to make conformant with the specification) will be released.

    NameServer-Side Class

    Sub-packages of uk.org.ogsadai.activity.

    		Client-Side Class

    Sub-packages of uk.org.ogsadai.client.toolkit.activities.

    		Reviewed?
    Block Activitiesblock.block. 
    ByteArraysResizeByteArraysResizeActivityByteArraysResizeYes
    CharArraysResizeCharArraysResizeActivityCharArraysResizeYes
    ControlledRepeatControlledRepeatActivityControlledRepeatYes
    ListConcatenateListConcatenateActivityListConcatenateYes
    ListControlledRepeatListControlledRepeatActivityListControlledRepeatYes
    ListMultiplyListMultiplyActivityListMultiply-
    SplitSplitActivitySplit-
    TeeTeeActivityTeeYes
    ListRemoveListRemoveActivityListRemoveYes
    Delivery Activitiesdelivery.delivery. 
    DeliverToDataSinkDeliverToDataSinkActivityDeliverToDataSinkYes
    DeliverToFTPDeliverToFTPActivityDeliverToFTPYes
    DeliverToHTTPDeliverToHTTPActivity--
    DeliverToRequestStatusDeliverToRequestStatusActivityDeliverToRequestStatusYes
    DeliverToSessionDeliverToSessionActivityDeliverToSession-
    DeliverToSMTPDeliverToSMTPActivityDeliverToSMTPYes
    ObtainFromFTPObtainFromFTPActivityObtainFromFTPYes
    ObtainFromHTTPObtainFromHTTPActivityObtainFromHTTP-
    ObtainFromSessionObtainFromSessionActivityObtainFromSession-
    ObtainFromDataSourceObtainFromDataSourceActivityObtainFromDataSourceYes
    ReadFromDataSinkReadFromDataSinkActivityReadFromDataSinkYes
    WriteToDataSourceWriteToDataSourceActivityWriteToDataSourceYes
    File Activitiesfile.file. 
    ReadFromFileReadFromFileActivityReadFromFileYes
    ListDirectoryListDirectoryActivityListDirectoryYes
    Management Activitiesmanagement.management. 
    CreateDataSinkCreateDataSinkActivityCreateDataSinkYes
    CreateDataSourceCreateDataSourceActivityCreateDataSourceYes
    CreateResourceGroupCreateResourceGroupActivityCreateResourceGroupYes
    Relational Activitiessql.sql. 
    ExtractTableSchemaExtractTableSchemaActivityExtractTableSchema-
    GetAvailableTableGetAvailableTableActivityGetAvailableTableYes
    SQLBagSQLBagActivitySQLBagYes
    SQLBulkLoadTupleSQLBulkLoadTupleActivitySQLBulkLoadTupleYes
    SQLParameterisedQuerySQLParameterisedQueryActivitySQLParameterisedQueryYes
    SQLParameterisedUpdateSQLParameterisedUpdateActivitySQLParameterisedUpdateYes
    SQLQuerySQLQueryActivitySQLQueryYes
    SQLResilientSQLResilientActivitySQLResilientYes
    SQLUpdateSQLUpdateActivitySQLUpdateYes
    Transformation Activitiestransform.transform. 
    CSVToTupleCSVToTupleActivityCSVToTupleYes
    NumberToStringActivityNumberToStringActivity--
    StringConcatenateActivityStringConcatenateActivity--
    StringTokenizeActivityStringTokenizeActivity--
    TupleProjectionTupleProjectionActivityTupleProjectionYes
    TupleSimpleMergeTupleSimpleMergeActivityTupleSimpleMergeYes
    TupleSplitTupleSplitActivityTupleSplitYes
    TupleToCSVTupleToCSVActivityTupleToCSVYes
    TableMetadataToXMLCharArraysTableMetadataToXMLCharArraysActivityTableMetadataToXMLCharArrays-
    TupleToWebRowSetCharArraysTupleToWebRowSetCharArraysActivityTupleToWebRowSetCharArraysYes
    WebRowSetCharacterDataToTupleWebRowSetCharacterDataToTupleActivityWebRowSetCharacterDataToTupleYes
    Utility Activitiesutil.util. 
    ActivityUserErrorProducerActivityUserErrorProducerActivityActivityUserErrorProducer-
    CloneableObjectProducerCloneableObjectProducerActivityCloneableObjectProducer-
    CounterCounterActivityCounter-
    DQPPatternDQPPatternActivityDQPPattern-
    EchoEchoActivityEchoYes
    HashCodeHashCodeActivityHashCodeYes
    ReadStringReadStringActivityReadStringYes
    SleepSleepActivitySleepYes
    TimestampTimestampActivityTimestampYes
    XSLTransformXSLTransformActivityXSLTransformYes
    XMLDB Activitiesxmldb.xmldb. 
    XMLListCollectionsXMLListCollectionsActivityXMLListCollectionsYes
    XMLListResourcesXMLListResourcesActivityXMLListResourcesYes
    XPathQueryXPathQueryActivityXPathQueryYes
    XQueryXQueryActivityXQueryYes

    Table 4.1. Status of activities in OGSA-DAI 3.0


    4.1.7. Tested platforms and databases

    OGSA-DAI 3.0 has been tested as follows. All tests are run on Linux Red Hat 9 under Java 1.4.2_12 and using Apache ANT 1.6.5.

    ReleaseUnit testsDatabase specific testsRelease build testsClient-server tests
    OGSA-DAI 3.0 Axis 1.4YesYesYesYes - with Tomcat 5.0
    OGSA-DAI 3.0 Axis 1.2.1YesYesYesNo

    Table 4.2. Tested platforms and databases


    Database-specific tests have been run upon all the databases listed in Chapter 3, Data resource products. Client-server tests were run using MySQL, eXist and file system resources.

    Prev   Next
    Chapter 3. Data resource products Home Chapter 5. Usage statistics