Appendix N. Upgrading from OGSA-DAI 2.2

N.1. Motivation
N.2. Changes from previous releases
N.2.1. Perform documents and requests
N.2.2. Architectural changes
N.2.3. Glossary of OGSA-DAI terminology and relationship to 2.2 and earlier releases
N.3. Migration guide for 2.2 to 3.0 for client applications
N.3.1. Migrating perform documents to requests
N.3.2. Mapping 2.2 relational activities to 3.0
N.3.3. Mapping 2.2 XMLDB activities to 3.0
N.3.4. Mapping 2.2 delivery activities to 3.0
N.3.5. Mapping 2.2 transformation activities to 3.0
N.3.6. Mapping 2.2 resource management activities to 3.0
N.3.7. Mapping 2.2 file activities to 3.0
N.3.8. Mapping 2.2 indexed file activities to 3.0
N.3.9. Mapping 2.2 miscellaneous activities to 3.0
N.3.10. Mapping 2.2 relational multi-resource activities to 3.0
N.3.11. Mapping 2.2 example activities to 3.0
N.3.12. DeliverToRequestStatus activity
N.3.13. Synchronous or asynchronous requests
N.3.14. Asynchronous data transport
N.3.15. Migrating a client that uses the client toolkit
N.3.16. Getting a client-side service proxy
N.3.17. Listing data resources
N.3.18. Building a request
N.3.19. Sending a request
N.3.20. Processing results
N.3.21. Performing a sequence of queries
N.3.22. Flow and sequence controls
N.3.23. Transferring data between OGSA-DAI servers
N.3.24. Processing partial results
N.4. Upgrading activity implementations from 2.2 to 3.0
N.4.1. Introduction
N.4.2. Define XML Schema
N.4.3. Writing an implementation class in Java
N.4.4. Activity implementation
N.4.5. Writing a corresponding client toolkit class in Java
N.4.6. Client toolkit implementation
N.4.7. Server-side activity deployment
N.5. Upgrading resource installer ANT properties files from 2.2 to 3.0

N.1. Motivation

OGSA-DAI 3.0 is a complete redesign and rewrite of the OGSA-DAI software. The reasons behind this are described in:

M. Antonioletti, N.P. Chue Hong, A.C. Hume, M. Jackson, K. Karasavvas, A. Krause, J.M. Schopf, M.P. Atkinson, B. Dobrzelecki, M. Illingworth, N. McDonnell, M. Parsons and E. Theocharopoulos. OGSA-DAI 3.0 The Whats and the Whys. Proceedings of the UK e-Science All Hands Meeting 2007, September 2007. http://www.ogsadai.org.uk/documentation/publications/AHM2007_OGSA-DAI_WhatAndWhy.pdf

N.2. Changes from previous releases

Despite the following changes the basic mode of interaction with OGSA-DAI remains unchanged - clients build a workflow from activities, submit this to an OGSA-DAI service and get back information as to the status of execution of the workflow and, depending on the request, data.

N.2.1. Perform documents and requests

The notion of an XML perform document and XML response document is no longer valid within the OGSA-DAI activity framework. This now only handles request objects and request status objects.

Keeping XML out of the OGSA-DAI activity framework simplifies the framework code and avoids problems arising from locating associated XML Schema (and accessing these locations). In addition, it allows the writing of applications that access OGSA-DAI functionality without the need to go through a web services presentation layer. It would be unreasonable and inefficient to have such applications construct an XML document only for this to be parsed again by OGSA-DAI when they could just construct the Java data structure used internally by OGSA-DAI directly and avoid XML altogether.

Perform documents have not been replaced by workflows. Perform documents are representations of workflows, they just were not termed as such. The difference is now that the workflows are represented in Java rather than in XML both client-side and server-side. The only time they are actually in an XML formt is when going from client to an OGSA-DAI web service via SOAP/HTTP.

N.2.1.1. Perform documents, requests and web services

If OGSA-DAI is exposed via web service (which most users will continue to do) then given that the WSDL definition of OGSA-DAI web services is an XML document and interacting with these services is basically an exchange of XML documents it is still possible to write clients in any language that can interact with web services. This has not changed from 2.2 or earlier versions.

In certain respects this should now be easier in the current release. In 2.2 and earlier the OGSA-DAI perform operation was defined as AnyType perform(AnyType) and the actual format of a validd workflow was not exposed in the OGSA-DAI service WSDL. In the current release the format of a workflow and the response/request status is fully defined in the WSDL for the execute (i.e. RequestStatus execute(Request)) operation which replaces the perform operation.

N.2.1.2. Perform documents, requests and clients

From a client perspective many users did not use perform documents directly but used the client toolkit which, in both previous and the current release allowed the construction of workflows via the connecting together of Java objects. So for many users the approach to writing clients remains unchanged from previous releases.

In 2.2 and earlier releases example clients were provided that submitted perform documents (specified at the command-line) to OGSA-DAI web services. These were primarily intende to be a quick way of experimenting with OGSA-DAI. The OGSA-DAI developers had always envisaged that most users would develop their own applications using the client toolkit to construct the workflows in Java, with the client toolkit building the perform document "behind the scenes".

Now in 2.2 the client toolkit assembled the SOAP/HTTP message XML in code and then used Apache Axis just to submit the constructed SOAP/HTTP message to the service and get the results. This allowed inter-op between OGSA-DAI on Axis services and OGSA-DAI on Globus Toolkit services to be supported but it is very difficult to maintain such code.

In the current release the client toolkit exploits the fact that the entire workflow XML Schema is exposed in the WSDL and Axis WSDL2Java and get a set of auto-generated classes (beans). Workflows constructed using the client toolkit are then mapped to the corresponding auto-generated classes and Axis is used to invoke the web service. i.e. the client toolkit dependence on Axis in 3.0 to manage interactions with OGSA-DAI web services has increased which means the overhead of developing Axis-compliant clients decreased.

Now support for executing workflows provided as documents (as in 2.2 and earlier releases) could be implemented. This would involve writing a workflow serializer which would take the workflow (formerly perform) document and convert it to the corresponding set of Axis beans and similarly a deserializer for the conversion of the Axis beans representing a request status into an XML response document. This would not be difficult to do just time consuming therefore it was not deemed a priority for the current release.

N.2.2. Architectural changes

Other changes are as follows:

  • Instead of 1 OGSA-DAI service there are now 6 types of OGSA-DAI service.
  • Instead of 1 OGSA-DAI resource there are now 6 types of OGSA-DAI resource.
  • There is no need for OGSA-DAI administrators to associate services with resources. Each OGSA-DAI service exposes and provides access to all resources of the corresponding type.
  • Multiple OGSA-DAI data resources can be targeted in a single request. This allows, for example, data to be queried from one database and streamed into another within the scope of a single request.

N.2.3. Glossary of OGSA-DAI terminology and relationship to 2.2 and earlier releases

Execute operation == Perform operation

The Execute operation of a data request execution resource takes an OGDSA-DAI request and returns an OGSA-DAI request status. In 2.2 and previous releases this was termed the Perform operation of a data service resource.

Data request execution resource (DRER) == Data service resource (DSR)

A data request execution resource accepts requests from clients, executes these and possibly returns results. In OGSA-DAI 2.2 and earlier releases this was termed a data service resource or DSR. One major difference is that in 2.2 and earlier a DSR was associated with exactly one data resource whereas a DRER is associated with 0 or more data resources.

Data request execution service (DRES) == Data service (DS)

A data request execution service is the web service via which clients access data request execution resources. In OGSA-DAI 2.2 and earlier releases this was termed a data service.

Data sink == Input stream

A data sink allows data to be streamed into an OGSA-DAI server by a client. In OGSA-DAI 2.2 and earlier releases this was termed an input stream. In 2.2 and earlier input streams were accessed via a data service, now they are accessed via a data sink service.

Data source == Output stream

A data source allows data to be streamed from an OGSA-DAI server to a client. In OGSA-DAI 2.2 and earlier releases this was termed an output stream. In 2.2 and earlier output streams were accessed via a data service, now they are accessed via a data source service.

Data source and data sink == Data transport

Data sources and data sinks are used to support asynchronous data transfer from and to an OGSA-DAI server. In OGSA-DAI 2.2 and earlier releases this was known as data transport.

Data source and data sink services == Data transport services

Data source and data sink services are used to support asynchronous data transfer from and to an OGSA-DAI server (operating in conjunction with the associated data sources and data sinks). In OGSA-DAI 2.2 and earlier releases this was done via the data transport operations of a data service.

OGSA-DAI Axis == OGSA-DAI WSI

OGSA-DAI Axis refers to an OGSA-DAI distribution compliant with a standalone Apache Axis release. In 2.2 or earlier such distributions were termed OGSA-DAI WSI.

OGSA-DAI GT == OGSA-DAI WSRF

OGSA-DAI GT refers to an OGSA-DAI distribution compliant with a Globus Toolkit release. In 2.2 or earlier such distributions were termed OGSA-DAI WSRF. The name was changed since OGSA-DAI Axis releases which do not require the Globus Toolkit are also WSRF-compliant.

Request == Perform document

A request is an OGSA-DAI workflow consisting of connected activities each of which may be targeted at an OGSA-DAI resource. In OGSA-DAI 2.2 and earlier this was referred to as a perform document. One major difference is that in a perform document each activity was targeted at the data resource associated with a DSR or no data resource whereas now each activity can be targeted at any resource known to an OGSA-DAI server. In addition in 2.2 DSRs accepted perform documents as just that - an XML document - now, however, DRERs accept request objects.

Request status == Response document

A request status is a representation of the status of execution of an OGSA-DAI request by a DRER. This includes the execution status of each activity and, depending upon the request, data itself. In OGSA-DAI 2.2 and earlier this was referred to as a response document. One major difference is in 2.2 data service resources produced response documents as just that - an XML document - now, however, DRERs produce request objects. Another difference now is that a request status is also accessible via a request resource.

N.3. Migration guide for 2.2 to 3.0 for client applications

This section attempts to provide some advice for developers migrating an OGSA-DAI application from OGSA-DAI 2.2 to OGSA-DAI 3.0. Many of the basic concepts used in OGSA-DAI have changed significantly in OGSA-DAI 3.0. It is strongly advised that the developer understands these concept changes before migrating code.

N.3.1. Migrating perform documents to requests

As described in Section N.2.1.2, “Perform documents, requests and clients” OGSA-DAI 3.0 does not provide support for using perform documents in it's end-to-end clients or client toolkit. If you currently use perform documents you will have to implement the workflows that these represent using the client toolkit or write a client toolkit component to accept an XML representation of OGSA-DAI workflows. Either way you should respect the changes in activities betwen releases as outlined in the following tables.

Migrating workflow/request from OGSA-DAI 2.2 to OGSA-DAI 3.0 requires replacing each activity with its OGSA-DAI 3.0 equivalent. How activities correspond between these releases is shown in the following tables.

N.3.2. Mapping 2.2 relational activities to 3.0

2.2 Activity Name3.0 Activity NameNotes
sqlQueryStatementSQLQuery or SQLParameterizedQuery 
sqlUpdateStatementSQLUpdate or SQLParameterizedUpdate 
sqlStoredProcedureNot included.May be in a future release.
sqlBulkLoadRowSetSQLBulkLoadTuple Uses tuples rather than WebRowSet. Requires use of OGSA-DAI 3.0 activity WebRowSetCharacterDataToTuples.
sqlResultsToXMLTupleToWebRowSetCharArrays 
sqlResultsToCSVTupleToCSV 
sqlResultToBytesTupleSplit Can be used to extract the BLOB from a tuple. Most activities that expect byte data can process a BLOB.
extractDatabaseSchemaGetAvailableTables and ExtractTableSchema ExtractTableSchema requires use of TableMetadataToXMLCharArrays to serialize data before returning to the client.
extractPhysicalSchemaNot included.An equivalent may be in a future release.
removeDuplicatesNot included.RemoveDuplicates and RemoveSortedDuplicates may be in a future release.
resultsetProjectionTupleProjection followed by TupleSplit 

N.3.3. Mapping 2.2 XMLDB activities to 3.0

2.2 Activity Name3.0 Activity NameNotes
xPathStatementXPathQuery 
xQueryStatementXQuery 
xUpdateStatementXUpdateStatement 
xmlCollectionManagementXMLListCollections Collections can be listed using XMLListCollections but creation and removal tasks are not supported. XMLCreateCollection and XMLRemoveCollection may be in a future release.
xmlResourceManagementXMLListResources Resources can be listed using XMLListResources but creation and removal tasks are not supported. XMLCreateResource and XMLRemoveResource may be in a future release.
xmlBulkLoadNot included.May be in a future release.

N.3.4. Mapping 2.2 delivery activities to 3.0

2.2 Activity Name3.0 Activity NameNotes
deliverFromURLObtainFromHTTP, ObtainFromFTP 
deliverToURLDeliverToFTPDeliverToHTTP may be in a future release.
deliverToGDTDeliverToDataSink 
deliverFromGDTObtainFromDataSource 
deliverToGFTPDeliverToGFTP 
deliverFromGFTPObtainFromGFTP 
deliverFromFileReadFromFile 
deliverToFileNot included.WriteToFile may be in a future release.
deliverToSMTPDeliverToSMTP 
inputStreamCreateDataSink and ReadFromDataSink 
outputStreamCreateDataSource and WriteToDataSource 
deliverToStream Not included. Accessing a data source via HTTP may be supported in a future release, with a servlet being written to interact with the data source. This would support the same functionality as deliverToStream and the OGSA-DAI 2.2 DeliverToStreamServlet.
deliverToNullNot included. DeliverToNull in may be in a future release. This activity was typically used in OGSA-DAI 2.2 to turn a perform document that would be executed synchronously into one that would be executed asynchronously. Synchronous or asynchronous execution can now be explicitly specified by the client and is no longer a function of the workflow.
deliverToResourcePropertyNot included.May be in a future release.
deliverToAttachmentNot included.DeliverToSOAPAttachment may be in a future release.

N.3.5. Mapping 2.2 transformation activities to 3.0

2.2 Activity Name3.0 Activity NameNotes
zipArchiveNot included.ZIPCompression and ZIPDecompression may be in a future release.
gzipCompressionNot included.GZIPCompression may be in a future release.
gzipDecompressionNot included.GZIPDecompression may be in a future release.
xslTransformXSLTransform 
stringTokenizerNot included.May be in a future release.
blockAggregatorNot included.May be in a future release.
bytesToTempFileNot included. This activity was used in the handling of BLOBS. The improvements in BLOB handling in 3.0 render this unnecessary.
webrowsetProjectionTupleProjection This activity operates on tuples rather than XML WebRowSet. There are activities to convert from XML WebRowSet to tuples and vice-versa.
csvProjectionTupleProjection 
frequencyDistributorNot included.FrequencyDistribute may be in a future release.
randomSampleNot included.RandomSample and ListRandomSample may be in a future release.

N.3.6. Mapping 2.2 resource management activities to 3.0

2.2 Activity Name3.0 Activity NameNotes
removeResourceNot included.DestroyResource may be in a future release.

N.3.7. Mapping 2.2 file activities to 3.0

2.2 Activity Name3.0 Activity NameNotes
directoryAccessActivityListDirectoryFullListDirectory may be in a future release.
fileAccessActivityReadFromFile 
fileManipulationActivityNot included.FileCreate, FileCopy, FileDelete, FileMove may be in a future release.
fileWritingActivityNot included.WriteToFile and FileReplaceBytes may be in a future release.

N.3.8. Mapping 2.2 indexed file activities to 3.0

2.2 Activity Name3.0 Activity NameNotes
addIndexFileNot included.CreateIndexFile may be in a future release.
searchIndexedFileNot included.LookUpIndexFile may be in a future release.
readFileNot included.ReadFromIndexedFile may be in a future release.

N.3.9. Mapping 2.2 miscellaneous activities to 3.0

2.2 Activity Name3.0 Activity NameNotes
dataStoreNot included. 
notificationNot included. 

N.3.10. Mapping 2.2 relational multi-resource activities to 3.0

2.2 Activity Name3.0 Activity NameNotes
sqlBagSQLBagMay also need use of CreateResourceGroup.
sqlResilientSQLResilientMay also need use of CreateResourceGroup.

N.3.11. Mapping 2.2 example activities to 3.0

2.2 Activity Name3.0 Activity NameNotes
helloWorldActivityNot included. 
demoExternalInputNot included. 
demoCreateInstanceNot included. 
demoCreateTransientInstanceNot included. 
demoQueryInstanceNot included. 

N.3.12. DeliverToRequestStatus activity

OGSA-DAI 3.0 includes the new activity, DeliverToRequestStatus. This activity writes its input to the request status. This is the only way data can be returned to the client within a SOAP response message from a data request execution service. In OGSA-DAI 2.2 any data written to an unconnected output was automatically written to the SOAP response message. In OGSA-DAI 3.0 such unconnected outputs are illegal and you have to explicitly connect them. If when migrating an application you encounter errors relating to unconnected outputs you should connect that output to a DeliverToRequestStatus activity if you wish to retrieve the data in the request status when the DRER finishes executing the workflow.

N.3.13. Synchronous or asynchronous requests

In OGSA-DAI 2.2 a request was executed synchronously if it contained one or more unconnected outputs. In OGSA-DAI 3.0 the client must explicitly specify synchronous or asynchronous execution.

If a request is executed synchronously the Execute operation will not return until the request has finished. The final request status will be returned in the SOAP response message. The request resource created for the request will be destroyed when the request is finished.

If a request is executed asynchronously the client must retrieve the final request status from the request resource that is created for the request. It is also the responsibility of the client to manager the lifetime of the request resource although by default it will expire 24 hours after creation.

N.3.14. Asynchronous data transport

OGSA-DAI 2.2 supports asynchronous data transport using the outputStream, inputStream, deliverToGDT and deliverFromGDT activities. Using these activities required the use of sessions.

In OGSA-DAI 3.0 the equivalent of asynchronous data transport is done through the use of data sinks and data sources. Clients can write data to a data sink using the data sink service. Data written to a data sink can be obtained and using within a workflow by the ReadFromDataSink activity. Similarly, clients can read data from a data source using the data source service. Data can be written to the data source from within a workflow using the WriteToDataSource activity.

Data can be passed between two OGSA-DAI servers by using the ObtainFromDataSource and DeliverToDataSink activities which, respectively, pull data from, and write data to, data sources and data sinks on remote OGSA-DAI servers.

Note that in OGSA-DAI 3.0 this type of data transport does not require use of sessions.

N.3.15. Migrating a client that uses the client toolkit

This section gives example of how code fragments written using the OGSA-DAI 2.2 client toolkit can be converted to code that uses the OGSA-DAI 3.0 client toolkit.

Before we start it must be stated that there is virtually is no consistency between the OGSA-DAI client toolkit classes in 2.2 and 3.0 - everything will need changed. The sections here describe some of the changes that will need to be made. The section headings are tasks described using OGSA-DAI 2.2 terminology.

N.3.16. Getting a client-side service proxy

In OGSA-DAI 2.2 the DataService interface was used to send perform documents to an OGSA-DAI server. The GenericServiceFetcher class was used to obtain the DataService if the client was unsure if the server was running OGSA-DAI WSI or OGSA-DAI WSRF.

The equivalent nearest equivalent of DataService in OGSA-DAI 3.0 is DataRequestExecutionResource. This is a resource proxy that provides the execute method that is the OGSA-DAI 3.0 equivalent of DataService.perform methods.

Code such as:

String handle = "http://localhost:8080/axis/services/ogsadai/DataService";
String id = "MySQLResource";
DataService service = GenericServiceFetcher.getInstance().getDataService(handle, id);

should be migrated to:

URL baseURL = new URL("http://localhost:8080/dai/services");
Server server = new ServerProxy();
server.setDefaultBaseServicesURL(baseURL);
ResourceID resourceID = new ResourceID("DataRequestExecutionResource");
DataRequestExecutionResource drer = 
    server.getDataRequestExecutionResource(resourceID)

[Note]Note

The Server.setDefaultBaseServicesURL method takes only the base URL that is common to all six OGSA-DAI services.

If the OGSA-DAI 2.2 code using the WSRFDataService or WSIDataService classes then these map to GTServer and AxisServer classes in OGSA-DAI 3.0. Code such as:

String handle = "http://localhost:8080/wsrf/services/ogsadai/DataService";
String id = "MySQLResource";
DataService service = new WSRFDataService(handle, id);

should be migrated to:

URL baseURL = new URL("http://localhost:8080/dai/services");
Server server = new GTServer();
server.setDefaultBaseServicesURL(baseURL);
ResourceID resourceID = new ResourceID("DataRequestExecutionResource");
DataRequestExecutionResource drer = server.getDataRequestExecutionResource(resourceID)

And similarly when going from WSIDataService to AxisServer.

N.3.17. Listing data resources

To obtain a list of data resources at an OGSA-DAI server in OGSA-DAI 2.2 required code like:

String handle = "http://localhost:8080/axis/services/ogsadai/DataService";
DataService service = GenericServiceFetcher.getInstance().getDataService(handle, null);
ResourceID[] resources = service.getResourceIDs();
for (int i=0; i<resources.length; i++) {
    System.out.println(resources[i].getName());
}

The equivalent of this for OGSA-DAI 3.0 is:

URL baseURL = new URL("http://localhost:8080/dai/services");
Server server = new ServerProxy();
server.setDefaultBaseServicesURL(baseURL);
ResourceID[] resources = server.listResources(ResourceType.DATA_RESOURCE);
for (int i=0; i<resources.length; i++) {
    System.out.println(resources[i].toString());
}

N.3.18. Building a request

To build an activity request that queries a relational database, converts the results to WebRowSet format and returns the result in the response message, OGSA-DAI 2.2 code would look like:

SQLQuery query = 
  new SQLQuery("select * from littleblackbook where id='3475'");
WebRowSet rowset = new WebRowSet(query.getOutput());
ActivityRequest request = new ActivityRequest();
request.add(query);
request.add(rowset);

In OGSA-DAI 3.0 we must build a Workflow rather than an ActivityRequest. The simplest type of Workflow is a Pipeline. Here is the equivalent OGSA-DAI 3.0 code:

SQLQuery query = new SQLQuery();
query.setResourceID("MySQLResource");
query.addExpression("select * from littleblackbook where id='3475'");

TupleToWebRowSetCharArrays tupleToWebRowSet =
    new TupleToWebRowSetCharArrays();
tupleToWebRowSet.connectDataInput(query.getDataOutput());

DeliverToRequestStatus deliverToRequestStatus = 
    new DeliverToRequestStatus();
deliverToRequestStatus.connectInput(tupleToWebRowSet.getResultOutput());

PipelineWorkflow pipeline = new PipelineWorkflow();
pipeline.add(query);
pipeline.add(tupleToWebRowSet);
pipeline.add(deliverToRequestStatus);

The are several important differences between these code snippets:

  • The SQLQuery activity has a setResourceID method to specify the resource this activity is to be targeted at. This is a new concept in OGSA-DAI 3.0. In OGSA-DAI 2.2 a DataService can be used to access only a single DataServiceResource which in turn gives access to a single data resource, for example a database. In OGSA-DAI 3.0 a single DataRequestExecutionResource can be used to access many data resources.Now that the DRER can give access to multiple data resources the resource ID of the data resource the SQL query is to be run up must be specified using the SQLQuery.setResourceID method.

  • The OGSA-DAI 3.0 workflow contains the DeliverToRequestStatus activity. This activity writes data to the Request resource's Request Status. In OGSA-DAI 2.2 any unconnected output was automatically written to the response document and the request was executed synchronously. In OGSA-DAI 3.0 unconnected outputs will result in an error.

N.3.19. Sending a request

To send a request to the server the OGSA-DAI 2.2 code would look like:

Response response = service.perform(request);

The OGSA-DAI 3.0 equivalent is:

RequestResource requestResource = 
  drer.execute(pipeline, RequestExecutionType.SYNCHRONOUS);

In OGSA-DAI 2.2 a perform document was executed synchronously if there was one or more unconnected outputs and asynchronously otherwise. In OGSA-DAI 3.0 requests, unconnected outputs are illegal and the client must explicitly specify synchronous or asynchronous execution.

Asynchronous execution is generally preferable to synchronous execution as it gives the client more control of the request. In particular it allows the client to terminate a request. To execute a workflow asynchronously, poll for the workflow to complete and then retrieve the final request status such that data can be read, the following code can be used:

int pollInterval = 500;     //   500ms = 0.5 seconds
int pollTimeout  = 60*1000; // 60000ms = 1 minute
RequestResource requestResource = 
  drer.execute(pipeline, RequestExecutionType.ASYNCHRONOUS);
requestResource.pollUntilRequestCompleted(
   POLL_INTERVAL, POLL_TIMEOUT);
requestResource.getRequestStatus();

The following code is an example of how to terminate a request during its execution.

RequestResource requestResource = 
  drer.execute(pipeline, RequestExecutionType.ASYNCHRONOUS);

// ... user decides the request is no longer required

// Terminate request
requestResource.destroy();

N.3.20. Processing results

In OGSA-DAI 2.2 results can be obtained from an activity via convenience methods in the client toolkit activity implementation. For example to get a ResultSet containing the results of an SQL query the code would look like:

SQLQuery query = 
  new SQLQuery("select * from littleblackbook where id='3475'");
WebRowSet rowset = new WebRowSet( query );
ActivityRequest request = new ActivityRequest();
request.add( query );
request.add( rowset );
service.perform( request );
ResultSet rs = rowset.getResultSet();

Obtaining the results from activities is much the same in OGSA-DAI 3.0. The major difference is most activities can now return multiple results where previously they could only return one. Most activities therefore adopt an iterator pattern to access the results. The OGSA-DAI code equivalent to the above is:

SQLQuery query = new SQLQuery();
query.setResourceID("MySQLResource");
query.addExpression("select * from littleblackbook where id='3475'");

TupleToWebRowSetCharArrays tupleToWebRowSet =
    new TupleToWebRowSetCharArrays();
tupleToWebRowSet.connectDataInput(query.getDataOutput());

DeliverToRequestStatus deliverToRequestStatus = 
    new DeliverToRequestStatus();
deliverToRequestStatus.connectInput(tupleToWebRowSet.getResultOutput());

PipelineWorkflow pipeline = new PipelineWorkflow();
pipeline.add(query);
pipeline.add(tupleToWebRowSet);
pipeline.add(deliverToRequestStatus);

RequestResource requestResource = 
  drer.execute(pipeline, RequestExecutionType.SYNCHRONOUS);

ResultSet rs = tupleToWebRowSet.nextResultAsResultSet();

N.3.21. Performing a sequence of queries

In OGSA-DAI 2.2 to process a sequence of SQL queries required the use of multiple SQLQuery activities. The code would look like:

 
SQLQuery query1  = 
  new SQLQuery("select * from littleblackbook where name like '%Krause'");
WebRowSet rowset1 = new WebRowSet( query1.getOutput() );
SQLQuery query2  = 
  new SQLQuery("select * from littleblackbook where name like '%Sugden'");
WebRowSet rowset2 = new WebRowSet( query2.getOutput() );
Request request = new ActivityRequest( 
    new RequestComponent[] { query1, rowset1, query2, rowset2 });
Response response = service.perform( request );

ResultSet rs1 = rowset1.getResultSet();
// code to use the result set goes here

ResultSet rs2 = rowset2.getResultSet();
// code to use the result set goes here

In OGSA-DAI 3.0 activities can generally produced multiple results. A single SQLQuery activity can now process a sequence of queries. The OGSA-DAI 3.0 equivalent of the above code is:

 
SQLQuery query = new SQLQuery();
query.setResourceID("MySQLResource");
query.addExpression("select * from littleblackbook where name like '%Krause'");
query.addExpression("select * from littleblackbook where name like '%Sugden'");

TupleToWebRowSetCharArrays tupleToWebRowSet =
    new TupleToWebRowSetCharArrays();
tupleToWebRowSet.connectDataInput(query.getDataOutput());

DeliverToRequestStatus deliverToRequestStatus = 
    new DeliverToRequestStatus();
deliverToRequestStatus.connectInput(tupleToWebRowSet.getResultOutput());

PipelineWorkflow pipeline = new PipelineWorkflow();
pipeline.add(query);
pipeline.add(tupleToWebRowSet);
pipeline.add(deliverToRequestStatus);

RequestResource requestResource = 
  drer.execute(pipeline, RequestExecutionType.SYNCHRONOUS);

while( tupleToWebRowSet.hasNextResult()) {
  ResultSet rs = tupleToWebRowSet.nextResultAsResultSet();
  
  // code to use the result set goes here
}

There are subtle differences between these two implementations. In the OGSA-DAI 2.2 version the two SQLQueries will be executed in parallel while in the OGSA-DAI 3.0 version they are executed in sequence. The OGSA-DAI 3.0 implementation of the client toolkit is based very much upon a streaming model. One side effect of this is that it is not possible to read from the two result sets at once.

N.3.22. Flow and sequence controls

In OGSA-DAI 2.2 it was possible to specify that groups of activities should be executed in parallel or sequentially using Flow and Sequence classes. For example, the following code creates the new table, loads data into this table and finally queries this new table in sequence.

 
SQLUpdate create = new SQLUpdate("create table mytable (id integer, number integer)");
SQLQuery query1  = 
   new SQLQuery("select id, phone from littleblackbook where name like '%Krause'");
WebRowSet rowset1 = new WebRowSet(query1.getOutput());
SQLBulkLoad bulkload = new SQLBulkLoad( rowset1.getOutput(), "mytable" );
SQLQuery query2 = new SQLQuery("select count(*) from mytable");
WebRowSet rowset2 = new WebRowSet(query2.getOutput());
Sequence sequence = new Sequence();
sequence.addChild(create);
sequence.addChild(query1);
sequence.addChild(rowset1);
sequence.addChild(bulkload);
sequence.addChild(query2);
sequence.addChild(rowset2);
service.perform(sequence);

In OGSA-DAI 3.0 these Flow and Sequence classes are replaced by ParallelWorkflow and SequenceWorkflow. The OGSA-DAI 3.0 equivalent of the above code is:

 
// Build 1st pipeline to create new table called 'mytable'
SQLUpdate sqlUpdate = 
    new SQLUpdate("create table mytable (id integer, number integer)");

DeliverToRequestStatus deliverToRequestStatus1 = 
    new DeliverToRequestStatus();
deliverToRequestStatus1.connectInput(sqlUpdate.getResultOutput());

PipelineWorkflow pipeline1 = new PipelineWorkflow();
pipeline1.add(sqlUpdate);
pipeline1.add(deliverToRequestStatus1);

// Build 2nd pipeline to query littleblack book and write to 'mytable'
SQLQuery query1 = new SQLQuery();
query1.setResourceID("MySQLResource");
query1.addExpression("select id, phone from littleblackbook where name like '%Krause'");

SQLBulkLoadTuples bulkload = new SQLBulkLoadTuples();
bulkload.addTableName("mytable");
bulkload.connectDataInput(query1.getDataOutput());

DeliverToRequestStatus deliverToRequestStatus2 = 
    new DeliverToRequestStatus();
deliverToRequestStatus2.connectInput(bulkload.getResultOutput());

PipelineWorkflow pipeline2 = new PipelineWorkflow();
pipeline2.add(query1);
pipeline2.add(bulkload);
pipeline2.add(deliverToRequestStatus2);

// Build third pipeline to query mytable
SQLQuery query2 = new SQLQuery();
query2.setResourceID("MySQLResource");
query2.addExpression("select count(*) from mytable");

TupleToWebRowSetCharArrays tupleToWebRowSet =
    new TupleToWebRowSetCharArrays();
tupleToWebRowSet.connectDataInput(query2.getDataOutput());

DeliverToRequestStatus deliverToRequestStatus3 = 
    new DeliverToRequestStatus();
deliverToRequestStatus3.connectInput(tupleToWebRowSet.getResultOutput());

PipelineWorkflow pipeline3 = new PipelineWorkflow();
pipeline3.add(query2);
pipeline3.add(tupleToWebRowSet);
pipeline3.add(deliverToRequestStatus);

// Put all pipelines in a sequence
SequenceWorkflow sequence = new SequenceWorkflow();
sequence.addChild(pipeline1);
sequence.addChild(pipeline2);
sequence.addChild(pipeline3);

// Execute the workflow
RequestResource requestResource = 
  drer.execute(sequence, RequestExecutionType.SYNCHRONOUS);

N.3.23. Transferring data between OGSA-DAI servers

In OGSA-DAI 2.2 the following code could be used transfer data between two data service resources on two different OGSA-DAI servers. In this example the data is pulled rather than pushed.

 
// Create two client DataService objects for the source 
// and sink data services.
DataService sourceService =
    GenericServiceFetcher.getInstance().getDataService(
      sourceHandle, sourceResourceID);
DataService sinkService =
    GenericServiceFetcher.getInstance().getDataService(
      sinkHandle, sinkResourceID);

// Assemble the activity request for the source data service. 
// Session requirements must be specified to ensure that a new 
// session is created.

SQLQuery sqlQuery =
    new SQLQuery("select * from littleblackbook where id<3000");
WebRowSet rowset = new WebRowSet(sqlQuery.getOutput());
DTOutputStream outputStream = new DTOutputStream();
outputStream.setInput(rowset.getOutput());
ActivityRequest sourceRequest = new ActivityRequest();
sourceRequest.add(sqlQuery);
sourceRequest.add(rowset);
sourceRequest.add(outputStream);
sourceRequest.setSessionRequirements(new JoinNewSession());

// Perform the request at the source data service resource
// and retrieve details of the session that the request has 
// been joined to.

Response response = sourceService.perform(sourceRequest);
Session session = response.getSession();

// At this stage the session stream has been created at the 
// source data service resource. Next a request must be assembled 
// to instruct the sink data service resource to read the result 
// data from the session stream.

DeliverFromDT deliverFromDT = new DeliverFromDT();
deliverFromDT.setDataTransportInput(outputStream.getDataTransport());
deliverFromDT.setDataTransportMode(DataTransportMode.BLOCK);
ActivityRequest sinkRequest = new ActivityRequest();
sinkRequest.add(deliverFromDT);

// We can now send the request to the sink data service resource 
// and display the response, which should contain the transported data.

response = sinkService.perform(sinkRequest);
System.out.println(response.getAsString());

// Finally, terminate the session that was created at the source 
// data service resource.

sourceRequest = new ActivityRequest();
sourceRequest.setSessionRequirements(new TerminateSession(session));
sourceService.perform(sourceRequest);

The equivalent code for OGSA-DAI 3.0 is:

 
// Get source and sink servers and DRERs

Server sourceServer = new ServerProxy();
sourceServer.setDefaultBaseServicesURL(sourceURL);
Server sinkServer = new ServerProxy();
sinkServer.setDefaultBaseServicesURL(sinkURL);
ResourceID resourceID = new ResourceID("DataRequestExecutionResource");
DataRequestExecutionResource sourceDRER = 
    server.getDataRequestExecutionResource(resourceID)
DataRequestExecutionResource sinkDRER = 
    server.getDataRequestExecutionResource(resourceID)

// Create a data source

DataSourceResource dataSource = sourceDRER.createDataSourceResource();
  
// Assemble workflow to write data to the data source

SQLQuery query = new SQLQuery();
query.setResourceID("MySQLResource");
query.addExpression("select * from littleblackbook where id<3000");

TupleToWebRowSetCharArrays tupleToWebRowSet =
    new TupleToWebRowSetCharArrays();
tupleToWebRowSet.connectDataInput(query.getDataOutput());

WriteToDataSource writeToDataSource = new WriteToDataSource();
writeToDataSource.setResourceID(dataSource.getResourceID());
writeToDataSource.connectInput(tupleToWebRowSet.getResultOutput());

PipelineWorkflow writeWorkflow = new PipelineWorkflow();
writeWorkflow.add(query);
writeWorkflow.add(tupleToWebRowSet);
writeWorkflow.add(writeToDataSource);

// Execute this workflow

RequestResource sourceRequestResource = 
  sourceDRER.execute(writeWorkflow, RequestExecutionType.ASYNCHRONOUS);
sourceRequestResource.pollUntilRequestStarted(POLL_INTERVAL);

// Assemble workflow to read from data source

ObtainFromDataSource obtainFromDataSource = new ObtainFromDataSource();
obtainFromDataSource.addResourceID(dataSource.getResourceID());
obtainFromDataSource.addURL(
    sourceServer.getDefaultDataSourceServiceURL());
obtainFromDataSource.addMode(ObtainFromDataSource.BLOCK);
obtainFromDataSource.addNumOfBlocks(1000);

DeliverToRequestStatus deliverToRequestStatus = 
    new DeliverToRequestStatus();
deliverToRequestStatus.connectInput(obtainFromDataSource.getDataOutput());

PipelineWorkflow readWorkflow = new PipelineWorkflow();
readWorkflow.add(obtainFromDataSource);
readWorkflow.add(deliverToRequestStatus);

// Execute this workflow

RequestResource sinkRequestResource = 
  sourceDRER.execute(readWorkflow, RequestExecutionType.SYNCHRONOUS);

// Destroy the resources
sourceRequestResource.destroy();
dataSource.destroy();

N.3.24. Processing partial results

If an activity encounters an error during its execution that activity will typically stop processing and the error will propagate through the workflow. Any data produced prior to the error occurring will still be available from the request status.

Consider the example where an SQLQuery activity is executes two queries. The first query is valid but the second is not. The following code shows how to retrieve the first result set and get full details of the error associated with the second one. In this example synchronous execution is used:

 
String[] expressions =
    new String[] {
"SELECT id, name, address, phone FROM littleblackbook " + 
"where id > 2 and id < 12 order by id",
"BAD QUERY"};

// Build the activities and activity chain
SQLQuery query = new SQLQuery();
query.setResourceID("MySQLResource");

for (int i=0; i<expressions.length; ++i)
{
    query.addExpression(expressions[i]);
}

TupleToWebRowSetCharArrays tupleToWebRowSet =
    new TupleToWebRowSetCharArrays();
tupleToWebRowSet.connectDataInput(query.getDataOutput());

DeliverToRequestStatus deliverToRequestStatus = 
    new DeliverToRequestStatus();
deliverToRequestStatus.connectInput(tupleToWebRowSet.getResultOutput());

// Build the workflow
PipelineWorkflow pipeline = new PipelineWorkflow();
pipeline.add(query);
pipeline.add(tupleToWebRowSet);
pipeline.add(deliverToRequestStatus);

RequestResource requestResource;
try
{
    requestResource = 
    mDRER.execute(pipeline, RequestExecutionType.SYNCHRONOUS);
}
catch(RequestExecutionException requestExecutionException)
{
    // Catch an request execution exception and get the request
    // resource from it.
    requestResource = requestExecutionException.getRequestResource();
    
    // Display status and any error messages for the activities
    
    System.out.println("Execution status: " + 
        requestResource.getRequestExecutionStatus());

    Activity[] activities = 
        new Activity[]{query, tupleToWebRowSet,deliverToRequestStatus};
    
    for (int i=0; i<activities.length; ++i)
    {
        if (activities[i].hasErrorMessages())
        {
            System.out.println(
                "Activity " + activities[i].getInstanceName() + 
                "of type " + activities[i].getActivityName() + 
                " has errors:");
    
            Message[] errorMessages = activities[i].getErrorMessages();
    
            for (int j=0; j<errorMessages.length; ++j)
            {
                System.out.println("\t" + errorMessages[j].toString());
            } 
        }
    }
}

// Process as many result sets as possible.

try
{
    while (tupleToWebRowSet.hasNextResult())
    {
        ResultSet rs = tupleToWebRowSet.nextResultAsResultSet();
        // Process the result set
        rs.close();
    }
}
catch(DataStreamErrorException dataStreamErrorException)
{
    // An error occurred when processing the data stream
}

N.4. Upgrading activity implementations from 2.2 to 3.0

This section attempts to provide some advice for developers migrating an OGSA-DAI activity from OGSA-DAI 2.2 to OGSA-DAI 3.0, henceforth referred to as 2.2 and 3.0 respectively. This does not seem to be straight-forward since basic concepts about activities have changed in the move to 3.0. It is advisable that developers understand these concept changes before migrating code.

Initially we are going to discuss the four different steps that used to take place when implementing an activity in 2.2 and see the differences compared to release 3.0. After that we will attempt a step by step migration of an activity on 2.2 into an activity into release 3.0.

N.4.1. Introduction

The most important point that needs the developer's attention, and one of the major differences of activities in the new release, is that there is no longer a requirement for XML that is activity specific. Therefore, the arbitrary XML that used to exist in order to specify inputs, outputs and parameters of activities has now been removed. The rationale behind this decision is that all data that used to come into the activity as XML are now coming directly as data to be read by the activity input. Having this in mind we can now move to some other issues before reaching the actual migration of activities.

N.4.2. Define XML Schema

As explained above all activities are now described by a common, unique, mininal XML Schema. This means that there no longer is a need to write an explicit XML Schema describing the XML syntax for each activity. As a concequence of this, instead of the XML validation for inputs, outputs and parameters of each activity the only concern is to set up inputs and outpus for this activity properly. The common schema that is describing all activities looks like:

<!-- Activity type -->

  <xsd:complexType name="ActivityType">
    <xsd:sequence>
      <xsd:element name="inputs" type="tns:InputsType"/>
      <xsd:element name="outputs" type="tns:OutputsType"/>
    </xsd:sequence>
    <xsd:attribute name="name" type="xsd:string" use="required"/>
    <xsd:attribute name="instanceName" type="xsd:ID" use="required"/>
    <xsd:attribute name="resource" type="xsd:string" use="optional"/>
  </xsd:complexType>

  <!-- Activity inputs type -->

  <xsd:complexType name="InputsType">
    <xsd:sequence>
      <xsd:element name="input" minOccurs="0" maxOccurs="unbounded">
        <xsd:complexType>
          <xsd:choice>
            <xsd:element name="inputStream" type="tns:InputStreamType"/>
            <xsd:element name="inputLiteral" type="tns:DataType" 
                         maxOccurs="unbounded"/>
          </xsd:choice>
          <xsd:attribute name="name" type="xsd:string" use="required"/>
        </xsd:complexType>
      </xsd:element>
    </xsd:sequence>
  </xsd:complexType>

  <!-- Activity outputs type -->

  <xsd:complexType name="OutputsType">
    <xsd:sequence>
      <xsd:element name="outputStream" type="tns:OutputStreamType"
                   minOccurs="0" maxOccurs="unbounded"/>
    </xsd:sequence>
  </xsd:complexType>

  <!-- Activity input stream type -->

  <xsd:complexType name="InputStreamType">
    <xsd:attribute name="pipe" type="xsd:ID" use="required"/>
  </xsd:complexType>
  
  <!-- Activity output stream type -->

  <xsd:complexType name="OutputStreamType">
    <xsd:attribute name="name" type="xsd:string" use="required"/>
    <xsd:attribute name="pipe" type="xsd:IDREF" use="required"/>
  </xsd:complexType>

  <!-- Data values -->

  <xsd:complexType name="ListBeginType"/>

  <xsd:complexType name="ListEndType"/>

  <xsd:complexType name="DataType">
    <xsd:choice>
      <xsd:element name="string" type="xsd:string"/>
      <xsd:element name="charArray" type="xsd:string"/>
      <xsd:element name="binary" type="xsd:base64Binary"/> 
      <xsd:element name="float" type="xsd:float"/> 
      <xsd:element name="double" type="xsd:double"/> 
      <xsd:element name="int" type="xsd:int"/> 
      <xsd:element name="long" type="xsd:long"/> 
      <xsd:element name="boolean" type="xsd:boolean"/> 
      <xsd:element name="date" type="xsd:dateTime"/> 
      <xsd:element name="listBegin" type="tns:ListBeginType"/>
      <xsd:element name="listEnd" type="tns:ListEndType"/>
    </xsd:choice>
  </xsd:complexType>

As one can see, an activity there is described by:

  • a number of inputs and outputs,

  • a name,

  • an instance name and

  • the resource that it is targetting, if any.

N.4.3. Writing an implementation class in Java

Before providing actual coding guidelines, let's we point out how the new activity model differs from the old one:

  1. As explained in the previous section, there is no more need for explict XML validation in the activity code to take place.

  2. As discused in other sections the activity framework is the enactment model that is part of the core engine and is responsible for performing tasks and streaming the data.

In 2.2 release the processing of blocks produced and consumed by activities is controlled by the pipe which is an external object to the activity. Therefore, processBlock() is called many times until the processing is completed by the activity framewor. The usual case is that one block is processed per call. Furthermore, an initialize() method preceeds the block processing and is invoked to perform any initialisation that the activity requires prior to it.

In contrast, in release 3.0 the processing of blocks is controlled by the activity itself. This means that a process() method is called only once. All actions necessary for the activity processing (initialisation, actual processing and clean up) take place in this single method. Of course, as is explained later on, useful abstract classes are provided that split the processing into different stages (initial processing, main processing, post processing and clean up) to help the developer implement an activity in a more convinient way. However, it should emphasized that all these auxiliary classes do is a fragmented manipulation of the process() method.

In addition, a consequence to the fact that the process() method is called only once, an activity can produce or consume as many blocks as necessary to maximise the streaming of the workflow of which it is a part.

N.4.4. Activity implementation

Detailed guidelines on how to write an activity implementation can be found in Chapter 24, How to write an activity

The remainder of this section will go though a step-by-step migration guide. The ReadFromFileActivity is used as an example for this (previously FileAccessActivity in the 2.2 release).

N.4.4.1. Class declaration

The activity is targetting a file resource and is of matched iterative pattern since its inputs have granularity at most one and they do not have multiple occurrences so its declaration in 3.0 would be:

public class ReadFromFileActivity extends
MatchedIterativeActivity implements ResourceActivity

N.4.4.2. Constructor

The constructor with Element parameter that used to present in 2.2 has gone in 3.0 since there is no more requirement for validation and parsing of XML to get the data from the parameters. In effect, there are no parameters required in constructors anymore as mentioned earlier on.

In release 3.0, we now have to get the inputs and outputs based on their names. For this reason, we define class variables with the input and output names as such:

/** File to be read */
public static final String INPUT_FILE = "file";

/** Offset in bytes */
public static final String INPUT_OFFSET= "offset";

/** Number of bytes to read  */
public static final String INPUT_LENGHT = "lenght";

/** Activity output name */
public static final String OUTPUT_DATA = "data";

Furthermore, we need to get the actual blockreaders and blockwriters that are mapped onto these names. The former are set up through the getIterationInput() method which will look like:

protected ActivityInput[] getIterationInputs() 
{
    return new ActivityInput[] {
    	new TypedOptionalActivityInput(INPUT_FILE, String.class, "/"),
        new TypedOptionalActivityInput(INPUT_OFFSET, Long.class, new Long(0)),
        new TypedOptionalActivityInput(INPUT_LENGHT, Long.class, new Long(-1)),
    };
}

The latter is validated for existence and set up as a blockwriter object in the processFirst() method.

N.4.4.3. Initialisation

The initialize method that existed in 2.2 has gone in 3.0. In 2.2 we can see that in this method the inputs and outputs as well as the access provider is set:

mOutput = mContext.getOutput(mInternalOutputs[0]);

and

mFile = new File(mTopDir,mFilename);
// Ensure user read access to this file
mFileAccessProvider.validateReadFile(mCredentials,mFile);

The set up of the accessor now takes place in the implementation of setTargetResourceAccessor((ResourceAccessor resourceAccessor) and getTargetResourceAccessorClass() provided by the ResourceActivity interface.These methods will be like:

public void setTargetResourceAccessor(ResourceAccessor resourceAccessor) 
{
    mResource = (FileAccessProvider) resourceAccessor;
}

and

public Class getTargetResourceAccessorClass()
{
    return FileAccessProvider.class;
}

N.4.4.4. Execution

ProcessFirst()

There is no need for this method in 2.2. In release 3.0, we use this method to validate the output. Note that although the two methods bear the same name, their purpose is different as explained previously.

processBlock()

processBlock() does not exist anymore. The main functionality of the activity now lies on processIteration(Object[] data) method.

In release 2.2 there were member variables to store the values of the parameter inputs (mFilename, mBase64encode, mOffset and mLength)through the XML parsing that took place in the contructor of the activity. Now we get these values in the processIteration method from the parameter array of this method by doing te following:

String filePath = (String) iterationData[0];
Long offset = (Long) iterationData[1];
Long lenght = (Long) iterationData[2];

Furthermore, if we want to write to the output write we now do:

output.write(block) in contrary to :
output.put(buf.toString());

Additionally, and although it is not visible in this example because FileAccessActivity does not have any inputs, processBlock() would typically look like:

if (mInput.hasNext())
{
    Object inputBlock = mInput.next();
    // do something with the object
}

and the engine would iteratively call processBlock() method. In 3.0, the iteration of the activity processing takes place in the activity and more precisely the processing algorithm of MatchedIterativeActivity takes care of it.

Finally, there is no need for explicit setCompleted() method in 3.0 since MatchedIterativeActivity will finalise the processing once no more data is available for consumption.

N.4.5. Writing a corresponding client toolkit class in Java

If you do not wish to interact with your new activity using the client toolkit then this step can be ommitted. Otherwise a client toolkit activity implementation class is needed. This is a client side class that corresponds to the server-side activity and makes it easier for clients to build workflows that include that activity.

N.4.6. Client toolkit implementation

Detailed guidelines on how to write a client toolkit implementation can be found in Chapter 24, How to write an activity

Let's go now though a step-by-step migration guide. We are going to use ReadFromFile (ReadFromFile in 2.2) as an example.

Having established that there is no more XML handling on activities describing explicitly inputs, outputs and parameters the following should easily make sense to the activity developer:

The generateXML() method no longer exists. What we do now is create client-side representations of activity inputs and outputs. To do this we declare their names which must match the input/output names of the server-side implementation. This looks like:

/** Directory input name */
public final static String FILE_INPUT = "file";
 
/** Directory input name */
public final static String OFFSET_INPUT = "offset";
 
/** Directory input name */
public final static String LENGTH_INPUT = "length";
 
/** operation status output name */
public final static String OUTPUT_DATA = "data";

in release 3.0. Furthermore we declare corresponding member variables of ActivityInput and ActivityOutput objects that are initialised in the activity constructor:

/** File input */
private ActivityInput mFileInput;

/** Offset input */
private ActivityInput mOffsetInput;

/** Recurse directories input */
private ActivityInput mLengthInput;

/** Data output */
private ActivityOutput mDataOutput;

and

/** Constructor.*/
public ReadFromFile()
{
    super(DEFAULT_ACTIVITY_NAME);
    mFileInput   = new SimpleActivityInput(FILE_INPUT);
    mOffsetInput = new SimpleActivityInput(OFFSET_INPUT, SimpleActivityInput.OPTIONAL);
    mLengthInput = new SimpleActivityInput(LENGTH_INPUT, SimpleActivityInput.OPTIONAL);
    mDataOutput = new SimpleActivityOutput(OUTPUT_DATA);
}

In 2.2 we used to have :

  • The addInput to add an input. This is now reflected by creating a new ActivityInput object in the constructor.

So addInput("readFileInput") will be replaced with:

/** Directory input name */
public final static String FILE_INPUT = "file";

/** File input */
private ActivityInput mFileInput;
mFileInput = new SimpleActivityInput(FILE_INPUT);

The setXXX methods to set the values of a parameter. Now these methods have been replaces by addXXX methods in release 3.0. So for example:

public void setFilename(String filename) 
{
    mFilename = filename;
}

will be replaced by:

public void addFile(final String filename)
{
    mFileInput.add(new StringData(filename));
}

The getXXX methods to get the value of a parameter. These do not exist anymore in 3.0.

The setInput(ActivityOutput input) methods where the output of another activity was connected to this input now use connectXXXInput(ActivityOutput input) methods for this purpose. So:

public void setInput(ActivityOutput input) 
{
    if (input == null) 
    {
        throw new IllegalArgumentException("input must not be null");
    }
    super.setInput(0, input.getName());
}

will be replaced by:

public void connectFileInput(SingleActivityOutput output)
{
    mFileInput.connect(output);
}
[Note]Note

In 3.0 the addXXX and connectXXX methods for an input are mutually exclusive, i.e. an activity can have exactly one of this two methods so long its a mandatory input.

Some activities used to provide getXXX methods to return the output data in a developer-friendly format. This concept exists in release 3.0 as well but the way we formulate such methods is described in detail in Chapter 24, How to write an activity

N.4.7. Server-side activity deployment

Once your activity has been written it can be deployed on the OGSA-DAI server. This is described in Section 16.1.8, “Deploying an activity”.

N.5. Upgrading resource installer ANT properties files from 2.2 to 3.0

OGSA-DAI 2.2 resource deployment ANT target made use of a data service resource properties file. This was a file provided to the OGSA-DAI deployResource target and had the following format:

dai.resource.id=RelationalResource
dai.data.resource.type=Relational
dai.product.name=MySQL
dai.product.vendor=MySQL
dai.product.version=1.0
dai.data.resource.uri=jdbc:mysql://myhost:3306/daitest
dai.driver.class=org.gjt.mm.mysql.Driver
dai.credential=
dai.user.name=someUser
dai.password=somePassword

OGSA-DAI 3.0 provides an ANT target to convert OGSA-DAI 2.2 ANT data service resource properties files for relational, XMLDB and file system data resources into ANT properties compliant with the 3.0 deployment targets:

To run the convertor target do:

$ ant convert -Ddai.resource.jar.dir=DATABASE_DRIVER_JAR_DIRECTORY \
              -propertyfile PATH_TO_2.2_PROPERTIES_FILE

where

The output file will be named after the value of the dai.resource.id property in the 2.2 properties file with the extension .properties e.g. for the excerpt above the 3.0 properties file will be called RelationalResource.properties.

For example:

$ ant convert -Ddai.resource.jar.dir=/home/user/mySQLDriver/ \
              -propertyfile /home/user/mySQL.data.service.resource.properties

$ ant convert -Ddai.resource.jar.dir=/home/user/existDriver/ \
              -propertyfile /home/user/eXist.data.service.resource.properties

$ ant convert -propertyfile /home/user/file.data.service.resource.properties

You can then use the file with the 3.0 targets using the -propertyfile argument as described in the documentation for those targets.