Overview
DES consists of a web service application and synchronization program. Due to size, the REST web service is selected because of its ability to steam and using on-the-fly compression. The data exchange format used is SDMX Cross-Sectional Format (see http://www.sdmx.org/ for further information about SDMX). The synchronization program (called ComtradeTools) is currently a command-line program running under the .NET Framework machine. In the future, ComtradeTools might be upgraded to have a better user interface.

Data Exchange Architecture


1. UN Site contains a web server and database server. Comtrade, Tariff Line, and Total Trade Data are available through the web services.

2. Web Services can stream compressed trade data in SDMX-ML format or element-based XML. The web services also have some metadata, such as the data availabilities and references.

3. Organization #1 connects to the internet and obtains trade data using the internet browser. He/she can save the data to XML-file.

4. Organization #2 uses ComtradeTools (developed by UNSD) to obtain the trade data and convert it into text files. Other tools can also be used.

5. Organization #3 set up the automatic one-way synchronization so that the local trade database will always be up to date with the UN site.

6. All three organizations are authenticated by IP Addresses and Web Service Access Control List.

Database Design
DES database has a similar design with UN Comtrade: one table holds data for a year. Multiple tables are needed for multiple years, and a partitioned view is created to cover them. There will be two tables: Tariff Line Tables and Comtrade Tables. Both of them have the same table design. In addition to that, several tables are created for footnotes, explanatory notes, and memorandum items.

Application Design: Approach and Strategies
1. The strategies of Trade Data Dissemination are
· Using a standard data format. SDMX (http://www.sdmx.org) version 1.0 has been released, and it is being used as one of the formats for trade data dissemination;
· Leveraging Internet as a low-cost transfer medium and for its standards, such as http protocol, web services, etc.
· Platform independent means that client can use any operating systems, any database, and program languages to obtain trade data;
· Be able to transfer extensive data. The REST Web Service and internet stream compression play an important role here;
· The possibility for unattended one-way synchronization.

2. The implementation of the strategies:
· Architecture: “Pull” architecture due to the flexibility and tight security (such as UN Server can’t initiate http request to any websites). The client has to initiate a request.
· Synchronization Approach: By comparing the data availability between two sites for creating the list of new/updated datasets. By using timestamps.
· Data Format: SDMX-ML version 1.0 is currently being used.
· Medium and Protocol: Internet (with http protocol) via Web Services.
· Compression: The SDMX-ML is compressed during the data transfer. It reduces 70-80% of the bandwidth usage and download time.
· Encryption: None at the moment. If it is necessary, the https will be implemented instead of http.
· Security: Based on IP Address and Access Control List (to use Web Service).
· Tools: A small command-line program (beta) has been created that downloads Comtrade SDMX-ML via Web Services and (can) import them into the SQL Server. If it is necessary, a complete client program can be developed. Please bear in mind that this tool is not the only one that can obtain trade data. The client can get the trade data using an Internet Browser such as Netscape or Internet Explorer.

Synchronization program: ComtradeTools

For a start, ComtradeTools is created as a command-line program. The main disadvantage is that it is not user-friendly, and the user needs to understand how to use it. However, the ComtradeTools is not necessary to get data. Users can create his/her program.

The advantages of ComtradeTools are providing one-way synchronization, compression, and direct import to SQL database.

Similar Pages: