|
|
Archive News | Quick Search | Contact Archivist | |
| NCSA BIMA Data Archive | ||||
The BIMA Data Archive currently contains 1085.094 gigabytes of data in 68267 datasets.
| About the Archive |
Archive Access
|
Related Links |
These three steps are explained in more detail in the next section. Note that in this discussion, we make two important assumptions:
Please note that passwords cannot be obtained from the archivist or through the archive web site.
Setting up a Download Client
To use the archive efficiently, it helps to have a download client
called DaRT (Data
Retrieval Tool). This Java application makes it possible to download
many datasets at once with considerable flexibility.
DaRT works with your browser as a so-called "helper app". This means your browser loads and executes DaRT from your local disk whenever it is needed (as opposed to an applet which is downloaded from the server). This means DaRT must be installed on your system before you can use it.
If you are at a consortium site, DaRT may already be installed on your system. Check with your local BIMA guru to find out its full pathname on disk. If are accessing the Archive from outside the consortium, you can obtain the latest version from the DaRT Home Page. No special (root) permissions are needed to install DaRT; it can happily be installed under your home directory for personal use if necesary.
While it is not necessary to add the directory that contains DaRT to your command search path, it is often helpful if you do. This will allow you to start DaRT from the command line which is handy when you want to delay the actual downloading for a later time.
In addition to DaRT, we also still support its predecessor,
xmrecvlist, as a download client. This program does not display a GUI
and does its downloading work silently. Thus, users who who still hooked
to xmrecvlist are strongly encouraged to upgrade
to DaRT since support for this program will be phased out as
the archive develops. Nevertheless, if you can't let go of
xmrecvlist just yet, you should consult the FAQ to ensure its propoer use.
Configuring your Browser
Once you know the location of DaRT on your system, you can configure
your web browser (e.g. Netscape) to use it whenever you request
multiple datasets from the archive. The easiest way to do this is to
edit the .mailcap file in your home directory on the
machine that you will run your broswer on. (It's best if you exit
from your browser before doing so.) This file needs to contain
the following line:
application/x-multiget; <pathname_to_dart> %s
where <pathname_to_dart> is the full pathname to the
DaRT program. For example, if the dart program is
located in /usr/local/bin, you put the follwing in your
.mailcap file:
application/x-multiget; /usr/local/bin/dart %s
Note that the %s is important: it represents the name of
the file that the archive sends to your browser when you request
selected datasets for downloading.
If you've used the Archive before, you might already have a line like
this that gives the pathname to xmrecvlist
(e.g. $MIRBIN/xmrecvlist); if so, you will need to
replace it with a line of the form above.
Netscape, like most broswers, provides a GUI interface for editing the
.mailcap file. To configure Netscape in this way, choose
"Preferences..." from the "Edit" menu; then in the popup window,
choose "Navigator->Applications" from the side menu. Search the
list for the entry, "application/x-multiget", select it, and press the
"Edit..." button. If the entry doesn't exist (because you've never
used the Archive from that machine before), add it as a new entry by
pressing the "New..." button. Choose "Application" under "Handled By"
and enter the dart pathname followed by a "%s". Clicking
"OK" saves the new setting.
If you have problems with your browser configuration (e.g. DaRT does not start up when retrieving data), consult QX of the FAQ document.
Using the Archive
As of this writing, typical interactions with the archive include
three activities:
Browsing Data
Datasets in the archive are organized into hierarchical collections,
and every collection has an ID associated with it. Collections are
divided into four main levels:
You can either select individual datasets for download or select all the datasets within a trial automatically by selecting the checkbox next to the trial's ID. Selecting a trial in this way will override any individual dataset selections within that trial.
Currently, three choices are provided as retrieval methods; you should select one based on the download client you are using:
When you use DaRT as your download client, you can take greater control of how the data is downloaded. You can set the directory where the data is unpacked, you can start and stop the download at anytime, you can even schedule the download for a later time. It displays all of your selected datasets, and--after you start the download--it shows the transfer progress. (See the DaRT User Manual for details on DaRT's capabilities.) Each dataset is typically sent as a tar file, so when the transfer is complete, DaRT will unpack the tar file within directory tree under the destination directory so that it is ready for processing.
If you are using DaRT version 3.0.2 or later, the data will be organized in a directory the same way as it is in the archive: seperated first by Project, then by Experiment, and then by Trial. Earlier versions of DaRT will unpack the data in the old style tree: first by date and then by observe script name.
Using xmrecvlist as your download client is not nearly as
helpful. xmrecvlist will silently begin downloading the
data into the directory from which you started netscape. The data
will be unpacked in the old date--observe file hierarchy. No feedback
is given indicating when the transfer is complete. Given this aloof
behavior, xmrecvlist users are strongly urged to
reconfigure their browsers to use DaRT.
Checking The Data Archive Status
You can also use your web browser to check the status of the BIMA
archive. The archive currently provides two types of status
information:
Many problems users experience can be traced to network difficulties happening anywhere between Hat Creek and NCSA or within NCSA itself. High network traffic is one such source. Occasionally, you may be unable to contact the archive server (with an error message like "Connection refused"). Such a problem often corrects itself after the traffic dies down. In this case, you should try your request later.
Note that if you are unable to access the archive as described above, sending us email is still helpful. It will alert the archivist to the problem, allowing him or her to ensure that the problem gets fixed as soon as possible. Note that problems in the network may also delay our receiving your message, thus delaying our response.
|
|
Quick Search | Contact Archivist |