[gram-user] How to take multiple output files generated during the execution of the job through gridfs

[Chandan Maheshwari]

Hi all,

I am new to this, I have one application in which I have to deploy that
application on different clusters and have to run that through GridWay from
the job submission node.

So the seen is that my executable is on one cluster I have compiled that
over there. Now there are two cases as below:

See let me give you the example,

CASE1:
I am into the CLUSTER . there is one folder ABC which has input files
and the executable.
So the hierarchy is
ABC-> BCD,CDE required input files more than one
ABC-> EXE.

Now when I run my executable through here only like directly ./EXE,
then i
get the output correctly.

CASE 2:

I am into the HEAD NODE and I have made JOBTEMPLATE as below:

EXECUTABLE = gsiftp://PATH/EXE.exe
ENVIRONMENT
=LD_LIBRARY_PATH=/opt/intel/Compiler/11.0/074/lib/intel64:/opt/intel/impi/
3.2.2.006/lib64:/home/PATH/lib
INPUT_FILES = gsiftp://PATH/BCD
INPUT_FILES = gsiftp://PATH/CDE
STDOUT_FILE = stdout_file.${JOB_ID}
STDERR_FILE = stderr_file.${JOB_ID}
REQUIREMENTS = HOSTNAME = "CLUSTER"
#RANK = CPU_MHZ
TYPE = "mpi"
NP = 2

The status is that job failed:

stdout

CLUSTER at gg-blr Chandan$ more stdout_file.20744
/opt/torque/aux//62289.CLUSTER

stderr file is

CLUSTER at gg-blr Chandan$ more stderr_file.20744

/home/.globus/f45a5c20-fd14-11df-9b95-88edc4a760c9/scheduler_pbs_cmd_script:
line 38: ./gsiftp://PATH/EXE.exe: No sucH file or directory

/home/.globus/f45a5c20-fd14-11df-9b95-88edc4a760c9/scheduler_pbs_cmd_script:
line 38: ./gsiftp:/gg-blr.tfg/home/PATH/EXE.exe: No such
file or
directory

>From the above I want to say that while running the job in the cluster being
all the dependencies in the same folder in which the executable is there ,
the code runs successfully.

But when I run the code through submitting the job template from Grdway
being the dependency on the CLUSTER and the mismatch in the path because the
temporary folder is created on the HEADNODE and that gets deleted after the
completion of jobe the job fails as above

SECOND PROBLEM:

How can I take multiple file output from executable.

Like I have one script that takes multiple files from the user and link
these files with anothere files and gives names like
http://NAME.AAANAME.AAB NAME.AAC and so on upto Z

now in the job template we have to give the name of the output file that
gets generated and I dont know that how much files the user is going to
give. And my script generates some output files and I want to copy these
files from the temporary folder to my Location.

To be clear:
suppose the hierarchy is
FOLDER
File1
File2
File3
File4

So after the execution of the script it will generate:

NAME.AAA
NAME.AAB
NAME.AAC
NAME.AAD

All these file are linked to the input given.

Now these files will be saved in the tempory folder which gets deleted after
the execution is done.

So I have to copy these these files through the job template only. But in
that to copy I have to specifically give the name of the Output files but
how to know the name of the files bexause it depends on the number of the
input files given
So how to handle this

Please help.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.globus.org/pipermail/gram-user/attachments/20101206/8a85460e/attachment.htm>