[workspace-user] gridftp Propagation - development question

[Tim Freeman]

On Tue, 27 Mar 2007 17:49:22 -0700
Duncan Penfold-Brown <dpb at uvic.ca> wrote:

> Hi -
> 
> I have a brief question about the development of some Workspace features 
> related to propagation using gridftp. I've noted that to successfully 
> use propagation with the gridftp (or gsiftp) protocol, both the 
> hypervisor node and the node submitting the workspace command (the 
> services node) must have valid proxies.

Hi Duncan,

What do you mean by "the node submitting the workspace command"?  I think you
mean the services node that is running the container?  It does submit the
workspace-control command but it does not need a valid proxy, it shouldn't have
anything to do with propagation.  The files are "pulled" from the hypervisor
nodes.

>  This seems somewhat impractical, 
> as giving the hypervisor node a proxy requires the ability to login and 
> execute a proxy-init command. This is inconvenient and causes some 
> maintenance (and possibly security) issues, and can be slightly 
> difficult to implement.

Agreed that would be impractical, but I don't think it should work like this.  

The intent, just like with the SCP propagation mechanism, is for
workspace-control to be a priviliged program that can access files on the image
node, ALL relevant files.  Like with SCP you could have static keys to do this
-- you can use EECs with GridFTP.  Or simply have a cron job refresh the proxy
as necessary.

The account used for workspace-control needs "control plane" priviliges, and
not just for propagation.  

Once a client has staged file(s) to the image node, they're out of its purview,
the infrastructure takes over and manages things from there.  (there are many
benefits to a managed model with respect to security and scheduling)

To really do staging + propagation right (with secret files at least), there
should be an agreed path structure on the image node where clients are given
rights to transfer files to specific directories on the image node and then the
workspace authorization policies only allow them to use files under specific
directories (this can be done with the authz callout).

There are possible ways around this where the service (or more likely an agent
of it) could create directories on the fly and hide such "internal" details
from clients entirely (if they chose... there are benefits to exposing paths,
e.g., being able to use out of band staging mechanisms).  But this is only
currently in the design phase.  It would be similar to what happens on the
hypervisor node with the temporary workspace-instance-specific file directories.

For non-secret files, the upcoming HTTP staging mechanism should provide a much
simpler entry into stage + propagate.

If possible, staging and replica location is probably better handled better by
higher level, orthogonal services.  But we do provide these hooks for staging
to allow basic "fire and forget" use of the workspace service without the need
to set up many other services and write workflows etc.

Tim