[gt-user] Important information on Globus events and plans

Ian Foster foster at anl.gov
Wed Oct 21 06:46:52 CDT 2009


Dear Colleagues:

Earlier this year we requested input to help us shape our plans for  
Globus' future.  (http://www.mail-archive.com/gt-user@lists.globus.org/msg00927.html 
). Since then we have been listening, discussing, exploring  
alternatives, and finally making decisions about our future efforts in  
Globus. This email is a long-overdue follow-up to that previous note,  
summarizing our plans.

Note that this mail describes plans only for those portions of Globus  
led primarily by U Chicago, Argonne, and USC/ISI.  The Globus  
community is larger than just these institutions, with substantial  
contributions from many individuals and organizations based on many  
distinct project and funding priorities.  Our plans highlighted here  
includes much input from these contributors, but do not speak for  
those individuals and organizations.

A) EVENTS

We are planning several events in the coming months where Globus'  
future (and present) can be further explained, discussed, and refined,  
including:

  * SC'09 (http://sc09.supercomputing.org/), November 15-20, Portland,  
Oregon, USA: Please come visit us at the Argonne National Laboratory  
booth.

  * Viva Globus, December 16 at Argonne (near Chicago, Illinois, USA):  
It has been a while since our last "Viva Globus" contributors  
meeting.  We will revive this meeting series on December 16.  More  
information will be forthcoming soon.  This meeting is intended for  
substantial contributors to the Globus software, as an opportunity to  
update each other on our respective activities, discuss future plans  
and collaborations, and make decisions regarding community governance  
and processes.  Of particular focus at this next Viva Globus will be  
discussions and decisions regarding the future of the Globus Alliance  
governance.

  * GlobusWorld, March 2-4 at Argonne (near Chicago Illinois, USA):  
This latest installment of the GlobusWorld user conference series will  
be focused on users of the Globus software, including talks and  
tutorials on using new (and old) Globus software, and presentations  
and discussions by Globus users on their experiences.  More  
information about this conference will become available starting at  
SC'09.

B) DECISIONS

The following points summarize the decisions that we have made over  
the past 6 months during the re-evaluation process:

1.  Of paramount importance is improving the quality of the Globus  
software and the service we provide to Globus users.  We will continue  
to maintain and support the existing Globus Toolkit version 4.x (GT4)  
software as long as demand exists and funding allows, minimally  
through the end of the CDIGS project in December 2010.  And in all  
changes and enhancements we are making to Globus going forward, we are  
placing a strong emphasis on issues related to helping our community  
migrate forward.

2.  We have made some long-overdue, difficult decisions about some of  
the more problematic portions of Globus, including:

2.a.  There have been long struggles and confusion over the GRAM2 vs.  
GRAM4 components. We have resolved these problems by re-investing in,  
fixing, and enhancing GRAM2. This new version, called GRAM5, is fully  
backward compatible with GRAM2 (with two minor exceptions*), but  
solves its scalability issues and adds numerous frequently requested  
features.  It is currently in alpha testing by several major users,  
who are using it with existing GT4 Java jGlobus/COG clients, C  
clients, and Condor-G.  It will be released soon as part of GT5. We  
will continue to support GRAM4 at least through December 2010 (perhaps  
longer, depending upon demand and funding), but have begun to assist  
GRAM4 users in migrating to GRAM5. If you are an existing GRAM4 user  
and would like to discuss migration issues to GRAM5, please contact  
us. For a more complete description of the GRAM5 alpha release, see http://dev.globus.org/wiki/GRAM/GRAM5 
.  We welcome additional testing and bug reports on this GRAM5 alpha  
release.

(*The two exceptions to GRAM5's backward compatibility with GRAM2 are:  
(i) no support for MPICH-G/MPIG job rendezvous; and (ii) GRAM5 stages  
out stdout/err at the end of the job rather than streaming them out  
while the job runs.)

2.b.  The Reliable File Transfer (RFT) service has been of  
considerable interest to many Globus users, but in practice has  
suffered from difficulties in both use and operation.  We have decided  
to replace the RFT functionality with a new Globus.org service,  
described below.

2.c.  GT4 Java Core is based heavily on obsolete technology (Apache  
Axis 1.x) and standards (WSRF), yet nonetheless continues to provide  
tremendous value-add to Web Services-based Grid builders, particularly  
in the area of security and stateful resource management. With the  
urging of, and in partnership with, some of our large Java Core users  
such as the caGrid team at Ohio State University, we have begun the  
Globus Crux effort to update our Java Web Services stack to newer  
technologies (e.g., Apache CXF), while preserving and enhancing our  
core value-add security capabilities as a plug-in to CXF and allowing  
for WSRF protocol compatibility. We expect to release an alpha version  
of Crux by the end of 2009. See http://confluence.globus.org/display/whi/Crux+for+GT+Developers 
  for details.

2.d.     While MDS is applicable to a broad range of monitoring and  
discovery tasks, in practice its predominant use has been to build  
service registries/catalogs for TeraGrid, caBIG, and BIRN, with  
limited adoption for systems monitoring in a few other communities.  
Since MDS4 is intimately intertwined with GT4 Java Core, it would  
require a substantial reimplementation effort to update it to Crux.   
Meanwhile, the state of monitoring tools has evolved considerably  
since MDS4 was first conceived, with the widespread adoption of highly  
capable, open source monitoring tools such as Nagios. Therefore we  
have begun work on a more focused effort to design and implement next  
generation service registry capabilities using Crux, which we are  
calling our Integrated Information Services (IIS) effort.  This IIS  
effort is still in the requirements gathering phase, with no releases  
planned until sometime in 2010.  We recommend that monitoring needs be  
met using other tools such as Nagios.

3. Development continues unabated for the other components of the  
Globus Toolkit (e.g., GridFTP, RLS, Myproxy, GSI-OpenSSH, etc) and the  
many other active dev.globus components (http://dev.globus.org).

4.  We plan to release Globus Toolkit version 5 (GT5) in late 2009, as  
recently announced (http://www.mail-archive.com/gt-user@lists.globus.org/msg01311.html 
). Like previous versions of GT, this version will continue to offer a  
collection of tools that Grid builders can use to create a wide  
variety of Grid solutions for specific communities. GT5 will include  
GridFTP, GRAM5, RSL, Myproxy, GSI-OpenSSH, and the relevant underlying  
C libraries such as GSSAPI, XIO, C Core, etc. Note that GT5 will not  
include Java Core. Instead, we will continue to support GT4 Java Core,  
and will work with our users to migrate GT4 Java Core services to Crux  
when it becomes available.

5. Globus Toolkit version 5.2, targeted for Q1 2010, will focus on  
repackaging the GT5 components into independent component releases  
that leverage OS-native packaging approaches (e.g., RPM), with  
assistance from other groups (e.g., KnowARC) who have already blazed  
this trail. Subsequent GT releases in the remainder of 2010 will focus  
primarily on usability and reliability, along with features required  
by Globus.org. The repackaging effort will not impact backward  
compatibility with GT 5.0.  We expect GT 5.2 clients and services to  
be fully compatible with GT 5.0.

C) THE GLOBUS.ORG ONLINE SERVICE

We are creating a new Globus.org online, hosted service (i.e.,  
Software-as-a-Service), to provide higher-level, end-to-end Grid  
capabilities, targeted to end users, as well as Grid builders looking  
for more complete solutions to build upon. Initial functionality of Globus.org 
  will focus on replacing and enhancing the RFT functionality of  
reliable, high-performance, fire-and-forget data transfer, but over  
time will grow to include more “collective layer” functionality (as  
described in the “Anatomy of the Grid” paper 1]). We plan to debut and  
demonstrate Globus.org at SC'09 next month.  We will begin operating a  
beta version of this service in November 2009 to a limited set of  
initial users, with a substantial ramp-up starting in early 2010.

During the remaining 14 months of the CDIGS project, we intend to  
focus more resources toward Grid data management problems.  We have  
seen tremendous growth in GridFTP usage over the past 2 years. Through  
a combination of usability and packaging improvements to GridFTP,  
along with the introduction of end-to-end Grid data management  
capabilities in Globus.org, we intend to substantially increase the  
value and resulting usage of the Globus data management software.

D) NEW LEADERSHIP

We are pleased to welcome returning and new leadership to our Globus  
team.  Steve Tuecke, who was the Globus lead architect for its first  
10 years before leaving five years ago to start a company, returned to  
the University of Chicago in January 2009, and has resumed both  
technical and project leadership for our Globus activities.  
Additionally, the University of Chicago recruited Paul Dave’ to a new  
Director of User Services position in August 2009, with a mandate of  
dramatically improving the quality and methods by which we provide  
services to Globus users, including support, consulting and  
operations. We have also re-organized our software development team  
and processes, with the introduction of widely adopted Agile Scrum  
development practices.

We are excited about this re-invogoration of the Globus software and  
community, and the increasing value the Globus community can bring to  
the many multi-institutional scientific and biomedical communities  
whose need for robust Grid computing middleware continues to grow  
unabated.

Viva Globus!

The Globus Team

[1] “The Anatomy of the Grid: Enabling Scalable Virtual  
Organizations,” Ian Foster, Carl Kesselman, Steven Tuecke.  
International Journal of Supercomputer Applications, 15(3), 2001.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.globus.org/pipermail/gt-user/attachments/20091021/b85e5a44/attachment.htm>


More information about the gt-user mailing list