[Globus] Help shape Globus' future: A call for use cases
foster at anl.gov
Tue Mar 31 01:05:35 CDT 2009
Over the past few months, within the context of the NSF Community
Driven Improvement of Globus Software (CDIGS: www.cdigs.org) project,
we have begun re-evaluating how we should invest toward Globus'
future. We believe we are starting to understand alternatives and
trade-offs. Of course, our understanding of the needs of the Globus
community is imperfect, and we do not want to make decisions without
broader consultation. Thus, we would now like to engage with the
Globus community to help us work through this evaluation.
A key concern at this point in the process is that we may not have a
complete enough set of uses cases that represent what people are doing
with Globus, or hope to do with Globus. Thus, we ask that you help us
by providing a description of how you use (or hope to use) Globus,
along with contact information so that we can follow up with you to
clarify details and/or participate in subsequent stages of this
effort. We provide below more details on what we are looking for.
Now the longer version...
Where are we?
We have had tremendous successes with Globus over the past 13 years,
along with some trying times. A challenging issue over the years has
been adapting to the changing technology landscape. Globus Toolkit
version 4 (GT4) was introduced approximately 5 years ago, and much has
changed since then. So what are the circumstances that factor into
our decisions on how to evolve?
Users: Many healthy, active, growing communities are relying on Globus
for their day-to-day work, including TeraGrid, OSG, ESG, LIGO, caBIG,
and many others. (Based on our anonymized usage reporting, it is
clear there are also many more Globus users than those we know
about.) These communities challenge us with the constant, healthy
tension between demands for stability and new functionality. There
are also new, large communities that are adopting Globus, such as BIRN
and other biomedical communities, which will continue to press these
sometimes conflicting demands.
Funding: We are fortunate to have significant NSF CDIGS funding for a
couple more years, which allows us to focus on support and development
of Globus to meet user needs. We also have a rich mix of grants for a
wide range of activities, ranging from computer science research, to
infrastructure deployment and support, to application integration, to
complete application development. However, these efforts are of
course finite in duration, and we must be constantly looking to
develop new programs to carry Globus forward.
Technology trends: The technology landscape in which we operate
continues to evolve at a dizzying pace, with technologies and
approaches such as Cloud Computing, Software-as-a-Service (SaaS),
OpenID, OpenCL, multi-core, etc. In the meantime, technology trends
that we have embraced in Globus, such as SOA and Web services,
continue to mature, for example with the widespread adoption of
lighter-weight REST approaches.
Aging foundations: Globus relies heavily on open source components
from other communities. Some component choices, such as OpenSSL, have
stood the test of time. Others have not: in particular, on the Java
web services front (e.g. GT4 Java Core), we find ourselves highly
dependent on libraries such as Apache Axis v1.x and PureTLS, which
most of the rest of the world has moved away from. Thus, not only do
we have to support these technologies entirely on our own, but it
becomes increasingly difficult to support our users and our own newer
activities that (rightly) want to exploit new open source software
libraries and approaches. These aging GT4 Java Core foundations are
also inhibiting our ability to effectively address a variety of issues
(e.g., scalability, REST support) in other Globus components, such as
GRAM4 (aka WS-GRAM), RFT, and MDS4.
Community: The Globus Alliance (dev.globus.org) community of
developers has made many valuable contributions that have seen wide
adoption, such as MyProxy, GridShib, Gaards, and GridWay. However,
the dev.globus.org community has not helped significantly with the
sustainability of existing components by bringing new developers into
to the community who add features or maintain those components.
Where do we go from here?
Thanks in particular to our NSF CDIGS funding, we have some time and
resources to address these issues, and we will not abruptly leave our
users out in the cold now or in coming years. One CDIGS mandate is to
continue supporting existing users, which we will clearly do. So
regardless of what decisions are made, we will continue to support our
GT4 users as long as we have demand and funding such as CDIGS to do
so. But CDIGS also has two other mandates: helping new users, and
working to ensure Globus' sustainability. And, of course, we have
many other projects that are contributing to development of new
functionality in Globus, which we can leverage as well.
With all of this in mind, our intent is to evolve Globus based on
Embrace widely adopted and community supported software and standards:
When GT4 Java Core was created over 5 years ago, Apache Axis 1.x was
the right choice, but as discussed above the world has moved on. (You
One might argue whether WSRF was the right choice at the time, but
there is strong support for managing/representing state as a first
class part of the web services model, as evidenced by adoption of WSRF
in large communities such as caBIG, and the popularity of REST
approaches throughout the Web 2.0 community.) We have begun evaluation
and prototyping efforts around the many alternative web services
toolkits now available, with goals of: being as conservative as
possible in our choice of technologies (e.g., looking for toolkits
with robust communities); being as flexible as possible in what
requirements we place on our users (e.g., looking to support multiple
web services toolkits); doing as little "invention" or variance from
the standard technology as possible (so we do not have to ship our own
modified version, and plug cleanly into existing systems); while
preserving core notions that Globus users have found compelling such
as rich security and first class management and representation of state.
Maintain backward compatibility as much as possible: We recognize that
protocol incompatibilities have been a recurring headache for our
community in the past. Sometimes these are unavoidable as one
progresses forward -- particularly on the tight budget of a non-profit
development group such as ours. But our decisions will be strongly
influenced by our ability to maintain backward compatibility. For
example, one consideration in evaluating web services toolkits is
whether we can provide WSRF (GT4.0 and/or GT4.2) backward
compatibility while simultaneously allowing evolution to newer SOAP-
and REST-based protocols.
Address long-term sustainability: As discussed above, one key aspect
of sustainability is moving Globus onto newer open source components
that enjoy wide support from a much broader community than just
Globus. Additionally, we have been considering: options for
simplifying Globus, such as removing little-used features which
present code maintenance, stability, and/or scalability issues (e.g.,
GRAM2&4 job rendezvous, GRAM2 stdout/err streaming); reinvesting
resources into GRAM2 since it is still heavily used; ceasing
additional development on some components such as GT4 Java Core, GRAM4
and RFT (e.g., only providing bug fixes, and working with users to
migrate to other options); and re-implementing components in a more
maintainable fashion (e.g., RFT). These examples are merely options
we are considering -- at this point no firm decisions have been made.
Of particular concern is how to move forward with GT4 Java Core, and
the GT4 services that are built on it. Some of our thoughts are:
GT4 Java Core: As discussed above, we are investigating various open
source web services frameworks. There are three primary compatibility
concerns: maintaining optional WSRF protocol compatibility while
allowing for migration to non-WSRF protocols; porting existing code
that uses GT4 Java Core to new frameworks; and adopting approaches
that allows for easier protocol transitions in the future.
GRAM: A large part of the community is still using GRAM2, and GRAM4
(aka WS-GRAM) has sustainability issues due to its use of GT4 Java
Core. We are considering re-investing in GRAM2 to solve some known
issues (e.g. scalability).
RFT: While many people have expressed interest in RFT, we have had
limited success with it for a variety of reasons. RFT also has
sustainability issues due to its use of GT4 Java Core. We are
experimenting with a replacement for RFT which is implemented using a
Software-as-a-Service model, and are now looking for users interested
in trying it out and giving feedback. Please contact Lisa Childers if
you are interested.
GridFTP: Address some persistent usability concerns, so that GridFTP
can be downloaded and operated "out of the box" with little or no
MDS4: Various communities use MDS for various purposes, including for
a service registry (e.g. caBIG, Teragrid), and for monitoring
services. Many good options for monitoring have emerged in the open
source community in recent years, but there still appears to be a need
for service registries. We are considering focusing our MDS energies
more on the service registry use cases.
Security: Many communities use Globus for its powerful and flexible
security (GSI). There are already activities underway for integrating
GSI with other technologies such as Shibboleth, SAML, XACML, and group
membership services (e.g. VOMS). However, we also need to understand
the role and potential integration of GSI in the context of emerging
technologies such as OpenID and OAuth.
RLS: RLS is used by several important Globus communities. We will
implement a number of community-requested features, including a
pluggable back end, improved logging, and more streamlined startup.
How you can help
We need to hear from our users directly about how you use, or want to
use, Globus. It is only through a clear understanding of these use
cases that we can be confident of our decisions. What we need is a
detailed description of how you use Globus, including:
what for (e.g., a science gateway, a data replication system for a
science experiment, etc.),
what features of those components (e.g., the GRAM2 features you rely
what clients (e.g., jGlobus for GRAM and GridFTP, Condor-G with GRAM2
or 4, etc.),
what sort of performance or scalability issues you are having or
would like to see,
Please email the following people (copied on this email) with your use
cases for the following components of Globus:
GRAM 2, GRAM 4, C security: Stu Martin
GridFTP, XIO: Raj Kettimuthu
Reliable File Transfer (RFT): Lisa Childers
GT4 Java Core, Java security: Ravi Madduri
Service registries (MDS4): Lee Liming
Replica Location Service (RLS): Ann Chervenak
Please include appropriate contact information, so that they can
follow up for clarification or more details if necessary.
In the coming months you can expect more communication from us as we
work through this activity. We will likely produce one or more
technical whitepapers that describe our proposed directions, on which
we will be looking for feedback. We will also produce prototypes that
you can try to provide early feedback. We may also try to organize a
workshop in Chicago in the near future so that we can discuss these
directions in person -- please let us know if you would be interested
in attending such a workshop.
Thanks in advance for your valuable input.
The Globus Team
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the announce