Untitled Document Untitled Document OSG October Newsletter, 2009
Please remember to mark your calendars for the next OSG All Hands Consortium Meeting on March 8-11th, 2010 located at Fermilab.
Firefly
Last May the University of Nebraska received Firefly, a 1152 node, primarily dual-core Opteron cluster as a result of a generous donation from the Holland Foundation. This became the largest resource in what is now known as the Holland Computing Center (HCC) at the university. Firefly is connected by SDR Infiniband and provides 150 TB of Panasas storage. Originally ranked at 43 on the Top500 list at 21.5 TFlops, it is located at the Peter Kiewit Institute (PKI) in Omaha. HCC at PKI is connected to further resources and staff at the University of Nebraska-Lincoln (UNL) and Internet 2 at 10 Gbps.
Firefly serves two primary communities, local Nebraska researchers running parallel codes via OpenMPI, and OSG VOs who opportunistically deploy serial code. Firefly has provided over 10 million CPU hours to these combined communities since UNL began operating it in June, with a rough CPU hour ratio of 2:1 for OSG vs. local parallel jobs.
Near complete utilization was achieved almost overnight. This surge in usage quickly exposed some scalability issues previously not encountered. Local parallel usage benefiting from more reliable infrastructure is increasing over time, but grid jobs remain a major component of Firefly usage. CMS, at over 2.3 millionCPU hours, is the largest user of any VO, primarily due to UNL's affiliation with that experiment; next is ENGAGE at over 760,000 CPU hours, followed by DZero (530,000), the local GPN VO (490,000), and LIGO (430,000). “Firefly is an example of the mutual benefit that can occur with opportunistic grid job deployment,” says Brian Bockelman, an OSG developer at UNL .
~ David Swanson
_____________________________________________________________________________________________________________________________
Bringing HighThroughput capabilities to ensembles of Parallel Applications
Working within the framework of the Open Science, researchers from the University of Chicago, RENCI , and the University of Wisconsin have teamed together to meet the needs of an emerging class of applications—large ensembles (hundreds to thousands) of modestly parallelized (less than 64 way) jobs. Funded by a recent STCI - NSF award, this team will use OSG resources and tools to support scientists with High Throughput Parallel ( HPTC ) applications across a growing range of scientific domains (e.g. weather forecasting, computational chemistry, flood-plain modeling, numerical relativity, and computational biophysics, to list a few.)
Our approach toward solving HTPC class problems is first to focus on three fundamental challenges: distributing and keeping track of HPTC jobs on remote resources; making the codes portable across heterogeneous system architectures; and optimizing these jobs on the multi-core system architectures that make up the majority of High Performance Computing resources today.
Traditionally, users of HPC resources have fallen into one of two categories: “capacity” computing, in which a single parallel job uses all (or a major fraction) of an HPC resource; or “throughput” computing, in which ensembles of single processor jobs are executed simultaneously. This research clearly attempts to move outside of traditional boundaries and introduce viable new computational models for scientists to utilize.~ Dan Fraser
____________________________________________________________________________________________________________________________
OSG at the University of Johannesburg (UJ)
The University of Johannesburg (UJ) Research Cluster is now reporting its presence on two grids—the South Africa Grid ( SA-Grid) and the OSG. UJ is somewhat unique in uniting the American and European grid flavours needed for our research. The VDT is needed for our participation in the ATLAS project while g-Lite is needed for the SA-Grid and EGEE. This means we have shared infrastructure and interoperability. Soon Witwatersrand University will collaborate with UJ in its OSG project with ATLAS.
Many thanks are due to our colleagues at Columbia University, the OSG, the UJ-ICS section and Sergio Ballestrero, and of course, lots of ATLAS, CERN and SA-Grid friends.
Within the next few days, we expect to grow from 96 to 280 compute cores.We are starting at 10TB storage, but we need to eventually reach 100TB. We expect to be doing (nearly) real-time event analysis from ATLAS data at the Large Hadron Collider starting in December. We will also look into co-affording bandwidth to be an effective Tier-3 site for ATLAS, amongst other memberships for other groupings in UJ.
See site status for SA-Grid and OSG Maps -check “all sites” and Update the Page.
~ Miron Livny and Ruth Pordes
____________________________________________________________________________________________________________________________
Grid Colombia Workshop
For the last two weeks in October we have been organizing the presentation of a hands-on grid school in Colombia.
This workshop, the second part in a three-phase program supporting the launch of Grid Colombia, covers material on the ground work of scientific computing and grid prototype building. There are ATLAS and CMS University Groups in Colombia who will benefit from this work.
The local participants are taking initial steps to build their own grid infrastructure—including the operations processes and services. In a few months they will work to deploy production facilities across multiple sites in Colombia.
Many members of the OSG are participating remotely through AdobeConnect or the regular “OSG daily Campfire chat,” and several sites are supporting the OSGEDU VO specifically for this period. We much appreciate their effort and participation!
~Jose Caballero, OSG South America Outreach, and Rob Gardner, Integration and Sites Coordinator
____________________________________________________________________________________________________________________________
MyOSG adopted by EGEE
At EGEE '09 in Barcelona, the EGEE SAM group announced the adoption of the MyOSG presentation framework for MyEGEE.
MyOSG, developed for OSG at Indiana University, is an information consolidation and presentation web tool used to create custom user views from several OSG data sources. These include information data, such as BDII and GIP Validation, monitoring data gathered by Resource and Service Validation ( RSV ) Probes, accounting data from Gratia, and administrative data from the OSG Information Management ( OIM ) database.
MyOSG selection criteria allow each user to have a unique view of content gathered from multiple sources within OSG . It also allows export into a generic widget format ( UWA ) to personalized workflow environments such as iGoogle, NetVibes, and other widget viewers, including mobile devices.
EGEE prototyped MyEGEE based on the framework developed by the Indiana team. MyEGEE works along with updates to their overall Service Availability Monitoring ( SAM ) infrastructure update that also consists of Nagios monitoring components. See a SAM presentation on YouTube.
Please send questions about the MyOSG software to the Grid Operations Center at Indiana University, goc@opensciencegrid.org, from where they will be forwarded to the MyOSG team.
~ Rob Quick
____________________________________________________________________________________________________________________________
Executive Director's Report
Over the past month, we've had a lot of interesting activity.
LIGO's Einstein @ Home has been using an increasing number of cycles and sites. US ATLAS and US CMS are sustaining a high level of OSG usage as they prepare for the beam to turn back on this coming month.We welcomed the first resource of the GlueX VO at the University of Connecticut .
On behalf of the OSG as a whole, I would like to express appreciation to the University of Wisconsin's Bart Miller, Jim Kupsch, Elisa Heymann Pignolo, and Zach Miller for their very successful tutorial—an important part of our outreach training for those contributing software for use on the OSG—on “Secure Programming and Vulnerability Assessment of Distributed Computing Middleware” at the Open Grid Forum in Banff. There were around fifty attendees, and Bart reported that the organizers received lots of comments about how great both the work done and the sharing of this information were. He also says there is a whole list of follow-up activities to keep them busy for a while! We look forward to future collaboration with the UW team.
Horst Severini travelled to Brazil for the “opening” of the GridUNESP, while Jose Caballero and Rob Gardner travelled to Colombia for the “Grid Colombia Workshop.” The goal for both these activities is to facilitate a distributed infrastructure locally.
~ Ruth Pordes____________________________________________________________________________________________________________________________
OSG Enterprise Document Repository
The OSG enterprise document repository, also known as DocDB, includes the official governance documents of the OSG .
This summer we completed a project to recategorize documents in the repository. We hope that this will make it easier to find documents of interest. While the OSG twiki contains technical documentation, the repository is the place for official documents, such as OSG -wide policies and blueprints, outreach materials, and OSG published papers. We encourage you to upload documents of this nature to the repository. To help you properly categorize your documents, we have some topic guidelines posted.
DocDB requires your IGTF certificate. The first time you access the non-public database to update or post a document, you will be prompted to apply for access. This is a quick and easy one-time process that requires only your email address and the access group/s you need membership in. (We will assist you if you are unsure of which group/s you need.)
~ Marcia Teckenbrock
___________________________________________________________________________________________________________________________
Public Documents Changed and/or Added in the last 30 days from the OSG Document Repository
OSG-doc-#
Title
Author(s)
Topic(s)
Last Updated
Joint Statement of Agreed Upon Principles between OSG and TeraGrid
21 Oct 2009
11 Oct 2009
OSG Newsletter Archive
Supported by the National Science Foundation and the U.S. Department of Energy's Office of Science