OSG 3.2 is here!
On November 12, 2013 the OSG Technology area released OSG 3.2.0- thereby launching the new OSG 3.2 release series. For the first time since the OSG began distributing RPMs, we now offer separate and parallel release areas for our software: OSG 3.1 and OSG 3.2. Having two release series allows us to make substantial software changes in 3.2, while supporting older versions in 3.1 for a while longer. This capability lets us serve our users better than with a single release series.
Currently, big changes to OSG 3.2 are as follows:
- Hadoop Distributed File System (HDFS) was significantly updated from 0.20 to 2.0, which included HDFS Federation.
- HTCondor was updated to its latest stable series, 8.0. For more information, see the HTCondor manual.
- HTCondor-CE is a new option for a site Compute Element; currently, it works only for sites that use HTCondor to manage their computing resources, but we plan to expand to other OSG batch systems (i.e. LSF, PBS, SGE, SLURM) in coming releases.
- glideinWMS was updated from 2.7 to 3.2; the new version supports Amazon EC2-based clouds, improves credential handling, and provides better debugging tools, and contains stability and performance improvements.
- gLite CE Monitor was dropped in favor of our own OSG Info Services software.
NOTE: Because these changes may be disruptive to a production site, you must take explicit action to update from the 3.1 to the 3.2 series; if you do nothing, package updates will continue to come from the 3.1 series.
The process for upgrading from 3.1 to 3.2 is documented here: https://twiki.opensciencegrid.org/bin/view/Documentation/Release3/OSGReleaseSeries
Back when we designed the initial OSG RPM plumbing, we knew that it would be incapable of supporting multiple release series. It was the right decision at the time, because we could not anticipate our needs years later, but we also knew that it would be a challenge to fix later. And it was.
The Software, Release, and Operations teams worked through the technical and social challenges of making this transition, and I am delighted to see how well it turned out. Please join me in congratulating the teams for their efforts.
Many people contributed in ways large and small, and without minimizing any contribution, I want to highlight a few individuals for outstanding effort.
- Mat Selmeci, who knows more about our build and deployment systems than anyone, was instrumental in designing the new system, identifying and qualifying deployment plans, figuring out how we would test changes in the face of a less-than-ideal testing infrastructure, and overseeing and executing many of the actual changes. In all, he provided solid technical leadership throughout, in his own, quiet way.
- Carl Edquist and Brian Lin, for coming up to speed on so many aspects of our infrastructure, which they had used previously but not modified, and then planning, making, and verifying many of the detailed changes that resulted in the new system.
- Tim Theisen, for tracking the fine details of so many different packages and their particular needs in both the old and new release series. (It was quite the learning experience, I suspect!)
- Soichi Hayashi and Scott Teige, for putting up with our bizarre requests and needs. Without Soichi’s help, we could not have made the internal repository changes so transparent to end users. And Scott patiently made sure that our remaining infrastructure needs were handled on time and with minimal fuss.
Thanks again to everyone involved!
~ Tim Cartwright