Transitioning to the HTCondorCE in OSG

Over the past year, the OSG Technology Team has been transitioning users from the Globus gatekeeper to HTCondor CE to improve scalability in satisfying future resource demands. An added benefit of the transition is that the OSG Software Team has close ties with the HTCondor developers, which can be used to influence the design and development of OSG’s job gateway software. However, the switch is not a complete abandonment of the Globus Toolkit; HTCondor CE still uses Globus libraries for authentication and file transfer (GridFTP).

Starting with a few test sites, the OSG Technology Team was able to find and fix bugs, add features, and fill holes in our documentation to get HTCondor CE where it is today. Thus far in the transition, HTCondor CE scalability tests have reached 10,000 jobs, over 10 sites are using HTCondor CE in production, and our information services are running out of Indiana University. In the coming months, we hope to have all U.S. Worldwide LHC Computing Grid sites transitioned by January 1, full support of LSF (Load Sharing Facility) and SGE (Sun Grid Engine) batch systems, and continual improvement to the documentation.

As more sites moved to HTCondor CE, the Operations Team needed tools and resources to be able to address incoming support tickets. So the OSG Technology Team organized a two-day training session at the Grid Operations Center at Indiana University where support staff learned about the inner workings of HTCondor CE and were given a myriad of hands-on troubleshooting exercises. The event was a success, and the OSG hopes to offer similar training sessions to system administrators in the future.

~ Brian Lin