One of the goals of the Advanced CyberInfrastructure – Research and Education Facilitation (ACI-REF) program focuses on supporting the needs of the steadily increasing “long tail” of ACI users/researchers who belong to disciplines that have not traditionally required access to ACI resources but who recognize that their research needs more compute power than can be provided by their desktop machines. Many of the universities involved in the ACI-REF program have been successful in reaching out to these communities, but success brings more challenges. For one, many of us are now faced with having a significant increase in the number of high throughput jobs running on our local resources, making it difficult for those with high bandwidth and MPI needs to run without long queue times.
To address this surge in high throughput jobs, the Cyberinfrastructure team from Clemson worked closely with Rob Gardner’s Open Science Grid (OSG) team to transparently connect Clemson users to OSG resources through OSG Connect. This partnership focused on developing a campus solution that could be easily implemented on other ACI-REF campuses, and thus help scientists on all campuses further their research by directing them to the right resources.
Since joining with the OSG in July, Clemson researchers have logged over two million core hours—including PhD student Paul Kilgo, who has leveraged the partnership to drastically speed up his work on volume rendering. “All of the computation I was doing would take 80 years of compute time,” said Kilgo. “Fortunately it takes two weeks for the OSG to turn that around. It’s been really nice to throw thousands of cores behind my research.” Kilgo, with Academy Award-winning effects professor Jerry Tessendorf, works to simulate light scattering through multiple points by computing the many different paths light can take as shown in the following figures.
Figure from Paul Kilgo, “A visualization of paths the algorithms produce.”
ACI-REF’s emphasis on sharing infrastructure and expertise does not always make for an easy task and the project hasn’t come without its growing pains. Both Barr von Oehsen, executive director of Clemson’s Cyberinfrastructure Technology Integration group, and Gardner believe in the importance of creating testbeds, such as this project at Clemson, before sharing with others to smooth out kinks before offering it as a campus solution. “It’s a pattern we want to replicate on other campuses,” said Gardner. “They’ve given great feedback and have been an excellent research partner on the concepts and method to move it forward. Other campuses can now adopt this pattern to easily connect research communities and compute resources to shared regional and national cyberinfrastructures such as the OSG.”
– Ryan Real