NRC-Compute Canada collaboration seeks to add value to ballooning astronomy data

Model could be used for other disciplines

Compute Canada (CC) and the National Research Council’s astronomy and astrophysics facilities have teamed up to give astronomers greater computational power needed to process and analyze data from powerful telescopes around the globe. The three-year, Three Cs Transition Project (C3TP) will see CC collaborate with the Canadian Advanced Network for Astronomical Research (CANFAR) and the NRC’s Canadian Astronomy Data Centre (CADC) in British Columbia to establish a cloud-based, open data management approach that will enhance the advanced computer services and networks available to researchers across all disciplines, from astronomy to medicine and the social sciences.

The CC-NRC project builds upon strong existing collaboration between the two organizations, prompted by the conviction that the platform needs to be revamped with a technology refresh, says Dr. Dugan O’Neil, CC’s chief scientific officer.

“This is a first-of-a-kind … The Compute Canada platform was not built with this in mind. It’s a badly fitting shoe requiring a shoehorn,” says O’Neil. “(Astronomical) data is already well curated and easy to find. Now, researchers can come in and analyze the data, send the complete job into our cloud, process it, give them a result and put it into their database.”

NRC is providing $2.5 million for the project which will be allocated to CC and the hiring of new developers that will make it easier for researchers to work with the data. Currently, data provided to CANFAR by the NRC’s CADC are only being archived and curated. As the volume of data increases exponentially, the astronomy research community is seeking more sophisticated and powerful cloud-based solutions.

O’Neil says they are working with NRC developers to build services that are agile and flexible and adaptable to other disciplines.

“We want to move strongly into the services layer above the hardware and rebuild all our systems from the ground up,” he says. “This is the moment to do it.”

Dr. Greg Fahlman, general manager of NRC Herzberg Astronomy and Astrophysics, credits CANARIE and its Network Enabled Platforms (NEP) program a decade ago for sowing the seeds which are leading to initiatives such the CANFAR and the CC-NRC collaboration.

[rs_related_article slug="CANARIE targeting academics with new cyberinfrastructure program"]

“The idea was to find a way to project these services out into environments where they can make better use of data … Our focus is on the end user to extract the full value of data generated by telescopes,” says Fahlman. “Rather than shift huge amounts of data around, move the user to where the data reside in the cloud, providing the user community with more power at their disposal.”

Fahlman says the CANARIE-generated concept attracted a lot of attention when it was first raised, with several European initiatives underway and companies such as Amazon and Microsoft launching cloud computing platform services.

O’Neil agrees that the CANARIE NEP program has helped to generate demand for the services CC and NRC developers are now creating. In addition to C3TP, NEP funding helped to establish organizations such as the iReceptor Research Software Platform at Simon Fraser Univ (immunogenetics), the GenAp Genetics and Genomics Analysis Platform at McGill Univ and the Univ of Sherbrooke and the Canadian Brain Imaging Research Network (C-BRAIN) headquartered at McGill’s Montreal Neurological Institute.

O’Neil says the C3TP project seamlessly addresses and aligns the needs of both organizations and, once completed in mid-2019, can be leveraged to the point where the cloud services will become more common in the future. The project’s initial focus is on platform architecture and design and new services are set for prototyping.

“There’s now a community of people building these things,” says O’Neil, adding that critical funding has also been provided by the Canada Foundation for Innovation’s Challenge 1 Cyberinfrastructure Initiative. “I hope it (C3TP) will serve as a model of what we can do … It’s special among all our projects.”

R$