Open Data Cube workshop
In early February, forty software developers, decision-makers and subject matter experts came together for the third annual Open Data Cube Conference. The week-long conference, which included strategic meetings and a four-day hackathon, was hosted in Canberra by Geoscience Australia from 11–15 February.
Alex Leith and Chris Morgan from FrontierSI’s research and innovation team were up at Geoscience Australia for the week. Alex said FrontierSI were heavily involved in the week, but especially in the technical stream.
“FrontierSI have been working with partners Geoscience Australia, supporting the Open Data Cube community for over 12 months,” Alex said. “The ODC is a mature open source software project, that enables people to easily work with large volumes of earth observation data. Core to the project are the foundational software library, a suite of applications, and the people that dedicate their time to ensuring the ODC project runs smoothly and relied upon by organisations, small and large, all across the globe.
Chris said it was great to bring together scientists and programmers with diverse backgrounds and skillsets from all over the world to use the ODC in this hackathon event.
“The week was really empowering for the developers, we were given free rein on the projects, with some background given to work towards building indicators for the United Nation’s Sustainable Development Goals,” Chris said. “And how we could improve on these indicators through the integration of data from multiple satellite platforms, including optical and radar sensors. On day one, six projects were pitched and individuals chose which problem they most wished to work on for the week.”
Three projects were worked on over the week. The Dask Python library was included in the Cube in a Box environment, which is an easy way to run the ODC. Dask proved relatively easy to include in the ODC stack, and was soon included in a development cluster of ten servers on AWS. This team was successfully able to calculate the average colour of Australia over 18 billion pixels, with the calculation taking just two minutes to process. The result, which is a little boring, is that Australia is a shade of brown known as ‘Millibrook’. The point of this work, though, is that we are now able to undertake non-trivial computation in a distributed way and in a public cloud environment, which opens up a number of new possibilities for the ODC.
The image of Australia created in the Dask project.
The second project was a task aimed at developing a unified water product from three platforms, Landsat 8, Sentinel 2 and Sentinel 1. This project used a year of data from each platform to identify areas that had surface water and compared the results between the three, in addition to bringing together a unified result. The use of radar data, from Sentinel 1, means that even in months where there are not enough cloud-free days captured in the optical platforms, we can build up a good view of surface water cover.
The final project involved the development of a decision tree over a range of data products, including DEA’s Fractional Cover and Water Observations from Space as well as the Bureau of Meteorology’s GeoFabric. The decision tree was carefully prepared to classify each pixel (25 m2) so we are able to detect changes in a broad range of land coverage, from forests to farmland, to natural and man-made water sources. These results enable analysis of landcover change in areas of Australia between 2010 and 2015.
Land cover classification with change detection included.
Both Alex and Chris agreed that the event was a great success, and initial feedback has been very positive.
“The hackathon teams delivered presentations on Friday morning wrapping up their work and all were successful in achieving their goals,” Alex said.
FrontierSI was pleased to play a more significant role, alongside Geoscience Australia and Symbios, in the organisation of this year’s conference.