QUINTON - QUerying and INTegrating Over Nested data

It has long been recognised that nested data models - in which information is modelled as collections of tuples whose attributes may in turn take values that are collections - are the most natural modelling formalism for a wide variety of information management scenarios. Query languages that support nested data were developed decades ago. However, even as emerging applications have made the need for querying of nested data more crucial, and even as many of the most important big data management frameworks assume programmatic interfaces based on nested data, processing large-scale nested data remains extremely cumbersome, radically more so than in the case of flat data. The research hypothesis for QUINTON is that fundamental problems in querying and integrating nested data need to be resolved for this situation to change. 

The QUINTON project will provide new foundations for both querying and integrating nested data. In terms of querying, it will establish a standard processing pipeline for queries over nested data. This will include a foundational study of the basic transformations involved in any such pipeline, such as the "shredding" of nested queries into relational queries. It will also include the development of algorithms and tools that implement this pipeline, working on top of scalable infrastructure for flat data, such as the Apache Spark project. On the side of integration, the project will establish the foundations of specifying and querying virtual data sources consisting of nested data, and develop middleware that can implement queries over virtual data on top of heterogenous nested data sources. 

The impact of QUINTON is both practical and foundational. The project will build infrastructure for querying and integration, but also investigate the fundamental problems of scalable querying over materialised and virtual data sources, providing the foundations that can guide the research community in future implementations. It will also drill down into a particular compelling and timely application of nested data integration and management, working with an industrial partner to build components and novel analyses in the area of management for biomedical data. The partner deals with unified interfaces to diverse biomedical data sources - clinical, imaging, and genomic data - and their use cases are a perfect fit for the QUINTON technology.