Our Work Data Management
A grey gloved hand reaches into the frame from the left. The hand is holding a black barcode scanner which is shining a bright green dot of light onto a barcode sticker, which is stuck to the edge of a shelf.

Barcodes and barcode scanners are used to keep track of record movement
With tens of millions of images to keep track of, good data management is vital

Data Management

As one of the largest collections of aerial imagery in the world, the National Collection of Aerial Photography (NCAP) holds 30 million high-quality analogue images that record key moments in world history, and places around the globe. 

image generation

As part of NCAP’s work digitising these tens-of-millions of images, large volumes of digital data are generated which need careful storage. NCAP is also committed to making these images accessible, which requires effective data retrieval processes. At full capacity, NCAP can generate terabytes (TB) of data per 24 hours through digitisation efforts. Storing this volume of data far exceeds NCAP’s server capacity, necessitating collaboration with external partners, notably the Edinburgh International Data Facility (EIDF). 

Working in Partnership

The EIDF is an extensive collection of computational and data management services, funded by the UK and Scottish Governments to offer data-driven products, services, and research support. NCAP’s partnership with the EIDF grants access to a wealth of expertise. This collaboration bolsters capacity to process rapidly growing volumes of high-resolution digital imagery through the use of high-performance computing (HPC) systems. By providing faster access to millions of high-resolution historical aerial images of places around the world, the partnership will provide the global research community with access to a growing library of datasets.

A thin strip of white light extends horizontally across the middle of the image. Above, the edge of a black and white aerial photo can be seen. Below is a label with measurement markings, a greyscale colour chart and the word ‘NCAP’.
The systematic use of tools such as identical calibration targets on all scanning equipment make the mass-processing of digital files an easier task for automated systems.

The team at NCAP values any opportunity to enhance its processes through innovative technologies and finding new approaches to traditional ways of working. Partnering with the EIDF has been hugely rewarding. Promoting knowledge exchange and cross-sector development in geospatial intelligence and data management across industry, academia, and government cultivates a thriving research base. This data-driven innovation holds immense potential to benefit various sectors.

geographical information systems (GIS)

Although GIS is widely used for managing mapping data, traditional systems struggle to handle the vast amount of data generated by NCAP's global digital outputs. To address this challenge, the NCAP data management team collaborates with GIS, data processing, and artificial intelligence sectors to enhance existing systems and develop new ones.

A screenshot of NCAP’s Air Photo Finder. On the left is a black and white oblique aerial photo of docks in Dundee, with buildings on the shore and a ship on the water. On the right is a map with the area shown in the photo marked by a grey polygon.
NCAP leverages traditional GIS data management systems to accurately map the coverage of each image for display on the Air Photo Finder.

Ease of Access

The digital assets created by NCAP's digitisation streams not only need to be stored for long-term preservation, but they also need to be easily accessed and made available as a public record. Smaller resolution copies, roughly the size of a postcard are made freely and openly available on the Air Photo Finder. The management of data for sales is equally important to many of NCAP’s everyday users. This requires a different set of processes. Given that image files stored at EIDF can be as large as 1GB per image file, they are impractical for traditional sharing methods. NCAP works with other partners to establish a more accessible dataflow set up to meet the needs of commercial customers. 

The NCAP Data Management Team works actively with partners and stays updated on industry innovations to ensure consistent and efficient management of various storage and access strategies.

A man with short brown hair and glasses faces the camera, looking at a computer screen. Between us and the man are two out of focus screens facing us. To the left another man with cropped hair and a ginger beard faces away from us, also out of focus.

Born-Digital and Digital Collections

As research institutions transition from analogue photographic film, and to digital capture, NCAP is also embracing this shift by actively acquiring born-digital collections. Furthermore, through project partnerships with major international archives like the National Archives and Records Administration (NARA) in the United States, NCAP is helping partner organisations in making their digital collections of aerial photography accessible to the public. 

NCAP has consistently enhanced photographic preservation and scanning processes. Likewise, it will strive to make further innovations in areas of data processing, artificial intelligence and digital asset management practices as collections expand. 

Related reading

IN THIS SECTION

A purple gloved hand reaches in from the left. They are using a brush with a white handle to brush the surface of a black and white photographic print towards the nozzle of a black vacuum cleaner. There are other images and boxes on the table.

Collections Care

Making sure photographic collections are cleaned and stable before any scanning has started underpins all work at NCAP.

A man wearing a blue lab coat and black gloves raises the lid of an image scanner on a desk to allow them to feed film across it. The film runs between two spools on either side. There is other scanning equipment out of focus in the background.

Digital Imaging

Using the latest scanning technology, skilled technicians and robotic systems NCAP operates a highly productive digitisation facility.

Two hands wearing grey gloves hold a slim cardboard box. They appear to be moving the box, and the bundle of boxes piled beneath it, on to the white shelving behind.

Cataloguing and Access

All collections held by NCAP are open public record, once catalogued they are made available to the public.