Data Management Dashboards: A Harvard University Case Study Using Coldfront and Starfish

Our group developed a streamlined self-service data management ecosystem, connecting a storage allocation management tool with a data visualization tool. By combining an open-source tool (Coldfront) with a vendor supported tool (Starfish), we reduced the amount of time and effort required by labs to review and manage research data on our filesystems. The tracking and monitoring of lab storage usage became increasingly important following the implementation of a paid storage model in 2022, which required labs to pay for existing storage allocations. The introduction of these data management tools helped reduce the number of internal bottlenecks, automated the storage request process, and gave labs the necessary information to make long-term decisions about their storage environments. The storage allocation tool provides key information about storage folder locations, usage amounts, member lists, storage limits, and overall costs, which are tailored to the individual lab. The data visualization tool offers further insight into the dataset metadata, including number of files, dataset size, and last accessed and modification times, which supplements the information found in the open-source tool. Interweaving the two tools allows for a more seamless transition, improving the overall user experience. The new ecosystem has proven to be extremely successful, with labs proactively planning for storage capacity expansions, upcoming data migrations, and associated costs, while also engaging in more thorough data review and cleanup efforts. Our group’s Data Manager will discuss the benefits of combining these two tools, including improvements to the user experience and the reduction in the amount of manual labor needed by members of the team to support the storage request process.