Data Guide
PEDP retrieves data from federal sources, either by downloading from publicly accessible websites or through FOIA requests. We store that data in external repositories such as Harvard Dataverse’s CAFE project (opens in a new tab), Zenodo (opens in a new tab), or SciOp (opens in a new tab).
We also copy Github repositories that store federal data, research, or source code, which can be accessed via our Github organization (opens in a new tab).
Notice any issues? Let us know.
How to use the PEDP data catalog
The PEDP data catalog is a browsable list of all the datasets that PEDP has archived since January 2025. Its goal is to make the data more discoverable, accessible, and usable. Use the search, filters, and sort functionality to find a dataset of interest, expand the dataset record to view the dataset’s summary and archive notes, and click “Open” to view the detailed dataset information and download the dataset from the external repository.
Term definitions
Agency: The original federal agency or organization that published the dataset.
Archive Date: The date that PEDP archived the dataset by storing it in an external repository.
Archive Notes: Any notes from PEDP archivers about the process of archiving the dataset, such as scripts used, changes made, missing records, etc.
Dataset Title: The name of the dataset as given by the publishing agency.
Last Updated: The date the original publishing agency last updated the dataset.
Metadata
PEDP created a new metadata standard in 2025, inspired by Harvard Dataverse’s CAFE project and with ISO metadata standards with FAIR principles at the forefront.
In doing so PEDP:
- standardized descriptions across document types and archived in different places online
- improved retrieval and sharing of information
- ensured interoperability across systems
- ensure the preservation of its information over time
- improved its archival descriptions
- demonstrated consistency across records management
PEDP’s Data Preservation Working Group are the stewards of the metadata for all archives and are ensuring that the PEDP Metadata Standard is upheld, as well as maintaining the Standard itself and updating it as necessary.
Schema
Every dataset is archived with a metadata document that can be accessed from the dataset’s record in the external repository. The schema for that metadata is documented here (opens in a new tab).