Building off the 1st and 2nd generation, we have reimagined what a Data Commons can be by utilizing cloud native thinking. Each microservice now lives in its own container providing isolation from the environment around it. Container orchestration is conducted dynamically to scale to the commons workload. Clouds are approached agnostically so that Gen3 can operate on any cloud, public or private.

 

MicroServices

Commons api

The Commons API service is responsible for handling user submissions, and GraphQL search queries. The submissions are quality controlled against the data dictionary to ensure all required fields are present and have appropriate data values. The GraphQL service allows Commons operators and users to precisely query only the information they are most interested in from the metadata collections. 

FENCe

The Fence service manages all user and service authentication in the data commons. Fence controls access to the metadata, submission, indexing, and data itself. Fence implements the data commons OAuth flow, and generating and tracking JWT auth tokens by which a client may authenticate with the other commons services to access data under the projects to which the client has access.

indexd

The Indexd service provides permanent digital IDs for data objects. These IDs can be used to retrieve the data, or query the metadata associated with the object. The Indexd service tracks the locations and hash of every asset (file) in the data commons object store.  It exports RESTful APIs for registering a new asset, and retrieving data for an existing asset.

PORTAL

The Portal service is an interactive website that allows users to explore, submit, and download data. The Portal service utilizes the Commons APIs offered by the data commons just as any other app built could.

 

Secure computing architecture

architecture.png
 

Open source technology

kubernetes-logo.png
python.png
mono_vertical_large.png
postgresql-logo.png
jupyter.png
GOPHER_SHARE.png
jenkins.png
packer.png
graphql.png
Screen Shot 2017-10-17 at 10.53.38 PM.png
React_logo_wordmark.png
 

Public and Private clouds

OpenStack-Logo-Vertical.png
GCP.png
Azure.png
PB_AWS_logo_RGB.jpg