Experiment Data¶

Data Submission¶

Access the Data Submission and Management module

Access https://platform.rd-connect.eu/datamanagement/#/

Left side-bar (1) - Side-bar fixed in all experiment submission and management steps - provides direct access to different experiment submission and management steps. Bottom menus (2) - Menus containing access to submit and manage your experiments.

The process to submit your experiments is the following (and in this order):

Provide experiment metadata
Provide file metadata
Upload sequencing files through RedIris server

Please mind that:

Steps 1 and 2 are performed following the “Submit Data” section.
Step 3 is performed outside the RD-Connect GPAP website.

Important

Be aware you will need the Phenostore IDs to begin the data submission process.

Provide experiment metadata¶

In the Experiment Submission Site enter the Submit Data section in the bottom menu. You will see 2 sections, one to submit an Experiment metadata and one to submit File metadata. Enter the “Submit Experiments” section.

You will enter the site to provide the experiment metadata.

The first step is to assign a group owner to your set of experiments. Click on your desired choice at the Group Owner selection menu and click next button.

There are 2 methods to submit the experiment metadata:

Individual upload using the website’s form (one row per experiment)
Bulk upload by using a .tsv file

In case of bulk upload from .tsv file you can get an excel template to guide you over the required fields and values.

_images/experiment_template_submission.png

Experiment submission view

Once the information appears in the form, click “Validate” to check for possible errors. If no errors appear, click on “Submit” to submit your data.

Important

This step will generate new “GPAP Experiment IDs” for your experiments.

A new page will appear with a summary for your submission.

Use the button “Download as csv” to download a file with a summary of your submission, including the newly generated Experiment IDs. Use the button “Go to Submit File(s)” to submit files metadata.

Provide file metadata¶

In the Experiment Submission Site enter the Submit Data section in the bottom menu. You will see 2 sections, one to submit an Experiment metadata and one to submit file metadata. Enter the “Submit Files Metadata” section.

The first step is to select the set of experiments to provide file metadata.

_images/select_experiments_for_submit_files.png

There are 2 methods to submit the file metadata:

Individual upload using the website’s form (one row per file)
Bulk upload by using a .tsv file

Once the information appears in the form, click “Validate” to check for possible errors. If no errors appear, click on “Submit” to submit your data.

Important

Be aware this step is to upload file metadata, not to upload the sequencing files, which is done in Step-3 “Upload Sequencing Files”.

Important

You can upload multiple files for a given experiment if they are FASTQ and paired end.
Our pipeline assumes just one experiment file submission in case of BAM or CRAM data.
If more files are provided then the analysis will not be done.
Analysis pipeline will not work for experiments submitted with multiple file types.

A new page will appear with a summary of the submission.

Use the button “Download as csv” to download a file with a summary of your submission, including the newly generated Experiment IDs.

Last step is Experiment file submission closed to mark your experiments as closed (no more files metadata will be provided) and able to trigger the pipeline analysis.

Mark the checkbox table for every experiment you want to close the file metadata submission. Unmark the checkbox for those experiments you will submit more file metadata.

Press CLOSE SUBMISSION FOR SELECTED EXPERIMENTS button to set the marked experiments as closed and ready to be analysed. Otherwise click SKIP button.

Upload sequencing files through RedIris server¶

After performing Step-2 “Upload File Metadata” you will immediately receive an email titled “Submission of Sample file/s” with a summary of your submission.

In the next 24 hours you will receive another email titled “Red Iris Account Created”. This email will contain:

Login credentials to use RedIris server (https://fileshare.rediris.es).
Instructions to upload sequencing files through Web.
Instructions to upload sequencing files through Command Line.

After you follow these instructions and the files are uploaded, we will download them from RedIris and begin the data processing.

Data Management¶

In the “Manage Data” section you will be able to manage:

Experiment metadata
File metadata
Statistics on Data Access
Experiment Analysis Status
Experiment QC Metrics

Experiment Metadata¶

In this section you can:

See the experiments you have submitted through Step-1 “Provide Experiment Metadata”.
Download data using the “Download Table to CSV” button.
Access information of each experiment by clicking in each Experiment ID.
Close file submission for your experiments.
Share/Unshare your experiments with other groups.
Share/Unshare your experiments with other projects.

To apply changes to your experiments you should:

1. Select the experiments you own by clicking row checkbox.
1. Select one of the available actions.
1. Click apply button.
1. Select new value to apply to your experiments.
1. Apply changes

File Metadata¶

In this section you can:

See the files you have submitted through Step-2 “Provide File Metadata”.
Download data using the “Download Table to CSV” button.

Statistics on Data Access¶

In this section you can:

See which experiments have been accessed by other users “Individually” or by the “Search across all” option.
Download data using the “Download Table to CSV” button.