ebook img

ISB Cancer Genomics Cloud Documentation PDF

234 Pages·2017·5.11 MB·English
by  
Save to my drive
Quick download
Download
Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.

Preview ISB Cancer Genomics Cloud Documentation

ISB Cancer Genomics Cloud Documentation Release 1.0.0 the ISB-CGC team May 29, 2020 GETTING STARTED 1 About the ISB-CGC Platform 3 2 ISB-CGC Data Overview 5 3 Quick-Start Guide 7 4 Getting Started with Analysis 9 5 How to Request Cloud Credits 13 6 Best Practices 15 7 Benefits of Using The Cloud 17 8 Mitelman Database 19 9 Programs and Data Sets 21 10 Data Access and Security Overview 47 11 Accessing Controlled Data 49 12 ISB-CGC BigQuery Tables 63 13 ISB-CGC BigQuery Table Search 85 14 ISB-CGC Data Explorer 93 15 ISB-CGC Web Interface (Web App) 95 16 ISB-CGC APIs 145 17 ISB-CGC Notebooks 149 18 Statistical Notebooks 151 19 Tutorials and How-To Guides 153 20 Release Notes 323 i 21 Quick Links 377 22 Frequently Asked Questions (FAQ) 381 23 Contact Us 389 ii ISB Cancer Genomics Cloud Documentation, Release 1.0.0 Democratizing access to cancer data in the cloud Contained within this documentation are descriptions of ISB-CGC features along with guides and tips for exploring data sets hosted on the Google Cloud Platform. The ISB-CGC aims to serve the needs of a broad range of cancer researchers ranging from scientists or clinicians who prefer to use an interactive web-based application to access and explore the rich TCGA, TARGET, CCLE, and COSMIC datasets, to computational scientists who want to write their own custom scripts using languages such as R or Python, accessing the data through APIs, and to algorithm developers who wish to spin up thousands of virtual machines to analyze hundreds of terabytes of sequence data. – the ISB-CGC team GETTING STARTED 1 ISB Cancer Genomics Cloud Documentation, Release 1.0.0 2 GETTING STARTED CHAPTER 1 About the ISB-CGC Platform The ISB Cancer Genomics Cloud (ISB-CGC) is one of three National Cancer Institute (NCI) Cloud Resources tasked with bringing cancer data and computation power together through cloud platforms. It is a collaboration between the Institute for Systems Biology (ISB) and General Dynamics Information Technology Inc. (GDIT). Since starting in 2014 as part of NCI’s Cloud Pilot Resource initiative, ISB-CGC has provided access to increasing amounts of cancer data in the cloud. 1.1 Exploring Cancer Data The ISB-CGC Platform enables a wide range of users to bring their analysis tools to the data in the cloud, eliminating the need to download and store large data sets. Built with the Google Cloud Platform, it provides several entry points for exploring and analyzing cancer data: • The ISB-CGC Web Application allows users to interactively create and explore cohorts of interest. • The ISB-CGC API gives users the ability to programmatically work with data such as cases, samples, cohorts, files and cloud projects. • The ISB-CGC BigQuery Table Search is a discovery tool that allows the user to explore and search for ISB- CGC Google BiqQuery tables. • On the Google Cloud Platform BigQuery Console, ISB-CGC tables can be viewed and queried directly. • Python and R can interface with the ISB-CGC tables, retrieving and analyzing data. • Using Google Compute Engines and VMs, workflows can be run to perform data analysis. Please see the USER GUIDE section to learn more about each of these tools and the MORE INFORMATION section to see examples, tutorials, Jupyter and R Notebooks, Frequently Asked Questions and more. 3 ISB Cancer Genomics Cloud Documentation, Release 1.0.0 Have feedback or corrections? Please email us at CHAPTER 2 ISB-CGC Data Overview ISB-CGC provides access to data from several research programs, such as The Cancer Genome Atlas (TCGA), Therapeutically Applicable Research to Generate Effective Treatments (TARGET), Cancer Cell Line Encyclo- pedia (CCLE) and Catalogue of Somatic Mutations in Cancer (COSMIC). The full list is available here. The majority of the data made available through ISB-CGC originates from NCI Genomic Data Commons (GDC). Users can access GDC data on the cloud through ISB-CGC. Users have access to both raw and processed data from cancer patients. In general, almost all raw data is controlled-access and is accessible through Google Cloud Storage buckets; only those users with proper authorization can access them. The GDC has established bioinformatics workflows/pipelines exe- cuted on the raw data to generate processed data. In this way, users can directly access the processed data without hav- ing to run compute-intensive workflows themselves. However, users who wish to run their own workflows/pipelines still have access to the raw data as well. GDC processed data, however, are generally open-access. ISB-CGC allows users to utilize this processed data in two ways on the platform: • Google Cloud Storage: All individual processed data files are accessible through GDC Google Cloud Storage buckets; ISB-CGC provides pointers to these files. • Google BigQuery: Processed data are consolidated by datatype (ex. Clinical, DNA Methylation, RNAseq, Somatic Mutation, etc.) and transformed into ISB-CGC Google BigQuery tables for ease of access and analysis. This novel approach allows our users to quickly analyze information from thousands of patients in our curated BigQuery tables. 5 ISB Cancer Genomics Cloud Documentation, Release 1.0.0 2.1 Google Cloud Storage Google Cloud Storage (GCS) is a cloud-based object-store that is used to store many types of (usually binary) data, typically processed by custom software pipelines. The data hosted by GDC is contained within Google Cloud Storage. Metadata stored within ISB-CGC BigQuery tables contains pointers to file locations in this GDC data. 2.2 Google BigQuery Google BigQuery (BQ) is a columnar database ideal for storing tabular data. Its query speed is automatically scaled by multiprocessing. Data is accessed using a powerful SQL language interface. ISB-CGC stores high-level clinical, biospecimen, and molecular data from the main NCI programs in the BigQuery project isb-cgc. It also stores a large amount of metadata about files that are stored in the GDC Google Cloud Storage, as well as genome reference sources (e.g. GENCODE, miRBase, etc.). All of these data sets and tables are completely open access and available to the research community. Have feedback or corrections? Please email us at

See more

The list of books you might like

Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.