---
title: "TCGAbiolinks: Clinical data"
date: "`r BiocStyle::doc_date()`"
vignette: >
  %\VignetteIndexEntry{"TCGAbiolinks: Clinical data"}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
knitr::opts_knit$set(progress = FALSE)
```

```{r message=FALSE, warning=FALSE, include=FALSE}
library(TCGAbiolinks)
library(SummarizedExperiment)
library(dplyr)
library(DT)
```


**TCGAbiolinks** has provided a few functions to search, download and parse clinical data.
This section starts by explaining the different sources for clinical information in GDC, followed by the necessary
function to access these sources and it finishes by showing the insconsistencies between those sources.


---
# Useful information


<div class="panel panel-info">
<div class="panel-heading">Different sources</div>
<div class="panel-body">

In GDC database the clinical data can be retrieved from two sources:

- indexed clinical: a refined clinical data that is created using the XML files.
- XML files

There are two main differences:

- XML has more information: radiation, drugs information, follow-ups, biospecimen, etc. So the indexed one is only a subset of the XML files
- The indexed data contains the updated data with the follow up informaiton. 
  For example: if the patient is alive in the first time clinical data was collect and the in the next follow-up he is dead, 
  the indexed data will show dead. The XML will have two fields, one for the first time saying he is alive (in the clinical part) and the follow-up saying he is dead. You can see this case here: 

</div>
</div>


# Get clinical indexed data

In this example we will fetch clinical indexed data.

```{r results='hide', echo=TRUE, message=FALSE, warning=FALSE}
clinical <- GDCquery_clinic(project = "TCGA-LUAD", type = "clinical")
```
```{r  echo=TRUE, message=FALSE, warning=FALSE}
datatable(clinical, filter = 'top', 
          options = list(scrollX = TRUE, keys = TRUE, pageLength = 5),  
          rownames = FALSE)
```


# Parse XML clinical data

In this example we will fetch clinical data directly from the clinical XML files.

```{r results = 'hide', echo=TRUE, message=FALSE, warning=FALSE}
query <- GDCquery(project = "TCGA-COAD", 
                  data.category = "Clinical", 
                  barcode = c("TCGA-RU-A8FL","TCGA-AA-3972"))
GDCdownload(query)
clinical <- GDCprepare_clinic(query, clinical.info = "patient")
```
```{r  echo = TRUE, message = FALSE, warning = FALSE}
datatable(clinical, options = list(scrollX = TRUE, keys = TRUE), rownames = FALSE)
```
```{r results = 'hide', echo=TRUE, message=FALSE, warning=FALSE}
clinical.drug <- GDCprepare_clinic(query, clinical.info = "drug")
```
```{r  echo = TRUE, message = FALSE, warning = FALSE}
datatable(clinical.drug, options = list(scrollX = TRUE, keys = TRUE), rownames = FALSE)
```
```{r results = 'hide', echo=TRUE, message=FALSE, warning=FALSE}
clinical.radiation <- GDCprepare_clinic(query, clinical.info = "radiation")
```
```{r  echo = TRUE, message = FALSE, warning = FALSE}
datatable(clinical.radiation, options = list(scrollX = TRUE,  keys = TRUE), rownames = FALSE)
```
```{r results = 'hide', echo=TRUE, message=FALSE, warning=FALSE}
clinical.admin <- GDCprepare_clinic(query, clinical.info = "admin")
```
```{r  echo = TRUE, message = FALSE, warning = FALSE}
datatable(clinical.admin, options = list(scrollX = TRUE, keys = TRUE), rownames = FALSE)
```

# Clinical data inconsistencies

<div class="panel panel-danger">
<div class="panel-heading">Clinical data inconsistencies</div>
<div class="panel-body">

Some inconsisentecies have been found in the indexed clinical data and are being
investigated by the GDC team. These inconsistencies are:

- ***Vital status*** field is not correctly updated 
- ***Tumor Grade*** field is not being filled
- ***Progression or Recurrence*** field is not being filled

</div>
</div>

## Vital status inconsistancie

```{r results = 'hide', echo=TRUE, message=FALSE, warning=FALSE}
# Get XML files and parse them
clin.query <- GDCquery(project = "TCGA-READ", data.category = "Clinical", barcode = "TCGA-F5-6702")
GDCdownload(clin.query)
clinical.patient <- GDCprepare_clinic(clin.query, clinical.info = "patient")
clinical.patient.followup <- GDCprepare_clinic(clin.query, clinical.info = "follow_up")

# Get indexed data
clinical.index <- GDCquery_clinic("TCGA-READ")
```
```{r  echo = TRUE, message = FALSE, warning = FALSE}
select(clinical.patient,vital_status,days_to_death,days_to_last_followup) %>% datatable
select(clinical.patient.followup, vital_status,days_to_death,days_to_last_followup) %>% datatable
# Vital status should be the same in the follow up table 
filter(clinical.index,submitter_id == "TCGA-F5-6702") %>% select(vital_status,days_to_death,days_to_last_follow_up) %>% datatable
```

## Progression or Recurrence and Grande inconsistancie

```{r results = 'hide', echo=TRUE, message=FALSE, warning=FALSE}
# Get XML files and parse them
recurrent.samples <- GDCquery(project = "TCGA-LIHC",
                             data.category = "Transcriptome Profiling",
                             data.type = "Gene Expression Quantification", 
                             workflow.type = "HTSeq - Counts",
                             sample.type = 	"Recurrent Solid Tumor")$results[[1]] %>% select(cases)
recurrent.patients <- unique(substr(recurrent.samples$cases,1,12))
clin.query <- GDCquery(project = "TCGA-LIHC", data.category = "Clinical", barcode = recurrent.patients)
GDCdownload(clin.query)
clinical.patient <- GDCprepare_clinic(clin.query, clinical.info = "patient") 
```
```{r  echo = TRUE, message = FALSE, warning = FALSE}
# Get indexed data
GDCquery_clinic("TCGA-LIHC") %>% filter(submitter_id %in% recurrent.patients) %>% 
    select(progression_or_recurrence,days_to_recurrence,tumor_grade) %>% datatable

# XML data
clinical.patient %>% select(bcr_patient_barcode,neoplasm_histologic_grade) %>% datatable

```