broadinstitute / catch / 40

94%

push

travis-ci

Update datasets using 2018-10 pull of viral sequence data

This updates viral datasets based on a pull of NCBI's
viral accession list on 2018-10-19 and a pull of NCBI's
Influenza virus database accessions on 2018-10-20.

The updated accessions add sequences and remove sequences
relative to what was previously used to generate datasets.
Furthermore, there are many more species included in this
pull than were previously present; this is due to more
species being present in the accession list as well as
the inclusion of additional species that were not previously
considered (in the pull) to have human as a host (including
a manual selection of species that are not listed by NCBI
as having human as a host). In particular, these datasets
encompass 588 viral species known to infect human.

This also fundamentally changes how datasets are composed.
Now, each dataset corresponds precisely to one species.
The dataset names reflect the species names, as listed
in NCBI's taxonomy (except a species name trailing in
' virus' has ' virus' removed).

There are small changes in the generation of the dataset
.py files compared to how they were previously written.

The data are now reorganized in two ways. First, every
FASTA is now compressed (gzip). Second, all segments
across all genomes of segmented viruses are now stored
in a single FASTA file; the sequence names in this file
are all modified to identify the segment and a genome.
(Previously, data for datasets corresponding to segmented
viruses were stored in separate directories, such that
in each directory every genome was stored in a separate
file that consisted of its segments. This could became
unwieldy for datasets with a lot of genomes.)

This commit makes several additional changes related
to the updated datasets: The datasets README.md is
updated to show the new datasets. Unit tests that
use these datasets are also updated.

1537 of 1721 branches covered (89.31%)

11 of 11 new or added lines in 2 files covered. (100.0%)

4713 of 4968 relevant lines covered (94.87%)

0.95 hits per line