• Home
  • Features
  • Pricing
  • Docs
  • Announcements
  • Sign In

broadinstitute / hellbender
71%
master: 81%

Build:
Build:
LAST BUILD BRANCH: dr_disentangle_spark_and_dataflow
DEFAULT BRANCH: master
Repo Added 06 Jan 2015 07:33PM UTC
Files 539
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

LAST BUILD ON BRANCH lb_dataflow
branch: lb_dataflow
CHANGE BRANCH
x
Reset
  • lb_dataflow
  • AddLogStatements
  • BRUTest
  • CTBuilder
  • ak_AddOrReplaceReadGroups_test
  • ak_Annotator
  • ak_CalculateReadGroupChecksum_tests
  • ak_CigarUtils_tests
  • ak_CollectBaseDistributionByCycle_tests
  • ak_CollectQualityYieldMetrics_test
  • ak_CollectWgsMetrics_test
  • ak_LeftAlignIndels
  • ak_LiftOverIntervalList_test
  • ak_MeanQualityByCycle_spark2
  • ak_MisencodedBaseQualityReadTransformer
  • ak_QualityScoreDistribution_tests
  • ak_ReadLikelihoodCalculationEngine
  • ak_ReplaceSamHeader_test
  • ak_VariantAnnotatorEngine
  • ak_add_cloudera
  • ak_add_google
  • ak_analyzeCovariates
  • ak_apply_vqsr_onTranchesBranch
  • ak_bqsr_NPE
  • ak_bqsr_multiple_site_lists
  • ak_bqsr_part2
  • ak_bqsr_pipeline_test
  • ak_clipReads
  • ak_clone
  • ak_clp_test
  • ak_codecs
  • ak_copyright
  • ak_countBases_spark
  • ak_countReads_spark
  • ak_countReads_spark_moreTests
  • ak_coverage_spark
  • ak_dataflow_codereading
  • ak_dead_code_removal
  • ak_delete_BamToBfq
  • ak_delete_code_not_being_ported_from_picard
  • ak_diamonds
  • ak_diff_engine
  • ak_disable_broken_picard_tests
  • ak_enable_ReadCovariatesUnitTest
  • ak_exampleVariantWalker
  • ak_feature_input
  • ak_flagstat_spark
  • ak_gatreport
  • ak_genotyper
  • ak_good_read_filter
  • ak_hc_assemblygraph
  • ak_improve_testing_coverage
  • ak_left_tail_bqsr
  • ak_log4j2_xml
  • ak_logging_info_in_readme
  • ak_malformed_read_filter
  • ak_mark_duplicates
  • ak_meanqualitybycycle_tests
  • ak_pairhmm
  • ak_pileup_work
  • ak_printReadsOptionsAndTests
  • ak_printReads_spark
  • ak_readClipperUnitTests
  • ak_read_additional_filters
  • ak_readme_on_testing
  • ak_remove_multiThreading_from_ExtractIlluminaBarcodes
  • ak_remove_readFilterTest
  • ak_replaceCustomGamma_withApache
  • ak_simplify_covariates
  • ak_small_readme_edits
  • ak_snacks
  • ak_some_code_cleanup
  • ak_sparkToolName
  • ak_splitNCigarReads
  • ak_test_LeftAlignIndels
  • ak_validate_variants
  • ak_variant_filtration
  • ak_variant_recalibrator
  • ak_variant_walker
  • ak_vqsr_tranche_doc
  • ak_vqsr_tranches
  • ak_vqsr_variant_data_management
  • ak_warnings
  • ak_wellformed_filter_tests
  • akiezun-README-guidelines
  • akiezun-readme-style_tweaks
  • bh_prune_vcf
  • cn_filter_reads
  • cn_gatk_read
  • cn_haplotype_bamwriter
  • cn_log4j_picard
  • cn_read_filters
  • cn_select_variants
  • cn_set_log4j_level
  • cn_split_reads
  • cs_basicqcmetrics
  • da_auth
  • da_copy_deps
  • da_credential_factory
  • da_credentials
  • da_dataflow_version_bump
  • da_genomics_pipeline_deps
  • da_hook_up_mark_dupes
  • da_keyed_transforms
  • da_less_brittle_dataflow_tests
  • da_mark_dupes_fusion_fun
  • da_mark_dupes_refactor
  • da_playground2
  • da_read_pipeline
  • da_spark_integration
  • da_spark_integration_test
  • da_spark_local
  • da_spark_mark_dupes
  • da_spark_markdupes
  • da_spark_skeleton
  • da_spark_tuning
  • da_variant_start
  • db_covariance
  • db_genotyper_logarithms
  • db_kill_median
  • db_natural_log
  • db_target_collection
  • dek_CollectJumpingLibraryMetrics_IT
  • dek_CollectTargetedPcrMetrics_IT
  • dek_RenameSampleInVCF_test
  • dek_normalizeFasta_IT
  • dr_activate_cloud_tests_for_real
  • dr_add_dbsnp_vcf_for_chr1
  • dr_add_intellij_setup_to_readme
  • dr_add_large_test_files_with_git-lfs
  • dr_add_tests_to_ref_api_source
  • dr_add_yarn_executor_overhead
  • dr_artificialreadutils_specify_contig_in_artificial_uuid_methods
  • dr_authenticated_pipeline_options_test_helper
  • dr_change_logging_level
  • dr_clarify_testing_requirements_for_tools
  • dr_customize_reference_window_in_dataflow_sharding
  • dr_disable_parallel_tests
  • dr_disentangle_spark_and_dataflow
  • dr_dummy_feature
  • dr_dynamic_window_in_referencecontext
  • dr_expose_feature_headers
  • dr_extract_readssource_class
  • dr_feature_support
  • dr_feature_traversal_by_intervals
  • dr_featureinput_tostring
  • dr_features_as_intervals
  • dr_fix_featuredatasource_caching
  • dr_fix_gradle_fatjar
  • dr_fix_ioutils_createtempfile
  • dr_fix_sequence_dict_check_in_bqsr_dataflow
  • dr_gatkcoder_test
  • dr_gatksparktool
  • dr_handle_reads_that_consume_no_ref_bases_in_readwalkers
  • dr_initial_traversal_engine
  • dr_interval_walker
  • dr_lb_spark_bqsr
  • dr_lb_spark_bqsr_bak
  • dr_lb_spark_test
  • dr_make_context_objects_non_optional
  • dr_more_contigs_in_artificialreadutils
  • dr_move_simpleinterval_to_hellbender
  • dr_output_logging_messages_during_engine_initialization
  • dr_port_tools_to_readwalker_interface
  • dr_read_adapter_equals_tweak
  • dr_read_common_interfaces
  • dr_reenable_broken_bqsr_test
  • dr_refactor_exceptions
  • dr_relax_sequence_dictionary_checks
  • dr_remove_com_sun_import
  • dr_remove_protective_copies_from_spark_bqsr
  • dr_rename_sam_utils_package
  • dr_rev_htsjdk_to_1.132
  • dr_rev_htsjdk_to_1.139
  • dr_rev_htsjkd_to_1.138
  • dr_spark_pipeline_markdupes_sharding_fix
  • dr_stub_reads_pipelineTest
  • dr_test_cram_with_intervals
  • ek_comment_802
  • ek_countreads_590
  • ek_craminttests_675
  • ek_cramreference_606
  • ek_cramreference_677
  • ek_fixtests_659
  • ek_fixwarnings_359_377
  • ek_locatable_559
  • ek_npe_273
  • ek_refdict_802
  • ek_seqdict_101
  • ek_unmapped_766
  • ek_wordcount_537
  • expand_dataflow_test_contigs
  • firstCheckin
  • fix_gradle_one_more_time
  • gvda_authorlist_sorting
  • gvda_current_gatk_links
  • issue-948-option-duplicate-reads-exomereadcounts
  • jp_ApiMeansNoSecrets
  • jp_ApplyBQSR
  • jp_ApplyBQSRCloud2
  • jp_BaseRecalOutputs
  • jp_CEU_test_data
  • jp_CloudTests
  • jp_DFBR_cloud
  • jp_DF_BaseRecalibrator
  • jp_FewReadsWriter
  • jp_ReadSourceStringency
  • jp_RecalibrationTablesMerger
  • jp_adding_first_test
  • jp_apply_skel_rebased
  • jp_badtype_repro
  • jp_bigger_apiref
  • jp_binarysearch
  • jp_bqsr_newtest
  • jp_bqsr_skel_rerebased
  • jp_consistentWhitespace
  • jp_enable_working_tests
  • jp_fixFatJar
  • jp_fix_BaseRecal_test
  • jp_fix_refapisource
  • jp_fix_secrets
  • jp_intervalsskiplist
  • jp_perf_apply2
  • jp_perf_br
  • jp_perf_markdups
  • jp_readstest
  • jp_sam2gatk
  • jp_skel_01
  • jp_skel_03
  • jp_start_with_mandatory_tests
  • jp_start_with_mandatory_tests_rebased
  • jp_start_with_mandatory_tests_rebased2
  • jp_testReadWrite
  • jp_uprev_genomics-dataflow
  • jpm_bqsr
  • jt_typo
  • ks_print_reads_by_read_group
  • lb_add_key_for_travis
  • lb_add_packing_command
  • lb_add_rm_rf_shutdownhook_for_da
  • lb_bqsr_scaling_tweaks
  • lb_change_cran_mirror
  • lb_codeship
  • lb_coveralls
  • lb_dataflow_dependencies
  • lb_delete_commandlineparse_exception
  • lb_delete_compare_metrics
  • lb_delete_illumina_test_data
  • lb_delete_unused_classes_relating_to_MarkDuplicates
  • lb_dockerizing_for_execution_service
  • lb_enable_circle_ci
  • lb_error_message_when_R_doesnt_exist
  • lb_exception_hidden_by_onShutdown_exception
  • lb_expose_test_classes
  • lb_fix_137_truncate_names_with_whitespace
  • lb_fix_ambigous_parse_with_optional_args
  • lb_fix_bqsr_concurrency_issue
  • lb_fix_build_failure_from_no_git
  • lb_fix_default_collections
  • lb_fix_documentation_error
  • lb_fix_last_4_warnings
  • lb_fix_monstrous_error_message
  • lb_fix_required_collections
  • lb_fix_travis_button
  • lb_gatktool_argument_collection_enhancement
  • lb_hide_secret_key
  • lb_htsjdk_to_130
  • lb_improve_cli_output_for_collections
  • lb_improve_parsing_errors
  • lb_improved_R_installation
  • lb_insertSizeMetricDataflow
  • lb_java_logger_levels
  • lb_killing_warnings_again
  • lb_limit_circle_ci_gradle_memory
  • lb_lower_git_lfs_usage
  • lb_lsystem
  • lb_mail_test_result
  • lb_make_working_cloud_tests_mandatory
  • lb_mavenization
  • lb_minor_utils_cleanups
  • lb_misencoded_base_test_update
  • lb_move_placement_of_travis_wait
  • lb_move_testSortingFile_somewhere_sensible
  • lb_move_unreliable_test_to_dataflow_todo
  • lb_only_download_gsalib_when_needed
  • lb_optimize_GATKRead_getLength
  • lb_option_to_run_all_tests
  • lb_output_failed_tests
  • lb_parallel_tests
  • lb_port_gvcfwriter
  • lb_r_installation_refactoring
  • lb_r_output_improvement
  • lb_reduce_test_time_for_SplitNCigarReads
  • lb_reinstate_transformers_and_filters
  • lb_remove_deprecated
  • lb_remove_encrypted_key_for_now
  • lb_remove_min_max_elems_cli
  • lb_remove_sudo
  • lb_remove_the_vampire_that_haunts_valentine
  • lb_remove_unecessary_gradle_download
  • lb_remove_unused_genotypeutils
  • lb_rename_option
  • lb_rename_usage_short
  • lb_replace_TestPipeline_with_GatkTestPipeline
  • lb_replace_file_create_temp_file
  • lb_return_values
  • lb_scale_back_R
  • lb_scaling
  • lb_simple_interval_constructor
  • lb_sort_dependencies
  • lb_stop_creating_test_file_in_basedir
  • lb_suppress_warnings
  • lb_test_speedup
  • lb_transfer
  • lb_travis_container_build
  • lb_travis_lfs
  • lb_unix_style_options
  • lb_update_ReadSource_for_public_consumption
  • lb_update_dependencies
  • lb_update_gradle_to_2.7
  • lb_update_htsjdk
  • lb_update_license_to_bsd
  • lb_update_sam_header
  • lb_update_to_central_htsjdk
  • lb_update_to_snapshot_google_http
  • lb_updating_dataflow_sdk_and_others
  • lb_upload_test_reports
  • lb_version_number
  • lbergelson-readme-update
  • master
  • mds_cleanup_picard_package
  • mds_fix_compare_sams
  • mds_fix_picard_issues
  • mds_migrate_picard_sam
  • mds_picard_sam_2
  • mds_port_all_the_things
  • mds_resync_picard_sam
  • mds_update_integration_test
  • mds_vectorize_coveralls_badge
  • tn_mark_duplicates
  • tw_hadoop_bam_writer
  • tw_hadoop_references
  • tw_hadoop_unmapped_reads
  • tw_hadoop_use_seq_dict
  • tw_hadoop_variants
  • tw_local_base_recalibrator_test
  • tw_parallelism_comment
  • tw_r_groups
  • tw_shared_spark_context
  • tw_spark_fix_adam_deps
  • tw_spark_mark_dupes_sharded
  • tw_spark_markdupes
  • tw_spark_markdupes_optimization
  • tw_spark_sam_record_optimization
  • tw_spark_shadowjar_fix
  • tw_spark_sort_bam
  • tw_spark_test_group
  • tw_test
  • tw_ul_spark_bqsr
  • ul_parquet_io
  • ul_random_uuid_bdg
  • vrr_exome_depth
  • vrr_exons_from_file
  • vrr_feature_iterators
  • vrr_help_generation_on_enum_options_change_toSring_to_name
  • vrr_mark_duplicates_erc
  • vrr_pcov_erc
  • vrr_remove_genomeloc_from_erc
  • wb_copyright

pending completion
195

push

travis-ci

lbergelson
Adding a dataflow pipeline for Read only tools
This includes wrappers to present SAMRecords to the tools themselves
Also adding 4 simple tools as examples
  FlagStatsDataflow
    It makes use of dataflow's built in hierarchical aggregation
  CountBases
    Simple walker that makes use of the SAMRecord conversion
  CountReads
    Does what it says
  PrintReads
    This is a very limited version of our print reads walker
    It prints SAMRecords as strings to an unordered text file
    It could potentially be useful as method for examining bam output before we have a proper bam writer

These tools exist in two parts:
  A transform extending from PTransformSAM (A subclass of PTransform<Read,O> which facilitates conversion to SAMRecord
  A command line tool implementing a complete pipeline
    These pipelines can apply arbitrary ReadFilters / ReadTransformers which are applied before the main transform
    (a list of transforms and a list of filters can be applied, it's currently not handled very efficiently though, better to pre-comine them into a single meta transform)

Currently, only tests which use local files are running on travis.
There is code included to run on files in buckets, but the tests for it are currently disabled due to travis configuration issues (will be resolved in a seperate ticket)

Some changes were made to existing classes to make them Serialize properly
Some test files were moved to help normalize test data locations (although not all tests are normalized, should be done in separate ticket)
  the new storage locations are based on the complete package name rather than just the tool name

16296 of 22998 relevant lines covered (70.86%)

0.71 hits per line

Relevant lines Covered
Build:
Build:
22998 RELEVANT LINES 16296 COVERED LINES
0.71 HITS PER LINE
Source Files on lb_dataflow
  • List 0
  • Changed 133
  • Source Changed 133
  • Coverage Changed 13
Coverage ∆ File Lines Relevant Covered Missed Hits/Line

Recent builds

Builds Branch Commit Type Ran Committer Via Coverage
195 lb_dataflow Adding a dataflow pipeline for Read only tools This includes wrappers to present SAMRecords to the tools themselves Also adding 4 simple tools as examples FlagStatsDataflow It makes use of dataflow's built in hierarchical aggregation Count... push 28 Apr 2015 07:08PM UTC lbergelson travis-ci pending completion  
157 lb_dataflow Adding a dataflow pipeline for Read only tools This includes wrappers to present SAMRecords to the tools themselves Also adding 4 simple tools as examples FlagStatsDataflow It makes use of dataflow's built in hierarchical aggregation Count... push 24 Apr 2015 09:13PM UTC lbergelson travis-ci pending completion  
156 lb_dataflow "saving everything in a horrible state" push 24 Apr 2015 09:03PM UTC lbergelson travis-ci pending completion  
112 lb_dataflow disablign bucket tests and non-dataflow ones push 22 Apr 2015 09:06PM UTC lbergelson travis-ci pending completion  
582 lb_dataflow fixing the generics issue with PipelineRunner push 10 Mar 2015 03:29PM UTC lbergelson travis-pro pending completion  
579 lb_dataflow changes to work on cloud push 10 Mar 2015 03:08PM UTC lbergelson travis-pro pending completion  
See All Builds (2517)
  • Repo on GitHub
STATUS · Troubleshooting · Open an Issue · Sales · Support · CAREERS · ENTERPRISE · START FREE · SCHEDULE DEMO
ANNOUNCEMENTS · TWITTER · TOS & SLA · Supported CI Services · What's a CI service? · Automated Testing

© 2025 Coveralls, Inc